Chunks get loaded while chunks are unloading leading to potential crashes

[mjwells2002]

Version Information

lithium-fabric-mc1.21-0.12.7

Expected Behavior

unloaded chunks should stay unloaded, not trigger game events that could potentially load them again

Actual Behavior

the chunks unload, and then enter a potentially endless loop of loading and unloading which could lead to a server crash

Reproduction Steps

build the shown setup with multiple mobs in a boat on a chunk boundry across to a chunk border with a sculk sensor next to it, flying away from this or otherwise unloading it will cause the chunks the setup occupied to then be loaded and unloaded every game tick

image from MCRcortex

Other Information

issue was orignally reported in the caffinemc discord in dev-lithium channel, following stack trace from our debug mod (yarn mappings) shows a case where this issue could potentially crash the server CheckingLongOpenHashSet replaces the normal LongOpenHashSet in ServerEntityManager, and you can see that remove was called while in the removeIf function

[13:09:22] [Server thread/ERROR]: IN PREDICATE REMOVAL
java.lang.Throwable: null
	at me.cortex.hc_debug.Hc_debug.ERR(Hc_debug.java:15) ~[hc_debug-1.0-SNAPSHOT(1).jar:?]
	at me.cortex.hc_debug.CheckingLongOpenHashSet.remove(CheckingLongOpenHashSet.java:47) ~[hc_debug-1.0-SNAPSHOT(1).jar:?]
	at net.minecraft.server.world.ServerEntityManager.updateTrackingStatus(ServerEntityManager.java:236) ~[server-intermediary.jar:?]
	at net.minecraft.server.world.ServerEntityManager.updateTrackingStatus(ServerEntityManager.java:226) ~[server-intermediary.jar:?]
	at net.minecraft.server.world.ServerWorld.md6c3bd0$lithium$lambda$combineWithLithiumChunkStatusTracker$0$8(ServerWorld.java:5265) ~[server-intermediary.jar:?]
	at net.minecraft.server.world.ServerChunkLoadingManager.onChunkStatusChange(ServerChunkLoadingManager.java:1209) ~[server-intermediary.jar:?]
	at net.minecraft.server.world.ChunkHolder.method_31412(ChunkHolder.java:274) ~[server-intermediary.jar:?]
	at java.base/java.util.concurrent.CompletableFuture$UniRun.tryFire(CompletableFuture.java:787) ~[?:?]
	at java.base/java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:482) ~[?:?]
	at net.minecraft.util.thread.ThreadExecutor.executeTask(ThreadExecutor.java:162) ~[server-intermediary.jar:?]
	at net.minecraft.server.world.ServerChunkManager$class_4212.executeTask(ServerChunkManager.java:552) ~[server-intermediary.jar:?]
	at net.minecraft.util.thread.ThreadExecutor.runTask(ThreadExecutor.java:136) ~[server-intermediary.jar:?]
	at net.minecraft.server.world.ServerChunkManager$class_4212.runTask(ServerChunkManager.java:561) ~[server-intermediary.jar:?]
	at net.minecraft.util.thread.ThreadExecutor.runTasks(ThreadExecutor.java:145) ~[server-intermediary.jar:?]
	at net.minecraft.server.world.ServerChunkManager$class_4212.runTasks(ServerChunkManager.java:525) ~[server-intermediary.jar:?]
	at net.minecraft.server.world.ServerChunkManager.getChunkBlocking(ServerChunkManager.java:1245) ~[server-intermediary.jar:?]
	at net.minecraft.server.world.ServerChunkManager.getChunk(ServerChunkManager.java:1166) ~[server-intermediary.jar:?]
	at net.minecraft.world.World.getChunk(World.java:208) ~[server-intermediary.jar:?]
	at net.minecraft.world.World.getChunk(World.java:5594) ~[server-intermediary.jar:?]
	at net.minecraft.world.World.getBlockState(World.java:6086) ~[server-intermediary.jar:?]
	at net.minecraft.world.BlockView.method_32881(BlockView.java:62) ~[server-intermediary.jar:?]
	at net.minecraft.world.BlockView.raycast(BlockView.java:153) ~[server-intermediary.jar:?]
	at net.minecraft.world.BlockView.raycast(BlockView.java:61) ~[server-intermediary.jar:?]
	at net.minecraft.Vibrations$ListenerDataclass_8516.isOccluded(Vibrations.java:318) ~[server-intermediary.jar:?]
	at net.minecraft.Vibrations$ListenerDataclass_8516.listen(Vibrations.java:269) ~[server-intermediary.jar:?]
	at net.minecraft.world.event.listener.GameEventDispatchManager.method_45492(GameEventDispatchManager.java:37) ~[server-intermediary.jar:?]
	at net.minecraft.world.event.listener.SimpleGameEventDispatcher.dispatch(SimpleGameEventDispatcher.java:79) ~[server-intermediary.jar:?]
	at net.minecraft.world.event.listener.GameEventDispatchManager.redirect$bbn000$lithium$handleNullDispatcher(GameEventDispatchManager.java:565) ~[server-intermediary.jar:?]
	at net.minecraft.world.event.listener.GameEventDispatchManager.redirect$bbn000$lithium$handleNullDispatcher$mixinextras$bridge$7(GameEventDispatchManager.java) ~[server-intermediary.jar:?]
	at net.minecraft.world.event.listener.GameEventDispatchManager.dispatch(GameEventDispatchManager.java:48) ~[server-intermediary.jar:?]
	at net.minecraft.server.world.ServerWorld.emitGameEvent(ServerWorld.java:1002) ~[server-intermediary.jar:?]
	at net.minecraft.world.WorldAccess.emitGameEvent(WorldAccess.java:111) ~[server-intermediary.jar:?]
	at net.minecraft.entity.Entity.emitGameEvent(Entity.java:1218) ~[server-intermediary.jar:?]
	at net.minecraft.entity.Entity.removePassenger(Entity.java:2292) ~[server-intermediary.jar:?]
	at net.minecraft.entity.Entity.dismountVehicle(Entity.java:2254) ~[server-intermediary.jar:?]
	at net.minecraft.entity.Entity.stopRiding(Entity.java:2259) ~[server-intermediary.jar:?]
	at net.minecraft.entity.LivingEntity.stopRiding(LivingEntity.java:2963) ~[server-intermediary.jar:?]
	at net.minecraft.entity.mob.MobEntity.stopRiding(MobEntity.java:2678) ~[server-intermediary.jar:?]
	at com.google.common.collect.ImmutableList.forEach(ImmutableList.java:422) ~[guava-32.1.2-jre.jar:?]
	at net.minecraft.entity.Entity.setRemoved(Entity.java:3708) ~[server-intermediary.jar:?]
	at net.minecraft.server.world.ServerEntityManager.unload(ServerEntityManager.java:333) ~[server-intermediary.jar:?]
	at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1709) ~[?:?]
	at java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:782) ~[?:?]
	at net.minecraft.server.world.ServerEntityManager.method_31858(ServerEntityManager.java:319) ~[server-intermediary.jar:?]
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1597) ~[?:?]
	at net.minecraft.server.world.ServerEntityManager.trySave(ServerEntityManager.java:300) ~[server-intermediary.jar:?]
	at net.minecraft.server.world.ServerEntityManager.unload(ServerEntityManager.java:316) ~[server-intermediary.jar:?]
	at net.minecraft.server.world.ServerEntityManager.method_31849(ServerEntityManager.java:343) ~[server-intermediary.jar:?]
	at me.cortex.hc_debug.CheckingLongOpenHashSet.lambda$removeIf$0(CheckingLongOpenHashSet.java:35) ~[hc_debug-1.0-SNAPSHOT(1).jar:?]
	at it.unimi.dsi.fastutil.longs.LongCollection.removeIf(LongCollection.java:274) ~[fastutil-8.5.12.jar:?]
	at me.cortex.hc_debug.CheckingLongOpenHashSet.removeIf(CheckingLongOpenHashSet.java:30) ~[hc_debug-1.0-SNAPSHOT(1).jar:?]
	at net.minecraft.server.world.ServerEntityManager.unloadChunks(ServerEntityManager.java:338) ~[server-intermediary.jar:?]
	at net.minecraft.server.world.ServerEntityManager.tick(ServerEntityManager.java:358) ~[server-intermediary.jar:?]
	at net.minecraft.server.world.ServerWorld.tick(ServerWorld.java:417) ~[server-intermediary.jar:?]
	at net.minecraft.server.MinecraftServer.tickWorlds(MinecraftServer.java:1021) ~[server-intermediary.jar:?]
	at net.minecraft.server.dedicated.MinecraftDedicatedServer.tickWorlds(MinecraftDedicatedServer.java:299) ~[server-intermediary.jar:?]
	at net.minecraft.server.MinecraftServer.tick(MinecraftServer.java:912) ~[server-intermediary.jar:?]
	at net.minecraft.server.MinecraftServer.runServer(MinecraftServer.java:697) ~[server-intermediary.jar:?]
	at net.minecraft.server.MinecraftServer.method_29739(MinecraftServer.java:281) ~[server-intermediary.jar:?]
	at java.base/java.lang.Thread.run(Thread.java:1570) [?:?]

if the call to remove shown above happens to cause the LongOpenHashSet to resize, the server will crash with a stack trace that looks like this

---- Minecraft Crash Report ----
// On the bright side, I bought you a teddy bear!

Time: 2024-08-21 22:06:02
Description: Exception ticking world

java.lang.ArrayIndexOutOfBoundsException: Index 1709 out of bounds for length 513
	at it.unimi.dsi.fastutil.longs.LongOpenHashSet$SetIterator.shiftKeys(LongOpenHashSet.java:572)
	at it.unimi.dsi.fastutil.longs.LongOpenHashSet$SetIterator.remove(LongOpenHashSet.java:593)
	at it.unimi.dsi.fastutil.longs.LongCollection.removeIf(LongCollection.java:275)
	at net.minecraft.class_5579.	(class_5579.java:338)
	at net.minecraft.class_5579.method_31809(class_5579.java:358)
	at net.minecraft.class_3218.method_18765(class_3218.java:417)
	at net.minecraft.server.MinecraftServer.method_3813(MinecraftServer.java:1021)
	at net.minecraft.class_3176.method_3813(class_3176.java:299)
	at net.minecraft.server.MinecraftServer.method_3748(MinecraftServer.java:912)
	at net.minecraft.server.MinecraftServer.method_29741(MinecraftServer.java:697)
	at net.minecraft.server.MinecraftServer.method_29739(MinecraftServer.java:281)
	at java.base/java.lang.Thread.run(Thread.java:1583)

[2No2Name]

Workaround (disabling the broken lithium optimization) is in the current releases

[2No2Name]

Still need to find out why exactly this happens and how to fix the optimization