Batch chunk requests, add region LOD, fix unbounded chunk growth - #86
Open
gh8sted wants to merge 3 commits into
Open
Batch chunk requests, add region LOD, fix unbounded chunk growth#86gh8sted wants to merge 3 commits into
gh8sted wants to merge 3 commits into
Conversation
Cuts the added commentary down to wire formats and non-obvious invariants. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Batched requests
requestMissingChunksscanned the visible rect and sent one 8-byte packet per chunk. It now collects the rect and sends a single batched packet, and parses the batched0x0Bresponse.Chunks are gathered in expanding rings from the middle of the view, so the area actually being looked at is requested first. At most 4,096 are in flight at once: a very zoomed-out screen can be tens of thousands of chunks, and committing to all of it up front means the server keeps sending chunks for where the camera used to be. The remainder is picked up in waves as chunks land.
(note: for bots that export large areas, you can do it faster now with requesting batches! like it goes from a minute to like mere seconds with a big area.)
unloadFarChunksnever existedmain.jsexposed it on the public API andoptions.unloadDistancewas set, but the method was not onWorld— calling it threw. Nothing ever freed a chunk while playing.Watched live, the chunk map grew past 224,000 entries with 65 cluster canvases (~260MB of canvas) and a 300MB heap, still climbing.
Implementing it was all that was missing; the rest of the teardown path (
chunk.remove()→chunkUnloaded→rmChunk→delChunk) already worked. After the fix, panning 24,000px takes the map from 20,559 chunks down to 10,691 and the heap from 91MB to 81MB — bounded and self-correcting rather than growing without limit.Region LOD
Previews are painted straight into the cluster canvas rather than becoming
Chunkobjects, so existing bookkeeping is untouched and real chunks simply overwrite the coarse pixels when they arrive. Chunks already held at full detail are skipped. Previews are requested only above 2,048 visible chunks, and only for regions not already fetched — while panning that is normally just the strip coming into view.Measured: 180× less data, with a complete coarse view in 35ms against 416ms for full detail.
Smaller fixes
getPixelreturns null for unloaded chunks and the pixel loop skipped them, so it exported whatever happened to be in memory rather than the selected region. It now loads the selection first, with progress shown, and reports in chat if anything could not be fetched. On a 512×512 area the client had never viewed, this went from 1,024 missing chunks to 0.fillFromBufcopies a row at a time rather than per element, and cluster uploads coalesce into oneputImageDataonce enough chunks are dirty.Caveat on the numbers
Transfer figures come from a test world of per-pixel random noise, which defeats the run-length encoding completely (~780 bytes per chunk). Real world data compresses far better, so the byte counts here are pessimistic. LOD's benefit likewise scales with view size — on a small area it is much closer to full detail, because both paths still load the same regions from disk.