Skip to content

Batch chunk requests, add region LOD, fix unbounded chunk growth - #86

Open
gh8sted wants to merge 3 commits into
OurSources:masterfrom
gh8sted:batched-chunk-loading
Open

Batch chunk requests, add region LOD, fix unbounded chunk growth#86
gh8sted wants to merge 3 commits into
OurSources:masterfrom
gh8sted:batched-chunk-loading

Conversation

@gh8sted

@gh8sted gh8sted commented Aug 15, 2026

Copy link
Copy Markdown

Batched requests

requestMissingChunks scanned the visible rect and sent one 8-byte packet per chunk. It now collects the rect and sends a single batched packet, and parses the batched 0x0B response.

Chunks are gathered in expanding rings from the middle of the view, so the area actually being looked at is requested first. At most 4,096 are in flight at once: a very zoomed-out screen can be tens of thousands of chunks, and committing to all of it up front means the server keeps sending chunks for where the camera used to be. The remainder is picked up in waves as chunks land.

(note: for bots that export large areas, you can do it faster now with requesting batches! like it goes from a minute to like mere seconds with a big area.)

unloadFarChunks never existed

main.js exposed it on the public API and options.unloadDistance was set, but the method was not on World — calling it threw. Nothing ever freed a chunk while playing.

Watched live, the chunk map grew past 224,000 entries with 65 cluster canvases (~260MB of canvas) and a 300MB heap, still climbing.

Implementing it was all that was missing; the rest of the teardown path (chunk.remove()chunkUnloadedrmChunkdelChunk) already worked. After the fix, panning 24,000px takes the map from 20,559 chunks down to 10,691 and the heap from 91MB to 81MB — bounded and self-correcting rather than growing without limit.

Region LOD

Previews are painted straight into the cluster canvas rather than becoming Chunk objects, so existing bookkeeping is untouched and real chunks simply overwrite the coarse pixels when they arrive. Chunks already held at full detail are skipped. Previews are requested only above 2,048 visible chunks, and only for regions not already fetched — while panning that is normally just the strip coming into view.

Measured: 180× less data, with a complete coarse view in 35ms against 416ms for full detail.

Smaller fixes

  • The export tool exported blank areas. getPixel returns null for unloaded chunks and the pixel loop skipped them, so it exported whatever happened to be in memory rather than the selected region. It now loads the selection first, with progress shown, and reports in chat if anything could not be fetched. On a 512×512 area the client had never viewed, this went from 1,024 missing chunks to 0.
  • Identical redeliveries fired the paste effect. Chunk data arriving for a chunk already held was treated as an update, repainting it and flashing the fade for no visible change. Now suppressed unless the pixels actually differ: 484 occurrences in one run went to 0.
  • Per-chunk decoding did two passes with two allocations; it is now a single pass straight to pixels. fillFromBuf copies a row at a time rather than per element, and cluster uploads coalesce into one putImageData once enough chunks are dirty.

Caveat on the numbers

Transfer figures come from a test world of per-pixel random noise, which defeats the run-length encoding completely (~780 bytes per chunk). Real world data compresses far better, so the byte counts here are pessimistic. LOD's benefit likewise scales with view size — on a small area it is much closer to full detail, because both paths still load the same regions from disk.

gh8sted and others added 3 commits August 15, 2026 01:34
Cuts the added commentary down to wire formats and non-obvious invariants.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant