Skip to content

Read render assets from a SC:R install in the wasm bindings - #7

Merged
tec27 merged 4 commits into
mainfrom
wasm-casc-source
Oct 25, 2026
Merged

tec27 merged 4 commits into
mainfrom
wasm-casc-source

Conversation

@2Pacalypse-

@2Pacalypse- 2Pacalypse- commented Oct 22, 2026 •

Copy link
Copy Markdown
Member

Lets the wasm bindings read render assets straight from a SC:R install's CASC storage. Hosts that can read files synchronously (Node, Electron) then need no prefetch and no extracted asset directory. ShieldBattery uses it to render previews of map files in a user's map folders that its server has never seen, using the user's own install, in the same style as server renders. This is B4 from the map render plan.

CascStorage (src/casc.rs)

  • new CascStorage(files, options?) runs broodcasc in wasm over five host callbacks: open(path), size(handle), read(handle, buffer, position), close(handle) and listDir(path). Paths are relative to the install directory. In Node they map one to one onto fs.openSync/fstatSync/readSync/closeSync/readdirSync; examples/casc-files.mjs is that adapter. TypeScript sees the callbacks as CascFiles and the options as CascStorageOptions.
  • read's buffer is a view of wasm memory, so reads land there directly instead of going through a JS buffer and a copy. As a result, the callbacks must not call back into the module.
  • broodcasc is a direct dependency at ea3556d, with default features off (fs doesn't build for wasm32) and fast-inflate on. broodmap-render's own broodcasc dependency only reaches the wasm build through dev-dependencies, so it doesn't supply that feature.
  • broodcasc's provider traits require Send + Sync, so the JS handles get unsafe impls. That's sound because a wasm32 target without atomics has one thread; a compile_error! guards the atomics case. On native targets, wasm-bindgen imports panic, so no handle can be created.
  • Each asset is decoded straight into the Arc<[u8]> that's shared from then on, via broodcasc's read_file_into: one allocation of exactly the decoded size, checked against broodcasc's decoded-size limit before allocating. broodmap-render's native CascSource does the same.
  • Decoded assets go in a cache shared by every MapRenderer loading from the same storage. It evicts the least recently used entries first, cacheBytes defaults to 128 MiB, and entries are shared without copies. Every render needs the ~38 MB SD/mainSD.anim, so without the cache each map would decode it again.
  • storage.version is the install's game version, for keying image caches; cachedBytes reports the cache's size.

MapRenderer.loadAssets(storage, options)

  • Runs both prefetch rounds against the storage.
  • Returns the art assets the install doesn't have; those drawables degrade, as they do with the prefetch.
  • Throws on a missing base asset (tileset or .dat table) or on any read error, so callers don't cache a degraded image because of a transient I/O failure.

Build

  • The workspace release profile now uses lto = true and codegen-units = 1. That makes the .wasm about 8% smaller (1.86 → 1.71 MB) at the same speed; incremental rebuilds are about as fast as before.
  • broodmap-render and broodmap-cli move to the same broodcasc rev. js-sys brings wasm-bindgen from 0.2.126 to 0.2.129 in the lockfile.

Testing

  • New native tests: cache eviction and budget, missing vs failed reads, and base-asset failures.

  • load_assets_matches_the_prefetch_flow checks that loadAssets and the prefetch flow give byte-identical PNGs from a real install. Like broodmap-render's real-asset tests, it needs BROODMAP_TEST_SCR_DIR. Those tests and this one pass against SC:R 1.23.10.13515.

  • pnpm run test:package: in CI, it checks that incomplete callbacks and a directory that isn't an install both throw. With BROODMAP_TEST_SCR_DIR set, it also renders from a real install and checks that a second render reads only from the cache. pnpm run test:types covers the new declarations.

  • pnpm test (111/111), cargo test --workspace, cargo fmt, and cargo clippy both native (--workspace --all-targets --all-features) and for wasm32-unknown-unknown.

  • Output is byte-identical to a render from an extracted fetch-assets directory.

  • Measured in Node on a desktop with 512 px original-style renders:

    Opening the install ~55 ms
    lt.scm, first render: loadAssets ~145 ms
    lt.scm, first render: total ~445 ms
    lt.scm, later render ~180 ms
    DSA, after other maps filled the cache ~230 ms
    wasm memory after opening the install 14 MB
    wasm memory after the first render 61 MB (44 MB of it cached assets)

To try it: pnpm run build, then node examples/node-casc.mjs "C:/Program Files (x86)/StarCraft" ../broodmap/assets/lt.scm.

2Pacalypse- added a commit to ShieldBattery/ShieldBattery that referenced this pull request Oct 23, 2026
…StarCraft install.

The app renders them with broodmap from the install's own assets, in the server's style, and caches them as JPEGs by file hash and game build. The vendored broodmap package is repacked from ShieldBattery/broodmap#7 for CascStorage.
2Pacalypse- added a commit to ShieldBattery/ShieldBattery that referenced this pull request Oct 23, 2026
…StarCraft install. (#1788)

* Show map files the server doesn't have with images rendered from the StarCraft install.

The app renders them with broodmap from the install's own assets, in the server's style, and caches them as JPEGs by file hash and game build. The vendored broodmap package is repacked from ShieldBattery/broodmap#7 for CascStorage.

* List maps on this PC from the files alone, asking the server about a file only when it's chosen.

With images rendered locally, the per-page mapsByHash lookup only added server stats and badges to the tab. A chosen file now checks its own hash: a known map is opened or picked without uploading, an unknown one is uploaded first.

* Render a batch of local map images in list order, newest batch first.

* Load local map images only for tiles on screen, a couple at a time in layout order.

Lazy-loaded images were requested well below the screen, so the renderer's queue couldn't tell which maps were visible. The client now decides, and the app renders requests in the order they arrive.

* Say that maps from this PC upload when first used, and show when the server is processing one.

* Remove mapFileIdToHash, which nothing uses anymore.
2Pacalypse- added a commit to ShieldBattery/ShieldBattery that referenced this pull request Oct 24, 2026
…StarCraft install. (#1788)

* Show map files the server doesn't have with images rendered from the StarCraft install.

The app renders them with broodmap from the install's own assets, in the server's style, and caches them as JPEGs by file hash and game build. The vendored broodmap package is repacked from ShieldBattery/broodmap#7 for CascStorage.

* List maps on this PC from the files alone, asking the server about a file only when it's chosen.

With images rendered locally, the per-page mapsByHash lookup only added server stats and badges to the tab. A chosen file now checks its own hash: a known map is opened or picked without uploading, an unknown one is uploaded first.

* Render a batch of local map images in list order, newest batch first.

* Load local map images only for tiles on screen, a couple at a time in layout order.

Lazy-loaded images were requested well below the screen, so the renderer's queue couldn't tell which maps were visible. The client now decides, and the app renders requests in the order they arrive.

* Say that maps from this PC upload when first used, and show when the server is processing one.

* Remove mapFileIdToHash, which nothing uses anymore.
Base automatically changed from wasm-map-summary to main October 24, 2026 22:48
2Pacalypse- and others added 2 commits October 24, 2026 15:53
CascStorage runs broodcasc over host-provided sync file callbacks, and MapRenderer.loadAssets fills both prefetch rounds from it. Decoded assets are shared across maps in a size-bounded cache, since every render needs the ~38 MB mainSD.anim.
broodcasc reserves at most 8 MiB for a decoded asset and doubles past
that, so the ~38 MB mainSD.anim went through 16, 32 and 64 MiB buffers
and left WASM memory at 163 MB after the first render. Reserving up to
64 MiB brings that to 116 MB; most of the rest is the Vec to Arc copy,
which needs a decode-into-buffer API in broodcasc to avoid.

read() now fills a view of WASM memory instead of a JS buffer that was
then copied in, and CascFiles/CascStorageOptions are declared for
TypeScript callers.
tec27 added 2 commits October 24, 2026 18:41
broodcasc's read_file now reserves the exact decoded size, so the
initial reserve workaround is gone, and read_file_into lets both
CascStorage and CascSource decode into the Arc they return instead of
copying a Vec into one. WASM memory after the first render drops from
116 MB to 61 MB, and after opening the install from 22 MB to 14 MB.

fast-inflate is enabled explicitly: broodmap-render's broodcasc
dependency only reaches the wasm build in dev-dependencies, so the
package wasn't getting it.
The wasm package comes out ~8% smaller (1.86 to 1.71 MB) at the same speed.
@tec27
tec27 merged commit 7447c4a into main Oct 25, 2026
16 checks passed
@tec27
tec27 deleted the wasm-casc-source branch October 25, 2026 01:54
2Pacalypse- added a commit to ShieldBattery/ShieldBattery that referenced this pull request Oct 27, 2026
…StarCraft install. (#1788)

* Show map files the server doesn't have with images rendered from the StarCraft install.

The app renders them with broodmap from the install's own assets, in the server's style, and caches them as JPEGs by file hash and game build. The vendored broodmap package is repacked from ShieldBattery/broodmap#7 for CascStorage.

* List maps on this PC from the files alone, asking the server about a file only when it's chosen.

With images rendered locally, the per-page mapsByHash lookup only added server stats and badges to the tab. A chosen file now checks its own hash: a known map is opened or picked without uploading, an unknown one is uploaded first.

* Render a batch of local map images in list order, newest batch first.

* Load local map images only for tiles on screen, a couple at a time in layout order.

Lazy-loaded images were requested well below the screen, so the renderer's queue couldn't tell which maps were visible. The client now decides, and the app renders requests in the order they arrive.

* Say that maps from this PC upload when first used, and show when the server is processing one.

* Remove mapFileIdToHash, which nothing uses anymore.
2Pacalypse- added a commit to ShieldBattery/ShieldBattery that referenced this pull request Nov 10, 2026
…StarCraft install. (#1788)

* Show map files the server doesn't have with images rendered from the StarCraft install.

The app renders them with broodmap from the install's own assets, in the server's style, and caches them as JPEGs by file hash and game build. The vendored broodmap package is repacked from ShieldBattery/broodmap#7 for CascStorage.

* List maps on this PC from the files alone, asking the server about a file only when it's chosen.

With images rendered locally, the per-page mapsByHash lookup only added server stats and badges to the tab. A chosen file now checks its own hash: a known map is opened or picked without uploading, an unknown one is uploaded first.

* Render a batch of local map images in list order, newest batch first.

* Load local map images only for tiles on screen, a couple at a time in layout order.

Lazy-loaded images were requested well below the screen, so the renderer's queue couldn't tell which maps were visible. The client now decides, and the app renders requests in the order they arrive.

* Say that maps from this PC upload when first used, and show when the server is processing one.

* Remove mapFileIdToHash, which nothing uses anymore.
2Pacalypse- added a commit to ShieldBattery/ShieldBattery that referenced this pull request Nov 15, 2026
…StarCraft install. (#1788)

* Show map files the server doesn't have with images rendered from the StarCraft install.

The app renders them with broodmap from the install's own assets, in the server's style, and caches them as JPEGs by file hash and game build. The vendored broodmap package is repacked from ShieldBattery/broodmap#7 for CascStorage.

* List maps on this PC from the files alone, asking the server about a file only when it's chosen.

With images rendered locally, the per-page mapsByHash lookup only added server stats and badges to the tab. A chosen file now checks its own hash: a known map is opened or picked without uploading, an unknown one is uploaded first.

* Render a batch of local map images in list order, newest batch first.

* Load local map images only for tiles on screen, a couple at a time in layout order.

Lazy-loaded images were requested well below the screen, so the renderer's queue couldn't tell which maps were visible. The client now decides, and the app renders requests in the order they arrive.

* Say that maps from this PC upload when first used, and show when the server is processing one.

* Remove mapFileIdToHash, which nothing uses anymore.
2Pacalypse- added a commit to ShieldBattery/ShieldBattery that referenced this pull request Nov 17, 2026
…StarCraft install. (#1788)

* Show map files the server doesn't have with images rendered from the StarCraft install.

The app renders them with broodmap from the install's own assets, in the server's style, and caches them as JPEGs by file hash and game build. The vendored broodmap package is repacked from ShieldBattery/broodmap#7 for CascStorage.

* List maps on this PC from the files alone, asking the server about a file only when it's chosen.

With images rendered locally, the per-page mapsByHash lookup only added server stats and badges to the tab. A chosen file now checks its own hash: a known map is opened or picked without uploading, an unknown one is uploaded first.

* Render a batch of local map images in list order, newest batch first.

* Load local map images only for tiles on screen, a couple at a time in layout order.

Lazy-loaded images were requested well below the screen, so the renderer's queue couldn't tell which maps were visible. The client now decides, and the app renders requests in the order they arrive.

* Say that maps from this PC upload when first used, and show when the server is processing one.

* Remove mapFileIdToHash, which nothing uses anymore.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants