Skip to content

Grow into memory instead of reserving it, and raise the ceiling - #4

Merged
CoderDayton merged 1 commit into
mainfrom
release/1.1.0
Aug 10, 2026
Merged

Grow into memory instead of reserving it, and raise the ceiling#4
CoderDayton merged 1 commit into
mainfrom
release/1.1.0

Conversation

@CoderDayton

Copy link
Copy Markdown
Owner

What changed

The banks move off .bss to computed offsets above the static data, and the module grows linear memory to reach them. An instance starts at 1.25 MiB (u32) or 1.75 MiB (u64) instead of 21 or 29, and grows with the table.

The hash widens to 64 bits so h1 can address the whole slot space, which is what capped the table at 2^20 slots. The ceiling is now 117,440,512 entries for u32 and 58,720,256 for u64, up from 917,504. They differ because three u64 banks run out of wasm32 address space an exponent earlier.

Placement

Bank 0 sits at the base of the heap, bank 1 two of its own lengths above it. Consecutive banks are always disjoint, which is all a rehash needs, and same-parity banks reuse committed pages. A reserve() crossing several doublings at once produces a bank too large for the anchor to clear, so that one goes directly above the live bank.

Address extent is 3x a bank on the odd index and 1.5x on the even one. The 3x case sets each module's ceiling, since three banks have to address inside 4 GiB. Resident memory is lower because same-parity banks share pages, measured at 1.8x to 1.9x a bank across a fill.

Bindings

A grow replaces the backing ArrayBuffer and detaches every view over it, so both bindings rebuild their cached views after any call that can rehash — including one that then fails, which is where the first draft was wrong. An open walk needs no such check: a grow only happens inside a rehash, and the walk already tests the generation counter before each window.

Breaking-ish

Each module now declares a --max-memory of 3.4 GiB (u32) or 2.4 GiB (u64). That is address space, not a reservation, and nothing is committed until a table reaches it — but a 32-bit or memory-constrained host may decline it at instantiation where the previous 21 MiB succeeded. Noted in SECURITY.md and the changelog. Build with a lower SWISS_MAX_CAPACITY_LOG2 for those hosts.

Memory is never returned to the host. clear() and shrinkToFit() ready a table for reuse, but an instance holds the high-water mark of every bank it used until dispose().

Tests

Two test-only modules cover the refusal paths, since the shipped ceiling is no longer reachable in bounded time: swiss_u32_capped (2^16 slots) for the slot ceiling, and swiss_u32_starved (2^20 slots against 8 MiB) for a host declining a grow. Neither is embedded or published.

Verification

  • bun run build — all four modules link
  • bun run typecheck — clean
  • bun test — 252 pass, 0 fail
  • bun run check:ubsan — 905,504 operations, no trap
  • Smoke on Node 24, Chrome 151, Firefox 153; all five examples run

Benchmarks reran (5 runtimes x 3 passes) and both charts were regenerated. Lookup is flat within noise, confirming the 64-bit hash costs nothing.

The banks move off .bss to computed offsets, and the module grows linear
memory to reach them. An instance starts at 1.25 MiB (u32) or 1.75 MiB (u64)
instead of 21 or 29.

The hash widens to 64 bits so h1 can address the whole slot space. The
ceiling is 117,440,512 entries for u32 and 58,720,256 for u64.

The bindings rebuild their cached views after any call that can rehash,
since a grow detaches them.
@CoderDayton CoderDayton self-assigned this Aug 10, 2026
@CoderDayton
CoderDayton merged commit e9958c4 into main Aug 10, 2026
9 checks passed
@CoderDayton
CoderDayton deleted the release/1.1.0 branch August 10, 2026 07:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant