Skip to content

Module cache: restore shared types by node, skip the finalize collect on cache hits - #4354

Merged
borisbat merged 4 commits into
masterfrom
aleksisch/ser-type-node-table
Oct 23, 2026
Merged

borisbat merged 4 commits into
masterfrom
aleksisch/ser-type-node-table

Conversation

@aleksisch

@aleksisch aleksisch commented Oct 23, 2026 •

Copy link
Copy Markdown
Collaborator

Module cache format change: version 234 - existing caches are discarded once and rewritten.

Why. A warm load rebuilt one type node per holder, then the module's finalize walked the whole module to merge them back. On a warm dasLLAMA load that cost about a quarter of the run.

What changes.

  • A type reference in a record names the node it points at; the reader returns the same pointer, so a served module comes back with exactly the sharing the cold compile left.
  • The content-keyed type table, its recursive hash/compare and the deep copy per reference are gone.
  • A module read from the cache carries the new fromModuleCache flag; ast_verify skips such a module, as its documentation already said.
  • A served module's finalize skips the GC collect - it has nothing left to merge or free.
  • tests/math/test_color_mapping.das compares tone_reinhard(1.0) with a tolerance: under fastmath it is 0.49999997, not 0.5 (the test came with Add linear PBR output and shared tone-mapping operators #4351).

Observable behavior.

  • warm tutorials/dasLLAMA/02_chat.das: 1.97-2.06 s -> 1.45-1.55 s (25-26% faster); cache 66.3 MB -> 64.5 MB
  • warm fio + match script: 890.6M -> 752.3M instructions (-15.5%)
  • cold compile: unchanged within noise
  • warm run requiring daslib/ast_verify: 11,304 false AST verify: lines would appear with shared types -> 0 (new test)

Where to look. AstSerializer::operator << ( TypeDeclPtr & ) in module_builtin_ast_serialize.cpp, and the skip in parseDaScriptEx (ast_parse.cpp).

Validation, claims, ledger

Validation

  • Metrics were measured against dd236c1fee (the base before the last rebase), interleaved master/branch runs on a shared 32-core box: dasLLAMA warm 15 runs each (median 1.966 s -> 1.453 s), cold 3 + 6 runs each; instructions from callgrind on a warm require daslib/fio + require daslib/match script.
  • Garbage check, local only: every test program compiled cold then warm with a diagnostic that ran the finalize collect on cache hits - 39,272 served module reads, 0 nodes left for the collect.
  • Full suite and the --ser/--deser sweep passed locally on the pre-rebase tip; the post-rebase tip ran the module-cache tests and the full suite.
  • Not run: the per-PR audit agents, make-pr/full preflight, and the full AOT suite - CI and #nightly cover them.

Claims - stated, not tested

  • A served module's decode allocates only attached nodes. Verified by the garbage check above, not by a test in the tree; a new source of garbage on a cache hit would show as memory held until the module unloads, not as a failure.

Not done

  • Macro-module simulation on a cache hit (fusion, debug info) is still the largest warm-load cost; untouched here.

#nightly

}

enum TypeRef : uint32_t { TypeRefNull = 0, TypeRefInline = 1, TypeRefFresh = 2, TypeRefFirstEntry = 3 };
enum TypeRef : uint32_t { TypeRefNull = 0, TypeRefPayload = 1, TypeRefFirstNode = 2 };

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@borisbat this is fine. Earlier we made type-merging in serializing. Now we do it in module after the parsing. So we don't need to merge types again.

Although, earlier we merged types only, and now we merge types + location. It has benefits: we do not allocate anything after deserialization.

@aleksisch
aleksisch force-pushed the aleksisch/ser-type-node-table branch 3 times, most recently from 468b29a to d71de05 Compare October 23, 2026 14:35
run_admin built one quoted command line for popen; on Windows cmd /C
strips the first and last quote of a line that starts with one and holds
more, so the child never started ("The filename, directory name, or
volume label syntax is incorrect."). run_and_capture spawns the argv
with no shell.
Under fastmath tone_reinhard(1.0) is 0.49999997, not 0.5.
A finished dependency module shares its equal types - the finalize's
remap merges equal content at an equal `at` - but the record streamed
types by content only and the reader deep-copied the table entry for
every reference, so every cache hit rebuilt one node per holder, its
child types taking the entry's `at`, and the finalize's remap merged
them again. A type reference now names the node it points at: the writer
numbers each node by its pointer, a repeat reference is the number
alone, and the reader answers it with the same pointer, so a record
reads back with exactly the sharing it was written with. A node takes
its number before its payload, so a same-module structure streamed
inline whose field holds the very type being written names it instead of
streaming a second copy. The content table, its recursive hash and
compare, and the tableable check go: the remap already decided which
holders share a node.

A module the reader builds carries the new `fromModuleCache` module
flag, and ast_verify's macros skip such a module, as its documentation
already said they do: its own macro module is simulated again on the
warm read, and its simulate macro walked the served tree, reported every
shared type as an alias (11,304 false `AST verify:` lines on a warm run
of a script requiring it) and un-shared them.
tests/module_cache/test_ast_verify_served.das pins that. The handmade
ModuleFlags descriptions were one line off past visibleEverywhere; they
are realigned.

Version 234.
A served module's record rebuilds the graph the writing compile's
finalize left - the type node table restores its sharing exactly, and
ast_verify no longer un-shares it - and its decode attaches every node
it allocates, so the finalize's collect finds nothing to merge and
nothing to free. A served module's finalize now only clears the lookup
caches. On a cold-then-warm run of every test program, none of the
module reads from the cache leaves a node for the collect.
@aleksisch
aleksisch force-pushed the aleksisch/ser-type-node-table branch 2 times, most recently from b0d6511 to 6b3bfae Compare October 23, 2026 21:42
@borisbat
borisbat merged commit 6261ec2 into master Oct 23, 2026
29 checks passed
@borisbat
borisbat deleted the aleksisch/ser-type-node-table branch October 23, 2026 23:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants