Skip to content

The metadata change log is a deque, so a full log drops a record without moving the rest - #631

Merged
matrixarkai merged 1 commit into
mainfrom
perf/metadata-change-deque
Nov 13, 2026
Merged

matrixarkai merged 1 commit into
mainfrom
perf/metadata-change-deque

Conversation

@matrixarkai

Copy link
Copy Markdown
Owner

What was wrong

MatrixMetaService keeps its change log as a Vec<MetadataChange> bounded at 1,024 records. Once the log has filled it is always at the bound, and every append drops the oldest record with drain(0..1), which shifts every record the log keeps. That is paid on every put, for ever, so it is invisible in a short profile and dominates a long one.

Callgrind of the journal door (three replicas, 1 KiB puts) found it by scale rather than by size: per-put instructions were not flat.

puts per put (before)
1,000 124,903
8,000 181,052 (+45%)

At 8,000 puts append_metadata_change was 27.7% of all instructions (~50,000 per put), memmove 33%, and the MetadataChange drop 27.6%. The object change feed had exactly this defect and was cured by holding it as a deque; this is the second place it was found.

The change

The log is a VecDeque<MetadataChange>. Dropping the oldest record moves nothing. The trim helper that took a Vec is replaced by one that takes a deque (nothing else used it). Behaviour is unchanged: the bound, the order and what a snapshot carries are the same, and a snapshot still holds a Vec at its edge (converted on export and install). About 15 call sites change mechanically (first/last become front/back, push becomes push_back).

Measured

Callgrind, journal door, three replicas, 1 KiB puts, instructions per put:

puts before after
1,000 124,903 124,484 (unchanged: the log hadn't filled)
8,000 181,052 131,073 (-27.6%)

Across an 8x larger store the per-put cost now rises 5% instead of 45%.

Tests

  • New the_metadata_change_log_is_a_deque_because_it_drops_one_record_per_append: pins the field declaration as a deque. Comments are excluded and the declaration count is asserted, so it cannot pass checking nothing. Planted check: with its expected type changed to Vec, it fails.
  • Full lib suite: 687 pass, 0 compiler warnings. The one failure, the_published_mutation_copy_figures_are_the_ones_this_tree_produces (README 2485 vs tree 2557), fails identically on unmodified main and is untouched here.

🤖 Generated with Claude Code

…out moving the rest

MatrixMetaService keeps its change log as a Vec bounded at 1,024 records. Once it has filled it
is always at the bound, and every append drops the oldest record with drain(0..1), which shifts
every record the log keeps. That is paid on every put, for ever.

A profile of the journal door at 8,000 puts of 1 KiB put 27.7% of its instructions in
append_metadata_change and 33% in memmove, and the per-put cost was 181,052 instructions against
124,903 at 1,000 puts: 45% more, from a term that was constant per put once the log was full.
The object change feed had exactly this defect and was cured by holding it as a deque; this is
the second place it was found.

The log is a VecDeque now, with the trim helper that took a Vec replaced by one that takes a
deque (nothing else used it). Behaviour is unchanged: the bound, the order and what a snapshot
carries are the same, and a snapshot still holds a Vec at its edge. The declaration is pinned by
a test that fails if the field goes back to a Vec; it excludes comments and asserts the
declaration exists once, so it cannot pass checking nothing.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@matrixarkai
matrixarkai merged commit ecd4753 into main Nov 13, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants