Skip to content

An append keeps the page's validation memo, so a read after it stops hashing the page - #633

Merged
matrixarkai merged 1 commit into
mainfrom
perf/an-append-keeps-the-page-validation-memo
Nov 14, 2026
Merged

matrixarkai merged 1 commit into
mainfrom
perf/an-append-keeps-the-page-validation-memo

Conversation

@matrixarkai

Copy link
Copy Markdown
Owner

What

A read of a page checks its bytes against a memo of one hash per 4 KiB page. The memo is keyed by (write_sequence, checksum), an append changes both, and the write also dropped the memo outright, so the first read after every append started from nothing: it hashed the whole payload and then every page of it again. For a page filled by appends and read after each one (a log that is followed while it is written), the cost of a read grew with the page.

An append leaves every byte before the old end untouched, so each page that was full at the old end keeps its hash. write_chunk now takes the memo out before the write and puts back one that keeps those hashes and recomputes only the page that held the old end and anything past it. A write that is not a pure append still drops the memo, exactly as before, and a memo that described anything other than the page just before the write is not carried.

Measured

Door under callgrind (journal mode, the shared-store harness driving it, instructions per write):

300 writes 1,200 writes growth
before 285.4K 435.3K +52.5%
this change 272.6K 302.4K +10.9%
ceiling: cold validation skipped (a measuring plant, never shippable) 230.4K 258.8K +12.3%

At 1,200 writes the run executes 30.5% fewer instructions (522.3M to 362.9M); the ceiling is 40.5%. Before the change, the replica read's own cost was 81% of the per-write growth (54.9K to 177K instructions per write, hashing inlined).

What is left of the growth is a separate term: invalidate_chunk_cache_bounded collects every cached read range of the page on each write (about 34K per write at 1,200 writes). Not part of this change.

Tests

  • The carried memo equals a cold build after every append, with widths that straddle the 4 KiB boundary (partial-page fill, exact boundary, multi-page span).
  • An overwrite drops it; a write past the end with a hole carries it and matches a cold build.
  • A byte damaged in a page whose hash was carried is still not served.

Planted defects, each caught: keeping the stale partial-page hash fails all three tests; carrying on a non-append write fails the overwrite test.

Checks run locally

The four named guards pass; cargo build --all-targets --locked passes with -D warnings, with and without --all-features; the library suite is 692 passed, 1 failed. The one failure, the_published_mutation_copy_figures_are_the_ones_this_tree_produces, fails with identical numbers on unmodified main (measured 2557 / 799,992, README 2485 / 768,312) and CI does not run it. The network consensus jobs I left to CI. cargo fmt --check reports many files, none of them new here; CI has no format step, so I did not reformat.

🤖 Generated with Claude Code

…hashing the page

A read of a page checks its bytes against a memo of one hash per 4 KiB page. The memo is
keyed by (write_sequence, checksum), an append changes both, and the write also dropped the
memo outright, so the first read after every append started from nothing: it hashed the whole
payload and then every page of it again. For a page filled by appends and read after each one
-- a log that is followed while it is written -- the cost of a read grew with the page.

An append leaves every byte before the old end untouched, so the hash of each page that was
full at the old end is still that page's hash. The write now takes the memo out before it,
and puts back one that keeps those hashes and recomputes only the page that held the old end
and anything past it. A write that is not a pure append still drops the memo, as before, and
a memo that described anything other than the page just before the write is not carried.

Measured on the door under callgrind, driven by the shared-store harness: the per-write cost
of the read path grew 2.5x between 300 and 1,200 writes, and its own cost inside the replica
read was 81% of the whole growth. With the cold validation skipped (a measuring plant, never
shippable) a 1,200-write run executed 40.5% fewer instructions.

Tests: the carried memo equals a cold build after every append of widths that straddle the
4 KiB boundary; an overwrite drops it and a write past the end with a hole carries it; and a
byte damaged in a page whose hash was carried is still not served.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@matrixarkai
matrixarkai merged commit d045dfa into main Nov 14, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants