Skip to content

An append resumes the hash of the partial page instead of rehashing it - #635

Merged
matrixarkai merged 1 commit into
mainfrom
perf/an-append-resumes-the-hash-of-the-partial-page
Nov 14, 2026
Merged

matrixarkai merged 1 commit into
mainfrom
perf/an-append-resumes-the-hash-of-the-partial-page

Conversation

@matrixarkai

Copy link
Copy Markdown
Owner

What

When an append carries a page's validation memo forward (#633), the page that held the old end was partial, and its hash was recomputed from the start of that page. A hash of a page is a left fold over its bytes, so the hash of the page as it is now is the hash it had carried on over the bytes appended to it; the page checksum already relies on that identity in write_chunk. The memo's partial-page hash is now resumed over the appended bytes, and only pages past it are hashed whole. Rehashing the partial page cost a page of hashing, per replica, to add a few bytes.

Measured

Door under callgrind (journal mode, the shared-store harness driving it):

300 writes 1,200 writes growth total at 1,200
before #633 and #634 285.4K 435.3K +52.5% 522.3M
main (both merged) 264.9K 270.4K +2.1% 324.5M
this change 251.6K 256.0K +1.8% 307.2M

carry_page_validation_over_append itself goes from 14,696 to 675 instructions per write (-95%). That is 5.3% of the door's instructions per write on top of main, and 41.2% fewer than before #633 and #634 at 1,200 writes.

Tests

No new test. The existing comparison of the carried memo against a cold build, over appends whose widths straddle the 4 KiB boundary (partial-page fill, exact boundary, multi-page span), is the oracle for exactly this change. Planted defects, each caught by it ("the carried memo differs from a cold build"): resuming over the wrong slice of the page, and not moving past the partial page after resuming it.

Checks run locally

The four named guards pass, cargo build --all-targets --locked passes with -D warnings (default and --all-features), and the library suite is 694 passed, 1 failed: the_published_mutation_copy_figures_are_the_ones_this_tree_produces, which fails with identical numbers on unmodified main and which CI does not run. The network consensus jobs are left to CI.

🤖 Generated with Claude Code

When an append carries the page's validation memo forward, the page that held the old end
was partial, and its hash was recomputed from the start of that page. A hash of a page is a
left fold over its bytes, so the hash of the page as it is now is the hash it had carried on
over the bytes appended to it; the page checksum already relies on that identity in
`write_chunk`. The memo's partial-page hash is now resumed over the appended bytes, and only
pages past it are hashed whole. Rehashing the partial page cost a page of hashing, per
replica, to add a few bytes.

No new test: the existing comparison of the carried memo against a cold build, over appends
whose widths straddle the page boundary, is the oracle for exactly this, and each planted
defect below is caught by it.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@matrixarkai
matrixarkai merged commit 58eed58 into main Nov 14, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants