Skip to content

A journaled put writes its frame inside its own transaction - #628

Merged
matrixarkai merged 1 commit into
mainfrom
perf/door-append-page-copy
Nov 13, 2026
Merged

matrixarkai merged 1 commit into
mainfrom
perf/door-append-page-copy

Conversation

@matrixarkai

Copy link
Copy Markdown
Owner

What was wrong

With a journal configured (MATRIXOBJECT_JOURNAL_PATH), put_object_with_options captured the whole store before every put, so a failed journal write could be undone. That capture clones the chunk service, which shares every page with the live store. The next write into a page then found a second holder on it and copied the whole page, once per replica.

Profiled with callgrind on the journal door at three replicas (the default), 1,000 puts of 1 KiB:

  • Arc<Vec<u8>>::make_mut -> memcpy was 62.9% of all instructions, called 2,997 times for 1,000 puts: every replica write copied its page.
  • The rest of the rollback capture (cloning objects, buckets, payloads, the meta service) was another ~25%.

A probe on the page-undo check confirmed the cause: it refused 15 of 18 replica writes with live=3 taken=3 (the page state had 3 holders; the narrow-undo path requires exactly 2).

The fix

The put already holds a narrow undo inside apply_put_storage_batched. The journal frame is now handed to it (a pending_journal_payload field set just before apply_command) and written after the mutation and before the batch ends. A journal failure then aborts the batch and restores through the same narrow undo any other failure of the put takes. No whole-store capture, so no extra page holder.

The pending frame is taken whatever happens, so a put that fails before reaching the journal cannot leave a frame standing for the next one. A release-build fallback writes a frame that was somehow left unwritten rather than dropping an acknowledged write.

Measured (callgrind, instructions per put, journal door, 3 replicas)

puts before after
250 629,902 184,999
1,000 1,548,804 158,269
2,000 (not measured before) 186,949

At 1,000 puts that is 9.8x fewer instructions per put, and the per-put cost is flat as the store fills instead of growing with it.

Tests

  • New a_put_whose_journal_write_fails_is_undone_and_leaves_nothing_behind: the journal works for a bucket and one put, then its path becomes a directory. The next put must return an error, leave nothing readable or listed, leave the earlier object byte-identical, and a second attempt must fail the same way. Planted check: with the in-transaction journal write skipped, this test fails (the put is acknowledged).
  • The footprint coverage guard caught the new field and now weighs it.
  • forget_persistent_journal_path becomes pub(crate) so the test can drop the cached append handle. Visibility only.
  • Full lib suite on the rebased tree: 681 passed, 1 failed. The one failure, the_published_mutation_copy_figures_are_the_ones_this_tree_produces (README says 2485 copied for the 250-object row, the tree measures 2557), fails identically on unmodified main and is not touched here.

🤖 Generated with Claude Code

With a journal configured, put_object_with_options captured the whole store before every put
so a failed journal write could be undone. That capture clones the chunk service, which shares
every page with the live store, so the next write to a page found a second holder on it and
copied the whole page for each of three replicas: 2,997 page copies for 1,000 puts, 62.9% of
the door's instructions, plus the cost of cloning every container in the store.

The frame is now handed to apply_put_storage_batched and written after the mutation and
before its batch ends, so a journal failure aborts the batch and restores through the narrow
undo the put already holds. The pending frame is taken whatever happens, so a put that fails
before reaching the journal cannot leave one standing for the next put.

Measured at three replicas on the journal door, instructions per put: 629,902 -> 184,999 at
250 puts and 1,548,804 -> 158,269 at 1,000. The new test fails when the journal write is
skipped. The footprint audit weighs the new field, and the journal-path helper it needs is
now visible to tests.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@matrixarkai
matrixarkai merged commit 3a09fd7 into main Nov 13, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants