Repository navigation
A journaled append writes its frame inside its own transaction, so an append stops copying its page - #632
Merged
Conversation
… append stops copying its page With a journal configured, append_object_with_options captured the whole store before every append so a failed journal write could be undone. The same defect was fixed for puts, and this is the path TemporalStore's shared store is made of: every write it makes is an append to a growing log object. The capture clones the chunk service, which shares every page with the live store, so each append then found a second holder on the page it was extending and copied the whole page, and cloned every container the store holds. The page being copied was the one the appends were growing, so the cost grew with every append. The frame is now handed to apply_append, which already holds the narrow undo an append needs, and is written after the mutation and before its batch ends. A journal failure aborts the batch and restores through that undo. The pending frame is taken whatever happens, and a frame left unwritten by a successful append is written rather than dropped. Measured with callgrind on the journal door at three replicas, 300-byte appends to one object, instructions per append: 448,272 -> 97,988 at 500 appends and 1,625,833 -> 98,806 at 4,000. Before the change the cost rose 3.6x across that range; after it, 0.8%. Two tests: a failed journal write leaves the object byte-identical at the same length, and 40 appends replay after a reopen to exactly the concatenation. Both fail when the journal write is skipped. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What was wrong
PR #628 fixed this for puts: with a journal configured, a put captured the whole store before it ran, so a failed journal write could be undone, and that capture shared every page with the live store, so the next write copied the page.
append_object_with_optionshad the identical pattern and was not fixed. It matters more, because every write TemporalStore's shared store makes is an append to a growing log object.Found by scale, not size. Running the TemporalStore harness through the door at 300, 1,200 and 3,000 operations per phase showed the door's CPU per write rising from 3.0 ms to 8.3 ms to 19.3 ms while the harness side stayed flat (5.2, 2.5, 2.4 ms). Reproduced with a pure append client (300-byte appends with
x-amz-write-offset-bytesto one object) under callgrind:Arc<Vec<u8>>::make_mutcopying memory was 53% and aBTreeMapclone 24%. The page being copied was the one the appends were growing, so the cost grew with every append.The change
The journal frame is handed to
apply_append(the samepending_journal_payloadmechanism as puts) and written after the mutation and before its batch ends.apply_appendalready holds the narrow undo an append needs, so a journal failure aborts the batch and restores through it. No whole-store capture, so no extra page holder.The pending frame is taken whatever happens, so an append that fails before the journal cannot leave a frame for the next write. A frame left unwritten by a successful append is written rather than dropped (unreachable today, but a silently dropped frame is an acknowledged write lost on restart). The placement variants take their own capture for their metadata commit and are not on this path; they are unchanged and the pending mechanism is inert for them.
Measured
Callgrind, journal door, three replicas, 300-byte appends to one object:
The cost was 3.6x higher at 4,000 appends than at 500; it is now 0.8% higher.
Tests
an_append_whose_journal_write_fails_is_undone_and_leaves_the_object_as_it_was: the journal works for one append, then its path becomes a directory. The next append must error, the object must read back byte for byte what it was at the same length, and a second failing append must find the same.journaled_appends_replay_to_the_object_they_built: 40 appends of varying sizes replay after a reopen to exactly the concatenation (each frame written once, in order).the_published_mutation_copy_figures_are_the_ones_this_tree_produces(README 2485 vs tree 2557), fails identically on unmodifiedmainand is untouched here.🤖 Generated with Claude Code