Repository navigation
pool, store: hold a template's record lock only for the re-promote commit - #430
Merged
Merged
Conversation
commitTemplate held lockRec across PublishDigested, which hashes the whole export and on S3 uploads every changed chunk, up to the 30 min budget. resolveGolden's rlockRec cannot be cancelled, so every claim of that template waited for the whole publish and stayed parked after its client left. The store now splits a digested publish into PrepareDigested (hash, upload chunks and manifest; nothing visible changes) and the commit it returns (the meta.json pointer). The owner re-check and the commit run under the lock; the rest runs outside it. Two promotes of one template still serialize at the commit, and the later commit wins as before.
CMGS
force-pushed
the
fix/promote-lock
branch
from
November 3, 2026 08:22
5073269 to
71db84e
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A re-promote held the template's record write lock across the whole publish: hashing every byte of the export, and on S3 uploading the changed chunks, up to the 30-minute transfer budget. Every claim served from that promoted template waited on the read lock for all of it, and that wait does not observe the claim's context, so goroutines stayed parked after clients left. Store generations already let a clone of the previous generation run beside a publish, so the lock only has to cover the pointer switch.
PublishDigestedbecomesPrepareDigested, which runs a publish up to the step that makes the generation current and returns that step as a commit function:meta.json.meta.jsonand removes staging. PlainPublishis prepare then commit, so checkpoints behave as before.commitTemplateprepares with no lock held, then takes the record lock, re-checks the owner, commits and clears the labels.Two promotes of one template still serialize at the commit and the later commit wins, as before; "later" is now the order their prepares finish. Claims stall at most for the commit, not the publish.
Tests: a claim with a 100 ms context during a re-promote blocked in prepare returns within its context (fails with the lock moved back above prepare); the store contract requires a prepared record to stay invisible until its commit on both backends. Prod +40 -34. Stacked on #420.