Repository navigation
cut: three mechanisms the rest of the system already covers - #424
Merged
Merged
Conversation
This was referenced Nov 3, 2026
…ng a second take handleOwner reaches AdoptArchived for a held claim only when its AgentSocket miss predates the flight that adopted it. A check under m.mu at the top of the flight answers that case before any registry write, so the duplicate branch and adoptHeld cannot run: no other path adds an id a live row names while the server serves.
… inside it A refused take no longer re-reads the row and retries within the pass. Any error leaves the node fenced and the next renewal, 5 s later, reads the rows again. An own claim is still never dropped on a refusal, and a contested pass costs replicated claims about 5 s more of 503.
Arm runs only from finalizeBatch on a sandbox startVM just built, and every claim path goes through startVM, which copies the tap from cocoon's clone or run record. Restart uses readoptEgressTap and Relock. An empty tap now fails the claim closed, which is the Arm contract, and outbound.Engine no longer needs Inspect.
…ogged
The line only repeated a documented rule: a claim that holds volumes is not replicated. GET /v1/sandboxes/{id}/volumes shows what a claim holds, so the state stays visible. replicaState collapses to the time of the last try.
ArmProxy's early return now closes what refill bound, so Prebind's re-check and the reload sweep have nothing left to cover. After a reload that drops every egress policy, an unclaimed warm VM keeps its listeners until it is claimed or removed. No tenant code runs in it before the claim, and removeVM, the only removal path, closes them too. An unguarded claim gains one uncontended lock and one map lookup, about 20 ns.
This reverts 413724c. Without the retry, one release landing on a claim not yet confirmed aborts the whole unfence pass, and every replicated claim on the node stays at 503 for another renewal. Simulated with 1.3 ms per Take: with 1000 replicated claims and 5 releases/s the fence lasted a mean 1m57s against 1.3 s with the retry, and with 1/s a mean 4.0 s and a max 19.6 s against 1.3 s. 16 lines do not pay for that.
This reverts b3f0964. runRecord swallows a parse failure of cocoon's output and returns an empty record, so without the fallback every egress claim on such a node answers 500 and burns its warm VM, where before it took one inspect and a warning: 5 of 5 claims succeeded with the lookup, 0 of 5 without. No cocoon version read today triggers it, but a test guaranteed the behaviour and 19 lines off a branch that never runs buy nothing.
CMGS
force-pushed
the
cut/overdesign-low
branch
from
November 3, 2026 08:22
5d66f91 to
36bfa20
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Three mechanisms whose justifying scenario came from review, not from an incident, and whose failure the rest of the system already handles. Each was attacked first: read every caller and test, then try to build an interleaving that loses a claim, serves two owners, leaves a free NIC or breaks a documented status code. None held, so they go, one commit each. Net production change is −44 lines.
2f6387dfadoptHeldand the duplicate branch: a held claim answers before reachingTakeAgentSocketmiss/ownerduring a release answers 200 where it answered 404, the same race as anAgentSockethit just before a releaseafb2cc33replicaState.skipLoggedand its log lineGET /v1/sandboxes/{id}/volumesshows what a claim holds8d2a837ePrebindandReloadedArmProxycloses them instead. After a reload to no policy, an unclaimed warm VM keeps two listeners until it is claimed or removedTests: rewritten where they drove a sequence production cannot produce, and named in each commit. Mutants confirm the replacements:
moved the row from epoch 3 to 4 under the wake.a prebound door is left open.Not cut, tried and reverted:
The tap lookup (
b3f0964e, reverted).runRecordswallows a parse failure of cocoon's output and returns an empty record. Without the lookup every egress claim on such a node answers 500 and burns its warm VM: 0 of 5 claims succeeded, against 5 of 5 with it. No cocoon version read so far triggers it, but a test guaranteed the behaviour, and 19 lines off a branch that never runs buy nothing.The retry inside
confirmOwnership(413724c5, reverted in the last commit). It looked redundant, because the next renewal retries. But without it, one release landing on a claim not yet confirmed aborts the whole unfence pass, and every replicated claim on the node stays at 503. Fake-clock simulation with 1.3 ms perTake, 20 trials per row:The rates above 1/s end early only because the simulation runs out of claims to release.
Evidence:
go build,go veton both GOOS,aslon both GOOS andgofmtare clean.go test ./...and-raceonpool,outboundandserverpass, as do the e2e module's tests.pool,outboundandserverpass in a Linux arm64 container as root.Hardware: two bare-metal nodes, a real PostgreSQL and S3-compatible storage, n=1 per step, at 8d2a837, which still carried the two later-reverted cuts:
/ownercalls answered 200 with the row byte-identical and nosandboxd_archivesstatement in that window.Not run: that hardware stage at this head. The full hardware round on the tree with every open PR merged covers it. An independent review reran each remaining cut against the base with differential tests on darwin, and kept all three.
Stacked on #420.
holdStaleVolumesstays: its 409 is part of the API.