[CMGS]

If sandboxd died during a capture, the guest could stay paused until someone ran vm start, or lose its memory to a cold boot. The next start now waits for the capture's CLI to exit and settles the claim from what the capture left behind. A failed hibernate or save answers at once and settles the same way in the background.

Closes #446. Closes #442.

Needs cocoon with cocoonstack/cocoon#306, now merged as 2f7252fa but not yet released, which removes the state where a VMM stays alive with no guest. The hardware below ran on that tree.

What happens now

The settle keeps the intent and holds the claim's transition, so calls to that sandbox wait while other claims are unaffected. Once no process carries the snapshot name as an argument, it reads vm list and snapshot list (no VMM API call, so a running dump cannot stall it):

Rendered state	Snapshot listed	Action
running	hibernate's, yes	vm stop --force, adopt as hibernated, wake with vm restore
running	no, or a save's	retrying vm start, then a silkd probe; the claim is marked failed after 30 minutes
stopped or error	hibernate's, yes	adopt as hibernated, wake with vm restore; never vm start
stopped	no	clear the intent; liveness handles it as before

Two rules hold this together:

• Wait for the CLI first. While it lives, vm start waits on cocoon's ops lock, then cold-boots a guest whose VMM the hibernate stopped, and still returns 0.
• A listed hibernate snapshot is the guest as it last ran. Persist happens inside the pause window, and a failed persist unrecords the snapshot, so it is the wake image whatever the VMM's state. A save resumes the guest before persist, so a save's snapshot is dropped and never adopted.

The intent stays until the guest is safe. If vm stop fails, the settle gives up, or sandboxd stops during the retries, the next wake or start finishes the work: every dangling intent goes through the same settle, and a wake never clears one without resuming the guest.

snapshot save now journals an intent too; a save cut short could not be seen after a restart.

Hardware

Bare metal, cocoon 2f7252fa, a 4 GiB guest with 3.5 GiB dirtied in /dev/shm. Every row checked the boot id, the PID of a running process and a marker hash, so a cold boot cannot pass.

Fault	main	this branch
sandboxd SIGKILLed, CLI dies on the broken pipe after persist	exec 502, VMM Paused, snapshot swept, vm start by hand	adopted, woken by vm restore, intact; survives a second SIGKILL before the wake
sandboxd and CLI killed during the dump	exec 502, vm start by hand	resumes itself, intact
killed after terminate, before the state flip	passes	passes, 0 vm start
restart while the CLI is alive; CLI dies	exec 502, vm start by hand	dial waited 115 s, then 200, intact, 0 vm start during the hold
restart while the CLI is alive; CLI finishes	cold boot, memory gone	vm restore, intact
checkpoint or fork, sandboxd killed	passes	passes; the snapshot is removed (4108 MiB reclaimed)
checkpoint or fork, sandboxd and CLI killed	exec 502, vm start by hand	resumes itself, intact
sandboxd alive, CLI killed during a hibernate or checkpoint	API 500, guest stuck, vm start by hand	API 500 within 0.1 s of the kill, then the guest resumes by itself, intact
sandboxd alive, CLI killed after persist	not run	API 204, adopted, 0 vm start, intact
SIGTERM during the resume retries	intent lost, guest stuck	intent kept on disk, next start resumes it
settle gives up, then an exec arrives	on the PR's first commit: exec 502, intent cleared, guest stranded	exec runs vm start, 200, intact; a later restart is quiet
vm start never succeeds	not applicable	claim failed after 30 minutes, 361 attempts

Successful captures leave no intent on disk, and later restarts run no settle. sandboxd-e2e.sh and archive-e2e.sh pass, and a clean stop still waits for the capture (about 2 s, and 26 s with a held capture).

Cost

Measured on bare metal NVMe, both arms pinned to one NUMA node, ABBA over 4 rounds, with an A/A control.

• Warm claim: no new work on its path.
• Each save from a running source: two more claims.json commits.

Claims	Per save, sync_claims off / on	Share of a 157 ms fork
1000	0.22 / 0.69 ms	0.1% / 0.4%
5000	0.63 / 1.81 ms	0.4% / 1.2%
20000	3.9 / 8.5 ms	2.5% / 5.4%

Against a real 4 GiB snapshot save of about 3.4 s, that is 0.02% to 0.25%.

• Warm claims while saves run: at 4 saves/s, claim p50 is unchanged at every size. The only difference is p99 at 20000 claims with sync off, +1.26 ms, just above the 0.94 ms the A/A control showed as noise. With saves in a tight loop, about 1000 times any real rate, claims wait behind at most about one commit.
• A failed capture: the API answers at once. The resume runs in the background and holds only that sandbox.

Not covered

• A real host loss or power cut. Every fault here is a process death.
• Guests over 4 GiB in this PR's runs. cocoon docs: performance covers the incremental claims.json commit and the buffered vsock handshake #306 measured stop and hibernate up to about 256 GiB; a guest above about 420 GiB of 4K pages would extrapolate past its 35 s wait.
• A VMM wedged past cocoon's grace, and these kills on a cloud image.
• A process that happens to carry the snapshot name, such as an operator's watch, holds the claim until it exits. The warning names the snapshot.

Tests and gates

Unit tests cover every row. Mutations of the decision logic each fail a test, among them not waiting for the CLI, answering only after the resume, swallowing a failed stop, starting a stopped VM, stopping a stopped one, skipping the probe, adopting a save's snapshot, and a wake clearing an intent through the old resolver.

• make go-lint: 10 0 issues. lines, 5 modules on linux and darwin, with a cold cache (a warm one panics staticcheck's SA4023).
• make go-test: 29 packages ok.
• asl on all five modules, linux and darwin: no findings.
• The WaitCapture test passes in a Linux container; it skips on macOS, which has no /proc.

Production +248 −28, of which about 40 lines are Reconcile's orphan sweeps moved to helpers to stay under the complexity limit. Tests +536 −22. Go comment lines +2 −2.