CRD path: upstream's claim controller is 1–2 s slower to Ready than the retired fork once the warm pool drains #33

Problem

Since #28 the Pod path runs upstream's agent-sandbox v1.0.3 claim controller. An interleaved A/B against the retired fork controller (0719d33) shows upstream slightly slower to a usable sandbox once the warm pool drains, and heavier on the control plane per claim.

Setup (two-node testbed, 2026-09-23): vk-sandbox nodes over sandboxd warm pools, SandboxWarmPool of 40, per arm two bursts of 200 SandboxClaims (create parallelism 1 and 20), arms interleaved upstream, fork, upstream, fork, 1600 of 1600 claims Ready. The harness times each claim to status.sandbox.name, to status.sandbox.podIPs and to the claim's Ready=True.

Per claim in the same bursts: upstream 23.2–24.2 apiserver writes and 19.6–20.6 etcd puts vs 22.1–22.7 and 19.1–20.0 on the fork (+5–9% writes, +3% puts); claim-controller reconciles 14.7–17.8 vs 7.5–8.0.

Two causes:

1. Pod-IP gate. Upstream binds a claim only to a warm Sandbox whose Pod IP it has observed; otherwise it retries every 100 ms (warmCandidateRetryInterval) for up to 2 s (warmCandidateGracePeriod), then falls back to cold creation (extensions/controllers/sandboxclaim_controller.go). Both are constants, not flags. The fork adopted the first valid member at once. vk-sandbox reports the Pod IP in the status it pushes at CreatePod, so on sandboxd nodes the gate mostly waits out status propagation.
2. Reconcile amplification. The fork's event trims (perf: hot-path round — event amplification, fan-out, and per-request costs #2, review: whole-repo round — owning-node index, cache-fed warm pool, scoped informers, indexed queue #11 — pool status writes no longer re-enqueue bound claims, member updates filtered to the transitions a reconcile reads) are not in upstream; its claim controller reconciles about twice per claim as often. That is controller CPU plus the extra writes above.

Not affected: the L3 path (aggregated apiserver + node-local sandboxd claims) — no controller is involved; test/l3bench is unchanged. In the drained window both arms are bound by the same refill pipeline (Sandbox → Pod → schedule → vk-sandbox → status, about 9 members/s), which dominates either way.

docs/performance.md ("Upstream's claim controller sits inside the fork's run-to-run spread; the CRD path lost nothing in the move") was written from the warm-hit window only and needs the drained-window numbers.

Plan

• Upstream (kubernetes-sigs/agent-sandbox): open an issue with these numbers and propose making warmCandidateGracePeriod and warmCandidateRetryInterval configurable, with a documented setting that binds the first valid member as before.
• Upstream: find which watch or enqueue path re-triggers bound claims in the claim controller and send the fork's filtering as a PR where it applies.
• Here: correct docs/performance.md with both windows, and state the sizing rule in the docs — SandboxWarmPool replicas at or above the expected burst; a burst beyond pool depth is refill-bound; bursts that need sandboxd latency go through the L3 path.

Acceptance

• The docs correction lands with the numbers above.
• After an upstream release carrying the change is pinned here, the same interleaved A/B shows drained-window Ready p50 within 5% of the fork's, claim reconciles at or below 9 per claim, and apiserver writes per claim within 2%.