feat: prefix KV reuse

[drunkcoding]

Description

Opt-in, correctness-preserving longest-prefix KV reuse wired into the active OpenAI continuous-batching path, with exact cold/warm equivalence and unchanged disabled or unsupported-runtime behavior.

Motivation

A PrefixCache scaffold existed, but the active request path performed no KV lookup, attachment, publication, or reuse. This PR implements the full path per the Momus-approved plan (docs/superpowers/plans/2026-08-21-prefix-kv-reuse.md).

What changed

• Prefix contract + index (serving/prefix_contract.py, serving/prefix_cache.py): single definition site for PrefixLease/PrefixMatch/PrefixLeaseProvider; namespace-scoped exact parent-path prefix tree with SHA-256 only as a bucket accelerator, pinned leases with retain/release, lease-aware subtree LRU eviction.
• Layer-complete KV storage (runtime/attention_backend.py, models/qwen3_paged_attention.py, serving/model_runner.py): one validated LayeredPagedKVStore across all layers with export/import/checkpoint/restore; per-module layer_idx plumbing; ModelRunner.get_prefix_reuse_capability with complete-registry validation.
• Query vs KV metadata (runtime/attention_types.py, serving/batch.py, serving/model_runner.py, runtime/attention_backend.py): canonical PagedBatchLengths(query_lengths, query_offsets, context_lengths, kv_seq_lengths); FlashInfer qo_indptr uses query lengths while KV pages use total; last_flashinfer_plan snapshot.
• Ownership + COW (serving/kv_cache.py): block refcounts, validated store binding that disables independent tensors, layer-complete copy-on-write for shared partial tails, checkpoint/restore-based swap.
• Transactional group admission (serving/scheduler.py, serving/sequence.py, serving/kv_cache.py): pin-before-evict, all-or-nothing SequenceGroup admission via prepare_group/commit_group/abort_group.
• Committed publication + DFlash reconciliation (serving/engine.py): advance committed_kv_tokens only after a successful forward, publish only block-complete committed prompt ranges, exclude reused-prefix and non-cold requests from DFlash delegation.
• Active-path gating + observability (serving/engine.py, entrypoints/openai/api_server_v2.py): capability + flag gate, validated store bound before scheduler construction, --prefix-cache-max-entries, /v1/reload invalidation, prefix_cache_* stats.
• Tests, benchmark, docs: exact-collision/lease, layer completeness, metadata parity, ownership/COW/preemption, atomic-group, committed-range/DFlash, engine gating, opt-in Qwen3/FlashInfer CUDA parity (test_prefix_cache_cuda.py), disabled/cold/warm benchmark with digest-mismatch abort (benchmarks/serving/prefix_cache_benchmark.py), and serving/benchmarking/README/ARCHITECTURE/CHANGELOG updates.

Scope / limitations

• Reuse activates only on the Qwen3 paged-attention path with a complete layer registry, matching geometry, and real FlashInfer prefill/decode; every other runtime keeps the unchanged cold path with a stable disabled reason.
• Shares canonical metadata, storage, and PrefixLeaseProvider contracts with chunked prefill; co-enablement requires future reconciliation tests.
• Real Qwen3-30B + FlashInfer CUDA parity and the real-model benchmark/rollback steps are environment-gated (GPU + FlashInfer + weights) and were not executed in this CPU CI environment; the CUDA test SKIPS without MOE_PREFIX_CACHE_CUDA=1, and the benchmark dry-run schema check passes.

Type of Change

• Production implementation
• Documentation update

Verification

• CPU suites green: tests/python/serving (174 passed, 3 CUDA/FlashInfer skips), tests/python/contextpilot (76 passed), touched tests/python/unit, tests/python/dflash, tests/python/integration.
• ruff check clean on all touched files; LSP reports no errors in serving/ or attention_backend.py.
• Benchmark dry-run: equal disabled/cold/warm token+logit digests, warm suffix geometry, mismatch aborts with exit 2 and no JSON.

Checklist

• Active-reference safety (pin-before-evict, lease-aware eviction)
• Reload/cancellation rollback
• Cold/warm equivalence path implemented and covered
• Disabled/unsupported runtime falls back to cold path unchanged

[drunkcoding]

Momus plan review: [OKAY] on the first review round.

[drunkcoding]

Independent implementation verification:

• Focused prefix ownership/admission/publication suites: 56 passed.
• Python LSP scan: 0 errors across 50 files.
• Branch clean and synchronized.

PR remains draft for real Qwen3/FlashInfer qualification.

[drunkcoding]

Productization findings (full report: docs/superpowers/reports/2026-08-30-productization-verdicts.md). Draft fix: #190.

The reported native segfault was the multi-model kTopologyHandle global (same root cause as #177; guard added in #187), not a prefix-COW bug. After isolating one model per process:

• device-index mismatch (cuda vs cuda:0) was rejecting the store → prefix cache now binds/activates (fixed);
• hit counters were tracked but never surfaced (fixed);
• FlashInfer plan dtype float16 vs bf16 (fixed).

Remaining correctness concern: with reuse active (query=8 / context=64 / kv=72), the warm run produces different generated tokens than cold on sm120 (e37a9d14 vs 3739c3b3). A read-only Oracle pass verified geometry, block tables, slot mapping, RoPE (K stored post-RoPE, reused at identical positions), COW, and FlashInfer causal/append masking are all correct on paper — most likely numerical greedy-argmax flips from the append kernel on Blackwell. Your parity test test_real_flashinfer_warm_suffix_matches_cold_logits is the decisive check but can't run here (loads 2 engines/process → the multi-model limit above). Please run it on reference hardware. #190 is a draft pending that.