Add verified page restore, shared prefixes, and tail-only publication - #39
Closed
FujitsuPolycom wants to merge 15 commits into
Closed
Add verified page restore, shared prefixes, and tail-only publication#39FujitsuPolycom wants to merge 15 commits into
FujitsuPolycom wants to merge 15 commits into
Conversation
Use the direct-file native restore function only for immutable flat snapshot objects and route reconstructed page-delta bytes to the in-memory C++/CUDA page-placement function. The two interfaces have distinct verified signatures: direct restore accepts manifest lookup and cache-root inputs; page placement accepts authenticated encoded pages and a decoded page plan. A rejected page-delta restore continues to recompute rather than serving unverified state. SparkCache wire values, digest salts, chunk geometry, and cache namespace are unchanged; deployment source receipts are rebound to the resulting source tree. Validation: 739 SparkCache tests passed with 7 skipped; 108 deployment tests passed with 1 skipped; Ruff and diff checks passed.
Expose SparkCache CUDA restore and CUDA placement names in connector configuration, environment variables, deployment profiles, CLI options, logs, and documentation. Preserve legacy native-restore keys and Python symbols as compatibility aliases; conflicting canonical and legacy values reject startup and legacy-only input warns once per process. Cache identity, digest salts, chunk geometry, and stored bytes are unchanged. Validation: 746 SparkCache tests passed with 7 skipped; 108 deployment tests passed with 1 skipped; Ruff passed.
Page-semantic extensions accept authenticated chunk boundaries that fall inside a larger HMA page. The delta reuses only byte-identical page prefixes, so a changed boundary-intersecting page and every following page are immutable replacement data. This preserves the longest verified publication base instead of falling back to a full snapshot. Restore continues to authenticate the embedded base graph and reconstructed result before placement; invalid geometry or bytes remain a cache miss and recomputation. Canonical SparkCache CUDA restore and placement configuration remains the only generated configuration vocabulary. Deployable SparkCache source SHA-256: bc7cae86732c869ee8b2205d48ac5be6f580ee8b77a3e4ffd4c69dcd4f1bfae5. Cache namespace impact: none. CacheIdentity values, digest salts, chunk geometry, and page-delta wire schemas are unchanged. Validation: python -m pytest sparkcache -q (747 passed, 7 skipped); python -m pytest deploy -q (108 passed, 1 skipped); python -m ruff check .; source-attestation tests (2 passed).
Per-file overhead dominated the 256K restore because physical delta objects followed the 256-token logical boundary. Page-delta manifest v2 stores ordered 64-MiB authenticated extents and reads or publishes them in bounded batches. Version 1 remains readable; cache identity, digest salts, logical chunk geometry, page sharing, and the page-tail namespace are unchanged. GPU-free tests cover exact restore, corruption, bounded memory, file count, compatibility, and capacity behavior.
SparkCache opts into vLLM recurrent-boundary hand-offs and accepts them only when request, group, token boundary, recurrent topology, and non-null physical block agree. Every aligned recurrent group must have exactly one proven block before a store plan is created. Absent, incomplete, contradictory, or preemption-stale metadata cancels publication. SparkCache never scans arithmetic, running, or speculative slots for a substitute. GPU-free coverage exercises the 6,912-token boundary with seven DFlash verification slots through SparkContextCacheConnector and ManifestStore, including malformed coverage and request-lifetime cleanup. The exact GLM vLLM lease contract accepts and requires the recurrent-boundary runtime postimages and SchedulerOutput interface. Cache namespace impact: none. CacheIdentity values, digest salts, chunk geometry, manifest schemas, and page-delta wire bytes are unchanged. Deployable SparkCache source SHA-256: 01cc59bf2c45af60f02813b771a484e65829e4f464eb27aa1756ca7067c82c9f. Validation: python -m pytest sparkcache -q (752 passed, 7 skipped); python -m pytest deploy -q (108 passed, 1 skipped); python -m ruff check .; source-attestation tests (2 passed). One concurrent validation run exceeded an existing sub-millisecond timing assertion; the isolated test and the following full SparkCache run passed.
The GLM recurrent lease document now names one final runtime state across all eleven files. Four recurrent postimages and seven unchanged files therefore share a coherent verifier state; the impossible pre-producer state is no longer advertised.\n\nThe four SparkCache patches remain exact preimages for the separate recurrent producer. Add regression coverage for coherent final-state verification and state that the final contract is valid only after producer composition.\n\nCache namespace impact: none. CacheIdentity values, digest salts, chunk geometry, manifest schemas, and page-delta bytes are unchanged.\n\nValidation: python -m pytest sparkcache -q (760 passed, 7 skipped); python -m pytest deploy -q (108 passed, 1 skipped); python -m ruff check .
vLLM exposes a hash-proven aligned recurrent block only after the prefill step that produces it, while nonaligned publication boundaries have no separate hand-off. Retain recurrent new-request store state through the following cached step, validate only when that step can publish, and require mappings only for recurrent groups exactly aligned at the store boundary. Nonaligned groups use the authoritative partial page in the accumulated request table; unexpected overrides and missing aligned proofs still fail closed. Cache namespace impact: none. CacheIdentity values, digest salts, 256-token geometry, manifest schemas, page-delta bytes, and the page-tail-cow-v1 namespace are unchanged. Validation: python -m pytest sparkcache -q (763 passed, 7 skipped); python -m pytest deploy -q (108 passed, 1 skipped); python -m ruff check .; git diff --check.
A recurrent partial page can be replaced after the initial request table is observed. Its durable publication source is the pinned block delivered by vLLM partial_tail_offloads, not the accumulated source ID. Treat absent per-request metadata as pending, latch complete validated mappings from any scheduler output, reject incomplete or conflicting evidence, and publish only after every recurrent group has a proven block. Preemption clears the latch; completion and quorum retire pending state. Cache namespace impact: none. CacheIdentity values, digest salts, 256-token geometry, manifest schemas, page-delta bytes, and page-tail-cow-v1 are unchanged. Validation: python -m pytest sparkcache -q (764 passed, 7 skipped after isolated timing rerun); python -m pytest deploy -q (108 passed, 1 skipped); python -m ruff check .; git diff --check.
The recurrent producer now identifies a replay-boundary block when one cache_blocks call crosses that boundary, rather than requiring the caller's token count to equal it. Advance the exact single_type_kv_cache_manager postimage and lease-contract receipt while retaining the same required symbol surface. Cache namespace impact: none. CacheIdentity values, digest salts, chunk geometry, manifest schemas, page-delta bytes, and page-tail-cow-v1 are unchanged. Validation: strict eleven-file verifier passed against the composed vLLM source; SparkCache 764 passed, 7 skipped; deploy 108 passed, 1 skipped; Ruff and diff checks passed.
An overshooting Mamba cache_blocks call has already nulled the earlier arithmetic boundary slots, so the crossing postimage cannot prove or recover that state. Restore the verified exact-stop producer postimage and lease contract. Scheduler-level regression coverage owns the invariant that aligned GLM prefill stops at the 2,304-token boundary; nonaligned 8,192 publication uses the next-step partial-tail CoW hand-off latched by SparkCache. Cache namespace impact: none. CacheIdentity values, digest salts, chunk geometry, manifest schemas, page-delta bytes, and page-tail-cow-v1 are unchanged.
vLLM can emit a valid aligned checkpoint while a request is still advancing toward a later SparkCache publication boundary. Treat well-formed entries below the store plan as intermediate evidence: do not latch or poison them. Continue waiting until every recurrent group supplies proof at the exact target boundary. Future, malformed, null, non-recurrent, incomplete, and conflicting target entries remain fail-closed. Cache namespace impact: none. CacheIdentity values, digest salts, 256-token geometry, manifest schemas, page-delta bytes, and page-tail-cow-v1 are unchanged. Validation: SparkCache 764 passed, 7 skipped; deploy 108 passed, 1 skipped; Ruff and diff checks passed.
Keep future-boundary evidence fail-closed, but include its observed boundary, target boundary, recurrent group, and physical block in the rejection reason. This makes the live scheduler hand-off diagnosable without changing acceptance semantics. Cache namespace impact: none. CacheIdentity values, digest salts, chunk geometry, manifests, page deltas, and page-tail-cow-v1 are unchanged. Validation: SparkCache 765 passed, 7 skipped; deploy 108 passed, 1 skipped; Ruff and diff checks passed.
Expose SparkCache's 256-token recurrent publication granularity and a side-effect-free per-request boundary proposal. Proposals exist only for eligible non-streaming recurrent stores and use the same aligned-span rule as the store plan. Advance the exact lease contract for vLLM's request-local target propagation, MultiConnector union, and exact secondary-hash CoW proof. Cache namespace impact: none. CacheIdentity values, digest salts, 256-token geometry, manifests, page deltas, and page-tail-cow-v1 are unchanged. Validation: SparkCache 766 passed, 7 skipped; deploy 108 passed, 1 skipped; exact 12-file verifier, Ruff, and diff checks passed.
Advance the exact scheduler postimage after fixing restored-prefix scheduling: a future connector proposal is passed to KVCacheManager only when the current allocation's finalized end reaches the target. Intermediate 8K allocations retain no request-local publication target. Cache namespace impact: none. CacheIdentity, digest salts, chunk geometry, manifests, page deltas, and page-tail-cow-v1 are unchanged. Validation: strict twelve-file verifier and focused source/profile tests passed; the preceding full SparkCache run passed 766 tests with 7 skips.
FujitsuPolycom
changed the base branch from
codex/nonaligned-recurrent-boundary
to
main
August 30, 2026 11:20
FujitsuPolycom
changed the title
This was referenced Aug 30, 2026
This was referenced Aug 31, 2026
Owner
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Resulting behavior
SparkCache persists verified prompt state on rank-local storage and restores the longest reusable prefix that every required rank can prove. Commit
65b6642df1afc64366430d3aef9aca01f5c5e1c3adds the complete opaque-page placement and recurrent-publication contract described below.SparkCache CUDA page placement
spark_cache_cuda_*names. Compatibility aliases remain accepted where the configuration parser documents them.Copy-on-write page tails and physical macro objects
Proven recurrent publication targets
recurrent_boundary_granularityadvertises SparkCache's immutable 256-token publication geometry without changing vLLM's internal hash size.get_recurrent_publication_boundaries(request)returns the exact eligible store target computed by the same aligned-span rule used for connector metadata. It returns no target when storage is unavailable, the request is outside policy, streaming publication is selected, or the topology has no recurrent group.BlockHashWithGroupIdremains attached to the pinned destination so the consumer can prove that the requested publication hash identifies the emitted block.Exact source identity
65b6642df1afc64366430d3aef9aca01f5c5e1c341ad0a119ba109fd28900a2dcc9f9b4d8c293809a2add45a9f97446f6c2a843355161da9a5499ff7501b4750d2163591785d73458adbdfa3fd4b06b213c3aab45255a0b039f1c9940a4b1fad0efd004d263227c9Compatibility and cache namespace
Cache namespace impact: none.
CacheIdentitywire values, digest salts, the 256-token logical geometry, page-tail storage key, model and topology identity fields, and SparkCache CUDA placement ABI are unchanged.Version 2 page-delta manifests add authenticated physical grouping while retaining version 1 readers. Compatible
page-tail-cow-v1entries remain eligible. The source identity, lease-contract identity, and composed image identity must change because the accepted runtime behavior and source bytes change. State whose identity or proof cannot be established is not published or restored; request execution recomputes it.Validation status
Status: qualified only for the exact artifact and bounded cases in SparkRing pull request #147.
That record binds SparkCache commit
65b6642df1afc64366430d3aef9aca01f5c5e1c3to SparkRing commitd93cb3d98305041081cf572521602625185112aeand local image IDsha256:ed60be066d6d9eadea267bc4597a0687869f3ddb95a3e5c6f86649893a838eb8. The exact image completed semantic generation, clean-restart 8K restore, 128K restore, 128K-to-256K tail publication, verified 256K restore, and one C16 shared exact-prefix cohort. The 256K tail used 13 authenticated macro objects. Restore timing remains research-only, and the record does not qualify DFlash response quality or a published OCI digest.GPU-free validation for this source head:
python -m pytest sparkcache -q: 766 passed, 7 skipped;python -m pytest deploy -q: 108 passed, 1 skipped;git diff --checkpassed;Historical review note
This main-based review contains the implementation previously divided among draft pull requests #25, #26, #29, #30, #32, #33, #35, #37, and #38. Pull requests #30 and #32 point to the same commit,
5ec6a9953ad5d39120298bbfc26e95a6fa4b1dc3. Those drafts remain open until this consolidated diff is reviewed; this metadata change does not close, delete, or merge them.