Add exact GLM-5.3 adaptive-MTP and DFlash7 SparkCache runtimes - #146
Closed
FujitsuPolycom wants to merge 16 commits into
Closed
Add exact GLM-5.3 adaptive-MTP and DFlash7 SparkCache runtimes#146FujitsuPolycom wants to merge 16 commits into
FujitsuPolycom wants to merge 16 commits into
Conversation
Compose both adaptive-MTP and DFlash7 Python-overlay images from SparkCache 08e2977 and the hash-proven recurrent-boundary vLLM producer. Require the producer label in resolved profiles, preserve fail-closed runtime attestation, and move the adaptive executable quickstart onto the capable builder.\n\nCache namespace impact: none. CacheIdentity values, digest salts, 256-token geometry, page-delta wire bytes, and the page-tail-cow-v1 namespace are unchanged. The lease-contract bytes advance so the exact recurrent producer postimages are accepted.\n\nValidation: exact source context prepare and verify passed; Ruff passed; GPU-free CI scope 1,973 passed, 9 skipped.
The PR35 lease contract requires recurrent producer symbols, so it cannot verify the intermediate source after only the SparkCache 041 patch. Run the single lease verification after the recurrent patch creates all required symbols and exact postimages.\n\nAdd GPU-free ordering coverage that requires the producer application and definitions to precede the only lease-verifier invocation.\n\nValidation: exact context prepare/verify passed; 1,974 passed, 9 skipped; Ruff passed.
Advance both GLM Python-overlay builders and executable profiles to SparkCache 49c517e and final lease contract f36ed14. Context preparation now executes SparkCache's strict lease verifier against the exact fully composed vLLM tree after all recurrent postimages exist.\n\nCache namespace impact: none. CacheIdentity values, digest salts, 256-token geometry, page-delta bytes, manifest schemas, and page-tail-cow-v1 remain unchanged.\n\nValidation: exact context prepare/verify and real final lease verifier passed; 1,975 passed, 9 skipped; Ruff passed.
Treat the composed verification report as one exact DFlash member plus an independent exact set of four recurrent postimages. Reject malformed, duplicate, missing, altered, or unexpected records.\n\nValidation: focused overlay/profile contracts 47 passed; Ruff passed. The broad GPU-free run passed 1,975 tests and hit one unrelated Windows atomic-status-file PermissionError; that isolated test passed immediately.
Mamba cache_blocks can receive a finalized token count beyond the replay boundary. Queue the hash-proven page when that boundary block enters the newly cached range instead of requiring num_tokens to equal it. Repin both GLM Python-overlay runtimes, profiles, and quickstarts to SparkCache 972b203 and its exact crossing-aware lease contract. Cache namespace impact: none. SparkCache identities, digest salts, 256-token geometry, manifest schemas, page-delta bytes, and page-tail-cow-v1 are unchanged. Validation: exact context preparation and eleven-file verifier passed; focused overlay/profile tests 47 passed; maintained-tree test run reached 1,975 passes with one corrected stale-pin failure; Ruff and diff checks passed.
An overshooting Mamba manager call has already nulled the earlier arithmetic page, so the attempted crossing predicate could not produce hash-proven state. Restore the exact-stop producer and fail closed for direct overshoot callers. Repin both runtimes and executable profiles to SparkCache bf7174e, which keeps the all-groups proof latch while restoring the verified f67 producer contract. Cache namespace impact: none. SparkCache identities, digest salts, 256-token geometry, manifests, page deltas, and page-tail-cow-v1 are unchanged.
Advance both GLM Python-overlay runtimes and executable profiles to SparkCache bd3eec1. The consumer now ignores valid earlier recurrent checkpoints while waiting for an exact later publication boundary, including the 6912 proof observed before the 8192 partial-tail CoW hand-off. The verified vLLM producer, scheduler regressions, lease contract, and cache namespace are unchanged. Validation: exact context preparation and repeat verification passed; maintained SparkRing scope 1976 passed, 9 skipped; Ruff and diff checks passed.
Advance both GLM Python-overlay runtimes and executable profiles to SparkCache c56f77f. Future recurrent proof remains rejected, and the diagnostic source now records its observed boundary, target boundary, group, and block. The verified vLLM producer and scheduler behavior are unchanged. Validation: exact context preparation and repeat verification passed; maintained SparkRing scope 1976 passed, 9 skipped; Ruff and diff checks passed.
Add a side-effect-free per-request recurrent publication proposal API with a MultiConnector union and stock empty default. The scheduler stops at the earliest exact connector target and passes it request-locally through allocation; native prefix-cache hash geometry is unchanged. Mamba partial-tail CoW carries the exact BlockHashWithGroupId and verifies the hash map names the emitted pinned block. Repin both GLM Python-overlay runtimes and executable profiles to SparkCache PR39. GPU-free exact tests use internal hash size 16, a 256-token target group, a hash-16 draft group, seven speculative slots, prompts 8209 through 8224, aligned publication, preemption, default behavior, and MultiConnector union. Validation: exact context preparation and repeat verification passed; exact vLLM target 61 passed; maintained SparkRing scope 1976 passed, 9 skipped; focused overlay/profile scope 47 passed; Ruff and diff checks passed.
Fix restored-prefix scheduling so a future connector proposal shapes chunk boundaries but is passed to KVCacheManager only when the current allocation's finalized end reaches it. Intermediate 8K allocations after a restored 128K prefix carry no target and cannot trip the allocator equality guard. Extend exact scheduler coverage to fresh and restored 256K prompts. Both paths reach 262144 through bounded chunks, set the request-local target only on the final allocation, and emit the exact CoW proof on the following schedule. Validation: exact context preparation and repeat verification passed; restored and fresh 256K scheduler regressions passed; maintained SparkRing scope 1976 passed, 9 skipped; Ruff and diff checks passed.
FujitsuPolycom
changed the base branch from
codex/glm53-pr35-recurrent-runtime
to
main
August 30, 2026 11:32
Name selected source contracts, profiles, checks, and recovery behavior directly. Correct the lease verifier size to twelve files and replace one test-patch description while preserving its behavior through updated exact checksums. Runtime source commits, cache identities, configuration, and serving behavior are unchanged.
FujitsuPolycom
changed the title
This was referenced Aug 30, 2026
This was referenced Aug 31, 2026
Owner
Author
|
Closing this draft as superseded by the source-pinned GLM-5.3 operator path now on main. The published runtime provides DFlash2 depth seven, DCP1/DCP2/DCP4 profiles, fastsafetensors target loading, media-aware SparkCache, CUDA restore/publication, shared-prefix retention, and the maintained four-rank quickstart. This branch remains available as historical research. Adaptive-MTP behavior that is still useful should be proposed separately against the maintained runtime so its source contract, performance evidence, and compatibility can be reviewed without carrying the obsolete image compositions and profile rewrites in this stack. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Resulting runtime
This draft provides two reproducible ARM64 GLM-5.3 Flash TP4/DCP1 compositions on NVIDIA DGX Spark:
Both compositions bind the same target checkpoint, vLLM Python behavior, B12X kernels, patched NCCL transport, SparkCache source, recurrent-publication contract, and four-rank launcher interface.
Exact vLLM Python overlay
The image retains compiled vLLM extensions, console entry point, wheel metadata, and generated version metadata from commit
da4d7be6c97434f6942292ed8abbf4b32dc44355. It overlays the production Python source from commit0b67266a0f37d6146a8403fb8482403c62f412d5.The
vllm/production-source diff fromda4d7be6through0b67266acontains exactly 31 Python files and no native-build-source changes. The overlay manifest contains those same 31 paths. Repository tests, documentation, and launch scripts outsidevllm/are not installed by the overlay. The image is therefore described as0b67266aPython over retainedda4d7be6native extensions, not as a source-built0b67266awheel.Every overlaid file is checked against its exact source blob. The retained ELF inventory, compiled-dispatch inventory, version metadata, wheel metadata, console entry point, and overlay manifest are independently hashed and recorded in the build receipt.
B12X, loading, and serving behavior
B12X commit
b1d541f9e71a35f030d45fae437630fff7507c2asupplies the trusted-metadata, request-sized live-tensor KDA interface required by the overlaid vLLM Python source. Image verification rejects the base B12X implementation because it lacks that interface.The adaptive-MTP profile uses embedded MTP with maximum depth five, initial depth three, and a 32-step acceptance window. It loads the target with fastsafetensors queue size one and reserves 20 GiB of FP8 KV per rank.
The qualified DFlash7 profile uses the external checkpoint
incoai/GLM-5.3-Flash-DFlash2@dc77ff1c99eeb2df044ee3d4f0094eb033fee410, seven speculative tokens, and draft TP4. The target uses fastsafetensors queue size one whiledraft_load_configkeeps the DFlash checkpoint on safetensors. An all-safetensors profile is implemented but has no live qualification on this composition.Both profiles use B12X attention, MoE, and linear backends; PYNCCL through the pinned NCCL library; asynchronous scheduling; 256-token vLLM blocks; chunked prefill; and the GLM tool and reasoning parsers. The DFlash7 image removes the unused
deep_ep==2.0.0+localdistribution after proving sole module ownership. Unsupported symmetric-memory and FlashInfer all-reduce probes are disabled. ModelOpt experimental-format and FP8 KV accuracy warnings remain visible because they describe real limitations.SparkCache tail publication and shared restore
SparkCache commit
65b6642df1afc64366430d3aef9aca01f5c5e1c3provides:tail-cow-v1publication, mapped topage-tail-cow-v1for opaque GLM pages;The scheduler carries an eligible recurrent target through one allocation. Full-page and partial-tail copy-on-write mappings are accepted only when request, group, boundary, block, ownership, and exact hash evidence agree. Missing per-request evidence keeps publication pending. Incomplete, malformed, conflicting, stale, null, or wrong-boundary evidence cancels the publication attempt. Request execution continues and unavailable state is recomputed.
The one-shot cache-removal option records its operation token only after SparkCache removes its owned entries successfully. Reusing the token on another start does not repeat the removal.
Exact identities
74a6a472e7f11a56029d52b879c9b3665aa60e01, tree5d8a87fb545a0cd6316d054dd9a2c35b186603e10b326f14578acaeed885637766fbc78f16df0a83, tree89cd07b46e3471227789d14b4025dbdc0d55e20dd93cb3d98305041081cf572521602625185112ae, tree867c43d0107856c3ba43500912462008ba149cc8ghcr.io/fujitsupolycom/sparkring-glm53-runtime@sha256:864adfe68f458223e186a19844ac80c7adc7365e5db1f25e109b85fc19850dcdda4d7be6c97434f6942292ed8abbf4b32dc44355, tree4a99033352bc1a6b00852dc0e8b1cecbdd9f0ebb0b67266a0f37d6146a8403fb8482403c62f412d5, treeba9484ccb33aa56e90ff2f447f15ca9b9da97639e5e528288b173399611a4930fecc4182b7208bc1564881d52ca5d2c5c4ae0f6ab1d541f9e71a35f030d45fae437630fff7507c2a, treec69cdec1c59a08e8e0e549f930fa8abcfb5134ae65b6642df1afc64366430d3aef9aca01f5c5e1c3, tree41ad0a119ba109fd28900a2dcc9f9b4d8c293809, source SHA-256a2add45a9f97446f6c2a843355161da9a5499ff7501b4750d2163591785d73458adbdfa3fd4b06b213c3aab45255a0b039f1c9940a4b1fad0efd004d263227c95a6561a5bbab990dcd03bfd6a485ea26c3b5a578c2fd61b76305767b16dbfba0587fc332917a8ffd5a29712dc5253d51e6051eca1166ed4a165e576a84f2e300local-inference-lab/GLM-5.3-Flash-NVFP4@520de24eabf507659eaef7c70f14fd584527faccb33c03475ba7322cf398828f2d8d1be376df30dc05c6b40c28c8ea8da23e410bCompatibility and cache namespace
Cache namespace impact from the recurrent-publication and macro-object integration: none. SparkCache
CacheIdentitywire values, digest salts, 256-token logical geometry, model and topology identity fields, page-tail storage key, and CUDA placement ABI are unchanged. Version 2 physical grouping does not alter logical chunk identity, and compatible version 1 page-delta manifests remain readable.Adaptive MTP and external DFlash use different draft-state identities, served-model names, JIT roots, SparkCache roots, and one-shot removal tokens. Their stored state cannot cross those identities. Loader selection does not change target or draft model state, but the profile templates use separate roots so loader observations remain isolated.
Image acceptance requires exact source, tree, patch, retained-binary, dependency, model, and image-label evidence. A mismatch prevents the image or stored state from being accepted; serving recomputes unavailable context. Existing GLM-5.2, public BF16 DFlash2, DeepSeek, and Qwen profiles remain unchanged.
Status and bounded evidence
DFlash7 status: qualified only for the artifact and cases recorded in SparkRing pull request #147.
That evidence binds local image ID
sha256:ed60be066d6d9eadea267bc4597a0687869f3ddb95a3e5c6f86649893a838eb8to runtime commitd93cb3d98305041081cf572521602625185112aeand SparkCache commit65b6642df1afc64366430d3aef9aca01f5c5e1c3. The image completed exact construction verification, four-rank health, semantic generation, clean-restart 8K restore, 128K restore, 128K-to-256K tail publication, verified 256K restore, and one C16 shared exact-prefix cohort. The 256K tail was represented by 13 authenticated macro objects.Adaptive-MTP status: implemented and GPU-free tested. Its image construction, exact source composition, profile resolution, fast loading configuration, and acceptance-policy interface are covered. Persistent restore, shared-prefix concurrency, performance, and output quality have no live qualification for the adaptive profile.
Limitations:
--prefill-schedule-interval 8was not part of the recorded profile.Validation
Validation of the main-integrated branch:
da4d7be6..0b67266aproduction-source audit: 31 Python changes and zero native-build-source changes;The exact patched vLLM target at runtime commit
d93cb3d98305041081cf572521602625185112aepassed 61 focused tests, including the 8,209-through-8,224 prompt matrix. Main-integration commit0b326f14578acaeed885637766fbc78f16df0a83changed only README text and preserves that runtime commit as its first parent. Review head74a6a472e7f11a56029d52b879c9b3665aa60e01changes repository prose, command error text, and one test-patch description with matching test-patch checksums. Production source commits, production patch bytes, cache identity, configuration, and serving behavior remain those qualified atd93cb3d98305041081cf572521602625185112ae.Historical review note
This main-based review contains runtime work previously divided among draft pull requests #128, #129, #130, #131, #133, #134, #136, #137, #139, #140, #141, #142, #143, #144, and #145. Pull request #147 consolidates the adaptive-versus-DFlash diagnostic from #132, predecessor artifact evidence from #138, and the exact
ed60...qualification. Pull request #148 consolidates direct-fabric image distribution from #135. Those evidence and distribution reviews remain separate from this runtime diff. No draft is closed or deleted by this consolidation.