Run eager GLM mHC prefill on each rank's token quarter - #240
Merged
Conversation
Emit optional admission, restore-finalization, accepted prompt-work, preemption and terminal events for the SparkCache request ledger. Preserve deferred GPU lease attribution and dispatch ranges across asynchronous scheduling. Apply an exact-source transform after verifying checkpoint ownership dependencies; retain immutable payloads and published receipts. Validation: 470 mesh tests passed, four optional skips, including five companion-ledger integration tests; Ruff and documentation links passed. Hardware validation requires a rebuilt image.
Install four byte-attested continuation sources after verifying checkpoint ownership, then apply the matching scheduler attribution transform. Preserve all unrelated dependencies and generate image receipts from the composed runtime. Enable both checkpoint flags and pin the SparkCache attribution source; published image receipts retain their identities. Avoid redundant empty prompt events during ordinary decode. Validation: 501 mesh tests passed, four optional skips, including both scheduler variants and companion-ledger cases; Ruff and 562 documentation links passed. Hardware validation remains pending.
… source Select the tested SparkCache package with same-pass survivor reconciliation, request-attribution accounting, and optional deletion pacing. Preserve immutable published image receipts and package the continuation runtime with the startup gate. Validation: 3,089 offline tests passed with 85 documented skips; Ruff, documentation links, and local context preparation passed. Image build and serving validation remain pending.
…ions Retain raw cold-prefill observations and bounded recurrence/cache evidence for the exact continuation image, including unpaired-control and clock-resolution limits. Execute packaged scheduler/allocator methods for source retention, reserve overlap, publication pins and cleanup, and provide a portable harness with offline readiness/timing coverage. No cache identity or runtime source bytes change. Validation: 512 scoped CPU tests passed, 14 documented optional skips; Ruff passed. The attribution-containing image rebuild remains unqualified.
Reduce-scatter conventional TP partials, run repeated mHC on each rank's contiguous token quarter, and all-gather full attention and FFN inputs. Preserve full final and auxiliary outputs, exclude MTP/decode/graph paths, and default the feature off. Package the nine exact tested runtime sources with strict preimages, ownership updates, licensed archives and a review diff. Retain bounded original-image timing and correctness evidence; the attribution-containing Docker rebuild is not qualified. Validation: 541 profile/harness CPU tests passed with 14 optional skips; 59 companion tests passed with one optional skip; Ruff, all eight source origins, archive privacy and publication hashes passed.
Reject absent, malformed, boolean or negative cached-token counts instead of accepting a missing field as zero. Apply the same accounting boundary to semantic reuse checks and cold timing. Preserve all raw observations: all 19 retained continuation rows contain explicit integer zero counts. Update portable-harness provenance. Validation: 14 offline readiness/timing tests pass, including six reproductions of missing or invalid accounting; Ruff passed. Runtime source packages are unchanged.
…into perf/token-sharded-mhc-prefill
Require explicit nonnegative integer cached-token counts in the portable mHC harness. Missing accounting can no longer enter a cold timing result as zero. All 32 retained mHC timing rows already supply integer zero counts; preserve their raw bytes and update the public harness provenance. Validation: 28 offline harness tests passed across continuation and mHC; Ruff, publication privacy, links and index-level hashes passed. The mHC runtime source package is unchanged.
FujitsuPolycom
marked this pull request as ready for review
September 9, 2026 01:06
FujitsuPolycom
changed the base branch from
feat/request-cache-attribution
to
main
September 9, 2026 01:10
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Run repeated manifold-constrained hyper-connection (mHC) operations on each GPU’s quarter of the tokens during eligible 8,192-token GLM prefills. Reduce-scatter full TP partials, compute mHC on 2,048 owner rows, then all-gather normalized inputs. Owner residual/post/combine state stays local between layers; final and auxiliary outputs return in full token order.
Status: research-only and installed default-off with
SPARK_MHC_PREFILL_SHARD=0. The optional performance builder composes this package after the continuation and request-attribution sources from #239. It accepts pure eager BF16[8192,4096]base-model prefills at TP4/DCP4/PP1/DP1/PCP1. Unsupported shapes, mixed/speculative batches, and graph paths retain their normal behavior. All ranks agree on admission; already-reduced or unsupported MoE results fail before reaching mHC.The manifest binds eight preimages and nine replacement files to image
sha256:65f2b9181acd77db660f9c105554c4fca5c4df89d87d1374276e17c6831d1359. Licensed archives and a readable diff preserve the exact source. Installation validates runtime and overlapping checkpoint-ownership preimages before writing, preserves symbol requirements, and records the composed inventory. Current shared-image sources, model profiles, numerical kernels, and published receipts remain unchanged.The serving record retains the bounded 8K–128K measurements and limitations. Compared 8K/16K/64K conditions were 5.26%/5.00%/4.59% faster than control, with sequential testing, a reboot, a fresh cache root, and an unexplained initial startup failure. These observations do not qualify the rebuilt attribution-containing composition; no decode gain is claimed.
Validation after main/continuation integration: 602 mesh/profile/portable-harness CPU tests passed with four documented optional skips, including the exact SparkCache companion attribution tests. Ruff and whitespace checks passed. Coverage includes owner state, complete auxiliary/final outputs, exactly-once reduction, MTP exclusion, collective agreement, fallback, preimages, and source/archive/ownership tampering. No image build, GPU test, or live-host action was performed for this integration.