Package GLM MTP3 compute fixes and stream-ordered mesh transport - #219
Closed
FujitsuPolycom wants to merge 26 commits into
Closed
Package GLM MTP3 compute fixes and stream-ordered mesh transport#219FujitsuPolycom wants to merge 26 commits into
FujitsuPolycom wants to merge 26 commits into
Conversation
… bind in the cycle launcher The four-Spark cycle launcher currently hardcodes --served-model-name deepseek-v4-flash-0731 and mounts only MODEL_HOST_PATH. Two operator-facing extensions, both backward compatible: * SERVED_MODEL_NAME: when set in the per-rank environment (or exported), it overrides the model name vLLM advertises and accepts; the default still reproduces recipes/deepseek-v4-flash-0731.json. The environment template documents the new key next to the other serving values, and the --check summary prints the effective name. * HF hub snapshot model directory: when MODEL_HOST_PATH points into an HF hub cache at <repo>/snapshots/<revision>/, every model file is a symlink whose relative target ../../blobs/<sha> resolves above the mounted snapshot tree. The launcher now requires the sibling blobs directory and binds it read-only at /blobs so those targets land on the real weight payloads. Plain checkpoint directories are unaffected. Offline contract tests cover the default name, the override reaching the rendered command, the snapshot blob bind, the plain-directory no-op, and the fail-closed missing-blobs path.
Package the tested CUDA 13.3 and B12X compute composition with exact source verification, retain the BF16 verifier head, and isolate its persistent-cache namespace. Publish the immutable image reference with quickstart and repeated benchmark evidence. Validation: 318 CPU tests passed, 3 skipped; ARM64 image verification, exact installed-package parity, and anonymous registry access passed.
There was a problem hiding this comment.
🔵 Needs a closer look
It introduces a new pinned compute supply chain (CUDA/tooling + source archives + offline install) and verification surface where small encoding/path-handling issues and provenance expectations warrant final human review.
Pull request overview
Updates the GLM-5.3 Flash NVFP4-Spark native-MTP3 mesh (research-only) profile to default to a separate runtime NVFP4 proposal head (BF16 activations), and introduces a manifest-bound compute composition (CUDA 13.3 + vLLM overrides + full B12X tree) with stronger in-image verification and reproducible build inputs.
Changes:
- Add a pinned, offline-installable compute payload (CUDA 13.3 + vLLM override set + B12X
b58f34ea) and verify it inside the mesh image. - Extend image verification/tests to validate layered vLLM overrides, complete B12X file sets, and required compute-selection environment.
- Update contracts/docs/records to reflect the NVFP4/BF16 proposal-head behavior, image identity, cache namespace separation, and measurement linkage.
File summaries
| File | Description |
|---|---|
| runtime/glm53-spark-mtp3-mesh/verify_mesh_image.py | Adds compute composition verification (vLLM layered overrides, complete B12X map, CUDA component + ptxas checks) to device-free image verification. |
| runtime/glm53-spark-mtp3-mesh/test_image.py | Adds unit tests for new verifier helpers and profile pin assertions for compute lock/environment. |
| runtime/glm53-spark-mtp3-mesh/README.md | Documents the NVFP4/BF16 proposal head, updated benchmark linkage, and updated contract scope for the new image composition. |
| runtime/glm53-spark-mtp3-mesh/public-image.json | Updates public image identity/metadata and links compute-equivalence + proposal-head record. |
| runtime/glm53-spark-mtp3-mesh/pins.json | Adds compute pin block and updates cache namespace/compatibility text for the new compute+proposal-head composition. |
| runtime/glm53-spark-mtp3-mesh/image-receipt.json | Updates the content receipt to include compute verification outputs and the new image identity/limitations. |
| runtime/glm53-spark-mtp3-mesh/IMAGE_BUILD.md | Updates build/reproduction flow to include prepared compute context and clarifies proposal-head memory impact and equivalence limits. |
| runtime/glm53-spark-mtp3-mesh/Dockerfile | Installs offline compute payload during build; sets required environment and emits additional labels for compute/proposal-head identity. |
| runtime/glm53-spark-mtp3-mesh/compute/verify_compute.py | Adds a standalone verifier for installed compute state vs. receipt/source lock. |
| runtime/glm53-spark-mtp3-mesh/compute/test_compute.py | Adds CPU tests for compute source-lock invariants, package mapping behavior, and installer failure cases. |
| runtime/glm53-spark-mtp3-mesh/compute/source-lock.json | Introduces the authoritative compute source lock binding vLLM override hashes, B12X identity/file-map hash, CUDA component hashes, and env contract. |
| runtime/glm53-spark-mtp3-mesh/compute/README.md | Documents compute source composition, provenance, offline preparation/install model, and line-ending normalization rationale. |
| runtime/glm53-spark-mtp3-mesh/compute/prepare_compute_source.py | Adds tool to fetch+verify CUDA/B12X inputs and build a prepared, checksum-bound compute context for offline Docker builds. |
| runtime/glm53-spark-mtp3-mesh/compute/apply_compute.py | Adds offline installer that lays down CUDA, patches vLLM via replacement archive, installs B12X, and writes an installed receipt. |
| runtime/glm53-spark-mtp3-mesh/compute-image-equivalence.json | Records compute-equivalence assertion between tested private image and published public image for package files + selected environment. |
| runtime/glm53-spark-mtp3-mesh/build_image.py | Extends context preparation to require and embed a prepared compute directory, and passes compute lock hash into the build. |
| recipes/glm53-spark-mtp3-managed-mesh-tp4.json | Updates recipe to bind compute contract/sha, proposal-head metadata, cache namespace, and evidence status/scope language. |
| README.md | Updates top-level profile descriptions and results table rows to reflect proposal-head configuration and revised record linkage. |
| performance/records/glm53-flash/spark-mtp3-nvfp4-proposal-head-20260905.md | Adds the proposal-head comparison record describing conditions/results/limitations and identities. |
| performance/records/glm53-flash/spark-mtp3-nvfp4-proposal-head-20260905.json | Adds machine-readable proposal-head comparison results and bound identities/receipt hashes. |
| docs/GLM53_SPARK_MTP3_MESH_QUICKSTART.md | Updates quickstart to reflect proposal-head behavior, revised evidence transfer limits, and new image IDs/links. |
| .gitattributes | Adds attributes to preserve binary archive bytes and patch whitespace handling for compute artifacts. |
Review details
Suppressed comments (2)
runtime/glm53-spark-mtp3-mesh/compute/prepare_compute_source.py:143
prepared-manifest.jsonis written without an explicit encoding/newline policy. For reproducible receipts (and consistency with other runtime receipts), write it as UTF-8 with\nnewlines.
manifest = destination / "prepared-manifest.json"
manifest.write_text(json.dumps(prepared, indent=2, sort_keys=True) + "\n")
return manifest
runtime/glm53-spark-mtp3-mesh/compute/apply_compute.py:149
- The installed compute receipt is written without an explicit encoding/newline policy. Since downstream verification compares raw bytes and other receipts are written as UTF-8 with
\nnewlines, this should be made explicit here too.
receipt.parent.mkdir(parents=True, exist_ok=True)
receipt.write_text(json.dumps(output, indent=2, sort_keys=True) + "\n")
return receipt
- Files reviewed: 22/24 changed files
- Comments generated: 3
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+8
to
+32
| from pathlib import Path | ||
|
|
||
|
|
||
| def _sha256(path: Path) -> str: | ||
| return hashlib.sha256(path.read_bytes()).hexdigest() | ||
|
|
||
|
|
||
| def _map_sha256(files: dict[str, str]) -> str: | ||
| payload = json.dumps(files, sort_keys=True, separators=(",", ":")).encode() | ||
| return hashlib.sha256(payload).hexdigest() | ||
|
|
||
|
|
||
| def verify(site_packages: Path, receipt: Path, source_lock: Path) -> dict: | ||
| lock = json.loads(source_lock.read_text()) | ||
| installed = json.loads(receipt.read_text()) | ||
| lock_hash = _sha256(source_lock) | ||
| if installed["source_lock_sha256"] != lock_hash: | ||
| raise ValueError("installed receipt uses a different compute source lock") | ||
| expected_vllm = {path: result for path, _, result in lock["vllm"]["files"]} | ||
| if installed["vllm_overrides"] != expected_vllm: | ||
| raise ValueError("installed receipt omits or changes vLLM overrides") | ||
| for relative, expected in expected_vllm.items(): | ||
| actual = _sha256(site_packages / relative) | ||
| if actual != expected: | ||
| raise ValueError(f"installed vLLM hash mismatch for {relative}: {actual}") |
Comment on lines
+14
to
+16
| HERE = Path(__file__).resolve().parent | ||
| LOCK = json.loads((HERE / "source-lock.json").read_text()) | ||
|
|
Preserve per-capture stream admission, sequence shared staging and output copies, and reject images without the pinned compute configuration. Record per-run benchmark variability and bundled vLLM provenance. Validation: 372 CPU tests passed, 21 skipped; ARM64 image content checks and anonymous publication passed. GPU stream/fault validation remains separate from these CPU checks.
FujitsuPolycom
marked this pull request as draft
September 5, 2026 22:55
FujitsuPolycom
changed the title
FujitsuPolycom
changed the base branch from
main
to
codex/managed-launch-memory-gate
September 6, 2026 01:05
The cycle launcher retains configured API model names and read-only Hugging Face blob mounts alongside the supported runtime profiles. Validation: 18 offline launcher tests pass.
Managed startup requires sufficient available RAM and contiguous allocator capacity on every rank. Memory preparation refuses active workloads and shares a host-local authorization lock with model startup. Direct managed starts check capacity without compacting memory. Validation: required CI checks pass; 324 local CPU tests pass with three documented skips. The bounded hardware evidence records refusal below threshold and successful four-rank startup after operator-authorized recovery.
The four-Spark launcher accepts a configured API model name and mounts readable sibling blob storage for Hugging Face snapshot checkpoints. Default vLLM arguments are preserved, and the effective model name is printed during input checks. Validation: 18 offline launcher tests and all required CI checks pass. No serving or GPU qualification is claimed.
The operator image builder synchronizes B12X histogram publication before block arrival and increments the kernel compile revision. Rank-zero readiness exercises stochastic sampling and reasoning; liveness detects sustained running requests without output progress. The published child image and checked receipts preserve native-library identities. Validation: required CI checks pass; bounded GB10 publication probes, installed-module tests, numerical checks, and graph replay evidence are recorded. Full-model soak and cold-cache specialization qualification are not claimed. Fixes #224.
Reject absolute, traversing, non-vLLM, and symlinked override paths before reading package bytes. Read JSON as UTF-8 and write receipts as UTF-8 with LF endings. Cache and compute source-lock identities are unchanged; published images are not rebuilt. Validation: five failing path regressions now pass; 16 compute tests and 353 mesh tests pass with one optional native-bundle skip.
FujitsuPolycom
changed the base branch from
codex/managed-launch-memory-gate
to
main
September 6, 2026 18:41
Owner
Author
|
Status: implemented in SparkRing main through merged #236. The GLM MTP3 compute and stream-ordering changes in this PR are incorporated; this PR is closed as superseded by that integration. The component head The native-MTP3 cache/checkpoint guide selects the integrated image and deployment inputs. Closing this component PR does not broaden the profile's research-only hardware qualification. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Behavior
Status: research-only serving profile with qualified bounded native, GPU stream, serving, restart, and persistent-cache checks for the published image.
The GLM-5.3-Flash-NVFP4-Spark profile uses native MTP3 with an NVFP4/BF16 proposal head and a BF16 target/verifier head. It includes owned deferred weights, independent draft/rejection RNG, shared NVFP4 MoE scales, and the top-k-512 selector. The source lock binds 24 vLLM replacement files, B12X revision
ef308bacwith three selector overrides from PR316, CUDA 13.3 components, and compute-selection settings. Donor identities, licenses, a readable diff, and exact replacement archives are included.RoCEnante orders shared staging across caller streams and preserves one-stream-per-capture admission. SIRCL, the hardware-forwarded ring, patched NCCL fallback, and SparkCache retain their documented routing roles. The cache namespace binds compute lock
139f3670and transport manifest69313e19.The startup-memory gate from #222 is merged into main. This draft targets main and preserves those host controls. Compute verification rejects override paths outside the vLLM package and reads/writes JSON as UTF-8. Published image bytes are unchanged; these verifier-tool changes have offline coverage, not rebuilt-image qualification.
Published image
Config-image ID:
sha256:2e41b1e934a85ff7c21b780532db2f0a0e978df081e52f4ae2bf11f8992fb24f.Anonymous manifest/config verification and Docker pull passed. The pulled config-image ID is the exact image tested on all four Sparks. The README, quickstart, recipe, source pins, content receipt, registry receipt, and compute-equivalence record identify these artifacts consistently.
Validation
run_native.py --mode streamsreproduces these checks in a stopped-model test window.The exact-image record links per-rank and per-cell evidence. All four model containers were healthy at the end of the restore check; rollback artifacts are preserved.
Compatibility and scope
Use the image and source-bound cache namespace together; do not rename entries from another compute composition into it. Reinstall changed host controls while the four-rank model is stopped. Model weights are obtained separately and are not distributed in the image.
Idle rank-loss containment does not establish in-flight stalled-collective containment or resolve upstream RoCEnante issue #313. One persistent recall fixture is not general model-accuracy or all-boundary cache qualification. This change does not enable RoCEnante all-gather, MXFP8 verifier-head quantization, or additional research optimizations.