Skip to content

Add exact GLM-5.3 adaptive MTP and DFlash7 SparkCache runtimes - #143

Closed
FujitsuPolycom wants to merge 1 commit into
mainfrom
codex/qualified-glm53-runtime
Closed

Add exact GLM-5.3 adaptive MTP and DFlash7 SparkCache runtimes#143
FujitsuPolycom wants to merge 1 commit into
mainfrom
codex/qualified-glm53-runtime

Conversation

@FujitsuPolycom

@FujitsuPolycom FujitsuPolycom commented Aug 30, 2026

Copy link
Copy Markdown
Owner

Resulting behavior

This PR adds two reproducible ARM64 GLM-5.3 runtime compositions that use the same exact vLLM Python source, B12X kernels, fast tensor loading, and SparkCache integration:

  • Adaptive embedded MTP5, intended for controlled evaluation of acceptance-based draft depth.
  • External DFlash7, the qualified TP4/DCP1 serving configuration.

Each builder verifies the retained compiled vLLM and CUDA components, the overlaid Python source, B12X source, SparkCache source, patch set, generated artifacts, and image labels before producing an image. Executable four-rank profiles and operator quickstarts cover ordinary safetensors and fastsafetensors loading. Runtime messages and configuration use SparkCache CUDA restore and SparkCache CUDA placement terminology.

The DFlash7 profile removes unused DeepEP, disables unsupported symmetric-memory and FlashInfer all-reduce probes, disables all-reduce RMS fusion, and selects language-model-only serving. ModelOpt and FP8 KV warnings remain visible because they describe real runtime limitations.

Status

  • DFlash7 runtime: qualified for the exact GLM-5.3 TP4/DCP1 image load, semantic serving, 131,072-token persistent restore, and 262,144-token correctness replay. C2/C8/C16 results belong to an earlier SparkCache source artifact and do not qualify this image.
  • Adaptive MTP runtime: implemented and GPU-free tested. Live generation has been observed, but its performance and persistent-cache behavior remain research-only.
  • Public OCI publication: unsupported by this PR. The qualified image exists only on the recorded deployment hosts.

Compatibility

The DFlash7 profile binds SparkCache commit 5ec6a9953ad5d39120298bbfc26e95a6fa4b1dc3, tree 94c236b9dfbf5f70075eb47877fd9caaa5d8c249, and source SHA-256 bc238f96e550c7ec27d4081dd1f2e741d404aaf5c8572d89ccc5e76812be4d63. That commit is the head of SparkCache consolidation draft PR #32.

The runtime compositions use distinct served-model names, cache roots, JIT roots, and runtime identities. Existing deployment profiles remain available. Stored state is accepted only when the complete model, topology, vLLM, B12X, SparkCache, KV layout, and runtime-option identity matches; other entries are rejected and recomputed.

Validation

  • Complete repository suite: 1,979 passed, 9 skipped
  • Maintained-tree suite: 1,945 passed, 9 skipped
  • Ruff: passed
  • Exact DFlash7 image loaded the target checkpoint in 53.93 seconds and the draft checkpoint in 4.85 seconds; the four-rank endpoint became ready in about 146 seconds
  • The exact final image restored a committed 131,072-token context correctly at C1; bounded-concurrency qualification for that image remains outstanding
  • SparkCache restored a committed 262,144-token context correctly; its 7.835-second latency identifies physical delta-object layout as research work outside this PR

This PR consolidates the implemented runtime surface previously presented across draft PRs #129, #130, #131, #134, #136, #137, #139, #140, and #141. Direct-fabric image distribution, runtime evidence, adaptive-versus-DFlash diagnostics, and recurrent-boundary metadata remain independent review units.

@FujitsuPolycom

Copy link
Copy Markdown
Owner Author

The resulting runtime, operator contract, and evidence are consolidated in retained draft stack #146#147#150. Independent page-base research remains in #149. Closing this superseded draft and deleting only its remote head branch.

@FujitsuPolycom
FujitsuPolycom deleted the codex/qualified-glm53-runtime branch August 31, 2026 01:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant