Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
ad1e255
Use NVFP4 MTP proposal heads in the GLM mesh profile
FujitsuPolycom Sep 5, 2026
a86aa58
Order RoCEnante staging and enforce mesh compute receipts
FujitsuPolycom Sep 5, 2026
322083f
List native MTP3 mesh in the GLM-5.3 profile table
FujitsuPolycom Sep 5, 2026
504c6b0
Describe MTP3 profile behavior and evidence by artifact identity
FujitsuPolycom Sep 5, 2026
3226102
Compose GLM loader ownership, independent RNG and B12X selector sources
FujitsuPolycom Sep 5, 2026
b94e442
Preserve artifact-specific evidence wording in compute composition
FujitsuPolycom Sep 5, 2026
001c48c
Bind GLM mesh profile to the validated compute and stream-safety image
FujitsuPolycom Sep 6, 2026
ec6cecd
Include memory-gated host startup in the validated mesh profile
FujitsuPolycom Sep 6, 2026
a5097d3
Validate DFlash profile content in the shared GLM table
FujitsuPolycom Sep 6, 2026
8e54e22
Enforce histogram publication ordering in the mesh indexer image
FujitsuPolycom Sep 6, 2026
b95030e
Record published mesh barrier image and bounded GPU qualification
FujitsuPolycom Sep 6, 2026
2fec09e
Expose cache resource controls and validate MTP3 prefix reuse
FujitsuPolycom Sep 6, 2026
320e9a5
Use a synthetic short credential in receipt-redaction tests
FujitsuPolycom Sep 6, 2026
4ea84a9
Model root-owned state in the Docker polling tests
FujitsuPolycom Sep 6, 2026
bd84cb9
Merge remote-tracking branch 'origin/main' into codex/pr219-review
FujitsuPolycom Sep 6, 2026
50d46f7
Constrain compute verification paths and encode receipts explicitly
FujitsuPolycom Sep 6, 2026
a0715d0
Distinguish cache reuse and barrier image compositions
FujitsuPolycom Sep 6, 2026
c2a023b
Explain reboot preparation for fragmented Spark memory
FujitsuPolycom Sep 6, 2026
031dc2c
Integrate native MTP3 compute and stream-safety profile
FujitsuPolycom Sep 6, 2026
f9788d8
Integrate cache controls and MTP3 reuse validation
FujitsuPolycom Sep 6, 2026
cbad8ff
Include source-checked MTP3 indexer barrier overlay
FujitsuPolycom Sep 6, 2026
e4d33af
Publish the integrated MTP3 cache and checkpoint profile
FujitsuPolycom Sep 6, 2026
2c18d6b
Preserve semantic CI checks alongside integrated MTP3 profiles
FujitsuPolycom Sep 6, 2026
160b9c5
Bind the LIL mesh descriptor to integrated compute pins
FujitsuPolycom Sep 6, 2026
22c8d48
Check exported MTP3 image against its verified descriptor
FujitsuPolycom Sep 6, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
*.patch text eol=lf
runtime/** text eol=lf
runtime/sparkring/*.gz -text
runtime/glm53-spark-mtp3-mesh/compute/*.tar.gz -text
runtime/glm53-spark-mtp3-mesh/compute/*.patch whitespace=-trailing-space
runtime/exl3/patches/*.patch whitespace=-trailing-space

# Runtime receipts hash these source files byte-for-byte before copying them,
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,12 +29,12 @@

| Model / predictor | Layout | Context | Sequences | Batch | Guide |
|---|---|---:|---:|---:|---|
| **GLM-5.3 Flash NVFP4-Spark · MTP3 mesh** | TP4/DCP4 | 1M | 16 | 8,192 | [Quickstart](docs/GLM53_SPARK_MTP3_MESH_QUICKSTART.md) |
| **GLM-5.3 Flash NVFP4-Spark · MTP3 cache/checkpoint mesh** | TP4/DCP4 | 1M | 16 | 8,192 | [Quickstart](docs/GLM53_MTP3_CACHE_CHECKPOINTS_QUICKSTART.md) |
| GLM-5.3 Flash NVFP4 · BF16 DFlash2 | TP4/DCP4; DCP1/2 | 1M | 16 | 8,192 | [Quickstart](docs/GLM53_JJ_R8_GB10_SPARKCACHE_TP4_QUICKSTART.md) |
| GLM-5.2 EXL3 3.5-bpw | TP4/DCP4 | 1M | 16 | 4,096 | [Quickstart](docs/GLM52_35BPW_QUICKSTART.md) |
| DeepSeek-V4-Flash-0731 | TP4/DCP1 | 1M | 32 | 4,096 | [Quickstart](docs/DEEPSEEK_V4_FLASH_QUICKSTART.md) |
| Qwen3.8-27B EXL3 K5/K6 | TP4/DCP1 | 1M | 64 | 8,192 | [Quickstart](docs/QWEN38_27B_EXL3_K5K6_QUICKSTART.md) |
* The GLM5.3 Flash MTP3 profile uses the new virtual meshing feature.

Check warning on line 37 in README.md

View workflow job for this annotation

GitHub Actions / docs links

Separate the list from preceding content with a blank line.
* Requires:[managed-mesh setup](runtime/glm53-spark-mtp3-mesh/MANAGED_MESH.md).
* DFlash2 profiles will be replaced by native-MTP. Also avoids: [separate CC BY-NC-ND 4.0 terms](https://huggingface.co/incoai/GLM-5.3-Flash-DFlash2#license).

Expand All @@ -45,7 +45,7 @@
| **GLM-5.3 Flash NVFP4-Spark · native MTP3** | TP2/DCP1 | 512K | 8 | 8,192 | [Quickstart](docs/GLM53_FLASH_SPARK_TP2_EXPERIMENTAL_QUICKSTART.md) |
| DeepSeek-V4-Flash-0731 | TP2/DCP1 | 1M | 32 | 4,096 | [Quickstart](docs/DEEPSEEK_V4_FLASH_QUICKSTART.md) |
| Qwen3.8-27B EXL3 K5/K6 | TP2/DCP1 | 1M | 32 | 8,192 | [Quickstart](docs/QWEN38_27B_EXL3_K5K6_PAIR_QUICKSTART.md) |
* GLM5.3 pair is **research-only**, uses 5 GiB KV per rank, and has a

Check warning on line 48 in README.md

View workflow job for this annotation

GitHub Actions / docs links

Separate the list from preceding content with a blank line.
[known video-color issue](https://github.com/FujitsuPolycom/sparkring/issues/229). In testing.
See the [profile index](docs/profiles/README.md) for evidence scopes and
[SparkCache compositions](recipes/sparkcache/README.md) for persistent-cache
Expand Down
31 changes: 28 additions & 3 deletions THIRD_PARTY_NOTICES.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,17 @@ two compatibility patches. The public GLM-5.3 builder does not consume them;
its `NCCL_SWITCHLESS_RING_ONLY` parameter and diagnostics are an independent
SparkRing implementation.

## 3. vLLM (referenced and patched)
## 3. vLLM (referenced, patched, and selected source included)

`runtime/glm53-spark-mtp3-mesh/compute/vllm-compute-files.tar.gz` includes
24 vLLM Python files for GLM metadata reuse, dense-kernel integration,
the NVFP4 proposal head, deferred-weight ownership, and independent draft
and rejection-sampling randomness. The adjacent patch provides a readable diff;
`source-lock.json` records the base, donor revisions, and exact file hashes.
These files derive from Local Inference Lab's vLLM fork at `3512b066` and
`a8c796f3`, loader correction `17e341b9`, and independent-RNG backport
`44e6766e` from [PR 653](https://github.com/local-inference-lab/vllm/pull/653),
under Apache-2.0 with their contributor notices retained.

The unified diffs under `runtime/deepseek0731-gb10/patches/` contain context
and removed lines from vLLM, pinned to the source revision recorded by that
Expand Down Expand Up @@ -245,8 +255,10 @@ artifacts that are not distributed in this repository:
repository notices.

SparkRing records these identities and validates compatible image content; it
does not redistribute the model weights, vLLM source, or B12X model-kernel
package. The selected B12X communication source in Section 11 is included. Operators
does not include model weights or the complete vLLM/B12X source trees in this
Git repository. The selected vLLM files in Section 3 and B12X communication
source in Section 11 are included. Published runtime images contain the
pinned vLLM and B12X packages under their respective licenses. Operators
must obtain each artifact under its own terms. The exact operator-image
composition is in `runtime/glm53-flash-jj-r8-gb10/pins.json` and
`runtime/glm53-flash-jj-r8-gb10/glm53-dcp4-sircl-public-image-receipt.json`.
Expand Down Expand Up @@ -287,3 +299,16 @@ The native-MTP3 profile references
Operators must obtain and use that checkpoint under its own license and
notices. Native MTP uses its included prediction layer and does not require
an external DFlash checkpoint.

## 12. B12X selector source and MoE scale sharing

The native-MTP3 compute package downloads B12X revision
`ef308bac0f3b3eb8fea63e4013afc0c2ea1c6301`, including its shared native NVFP4
scales for A4/A16 MoE paths. Three selector Python files are included in
`runtime/glm53-spark-mtp3-mesh/compute/b12x-selector-files.tar.gz` from
[B12X PR 316](https://github.com/local-inference-lab/b12x/pull/316), revision
`9ac142824b4edb750892a0fb63d914230086495d`. They implement the top-k-512
candidate buffer, exact overflow handling, and omission of unused terminal
scores. These files are licensed under Apache-2.0; source notices are retained
and the downloaded B12X archive supplies the license. The compute source lock
binds the source archive and each base/result file hash.
27 changes: 23 additions & 4 deletions docs/GLM53_JJ_R8_GB10_SPARKCACHE_TP4_QUICKSTART.md
Original file line number Diff line number Diff line change
Expand Up @@ -305,6 +305,11 @@ SPARKCACHE_ACCESS_MODE=read-write # restore existing entries and publish new o
SPARKCACHE_ACCESS_MODE=restore-only # restore existing entries; never capture new prompts
```

Keep `SPARKCACHE_ASYNC_PAGE_CAPTURE=auto` from the environment template to
disable capture automatically in restore-only mode or when SparkCache is off.
An explicit `1` with either setting is rejected. Explicit `0` in read-write
mode selects synchronous publication.

The GLM-5.3 profile retains a verified shared GPU prefix for up to five
minutes so one restore can serve an extended request queue:

Expand Down Expand Up @@ -360,20 +365,34 @@ and `IMAGE_ID`; keep the page-tail settings unchanged.
The recommended DCP4 profile enables bounded asynchronous page capture:

```bash
SPARKCACHE_ASYNC_PAGE_CAPTURE=1
SPARKCACHE_ASYNC_PAGE_CAPTURE=auto
SPARKCACHE_ASYNC_CAPTURE_SLOT_BYTES=auto
SPARKCACHE_ASYNC_CAPTURE_SLOT_COUNT=2
```

The `auto` slot policy selects 8 GiB for DCP1, 5 GiB for DCP2, or 3 GiB for
DCP4. Two capture slots let the background publisher consume one completed
capture while a later capture uses the other. Restore separately overlaps
bounded NVMe reads and CUDA placement through two 256 MiB mapped arenas. More
restore arenas are not part of this profile because measured arena waits did
not justify the additional unified-memory pressure. DCP1 and DCP2 page-tail
bounded NVMe reads and CUDA placement through two 256 MiB mapped arenas per
load lane. Eight lanes reserve 4 GiB per rank; adding the DCP4 capture slots
gives 10 GiB per rank (40 GiB across TP4) of configured payload capacity,
before KV, models, metadata, retained objects and transport. Restore-only
omits the 6 GiB capture ring. DCP1 and DCP2 page-tail
capture have no matching live record; use complete snapshots or test those
layouts separately.

Print the resolved memory plan without host or GPU access before launching:

```bash
SPARKRING_PRINT_MEMORY_PLAN=1 bash runtime/glm53-flash-jj-r8-gb10/launch-rank.sh \
0 /path/to/rank.env
```

Set `SPARKCACHE_BUFFER_BUDGET_BYTES` in that configuration to enforce a
per-rank restore-plus-capture payload ceiling; zero disables it. The report
shows KV separately and names excluded memory. It does not predict total
resident memory or replace a hardware capacity test.

The environment template enables `DFLASH_WARMUP=1`. Rank 0 waits for the API,
then exercises every concurrency from C1 through C16 and scheduled prompt
spans covering DFlash's Triton block-size specializations. DFlash depth seven
Expand Down
98 changes: 98 additions & 0 deletions docs/GLM53_MTP3_CACHE_CHECKPOINTS_QUICKSTART.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
# GLM-5.3 native MTP3 with verified caching and recurrent checkpoints

Status: **research-only**. The image build and source checks are implemented.
The published image's vLLM, B12X, transport, and warmup files match the deployed
composition in 5,308 byte comparisons. It incorporates SparkCache main at the
revision below. Source equivalence is not a serving soak of this rebuilt image.

## Prerequisites

Use four NVIDIA Sparks with the [managed mesh host setup](GLM53_SPARK_MESH_HOST_SETUP.md).
The model uses TP4/DCP4 and native MTP depth three. Keep the reviewed fabric,
driver, GID and interface settings; this guide does not reconfigure NICs.
Follow [host prerequisites](PREREQUISITES.md#four-spark-managed-hardware-forwarded-mesh),
including optional reboot preparation after large GPU workloads. Stop model
workloads through the managed lifecycle before replacing containers or services.

Download the target `local-inference-lab/GLM-5.3-Flash-NVFP4-Spark` at revision
`df116c4fb16b1d37ae43d2cfd624de26ffbc832e`. No external draft is required.

## Pull and verify the image

Use the same immutable image on all four ranks:

```bash
image='ghcr.io/fujitsupolycom/sparkring-glm53-sparkcache@sha256:11a556a54041fd823d152a7f051ac4f7c617dc539030df26e93008392fee0746'
image_id='sha256:6921a6c163ea40b603e19a0332330efe3dbccbf4dce9f6cbbf6b756c9231835a'
docker pull "$image"
test "$(docker image inspect "$image" --format '{{.Id}}')" = "$image_id"
docker run --rm --network none --entrypoint python3 "$image" \
-S -B /opt/sparkring/bin/verify-performance.py
```

The [public image contract](../runtime/glm53-spark-mtp3-mesh/performance/public-image.json)
is the renderer and installer input for this composition. Do not substitute the
compute parent's `image-receipt.json`. Source builds use
[the performance build instructions](../runtime/glm53-spark-mtp3-mesh/performance/README.md).

## Extract and render

Choose unused artifact and export-container names. The following commands do
not start a model. Create `/srv/sparkring/artifacts` with operator permissions:

```bash
docker create --name mtp3-artifact-export "$image"
docker cp mtp3-artifact-export:/opt/spark-sircl /srv/sparkring/artifacts/cache-checkpoints-bundle
docker cp mtp3-artifact-export:/opt/sparkring/bin/mlx5-rdma-tx-marker /srv/sparkring/artifacts/mlx5-rdma-tx-marker
docker rm mtp3-artifact-export
cp runtime/glm53-spark-mtp3-mesh/performance/public-image.json /srv/sparkring/verified-image-receipt.json
```

Edit the private site and fabric inputs described in the
[managed deployment guide](../runtime/glm53-spark-mtp3-mesh/MANAGED_MESH.md).
Set the site's bundle root to the extracted bundle and its marker path to the
extracted marker. The marker SHA-256 is
`2828c07e4255c4962c77425be2c88969e7eb7dd4b1bf9e36485bc705bb5d6d64`.
Use a distinct container prefix and cache directory; do not rename entries from
another cache namespace into this one.

```bash
python runtime/glm53-spark-mtp3-mesh/profile.py render \
--site /srv/sparkring/site/mtp3-mesh.json \
--bundle /srv/sparkring/artifacts/cache-checkpoints-bundle \
--image-receipt /srv/sparkring/verified-image-receipt.json \
--output build/mtp3-cache-checkpoints-launch
```

The renderer selects the matching image, transport manifest, optimized native
placement digest, and cache namespace. Review the generated rank environments
and plan, then follow the managed guide's create-only container, installation,
and coordinated startup steps with this launch directory and verified receipt.
Do not use direct `docker start` to bypass the memory and four-rank gates.

## Runtime behavior and limits

SparkCache revision `48bbd2be4a7b972e56632a2d7b934bac5460f272` provides bounded
restoration, tiled CUDA placement, authenticated history reconstruction,
publication dependency protection, and publication-backlog gauges. Periodic
full captures remain off by default. They trade more writes for shorter
history processing and require a separately selected workload configuration.

The profile retains 24 GiB KV capacity per rank, an 8,192-token batch budget,
40 GiB persistent-cache maximum and 32 GiB low watermark. Those capacity
defaults are not evidence of a 40 GiB multimodal stress test. The recorded
cache-pressure soak used 2 GiB per rank and an opt-in full-capture cadence.

The image includes explicit two-checkpoint prefill, complete convolution-state
exports, stream-ordered transport, CTA publication synchronization, delayed
proxy-thread startup, and linked payload/doorbell posts. Checkpoint source and
ownership hashes are preserved by the build rather than inferred from labels.

GLM-5.3 supports `reasoning_effort=low`, `high`, or `max`. Explicit thinking-off
chat requests return HTTP 400; warmup uses supported reasoning settings. This
does not implement reasoning-free generation. Responses API behavior is not
qualified by the chat-completion checks.

Run bounded semantic, repeated-prefix, multimodal, and idle-probe checks after
deployment. Neither source equivalence nor the parent's measurements qualify
unattended availability, every cache boundary, or a universal speedup.
Loading
Loading