Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ jobs:
--index-url https://download.pytorch.org/whl/cpu \
"torch==${TORCH_VERSION}"
- name: Test maintained Python trees
run: python -m pytest spark_transport runtime/exl3-r7 runtime/glm53-flash runtime/deepseek0731-gb10 runtime/qwen38 runtime/test_public_overlay.py performance/harnesses scripts -q -rs
run: python -m pytest spark_transport runtime/exl3-r7 runtime/glm53-flash runtime/glm53-flash-b12x-kda-adaptive-mtp runtime/deepseek0731-gb10 runtime/qwen38 runtime/test_public_overlay.py performance/harnesses scripts -q -rs

docs-links:
name: docs links
Expand Down
2 changes: 1 addition & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,7 @@ test imports torch:
python -m pip install -r requirements-dev.txt
python -m pip install --index-url https://download.pytorch.org/whl/cpu "torch==2.11.0"
ruff check --select E,F,W --ignore E501 spark_transport runtime scripts performance
python -m pytest spark_transport runtime/exl3-r7 runtime/glm53-flash runtime/deepseek0731-gb10 runtime/qwen38 runtime/test_public_overlay.py performance/harnesses scripts -q -rs
python -m pytest spark_transport runtime/exl3-r7 runtime/glm53-flash runtime/glm53-flash-b12x-kda-adaptive-mtp runtime/deepseek0731-gb10 runtime/qwen38 runtime/test_public_overlay.py performance/harnesses scripts -q -rs
```

The test suite is CPU-only contract coverage. It does not validate CUDA,
Expand Down
39 changes: 26 additions & 13 deletions docs/GLM53_B12X_KDA_ADAPTIVE_MTP_SPARKCACHE_TP4_QUICKSTART.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,17 @@
# Serve GLM-5.3 with adaptive MTP, live-tensor B12X KDA, and SparkCache

Status: **implemented, not qualified**. This guide builds vLLM commit
`0b67266a0f37d6146a8403fb8482403c62f412d5` and the SparkCache overlay from
commit `20838ace3ebda570ca039cb7f1976c29da554b39` for four DGX Spark systems at
TP4/DCP1.
`0b67266a0f37d6146a8403fb8482403c62f412d5`, B12X commit
`b1d541f9e71a35f030d45fae437630fff7507c2a` with Git tree
`c69cdec1c59a08e8e0e549f930fa8abcfb5134ae`, and the SparkCache overlay
from commit `20838ace3ebda570ca039cb7f1976c29da554b39` for four DGX Spark systems
at TP4/DCP1.

The B12X implementation accepts `kda_metadata_validation="trusted"` and binds
each request's live projection, metadata, and output tensors at their actual
capacities. B12X commit `2fcf23a0ce269be27b2e03fece73d46e90e6aeea`
is **unsupported** with this vLLM revision because it lacks both parts of that
interface.

The serving profile uses embedded MTP with maximum depth five, initial depth
three, and a 32-step acceptance window. Fastsafetensors uses queue size one.
Expand Down Expand Up @@ -55,9 +63,11 @@ test "${#native_sha256}" -eq 64
```

The runtime builder verifies the complete first-parent vLLM history from
`da4d7be` through adaptive MTP and the three live-tensor B12X KDA commits. The
SparkCache build verifies LF Linux preimages, four exact patches, and eleven
postimage source files.
`da4d7be` through adaptive MTP and the three vLLM live-tensor KDA commits. It
also verifies the B12X first-parent history from `2fcf23a` through `b1d541f9`,
the exact KDA source hashes, trusted metadata selection, and request-sized
live-tensor binding. The SparkCache build verifies LF Linux preimages, four
exact patches, and eleven postimage source files.

## Resolve the TP4 profile

Expand Down Expand Up @@ -125,10 +135,13 @@ NCCL error, or traceback.

The overlay does not change SparkCache wire fields, digest salts, 256-token
geometry, or stored object schemas. Its embedded-MTP digest is SHA-256 over
`glm53-embedded-mtp-runtime-v1`, the target identity, the full vLLM commit,
maximum depth five, and `adaptive:3:32`, separated by zero bytes.

Including the vLLM revision gives this runtime a distinct draft-state cache
identity from the e105 adaptive-MTP profile. Stored entries therefore
recompute instead of crossing the KDA source boundary without byte-equivalence
evidence.
`glm53-embedded-mtp-vllm-b12x-runtime-v1`, the target identity, the full vLLM
commit, the full B12X commit, maximum depth five, and `adaptive:3:32`, separated
by zero bytes. The resulting digest is
`2761488b43742f849e2cb7ac4977dc28d771faf5afef1951044327948d6ad71e`.

Including both implementation revisions gives this runtime a distinct
draft-state cache identity from the e105 adaptive-MTP profile and from any
0b67266 runtime paired with another B12X source. Stored entries therefore miss
and recompute instead of crossing a KDA implementation boundary without
byte-equivalence evidence.
2 changes: 1 addition & 1 deletion runtime/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ allowlist and validating the generated manifest through the builder.
The builder test suites cover the GLM, DeepSeek, and Qwen runtime contracts:

```bash
python -m pytest runtime/exl3-r7 runtime/glm53-flash runtime/deepseek0731-gb10 runtime/qwen38 -q
python -m pytest runtime/exl3-r7 runtime/glm53-flash runtime/glm53-flash-b12x-kda-adaptive-mtp runtime/deepseek0731-gb10 runtime/qwen38 -q
```

[`runtime/glm53-flash/`](glm53-flash/README.md) builds the source-pinned
Expand Down
8 changes: 7 additions & 1 deletion runtime/glm53-flash-b12x-kda-adaptive-mtp/Containerfile
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,10 @@ ARG ARM_BUILDER

FROM ${ARM_BUILDER} AS b12x-wheel
ARG B12X_COMMIT
ARG B12X_TREE
COPY bundle/sources/b12x /build/b12x
RUN test "$(git -C /build/b12x rev-parse HEAD)" = "${B12X_COMMIT}" \
&& test "$(git -C /build/b12x rev-parse 'HEAD^{tree}')" = "${B12X_TREE}" \
&& /opt/python/cp312-cp312/bin/python -m pip wheel \
--no-deps --wheel-dir /out/wheels /build/b12x

Expand All @@ -27,6 +29,8 @@ RUN test "$(git -C /build/nccl rev-parse HEAD)" = "${NCCL_COMMIT}" \
FROM ${VLLM_BASE}
ARG VLLM_COMMIT
ARG B12X_COMMIT
ARG B12X_TREE
ARG B12X_KDA_CONTRACT
ARG NCCL_COMMIT
ARG NCCL_PATCHED_TREE
ARG NCCL_PATCH_SHA256
Expand All @@ -35,7 +39,7 @@ ARG SPARKRING_REVISION

COPY --from=b12x-wheel /out/wheels/ /opt/sparkring/wheelhouse/
RUN uv pip install --system --no-deps /opt/sparkring/wheelhouse/b12x-*.whl \
&& python3 -c "import b12x; from vllm.utils.b12x import get_b12x_sparse_mla, get_b12x_dsa_indexer, get_b12x_gdn_decode, get_b12x_mhc; assert get_b12x_sparse_mla(); assert get_b12x_dsa_indexer(); assert get_b12x_gdn_decode(); assert get_b12x_mhc()"
&& python3 -c "import b12x, inspect; from vllm.utils.b12x import get_b12x_sparse_mla, get_b12x_dsa_indexer, get_b12x_gdn_decode, get_b12x_mhc; gdn = get_b12x_gdn_decode(); assert get_b12x_sparse_mla(); assert get_b12x_dsa_indexer(); assert gdn; assert get_b12x_mhc(); assert gdn.Caps.__dataclass_fields__['kda_metadata_validation'].default == 'transactional'; bind_source = inspect.getsource(gdn.bind_kda); run_source = inspect.getsource(gdn.run_kda); assert 'mixed_qkv.shape[0]' in bind_source; assert 'state_indices.shape[0]' in bind_source; assert 'caps.kda_metadata_validation == \"transactional\"' in run_source"

COPY --from=nccl-build /out/nccl/ /opt/sparkring/nccl/
RUN cd /opt/sparkring/nccl \
Expand Down Expand Up @@ -77,6 +81,8 @@ LABEL org.opencontainers.image.title="SparkRing GLM-5.3 Flash ARM64 runtime" \
org.jovian.architecture="linux-arm64-sm121" \
org.jovian.vllm.commit="${VLLM_COMMIT}" \
org.jovian.b12x.commit="${B12X_COMMIT}" \
org.jovian.b12x.tree="${B12X_TREE}" \
org.jovian.b12x.kda-contract="${B12X_KDA_CONTRACT}" \
org.jovian.transport="sparkring-nccl-2.30.7-source-built" \
org.sparkring.nccl.commit="${NCCL_COMMIT}" \
org.sparkring.nccl.patched-tree="${NCCL_PATCHED_TREE}" \
Expand Down
32 changes: 22 additions & 10 deletions runtime/glm53-flash-b12x-kda-adaptive-mtp/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,16 +2,26 @@

Status: **implemented**. The builder pins
`local-inference-lab/vllm@0b67266a0f37d6146a8403fb8482403c62f412d5`
and Git tree `ba9484ccb33aa56e90ff2f447f15ca9b9da97639`. Four-rank
TP4/DCP1 serving remains **unqualified** until an immutable image digest has a
live qualification receipt.
at Git tree `ba9484ccb33aa56e90ff2f447f15ca9b9da97639` and
`local-inference-lab/b12x@b1d541f9e71a35f030d45fae437630fff7507c2a`
at Git tree `c69cdec1c59a08e8e0e549f930fa8abcfb5134ae`. Four-rank TP4/DCP1
serving remains **unqualified** until an immutable image digest has a live
qualification receipt.

The pinned vLLM history contains the complete GLM-5.3 source runtime from
`da4d7be6c97434f6942292ed8abbf4b32dc44355` through acceptance-based
adaptive MTP at `e10536aadf02a18fccddda7ec939c33147e8b0b3`, followed by three
commits that bind B12X KDA metadata once and operate on live layer tensors.
`pins.json` records and verifies the exact first-parent sequence.

The B12X pin accepts `kda_metadata_validation="trusted"` and derives tensor
capacity from each live request's projection and recurrent-state metadata.
B12X commit `2fcf23a0ce269be27b2e03fece73d46e90e6aeea` is **unsupported** for this
runtime: its `Caps` interface has no trusted-metadata selection, and its
`bind_kda` interface requires plan-sized tensors. The source preparer rejects
that byte-exact implementation and any source that lacks the required call
contract.

The runtime also pins B12X, InstantTensor, CUDA, and SparkRing's source-built
NCCL transport. The fastsafetensors TP4 profile uses loader queue size one.
The vLLM implementation selects `nogds=True` when tensor parallelism exceeds
Expand All @@ -26,15 +36,17 @@ BUILD_RECEIPT="$PWD/glm53-b12x-kda-adaptive-mtp-runtime-receipt.json" \
bash runtime/glm53-flash-b12x-kda-adaptive-mtp/build-image.sh
```

The builder verifies commits, Git trees, the complete vLLM lineage, source
licenses, patched NCCL bytes, output labels, and required Python imports. Its
receipt proves image construction only. Startup, semantic generation,
SparkCache restore, shared-prefix concurrency, and fatal-log checks require a
separate four-rank receipt.
The builder verifies commits, Git trees, the complete vLLM and B12X lineages,
the trusted-metadata request-sized KDA call contract, source licenses, patched
NCCL bytes, output labels, and required Python imports. Its receipt proves
image construction only. Startup, semantic generation, SparkCache restore,
shared-prefix concurrency, and fatal-log checks require a separate four-rank
receipt.

The matching SparkCache overlay is pinned to
`FujitsuPolycom/sparkcache@20838ace3ebda570ca039cb7f1976c29da554b39`.
Its Linux-byte-exact vLLM contract is
`vllm-kv-block-lease-contract-glm53-b12x-kda-adaptive-mtp.json`. The
runtime-bound embedded-MTP identity prevents this profile from reusing e105
adaptive-MTP entries until byte-equivalence across the KDA revisions is proven.
embedded-MTP identity binds both the vLLM and B12X revisions. Entries produced
by another KDA implementation therefore miss and recompute instead of crossing
an unproven numerical boundary.
4 changes: 4 additions & 0 deletions runtime/glm53-flash-b12x-kda-adaptive-mtp/build-image.sh
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,8 @@ arm_builder="$(read_pin public_image_build.base_images.arm_builder.reference)"
cuda_runtime="$(read_pin public_image_build.base_images.cuda_runtime.reference)"
vllm_commit="$(read_pin public_image_build.sources.vllm.commit)"
b12x_commit="$(read_pin public_image_build.sources.b12x.commit)"
b12x_tree="$(read_pin public_image_build.sources.b12x.tree)"
b12x_kda_contract="$(read_pin public_image_build.vllm_b12x_kda.schema)"
nccl_commit="$(read_pin public_image_build.sources.nccl.commit)"
nccl_patched_tree="$(read_pin public_image_build.sources.nccl.patched_tree)"
nccl_patch_sha256="$(read_pin public_image_build.sources.nccl.patches.0.sha256)"
Expand Down Expand Up @@ -106,6 +108,8 @@ vllm_image="sparkring-glm53-vllm:${vllm_commit:0:12}-arm64"
--build-arg "ARM_BUILDER=${arm_builder}" \
--build-arg "VLLM_COMMIT=${vllm_commit}" \
--build-arg "B12X_COMMIT=${b12x_commit}" \
--build-arg "B12X_TREE=${b12x_tree}" \
--build-arg "B12X_KDA_CONTRACT=${b12x_kda_contract}" \
--build-arg "NCCL_COMMIT=${nccl_commit}" \
--build-arg "NCCL_PATCHED_TREE=${nccl_patched_tree}" \
--build-arg "NCCL_PATCH_SHA256=${nccl_patch_sha256}" \
Expand Down
48 changes: 46 additions & 2 deletions runtime/glm53-flash-b12x-kda-adaptive-mtp/pins.json
Original file line number Diff line number Diff line change
Expand Up @@ -49,8 +49,29 @@
},
"b12x": {
"repository": "https://github.com/local-inference-lab/b12x.git",
"commit": "2fcf23a0ce269be27b2e03fece73d46e90e6aeea",
"tree": "58a046fc8faa747346f40f87166cda7e0f67ff47",
"commit": "b1d541f9e71a35f030d45fae437630fff7507c2a",
"tree": "c69cdec1c59a08e8e0e549f930fa8abcfb5134ae",
"source_lineage": {
"base_commit": "2fcf23a0ce269be27b2e03fece73d46e90e6aeea",
"included_commits": [
"f1ddd12fe51667ade19a6fb6b460e338f0e94461",
"d56c1163b6e019d828ed24f135c2efd05fdca6ea",
"f2fd5113db6344a117074424e5461329ba33704f",
"11814a27cabc3501f3ab8d20f159a80f6d29d0af",
"ff9abf8a08fd4af0d3f7b3bab69d500e924bab2e",
"e8905c1afaedb1b905dc8dd80537dbc25d35453b",
"5c20a7f3190f92fabd290ddbd2a77390fe2b0f8f",
"501ffb9a9b118009b8ea2ea20b8d43f57ba4f221",
"98149b404f59498324d3193ccbd1a49aa8df21a8",
"cd9c3431f760f965fcf5f268b6a8d6ef1de9b6db",
"80d68e5702e7c9e99ca37988f87c964021d7e1b4",
"4e65ed4d7a57b708bc213580c19a4e9cf0a9ca82",
"12aea7d96928f540a64259e4e24ef7688093b515",
"f1a37c87a1b81cfe7b302b0a8c148ca4302ca8dc",
"00f7d4be8c080e6faf359c089d1439839565b87e",
"b1d541f9e71a35f030d45fae437630fff7507c2a"
]
},
"license": "Apache-2.0"
},
"nccl": {
Expand All @@ -69,6 +90,29 @@
]
}
},
"vllm_b12x_kda": {
"schema": "sparkring-vllm-b12x-live-tensor-kda/v1",
"status": "implemented",
"metadata_validation": "trusted",
"tensor_binding": "request-sized-live-tensors",
"vllm_source": {
"path": "vllm/model_executor/layers/mamba/gdn/kimi_gdn_linear_attn.py",
"sha256": "8bf8bc579dd4a80224dc1633e7513f2a0c58e07db72a736c7e41d28d3c35f3b9"
},
"b12x_source": {
"path": "b12x/sequence/gdn_decode/_impl.py",
"sha256": "a2c81a486eb4d86a59f39e4f03855382583d39cdadd5fc8fd1c685ba28b1b56e"
},
"unsupported_b12x_sources": [
{
"status": "unsupported",
"commit": "2fcf23a0ce269be27b2e03fece73d46e90e6aeea",
"tree": "58a046fc8faa747346f40f87166cda7e0f67ff47",
"source_sha256": "b1f072405ad2f3bba83e720419a59ae791e1269dfccc9fc4a279889f8bd07d6e",
"limitation": "Caps does not accept kda_metadata_validation, and bind_kda requires plan-sized tensors instead of request-sized live tensors."
}
]
},
"instanttensor": {
"version": "0.1.9",
"sdist_url": "https://files.pythonhosted.org/packages/37/69/a4dc4e0f0018a0e558b716e90bcb19cb6a8c7506f68e89a1493608cf5e62/instanttensor-0.1.9.tar.gz",
Expand Down
Loading
Loading