Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,8 +65,9 @@ See the [profile registry](docs/profiles/README.md) for recipe identities and ev
### GLM-5.3 Flash research observation

**Research-only — 16K context, single observation.** The SparkCache-enabled
profile recorded 2,371 tok/s prefill and 36.06 tok/s sustained
C1 decode on random tokens. No A/B baseline has been completed.
profile recorded 2,371 tok/s prefill and 36.06 tok/s sustained C1 decode on
random tokens. No A/B baseline has been completed. C4 and C8 were capacity-limited
and are omitted rather than reported as throughput results.

| Profile | Prefill | C1 decode | C8 decode | Highest valid decode | Coding peak |
|---|---:|---:|---:|---:|---:|
Expand Down
8 changes: 4 additions & 4 deletions docs/GLM53_B12X_KDA_ADAPTIVE_MTP_SPARKCACHE_TP4_QUICKSTART.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ three, and a 32-step acceptance window. Fastsafetensors uses queue size one.
TP4 makes the pinned vLLM loader select `nogds=True`, so model loading uses
pipelined host I/O without GPU Direct Storage.

The profile reserves 20 GiB of FP8 KV per rank and enables SparkCache native
The profile reserves 20 GiB of FP8 KV per rank and enables SparkCache CUDA
restore, tail-only publication, shared restore trunks, and bounded shared GPU
prefix leases. Image construction and distribution do not require stopping an
existing service. Do not run the launch command until all four ranks have the
Expand Down Expand Up @@ -49,10 +49,10 @@ Record immutable local identities:
```bash
sparkcache_image='sparkring-glm53-sparkcache:b12x-kda-adaptive-mtp-0b67266a-arm64'
sparkcache_image_id="$(docker image inspect --format '{{.Id}}' "${sparkcache_image}")"
native_sha256="$(docker run --rm --entrypoint sha256sum "${sparkcache_image}" \
cuda_placement_sha256="$(docker run --rm --entrypoint sha256sum "${sparkcache_image}" \
/opt/sparkcache-src/sparkcache/native/build-cuda/libspark_cache_placement.so \
| cut -d ' ' -f1)"
test "${#native_sha256}" -eq 64
test "${#cuda_placement_sha256}" -eq 64
```

The runtime builder verifies the complete first-parent vLLM history from
Expand All @@ -72,7 +72,7 @@ python sparkring/scripts/prepare_glm53_b12x_kda_adaptive_mtp_profile.py \
--image-id "${sparkcache_image_id}" \
--parent-image "${runtime_image}" \
--parent-image-id "${runtime_image_id}" \
--native-library-sha256 "${native_sha256}" \
--cuda-placement-library-sha256 "${cuda_placement_sha256}" \
--profile-output profile.json \
--site-output site.yaml
```
Expand Down
10 changes: 5 additions & 5 deletions docs/GLM53_E10536A_SPARKCACHE_TP4_QUICKSTART.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,8 @@ pipelined shard loading without GPU Direct Storage. The queue can retain one
additional shard-sized device buffer during model loading.

All profiles use TP4/DCP1, 20 GiB FP8 KV per rank, a 524,288-token request
limit, 32 sequences, native direct restore, two restore lanes, eight native
I/O workers, two 256 MiB arenas, shared GPU-prefix leases, and one-shot cache
limit, 32 sequences, SparkCache CUDA restore, two restore lanes, eight CUDA
restore I/O workers, two 256 MiB placement arenas, shared GPU-prefix leases, and one-shot cache
clearing.

## Build immutable inputs
Expand Down Expand Up @@ -57,10 +57,10 @@ Record the exact output identities:
```bash
sparkcache_image='sparkring-glm53-sparkcache:e10536a-source-arm64'
sparkcache_image_id="$(docker image inspect --format '{{.Id}}' "${sparkcache_image}")"
native_sha256="$(docker run --rm --entrypoint sha256sum "${sparkcache_image}" \
cuda_placement_sha256="$(docker run --rm --entrypoint sha256sum "${sparkcache_image}" \
/opt/sparkcache-src/sparkcache/native/build-cuda/libspark_cache_placement.so \
| cut -d ' ' -f1)"
test "${#native_sha256}" -eq 64
test "${#cuda_placement_sha256}" -eq 64
```

## Resolve a serving profile
Expand Down Expand Up @@ -88,7 +88,7 @@ python sparkring/scripts/prepare_glm53_e105_profile.py \
--image-id "${sparkcache_image_id}" \
--parent-image "${runtime_image}" \
--parent-image-id "${runtime_image_id}" \
--native-library-sha256 "${native_sha256}" \
--cuda-placement-library-sha256 "${cuda_placement_sha256}" \
--profile-output profile.json \
--site-output site.yaml
```
Expand Down
2 changes: 1 addition & 1 deletion recipes/sparkcache/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@ Limitations: The published receipts use a 4,096-token scheduler budget; other
budgets are operator choices whose performance and capacity behavior is not
recorded here, except that the GLM-5.3 Flash receipt uses 8,192. The DeepSeek
receipts cover DCP1, not DCP2 or DCP4. These
recipes disable streaming snapshots and native restore, so the receipts do not
recipes disable streaming snapshots and SparkCache CUDA restore, so the receipts do not
cover either mode. Other images, checkpoints, and cache geometries are also
outside the recorded evidence. The GLM-5.3 Flash receipt covers an 8,192-token
restored span and does not establish throughput neutrality or larger-span
Expand Down
4 changes: 2 additions & 2 deletions recipes/sparkcache/deepseek-v4-flash-0731-tp2-dcp1.json
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@
"min_span_tokens": 256,
"max_span_tokens": 1048576,
"streaming_snapshots": false,
"native_restore": false,
"cuda_restore": false,
"root_requirement": "Use a dedicated rank-local host directory mounted at the same container path on both ranks."
},
"evidence": {
Expand All @@ -78,7 +78,7 @@
"The 17179869184-byte reservation, block size 256, and 4096-token scheduler budget carry forward unchanged, but their capacity with the larger context and sequence target must be measured.",
"The complete checkpoint manifest is generated from the pinned revision and must reproduce the recorded digest. A different digest is a different composition and does not reproduce this qualification.",
"The published receipt covers DeepSeek DCP1, not DCP2 or DCP4.",
"This recipe disables streaming snapshots and native restore; the published receipt does not cover either mode."
"This recipe disables streaming snapshots and SparkCache CUDA restore; the published receipt does not cover either mode."
],
"record": "recipes/sparkcache/README.md",
"artifact_scope": "The exact SparkCache wheel, runtime image, checkpoint identity, topology, and historical max_model_len=131072, max_num_seqs=6 serving values stated in evidence.conditions.",
Expand Down
4 changes: 2 additions & 2 deletions recipes/sparkcache/deepseek-v4-flash-0731-tp4-dcp1.json
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@
"min_span_tokens": 256,
"max_span_tokens": 1048576,
"streaming_snapshots": false,
"native_restore": false,
"cuda_restore": false,
"root_requirement": "Use one dedicated rank-local host directory per physical rank, mounted at the same container path on all four ranks."
},
"evidence": {
Expand All @@ -68,7 +68,7 @@
"The 34359738368-byte reservation, 32-sequence limit, block size 256, and 4096-token scheduler budget carry forward unchanged, but capacity at the larger context must be measured.",
"The complete checkpoint manifest is generated from the pinned revision and must reproduce the recorded digest. A different digest is a different composition and does not reproduce this qualification.",
"The published receipt covers DeepSeek DCP1, not DCP2 or DCP4.",
"This recipe disables streaming snapshots and native restore; the published receipt does not cover either mode."
"This recipe disables streaming snapshots and SparkCache CUDA restore; the published receipt does not cover either mode."
],
"record": "recipes/sparkcache/README.md",
"artifact_scope": "The exact SparkCache wheel, runtime image, checkpoint identity, topology, and historical max_model_len=524288 serving values stated in evidence.conditions.",
Expand Down
4 changes: 2 additions & 2 deletions recipes/sparkcache/glm52-exl3-r7-3.5bpw-tp4-dcp4.json
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@
"min_span_tokens": 256,
"max_span_tokens": 1048576,
"streaming_snapshots": false,
"native_restore": false,
"cuda_restore": false,
"root_requirement": "Use one dedicated rank-local host directory per physical rank, mounted at the same container path on all four ranks."
},
"restart_contract": {
Expand All @@ -75,7 +75,7 @@
"The qualified receipt covers max_model_len=262144; the current max_model_len=1048576 target requires a larger CKV-gather workspace and live memory validation.",
"The qualified operator image is identified by local image ID but has no published immutable image reference. A clean-checkout rebuild has implemented status until it passes the promotion checklist.",
"The qualified lane requires preserving and regenerating the 40-query-row exact-state receipt, whose path contains q40-exact-state-serving-v1, before every coordinated restart.",
"This recipe disables streaming snapshots and native restore; the published receipt does not cover either mode.",
"This recipe disables streaming snapshots and SparkCache CUDA restore; the published receipt does not cover either mode.",
"Full GLM reasoning-trace equality is an inconclusive diagnostic under this runtime: repeated fixed-seed requests produced different non-empty reasoning bodies with identical final content and finish reason stop."
],
"record": "https://github.com/FujitsuPolycom/sparkcache/blob/999bc13d6c0b52a6cee3c90487ac9b8643ba3f99/GLM52_A2_LIVE_VALIDATION.md",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@
"min_span_tokens": 4096,
"max_span_tokens": 524288,
"streaming_snapshots": false,
"native_restore": false,
"cuda_restore": false,
"root_requirement": "Use one dedicated rank-local host directory per physical rank, mounted at /cache/jit on every rank."
},
"evidence": {
Expand All @@ -97,7 +97,7 @@
"A rebuilt image has implemented status until its immutable digest passes the same live qualification.",
"The first request after API startup recomputed because the scheduler had not received a complete four-rank inventory checkpoint; the following identical request formed quorum and restored.",
"The receipt does not establish throughput neutrality or restore performance for spans larger than 8192 tokens.",
"Streaming snapshots, native direct restore, MTP drafting, and other DFlash checkpoints are unsupported by this evidence.",
"Streaming snapshots, SparkCache CUDA restore, MTP drafting, and other DFlash checkpoints are unsupported by this evidence.",
"The source-built runtime uses the Triton KDA prefill backend and stock safetensors loading; FlashKDA prefill and InstantTensor checkpoint loading are unsupported by this image.",
"The optional deep_ep import emits a duplicate-NCCL warning. vLLM selects the source-built NCCL library through VLLM_NCCL_SO_PATH and serving proceeds.",
"The target repository does not record its base-checkpoint revision."
Expand Down
34 changes: 17 additions & 17 deletions runtime/glm53-flash-adaptive-mtp-python-overlay/Containerfile
Original file line number Diff line number Diff line change
Expand Up @@ -13,18 +13,18 @@ RUN test "$(git -C /build/b12x rev-parse HEAD)" = "${B12X_COMMIT}" \
--no-deps --wheel-dir /out/wheels /build/b12x \
&& sha256sum /out/wheels/b12x-*.whl > /out/wheels/b12x-wheel.sha256

FROM ${ARM_BUILDER} AS sparkcache-native
COPY bundle/sources/sparkcache/sparkcache/native /build/sparkcache-native
RUN cmake -S /build/sparkcache-native \
-B /build/sparkcache-native/build-cuda \
FROM ${ARM_BUILDER} AS sparkcache-cuda-placement
COPY bundle/sources/sparkcache/sparkcache/native /build/sparkcache-cuda-placement
RUN cmake -S /build/sparkcache-cuda-placement \
-B /build/sparkcache-cuda-placement/build-cuda \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_CUDA_ARCHITECTURES=121 \
-DSPARK_CACHE_PLACEMENT_ENABLE_CUDA=ON \
&& cmake --build /build/sparkcache-native/build-cuda \
&& cmake --build /build/sparkcache-cuda-placement/build-cuda \
--target spark_cache_placement -j"$(nproc)" \
&& test -s /build/sparkcache-native/build-cuda/libspark_cache_placement.so \
&& sha256sum /build/sparkcache-native/build-cuda/libspark_cache_placement.so \
> /build/sparkcache-native/build-cuda/libspark_cache_placement.so.sha256
&& test -s /build/sparkcache-cuda-placement/build-cuda/libspark_cache_placement.so \
&& sha256sum /build/sparkcache-cuda-placement/build-cuda/libspark_cache_placement.so \
> /build/sparkcache-cuda-placement/build-cuda/libspark_cache_placement.so.sha256

FROM ${PUBLIC_BASE}

Expand All @@ -43,7 +43,7 @@ ARG SOURCE_RECEIPT_SHA256
ARG OVERLAY_MANIFEST_SHA256
ARG NATIVE_ELF_MANIFEST_SHA256
ARG NATIVE_DISPATCH_MANIFEST_SHA256
ARG SPARKCACHE_NATIVE_SHA256
ARG SPARKCACHE_CUDA_PLACEMENT_SHA256

ENV PYTHON_OVERLAY_ROOT=/opt/sparkring/runtime/python-overlay \
VLLM_CACHE_ROOT=/cache/jit/vllm/py-0b67266-native-da4d7be \
Expand Down Expand Up @@ -98,11 +98,11 @@ RUN actual_source="$(python3 -c 'import importlib.util, pathlib; spec = importli
&& test "${actual_source}" = "${SPARKCACHE_SOURCE_SHA256}" \
&& printf '%s\n' "${actual_source}" \
> "${PYTHON_OVERLAY_ROOT}/sparkcache-source-tree.sha256"
COPY --from=sparkcache-native \
/build/sparkcache-native/build-cuda/libspark_cache_placement.so \
COPY --from=sparkcache-cuda-placement \
/build/sparkcache-cuda-placement/build-cuda/libspark_cache_placement.so \
/opt/sparkcache-src/sparkcache/native/build-cuda/libspark_cache_placement.so
COPY --from=sparkcache-native \
/build/sparkcache-native/build-cuda/libspark_cache_placement.so.sha256 \
COPY --from=sparkcache-cuda-placement \
/build/sparkcache-cuda-placement/build-cuda/libspark_cache_placement.so.sha256 \
/opt/sparkcache-src/sparkcache/native/build-cuda/libspark_cache_placement.so.sha256
COPY bundle/sources/sparkcache/patches/vllm-glm53-b12x-kda-adaptive-mtp \
/opt/sparkcache-vllm-patches
Expand All @@ -112,9 +112,9 @@ COPY bundle/sources/b12x/LICENSE /usr/share/licenses/B12X/LICENSE
COPY bundle/sources/sparkcache/LICENSE /usr/share/licenses/SparkCache/LICENSE

RUN set -eu; \
actual_native="$(sha256sum /opt/sparkcache-src/sparkcache/native/build-cuda/libspark_cache_placement.so | cut -d' ' -f1)"; \
printf 'sparkcache_native_sha256=%s\n' "${actual_native}"; \
test "${actual_native}" = "${SPARKCACHE_NATIVE_SHA256}"; \
actual_cuda_placement="$(sha256sum /opt/sparkcache-src/sparkcache/native/build-cuda/libspark_cache_placement.so | cut -d' ' -f1)"; \
printf 'sparkcache_cuda_placement_sha256=%s\n' "${actual_cuda_placement}"; \
test "${actual_cuda_placement}" = "${SPARKCACHE_CUDA_PLACEMENT_SHA256}"; \
root=/usr/local/lib/python3.12/dist-packages; \
config="${root}/vllm/config/vllm.py"; \
config_sha256="$(sha256sum "${config}" | cut -d' ' -f1)"; \
Expand Down Expand Up @@ -171,7 +171,7 @@ LABEL org.opencontainers.image.title="SparkRing GLM-5.3 adaptive-MTP Python over
org.sparkcache.source-revision="${SPARKCACHE_COMMIT}" \
org.sparkcache.source-tree="${SPARKCACHE_TREE}" \
org.sparkcache.source-sha256="${SPARKCACHE_SOURCE_SHA256}" \
org.sparkcache.native-library-sha256="${SPARKCACHE_NATIVE_SHA256}" \
org.sparkcache.cuda-placement-library-sha256="${SPARKCACHE_CUDA_PLACEMENT_SHA256}" \
org.sparkcache.vllm-contract-sha256="6defde9551cbb586fd09bb2d3020495531b6573397875a767eaae1dbad126024" \
org.sparkcache.deployment-profile="glm53-flash-adaptive-mtp-python-overlay" \
org.sparkring.source-receipt-sha256="${SOURCE_RECEIPT_SHA256}"
6 changes: 3 additions & 3 deletions runtime/glm53-flash-adaptive-mtp-python-overlay/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ placement library is commit
`e864ed9ad64f771188fdb59aa9738e348134d636`. The builder verifies clean
deployable-source SHA-256
`f7c0565521fddeff7085e4cc08043cb8d1e2bde33abc67f83b8608a162d05b88`
before generating the native library. It applies the VMM exemption,
before generating the SparkCache CUDA placement library. It applies the VMM exemption,
load-failure recovery, shared-prefix retention, and follower-attachment
patches in order, then runs the eleven-file lease-contract verifier. The test
profile selects `spark_cache_publication_schema=tail-cow-v1`, which maps opaque
Expand All @@ -50,7 +50,7 @@ bash runtime/glm53-flash-adaptive-mtp-python-overlay/build-image.sh

The script pulls and verifies the immutable public base, fetches exact source
commits into a temporary build context, builds a pure B12X wheel and the
SparkCache placement library, creates the composed image, and writes a local
SparkCache CUDA placement library, creates the composed image, and writes a local
receipt. It does not push an image or contact serving hosts.

## Resolve and inspect the four-rank plan
Expand All @@ -68,7 +68,7 @@ python scripts/prepare_glm53_public_python_overlay_profile.py \
--site-template /path/to/resolved-glm53-site.yaml \
--image "$image" \
--image-id "$(jq -r .image_id "$receipt")" \
--native-library-sha256 "$(jq -r .artifacts.sparkcache_native_sha256 "$receipt")" \
--cuda-placement-library-sha256 "$(jq -r .artifacts.sparkcache_cuda_placement_sha256 "$receipt")" \
--native-elf-manifest-sha256 "$(jq -r .runtime_contract.native_elf_manifest_sha256 "$receipt")" \
--native-dispatch-manifest-sha256 "$(jq -r .runtime_contract.native_dispatch_manifest_sha256 "$receipt")" \
--source-receipt-sha256 "$(jq -r .artifacts.source_receipt_sha256 "$receipt")" \
Expand Down
14 changes: 7 additions & 7 deletions runtime/glm53-flash-adaptive-mtp-python-overlay/build-image.sh
Original file line number Diff line number Diff line change
Expand Up @@ -80,17 +80,17 @@ mkdir -p "${context}/base-probe"
--output /out/retained-native.json >/dev/null
native_elf_manifest_sha256="$(python3 -c 'import json,sys; print(json.load(open(sys.argv[1], encoding="utf-8"))["native_elf_manifest_sha256"])' "${context}/base-probe/retained-native.json")"
native_dispatch_manifest_sha256="$(python3 -c 'import json,sys; print(json.load(open(sys.argv[1], encoding="utf-8"))["native_dispatch_manifest_sha256"])' "${context}/base-probe/retained-native.json")"
native_stage="sparkring-sparkcache-native:${sparkcache_commit:0:12}-${sparkring_revision:0:12}"
cuda_placement_stage="sparkring-sparkcache-cuda-placement:${sparkcache_commit:0:12}-${sparkring_revision:0:12}"
"${engine}" build \
--platform linux/arm64 \
--target sparkcache-native \
--target sparkcache-cuda-placement \
--file "${context}/bundle/runtime/Containerfile" \
--build-arg "ARM_BUILDER=${arm_builder}" \
--tag "${native_stage}" \
--tag "${cuda_placement_stage}" \
"${context}"
sparkcache_native_sha256="$("${engine}" run --rm --entrypoint sha256sum \
"${native_stage}" \
/build/sparkcache-native/build-cuda/libspark_cache_placement.so | cut -d' ' -f1)"
sparkcache_cuda_placement_sha256="$("${engine}" run --rm --entrypoint sha256sum \
"${cuda_placement_stage}" \
/build/sparkcache-cuda-placement/build-cuda/libspark_cache_placement.so | cut -d' ' -f1)"

"${engine}" build \
--platform linux/arm64 \
Expand All @@ -111,7 +111,7 @@ sparkcache_native_sha256="$("${engine}" run --rm --entrypoint sha256sum \
--build-arg "OVERLAY_MANIFEST_SHA256=${overlay_manifest_sha256}" \
--build-arg "NATIVE_ELF_MANIFEST_SHA256=${native_elf_manifest_sha256}" \
--build-arg "NATIVE_DISPATCH_MANIFEST_SHA256=${native_dispatch_manifest_sha256}" \
--build-arg "SPARKCACHE_NATIVE_SHA256=${sparkcache_native_sha256}" \
--build-arg "SPARKCACHE_CUDA_PLACEMENT_SHA256=${sparkcache_cuda_placement_sha256}" \
--tag "${image}" \
"${context}"

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -154,8 +154,10 @@ def test_containerfile_reuses_vllm_and_nccl_native_artifacts() -> None:
)
source_verification = recipe.index('printf \'sparkcache_source_sha256=%s')
source_receipt = recipe.index("sparkcache-source-tree.sha256")
native_copy = recipe.index("COPY --from=sparkcache-native", source_verification)
assert source_copy < source_verification < source_receipt < native_copy
cuda_placement_copy = recipe.index(
"COPY --from=sparkcache-cuda-placement", source_verification
)
assert source_copy < source_verification < source_receipt < cuda_placement_copy


def test_image_verifier_reads_the_clean_sparkcache_source_receipt() -> None:
Expand Down
Loading
Loading