Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 37 additions & 8 deletions docs/GLM53_DFLASH7_PYTHON_OVERLAY_SPARKCACHE_TP4_QUICKSTART.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,35 @@
# Serve GLM-5.3 with external DFlash7 and the exact Python-overlay runtime

Status: **implemented, not qualified** for the selected source contract. The
image builder, profile resolver, and four-rank dry-run contract pass without
GPUs. Historical local image ID
Status: **implemented** for reproducible image construction, profile
resolution, and four-rank dry-run planning from the selected source contract.
Exact local image
`sparkring-glm53-sparkcache:dflash7-pr39-reaching-d93cb3d-arm64`, image ID
`sha256:ed60be066d6d9eadea267bc4597a0687869f3ddb95a3e5c6f86649893a838eb8`,
is **qualified** only for the bounded cases in the
[exact-artifact validation record](../performance/records/glm53-flash/pr146-recurrent-publication-live-validation.md)
from [pull request #147](https://github.com/FujitsuPolycom/sparkring/pull/147).
It binds SparkRing `d93cb3d98305041081cf572521602625185112ae` and
SparkCache `65b6642df1afc64366430d3aef9aca01f5c5e1c3`.

The exact image completed the fixed semantic canary, a clean-restart
8,192-token persistent restore, a 131,072-token persistent restore, tail-only
copy-on-write publication from 131,072 to 262,144 tokens, a verified
262,144-token persistent restore, and one C16 cohort in which 16 distinct
request tails shared one restored 131,072-token segment. Restore timings are
**research-only**. The fixed semantic canary is not a DFlash response-quality
benchmark, so response quality is **unsupported**. Public OCI publication is
also **unsupported**; the image has no published digest. No rebuild inherits
these observations without its own exact evidence.

The `ed60...` artifact uses `block_pages_v1`. Its C16 case proves that one
exact 131,072-token prefix was restored once for requests with 16 distinct
tails. It does not exercise `per_token_rows` different-root descriptor-segment
coalescing. That behavior is implemented with GPU-free coverage only and is
absent from this artifact's live qualification.

### Separate historical artifact

Historical local image ID
`sha256:eef863d8bc578815a80b0e2d9f0d745102b6363415225101fd92171a2e5a55cb`
is **qualified** only for the TP4/DCP1 startup, health, semantic generation,
arbitrary page-boundary replay, and 131,072- and 262,144-token restore cases
Expand All @@ -20,7 +47,7 @@ identity and requires its own live checks.
| vLLM native extensions and wheel metadata | `da4d7be6c97434f6942292ed8abbf4b32dc44355` |
| vLLM Python source | `0b67266a0f37d6146a8403fb8482403c62f412d5`, tree `ba9484ccb33aa56e90ff2f447f15ca9b9da97639` |
| B12X | `b1d541f9e71a35f030d45fae437630fff7507c2a`, tree `c69cdec1c59a08e8e0e549f930fa8abcfb5134ae` |
| SparkCache shared-segment restore, tail-only copy-on-write publication, canonical CUDA configuration, and bounded page-delta reads | `65b6642df1afc64366430d3aef9aca01f5c5e1c3`, tree `41ad0a119ba109fd28900a2dcc9f9b4d8c293809`, clean source SHA-256 `a2add45a9f97446f6c2a843355161da9a5499ff7501b4750d2163591785d7345` |
| SparkCache exact prefix restore, GPU-free row-descriptor coalescing, tail-only copy-on-write publication, canonical CUDA configuration, and bounded page-delta reads | `65b6642df1afc64366430d3aef9aca01f5c5e1c3`, tree `41ad0a119ba109fd28900a2dcc9f9b4d8c293809`, clean source SHA-256 `a2add45a9f97446f6c2a843355161da9a5499ff7501b4750d2163591785d7345` |
| Recurrent replay-boundary producer | Patch SHA-256 `5a6561a5bbab990dcd03bfd6a485ea26c3b5a578c2fd61b76305767b16dbfba0`; produces the four postimages accepted by SparkCache lease contract SHA-256 `8adbdfa3fd4b06b213c3aab45255a0b039f1c9940a4b1fad0efd004d263227c9` |
| DFlash draft-loader separation | Patch SHA-256 `39b567013ee7aed79f63200ed460129587933dc77fb430decdf19f78178de279`, postimage SHA-256 `98acbae2b3bb4482d83f9637c163ce7c92707ccdf6561b7e431f23337f151cf4` |
| Unused DeepEP removal | Distribution `deep_ep==2.0.0+local`, removal receipt SHA-256 `65514f44829e7d176b0b2cacc9559ed22724e525b7041a8bcd4d2e02d1f372e3` |
Expand Down Expand Up @@ -68,7 +95,7 @@ Two profiles share the same image and DFlash7 cache identity:
| Profile | Status | Loader behavior |
|---|---|---|
| `glm53-flash-dflash7-python-overlay-safetensors-sparkcache-tp4-dcp1.example.json` | **implemented**, not qualified | Uses global safetensors for target and draft. This follows the qualified-compatible loader shape but still requires live qualification on the composed 0b image. |
| `glm53-flash-dflash7-python-overlay-fastsafetensors-sparkcache-tp4-dcp1.example.json` | **implemented**, not qualified for the selected source contract | Uses global fastsafetensors with queue size one for the target and `draft_load_config={"load_format":"safetensors"}` for DFlash. The historical receipt remains evidence only for its recorded image. |
| `glm53-flash-dflash7-python-overlay-fastsafetensors-sparkcache-tp4-dcp1.example.json` | **qualified** only for exact image `sha256:ed60be066d6d9eadea267bc4597a0687869f3ddb95a3e5c6f86649893a838eb8` and the bounded cases above; rebuilds are not qualified | Uses global fastsafetensors with queue size one for the target and `draft_load_config={"load_format":"safetensors"}` for DFlash. |

The image applies an exact-input vLLM patch that passes
`SpeculativeConfig.draft_load_config` to the DFlash model loader. The image
Expand Down Expand Up @@ -115,7 +142,8 @@ python scripts/sparkring_generic_launcher.py \
plan
```

`plan` is offline. Inspect every rank action before a lifecycle command.
`plan` is offline. Inspect every rank action before a command that changes
containers.

## Start and observe the four-rank service

Expand Down Expand Up @@ -179,8 +207,9 @@ different cache root and one-shot clear token so loader observations remain
isolated.

The pinned SparkCache source combines canonical CUDA configuration names,
authenticated shared-segment restore, an eight-worker ordered page-delta
reader, and tail-only copy-on-write publication. Cache identities, digest
authenticated exact prefix restore, GPU-free row-descriptor coalescing, an
eight-worker ordered page-delta reader, and tail-only copy-on-write
publication. Cache identities, digest
salts, 256-token chunk geometry, page-delta wire bytes, and the CUDA placement
ABI are unchanged. The source change does not change the namespace, so
compatible `page-tail-cow-v1` entries remain eligible.
Expand Down
1 change: 1 addition & 0 deletions performance/receipts/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ qualified functional claim or a research-only observation.
| [`glm-3.5bpw/temp1/`](glm-3.5bpw/temp1/) | 10 four-Spark TP4/DCP4 sustained-decode and Coding Peak receipts |
| [`glm53-flash/sparkcache-dflash2-bf16-tp4-20260828/`](glm53-flash/sparkcache-dflash2-bf16-tp4-20260828/) | Sanitized post-restore semantic canary for the TP4/DCP1 SparkCache validation |
| [`glm53-flash/sparkcache-dflash2-bf16-tp4-20260829/`](glm53-flash/sparkcache-dflash2-bf16-tp4-20260829/) | One accepted 16K prefill and C1 decode observation; capacity-limited C4 and C8 cells are excluded |
| [`glm53-flash/adaptive-mtp-vs-dflash7-20260829/`](glm53-flash/adaptive-mtp-vs-dflash7-20260829/) | Research-only server-log diagnostic; prompt and output receipts are absent |
| [`glm53-flash/sparkcache-dflash2-bf16-tp4-20g-20260829/`](glm53-flash/sparkcache-dflash2-bf16-tp4-20g-20260829/) | One accepted functional observation with 20 GiB of GPU KV memory per rank |
| [`qwen38-27b/temp1/`](qwen38-27b/temp1/) | 13 two-Spark and 16 four-Spark accepted prefill, decode, and Coding Peak receipts |

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
{
"schema": "sparkring-glm53-decode-log-diagnostic/v1",
"status": "research-only",
"target_model": {
"repository": "local-inference-lab/GLM-5.3-Flash-NVFP4",
"revision": "520de24eabf507659eaef7c70f14fd584527facc",
"cache_identity_sha256": "a35e6bf2875c1875609b8deaec404c07c6cc80259e4222fc0b51e649498bd6b9"
},
"topology": {
"systems": 4,
"tensor_parallel_size": 4,
"decode_context_parallel_size": 1,
"pipeline_parallel_size": 1
},
"adaptive_mtp": {
"container": "glm53-flash-public-python-overlay-mtp5-adaptive-fastsafetensors-sparkcache-tp4-r0",
"image_id": "sha256:d209cf986c1f14f320e53d8b425c5e3a255eef9320f25b632af50f5b5c977314",
"served_model": "glm-5.3-flash-nvfp4-python-overlay-0b67266-on-da4d7be-b12x-b1d541f-mtp5-adaptive-tp4",
"vllm_compiled_commit": "da4d7be6c97434f6942292ed8abbf4b32dc44355",
"vllm_python_commit": "0b67266a0f37d6146a8403fb8482403c62f412d5",
"b12x_commit": "b1d541f9e71a35f030d45fae437630fff7507c2a",
"sparkcache_commit": "20838ace3ebda570ca039cb7f1976c29da554b39",
"sparkring_revision": "914c94d084d6881e90660305dedaa410ef02b167",
"draft_identity_sha256": "2e06d909ce5bb71c0c0e3e8be74a70e3b41d92ba4c30196cfb0957fb812acef6",
"maximum_depth": 5,
"initial_depth": 3,
"observation_window_steps": 32,
"log_interval_utc": {
"start": "2026-08-30T03:23:25Z",
"end": "2026-08-30T03:25:15Z"
},
"observations": [
{"timestamp": "2026-08-30T03:23:25Z", "generation_tokens_per_second": 38.7, "depth": 4, "draft_acceptance_percent": 74.2},
{"timestamp": "2026-08-30T03:23:35Z", "generation_tokens_per_second": 30.7, "depth": 3, "draft_acceptance_percent": 65.9},
{"timestamp": "2026-08-30T03:23:45Z", "generation_tokens_per_second": 34.1, "depth": 3, "draft_acceptance_percent": 62.2},
{"timestamp": "2026-08-30T03:23:55Z", "generation_tokens_per_second": 29.5, "depth": 4, "draft_acceptance_percent": 77.0},
{"timestamp": "2026-08-30T03:24:05Z", "generation_tokens_per_second": 34.6, "depth": 4, "draft_acceptance_percent": 72.4},
{"timestamp": "2026-08-30T03:24:15Z", "generation_tokens_per_second": 31.4, "depth": 3, "draft_acceptance_percent": 59.4},
{"timestamp": "2026-08-30T03:24:25Z", "generation_tokens_per_second": 36.3, "depth": 3, "draft_acceptance_percent": 67.5},
{"timestamp": "2026-08-30T03:24:35Z", "generation_tokens_per_second": 36.8, "depth": 4, "draft_acceptance_percent": 72.0},
{"timestamp": "2026-08-30T03:24:45Z", "generation_tokens_per_second": 28.0, "depth": 3, "draft_acceptance_percent": 58.2},
{"timestamp": "2026-08-30T03:24:55Z", "generation_tokens_per_second": 24.5, "depth": 2, "draft_acceptance_percent": 48.9},
{"timestamp": "2026-08-30T03:25:05Z", "generation_tokens_per_second": 31.9, "depth": 3, "draft_acceptance_percent": 75.1},
{"timestamp": "2026-08-30T03:25:15Z", "generation_tokens_per_second": 37.1, "depth": 4, "draft_acceptance_percent": 81.2}
],
"sparkcache_events_in_interval": 0
},
"dflash7": {
"container": "glm53-flash-dflash2-bf16-sparkcache-public-tp4-r0-dflash7-stopped-20260829",
"rank_image_ids": [
"sha256:80b407904d21211d9a6b435c8ee4292ce66bfebcf802d94deb6605cdbc94d648",
"sha256:de4e219db4bdc2dfe4a2bcf2d5adb11655da22b5d40bf42494028c5ed128ea83",
"sha256:b5c29a1fc97fe7e086b62cffd832d90b292dc4f60cbf10c5593ef2f3debea365",
"sha256:9aeaf31b13af8a940734e31ec2963ce2714db535430ce7953a8baa5cf6dd4401"
],
"served_model": "glm-5.3-flash-nvfp4-dflash7-bf16-tp4",
"vllm_commit": "da4d7be6c97434f6942292ed8abbf4b32dc44355",
"b12x_commit": "2fcf23a0ce269be27b2e03fece73d46e90e6aeea",
"sparkcache_commit": "2b86fb9d02fa3595cca5caa864b81aedce44b8bb",
"draft_repository": "incoai/GLM-5.3-Flash-DFlash2",
"draft_revision": "dc77ff1c99eeb2df044ee3d4f0094eb033fee410",
"draft_weights_sha256": "b33c03475ba7322cf398828f2d8d1be376df30dc05c6b40c28c8ea8da23e410b",
"depth": 7,
"log_interval_utc": {
"start": "2026-08-29T21:13:57Z",
"end": "2026-08-29T21:15:07Z"
},
"generation_tokens_per_second": [40.6, 48.2, 52.6, 94.9, 111.8, 69.8, 133.6, 136.0],
"sparkcache_events_in_interval": 0,
"post_switch_validation": {
"log_interval_utc": {
"start": "2026-08-30T04:02:49Z",
"end": "2026-08-30T04:04:09Z"
},
"observations": [
{"timestamp": "2026-08-30T04:02:49Z", "generation_tokens_per_second": 48.8, "mean_acceptance_length": 4.82, "draft_acceptance_percent": 54.6},
{"timestamp": "2026-08-30T04:02:59Z", "generation_tokens_per_second": 61.3, "mean_acceptance_length": 4.61, "draft_acceptance_percent": 51.6},
{"timestamp": "2026-08-30T04:03:09Z", "generation_tokens_per_second": 74.5, "mean_acceptance_length": 5.69, "draft_acceptance_percent": 67.0},
{"timestamp": "2026-08-30T04:03:19Z", "generation_tokens_per_second": 60.9, "mean_acceptance_length": 4.54, "draft_acceptance_percent": 50.5},
{"timestamp": "2026-08-30T04:03:29Z", "generation_tokens_per_second": 61.3, "mean_acceptance_length": 4.51, "draft_acceptance_percent": 50.1},
{"timestamp": "2026-08-30T04:03:39Z", "generation_tokens_per_second": 56.3, "mean_acceptance_length": 4.33, "draft_acceptance_percent": 47.6},
{"timestamp": "2026-08-30T04:03:49Z", "generation_tokens_per_second": 64.4, "mean_acceptance_length": 5.07, "draft_acceptance_percent": 58.1},
{"timestamp": "2026-08-30T04:03:59Z", "generation_tokens_per_second": 72.1, "mean_acceptance_length": 5.30, "draft_acceptance_percent": 61.4},
{"timestamp": "2026-08-30T04:04:09Z", "generation_tokens_per_second": 72.2, "mean_acceptance_length": 5.47, "draft_acceptance_percent": 63.9}
],
"sparkcache_events_in_interval": 0
}
},
"user_reported_matched_coding_peak": {
"dflash7_tokens_per_second": 70.0,
"adaptive_mtp_tokens_per_second": 33.5,
"receipt_present": false
},
"prompt_receipt_present": false,
"output_receipt_present": false
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,135 @@
{
"schema": "sparkring-glm53-dflash7-pr25-live-validation/v1",
"status": {
"artifact_construction": "implemented",
"startup_health": "qualified",
"semantic_smoke": "qualified",
"flat_restore": "qualified",
"page_delta_restore_correctness": "qualified",
"page_delta_restore_performance": "research-only",
"persistent_128k_restore": "qualified",
"shared_prefix_waves": "qualified",
"dflash_quality": "unsupported"
},
"artifact": {
"image": "sparkring-glm53-sparkcache:dflash7-vllm-python-0b67266-native-da4d7be-b12x-b1d541f-arm64",
"image_id": "sha256:9faa36a9f37aee16d97ab9214ef3153b4d200121126e6b2dee5ebb63109fea18",
"oci_revision_label": "e2d92fdc7d0306d664d6fd9f296dc2adcaf0fe05",
"sparkring_revision": "e2d92fdc7d0306d664d6fd9f296dc2adcaf0fe05",
"vllm_compiled_revision": "da4d7be6c97434f6942292ed8abbf4b32dc44355",
"vllm_python_revision": "0b67266a0f37d6146a8403fb8482403c62f412d5",
"b12x_revision": "b1d541f9e71a35f030d45fae437630fff7507c2a",
"sparkcache_revision": "5d571018de5b63a9a90e5c11e6d6e86bbff4a957",
"sparkcache_tree": "e864ed9ad64f771188fdb59aa9738e348134d636",
"sparkcache_source_sha256": "f7c0565521fddeff7085e4cc08043cb8d1e2bde33abc67f83b8608a162d05b88",
"cuda_placement_library_sha256": "a2e495162bf3d58b01613cd82ac15c8e15031dd7d6de7299700d2c58d905ada8",
"dflash_loader_patch_sha256": "39b567013ee7aed79f63200ed460129587933dc77fb430decdf19f78178de279",
"dflash_loader_postimage_sha256": "98acbae2b3bb4482d83f9637c163ce7c92707ccdf6561b7e431f23337f151cf4"
},
"conditions": {
"hardware": "four NVIDIA DGX Spark systems",
"topology": {
"tensor_parallel_size": 4,
"decode_context_parallel_size": 1,
"pipeline_parallel_size": 1
},
"target_loader": "fastsafetensors",
"draft_loader": "safetensors",
"draft_method": "dflash",
"draft_tokens": 7,
"draft_tensor_parallel_size": 4,
"kv_cache_dtype": "fp8",
"vllm_block_size": 256,
"max_num_seqs": 32,
"sparkcache_publication_schema": "tail-cow-v1",
"sparkcache_effective_publication_schema": "page-tail-cow-v1",
"sparkcache_config_surface": "PR25 legacy-key compatibility",
"sparkcache_accepted_legacy_keys": [
"spark_cache_native_restore",
"spark_cache_native_library",
"spark_cache_native_library_sha256",
"spark_cache_native_arena_bytes",
"spark_cache_native_io_workers"
]
},
"measurements": {
"startup": {
"target_fastsafetensors_seconds": 69.52,
"draft_safetensors_seconds": 4.18,
"total_model_load_seconds": 79.27,
"warm_ready_seconds_approx": 190.0,
"healthy_ranks": 4,
"restart_count_by_rank": [0, 0, 0, 0],
"oom_killed_by_rank": [false, false, false, false]
},
"semantic_smoke": {
"raw_completion_tokens": [2, 2],
"scope": "continued raw completion only; no quality score"
},
"flat_restore": {
"restored_tokens": 11520,
"rank": 0,
"restore_milliseconds": 29.258
},
"page_delta_restore": {
"restored_tokens": 17152,
"restore_milliseconds": 703.826,
"restore_read_milliseconds": 579.259,
"chunk_count": 67,
"type_error_observed": false
},
"prime_128k": {
"request_seconds": 55.522,
"completion_token": 13,
"snapshot_milliseconds": 2440.7,
"commit_milliseconds": 1687.7
},
"unrelated_sentinel": {
"completion_token": 271
},
"persistent_restore_128k": {
"restored_tokens": 131072,
"bytes_per_rank": 813068464,
"rank_restore_milliseconds": {
"minimum": 123.69,
"maximum": 153.253
},
"logical_tokens_per_second_at_slowest_rank": 855265.476
},
"shared_prefix_waves": [
{
"concurrency": 2,
"wall_seconds": 0.808,
"http_200": 2,
"completion_token": 13,
"restore_events_observed": 1,
"retention_observed": true
},
{
"concurrency": 8,
"wall_seconds": 1.506,
"http_200": 8,
"completion_token": 13,
"restore_events_observed": 1,
"retention_observed": true
},
{
"concurrency": 16,
"wall_seconds": 1.781,
"http_200": 16,
"completion_token": 13,
"restore_events_observed": 1,
"retention_observed": true
}
]
},
"limitations": [
"The 7168-to-12032 base-geometry incompatibility is addressed only in draft SparkCache PR28 and remains present in this artifact.",
"Null-block publication rejection at 6912 tokens remains under investigation.",
"Page-delta read and reconstruction latency is too high for a performance claim.",
"No DFlash quality benchmark was run on this artifact.",
"One observed wave per concurrency does not establish latency variability, throughput, or soak behavior.",
"The collection timestamp, complete command transcript, and independent clock audit are not part of this bounded receipt.",
"This receipt qualifies only image sha256:9faa36a9f37aee16d97ab9214ef3153b4d200121126e6b2dee5ebb63109fea18; it does not qualify image sha256:eef863d8bc578815a80b0e2d9f0d745102b6363415225101fd92171a2e5a55cb or image sha256:ed60be066d6d9eadea267bc4597a0687869f3ddb95a3e5c6f86649893a838eb8."
]
}
Loading
Loading