Skip to content

Record bounded DFlash7 Python-overlay live validation #138

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
FujitsuPolycom wants to merge 2 commits into from
Closed

Record bounded DFlash7 Python-overlay live validation #138

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
e53bfaa
Document bounded DFlash7 Python-overlay qualification
FujitsuPolycom Aug 30, 2026
7707aea
Record the 128K logical restore rate
FujitsuPolycom Aug 30, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
93 changes: 83 additions & 10 deletions docs/GLM53_DFLASH7_PYTHON_OVERLAY_SPARKCACHE_TP4_QUICKSTART.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,9 +1,12 @@
# Serve GLM-5.3 with external DFlash7 and the exact Python-overlay runtime

Status: **implemented**, not qualified. The image builder, profile resolver,
and four-rank dry-run contract pass without GPUs. No image digest from this
path has completed TP4/DCP1 model loading, semantic generation, SparkCache
store/restart/restore, or concurrency qualification.
Status: **implemented** for the builder and profile resolver. Image ID
`sha256:9faa36a9f37aee16d97ab9214ef3153b4d200121126e6b2dee5ebb63109fea18`
is **qualified** only for the bounded startup, health, semantic smoke, and
restore cases in the
[live-validation record](../performance/records/glm53-flash/dflash7-python-overlay-pr25-live-validation.md).
DFlash response quality and serving configurations outside that record are
**unsupported** by its evidence.

## Runtime contract

Expand Down Expand Up @@ -43,16 +46,21 @@ Two profiles share the same image and DFlash7 cache identity:

| Profile | Status | Loader behavior |
|---|---|---|
| `glm53-flash-dflash7-python-overlay-safetensors-sparkcache-tp4-dcp1.example.json` | **implemented**, not qualified | Uses global safetensors for target and draft. This follows the qualified-compatible loader shape but still requires live qualification on the composed 0b image. |
| `glm53-flash-dflash7-python-overlay-fastsafetensors-sparkcache-tp4-dcp1.example.json` | **implemented**, not qualified | Uses global fastsafetensors with queue size one for the target and `draft_load_config={"load_format":"safetensors"}` for DFlash. |
| `glm53-flash-dflash7-python-overlay-safetensors-sparkcache-tp4-dcp1.example.json` | **implemented** | Uses global safetensors for target and draft. This profile has no live evidence on the composed image. |
| `glm53-flash-dflash7-python-overlay-fastsafetensors-sparkcache-tp4-dcp1.example.json` | **qualified** for the recorded image and bounded gates | Uses global fastsafetensors with queue size one for the target and `draft_load_config={"load_format":"safetensors"}` for DFlash. |

The image applies an exact-input vLLM patch that passes
`SpeculativeConfig.draft_load_config` to the DFlash model loader. The image
receipt verifies patch SHA-256
`39b567013ee7aed79f63200ed460129587933dc77fb430decdf19f78178de279` and
postimage SHA-256
`98acbae2b3bb4482d83f9637c163ce7c92707ccdf6561b7e431f23337f151cf4`.
Both profiles remain unqualified until live four-rank gates pass.
The all-safetensors profile is implemented and not qualified; it has no live
four-rank evidence.
The fastsafetensors profile completed the bounded four-rank gates recorded for
image ID
`sha256:9faa36a9f37aee16d97ab9214ef3153b4d200121126e6b2dee5ebb63109fea18`.
That result does not qualify other image IDs or the all-safetensors profile.

## Resolve the profile and inspect the plan

Expand Down Expand Up @@ -85,11 +93,76 @@ python scripts/sparkring_generic_launcher.py \

`plan` is offline. Inspect every rank action before a lifecycle command.

## Launch the recorded image with SparkCache pull request 25 keys

The validated image ID
`sha256:9faa36a9f37aee16d97ab9214ef3153b4d200121126e6b2dee5ebb63109fea18`
contains SparkRing commit
`e2d92fdc7d0306d664d6fd9f296dc2adcaf0fe05` and
[SparkCache pull request 25](https://github.com/FujitsuPolycom/sparkcache/pull/25)
commit `5d571018de5b63a9a90e5c11e6d6e86bbff4a957`. That SparkCache commit accepts
the configuration names whose literals begin `spark_cache_native_`. The
profile templates expose the canonical CUDA names, so translate the resolved
profile only when launching this exact image:

| Canonical profile key | Key accepted by the recorded image |
|---|---|
| `spark_cache_cuda_restore` | `spark_cache_native_restore` |
| `spark_cache_cuda_placement_library` | `spark_cache_native_library` |
| `spark_cache_cuda_placement_library_sha256` | `spark_cache_native_library_sha256` |
| `spark_cache_cuda_placement_arena_bytes` | `spark_cache_native_arena_bytes` |
| `spark_cache_cuda_restore_io_workers` | `spark_cache_native_io_workers` |

Apply the translation after resolving the profile and before running
`plan`. The program fails if a source key is absent or a destination key is
already present:

```bash
python - /path/to/glm53-dflash7-profile.json <<'PY'
import json
import pathlib
import sys

path = pathlib.Path(sys.argv[1])
profile = json.loads(path.read_text())
args = profile["extra_vllm_args"]
index = args.index("--kv-transfer-config") + 1
config = json.loads(args[index])
extra = config["kv_connector_extra_config"]
mapping = {
"spark_cache_cuda_restore": "spark_cache_native_restore",
"spark_cache_cuda_placement_library": "spark_cache_native_library",
"spark_cache_cuda_placement_library_sha256": "spark_cache_native_library_sha256",
"spark_cache_cuda_placement_arena_bytes": "spark_cache_native_arena_bytes",
"spark_cache_cuda_restore_io_workers": "spark_cache_native_io_workers",
}
for source, destination in mapping.items():
if source not in extra or destination in extra:
raise SystemExit(f"refusing ambiguous SparkCache key translation: {source} -> {destination}")
extra[destination] = extra.pop(source)
args[index] = json.dumps(config, separators=(",", ":"))
path.write_text(json.dumps(profile, indent=2) + "\n")
PY

python scripts/sparkring_generic_launcher.py \
--site /path/to/glm53-dflash7-site.yaml \
--profile /path/to/glm53-dflash7-profile.json \
plan
```

[SparkRing pull request #137](https://github.com/FujitsuPolycom/sparkring/pull/137)
changes the image contract to SparkCache pull request 26 and canonical CUDA
configuration names. The image specified by pull request #137 has not been
built or live-validated. Do not use the translation above for an
image whose receipt binds SparkCache pull request 26 or a later source
contract.

## Cache namespace impact

The external DFlash weights SHA-256 is stored as
`spark_cache_draft_checkpoint_sha256`. It cannot share entries with embedded
MTP profiles. `tail-cow-v1` also separates these entries from snapshot-v1
manifests. The two target-loader profiles share a namespace because loader
choice does not change target or draft model state; each profile uses a
different cache root and one-shot clear token while qualification is pending.
manifests. The two target-loader profiles could share a namespace because
loader choice does not change target or draft model state. Their templates use
different cache roots and one-shot clear tokens so observations from one
loader do not enter validation of the other loader.
134 changes: 134 additions & 0 deletions performance/receipts/glm53-flash/dflash7-python-overlay-pr25/validation.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,134 @@
{
"schema": "sparkring-glm53-dflash7-pr25-live-validation/v1",
"status": {
"artifact_construction": "implemented",
"startup_health": "qualified",
"semantic_smoke": "qualified",
"flat_restore": "qualified",
"page_delta_restore_correctness": "qualified",
"page_delta_restore_performance": "research-only",
"persistent_128k_restore": "qualified",
"shared_prefix_waves": "qualified",
"dflash_quality": "unsupported"
},
"artifact": {
"image": "sparkring-glm53-sparkcache:dflash7-vllm-python-0b67266-native-da4d7be-b12x-b1d541f-arm64",
"image_id": "sha256:9faa36a9f37aee16d97ab9214ef3153b4d200121126e6b2dee5ebb63109fea18",
"oci_revision_label": "e2d92fdc7d0306d664d6fd9f296dc2adcaf0fe05",
"sparkring_revision": "e2d92fdc7d0306d664d6fd9f296dc2adcaf0fe05",
"vllm_native_revision": "da4d7be6c97434f6942292ed8abbf4b32dc44355",
"vllm_python_revision": "0b67266a0f37d6146a8403fb8482403c62f412d5",
"b12x_revision": "b1d541f9e71a35f030d45fae437630fff7507c2a",
"sparkcache_revision": "5d571018de5b63a9a90e5c11e6d6e86bbff4a957",
"sparkcache_tree": "e864ed9ad64f771188fdb59aa9738e348134d636",
"sparkcache_source_sha256": "f7c0565521fddeff7085e4cc08043cb8d1e2bde33abc67f83b8608a162d05b88",
"cuda_placement_library_sha256": "a2e495162bf3d58b01613cd82ac15c8e15031dd7d6de7299700d2c58d905ada8",
"dflash_loader_patch_sha256": "39b567013ee7aed79f63200ed460129587933dc77fb430decdf19f78178de279",
"dflash_loader_postimage_sha256": "98acbae2b3bb4482d83f9637c163ce7c92707ccdf6561b7e431f23337f151cf4"
},
"conditions": {
"hardware": "four NVIDIA DGX Spark systems",
"topology": {
"tensor_parallel_size": 4,
"decode_context_parallel_size": 1,
"pipeline_parallel_size": 1
},
"target_loader": "fastsafetensors",
"draft_loader": "safetensors",
"draft_method": "dflash",
"draft_tokens": 7,
"draft_tensor_parallel_size": 4,
"kv_cache_dtype": "fp8",
"vllm_block_size": 256,
"max_num_seqs": 32,
"sparkcache_publication_schema": "tail-cow-v1",
"sparkcache_effective_publication_schema": "page-tail-cow-v1",
"sparkcache_config_surface": "PR25 legacy-key compatibility",
"sparkcache_accepted_legacy_keys": [
"spark_cache_native_restore",
"spark_cache_native_library",
"spark_cache_native_library_sha256",
"spark_cache_native_arena_bytes",
"spark_cache_native_io_workers"
]
},
"measurements": {
"startup": {
"target_fastsafetensors_seconds": 69.52,
"draft_safetensors_seconds": 4.18,
"total_model_load_seconds": 79.27,
"warm_ready_seconds_approx": 190.0,
"healthy_ranks": 4,
"restart_count_by_rank": [0, 0, 0, 0],
"oom_killed_by_rank": [false, false, false, false]
},
"semantic_smoke": {
"raw_completion_tokens": [2, 2],
"scope": "continued raw completion only; no quality score"
},
"flat_restore": {
"restored_tokens": 11520,
"rank": 0,
"restore_milliseconds": 29.258
},
"page_delta_restore": {
"restored_tokens": 17152,
"restore_milliseconds": 703.826,
"restore_read_milliseconds": 579.259,
"chunk_count": 67,
"type_error_observed": false
},
"prime_128k": {
"request_seconds": 55.522,
"completion_token": 13,
"snapshot_milliseconds": 2440.7,
"commit_milliseconds": 1687.7
},
"unrelated_sentinel": {
"completion_token": 271
},
"persistent_restore_128k": {
"restored_tokens": 131072,
"bytes_per_rank": 813068464,
"rank_restore_milliseconds": {
"minimum": 123.69,
"maximum": 153.253
},
"logical_tokens_per_second_at_slowest_rank": 855265.476
},
"shared_prefix_waves": [
{
"concurrency": 2,
"wall_seconds": 0.808,
"http_200": 2,
"completion_token": 13,
"restore_events_observed": 1,
"retention_observed": true
},
{
"concurrency": 8,
"wall_seconds": 1.506,
"http_200": 8,
"completion_token": 13,
"restore_events_observed": 1,
"retention_observed": true
},
{
"concurrency": 16,
"wall_seconds": 1.781,
"http_200": 16,
"completion_token": 13,
"restore_events_observed": 1,
"retention_observed": true
}
]
},
"limitations": [
"The 7168-to-12032 base-geometry failure is fixed only in draft SparkCache PR28 and is not fixed in this artifact.",
"Null-block publication failures at 6912 tokens remain under investigation.",
"Page-delta read and reconstruction latency is too high for a performance claim.",
"No DFlash quality benchmark was run on this artifact.",
"One observed wave per concurrency does not establish latency variability, throughput, or soak behavior.",
"The collection timestamp, complete command transcript, and independent clock audit are not part of this bounded receipt."
]
}
100 changes: 100 additions & 0 deletions performance/records/glm53-flash/dflash7-python-overlay-pr25-live-validation.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
# GLM-5.3 DFlash7 Python-overlay bounded live validation

Status: **qualified** only for the startup, health, semantic smoke, and exact
SparkCache restore cases described below. Page-delta restore performance is
**research-only**. DFlash response quality is **unsupported** by this record.

## Conditions

- Four NVIDIA DGX Spark systems; TP4, DCP1, and PP1.
- Local image ID
`sha256:9faa36a9f37aee16d97ab9214ef3153b4d200121126e6b2dee5ebb63109fea18`
on all four ranks. Its `org.opencontainers.image.revision` label and the
SparkRing source commit are
`e2d92fdc7d0306d664d6fd9f296dc2adcaf0fe05`.
- vLLM native extensions from
`da4d7be6c97434f6942292ed8abbf4b32dc44355`, vLLM Python source from
`0b67266a0f37d6146a8403fb8482403c62f412d5`, and B12X from
`b1d541f9e71a35f030d45fae437630fff7507c2a`.
- [SparkCache pull request 25](https://github.com/FujitsuPolycom/sparkcache/pull/25)
commit
`5d571018de5b63a9a90e5c11e6d6e86bbff4a957`, Git tree
`e864ed9ad64f771188fdb59aa9738e348134d636`, and clean deployable-source
SHA-256
`f7c0565521fddeff7085e4cc08043cb8d1e2bde33abc67f83b8608a162d05b88`.
- GLM-5.3 NVFP4 target loaded with fastsafetensors. The external BF16 DFlash
checkpoint loaded with safetensors through the exact draft-loader patch.
Serving used seven speculative tokens, draft TP4, FP8 KV, 32 sequences,
and 256-token vLLM blocks.
- SparkCache publication schema `tail-cow-v1`, resolved internally to
`page-tail-cow-v1` for opaque GLM pages.
- The corrected launch translated the canonical CUDA restore settings to the
names accepted by the recorded SparkCache commit. The exact mapping is
documented in the
[DFlash7 Python-overlay quickstart](../../../docs/GLM53_DFLASH7_PYTHON_OVERLAY_SPARKCACHE_TP4_QUICKSTART.md#launch-the-recorded-image-with-sparkcache-pull-request-25-keys).

The machine-readable identities, conditions, measurements, and scoped status
are retained in
[`validation.json`](../../receipts/glm53-flash/dflash7-python-overlay-pr25/validation.json).

## Measurement

Startup timings came from loader and readiness logs. Health gates checked all
four containers for HTTP health, restart count, and the runtime OOM-killed
flag. Two raw-completion requests checked that generation continued; they were
not scored for response quality.

Restore timings came from SparkCache logs for one flat restore, one
reconstructed page-delta restore, and one persistent 128K-class restore. The
persistent case first primed the cache, issued an unrelated sentinel request,
and then restored 813,068,464 bytes per rank. One retained-prefix wave was
observed at each of concurrency 2, 8, and 16. Each wave recorded one restore
event followed by retained-prefix followers; it does not assert one restore
per request. Wall-clock and component timing values are single observations,
so no variability estimate is available.

## Result

| Gate or measurement | Observed result |
|---|---|
| Target fastsafetensors load | 69.52 seconds |
| Draft safetensors load | 4.18 seconds |
| Total model load | 79.27 seconds |
| Warm readiness | approximately 190 seconds |
| Four-rank health | 4 healthy; restart counts `0,0,0,0`; OOM-killed flags `false,false,false,false` |
| Raw semantic smoke | completion tokens `2,2` |
| Flat restore | 11,520 tokens; rank 0; 29.258 ms |
| Reconstructed page-delta restore | 17,152 tokens; 703.826 ms total; 579.259 ms read; 67 chunks; no `TypeError` |
| 128K-class prime | 55.522 seconds; completion token 13; snapshot 2,440.7 ms; commit 1,687.7 ms |
| Unrelated sentinel | completion token 271 |
| Persistent restore | 813,068,464 bytes per rank; rank range 123.690-153.253 ms |
| Slowest-rank logical restore rate | 855,265 tokens/s (`131,072 / 0.153253`); ranks are shards of one context and are not summed |
| Concurrency 2 retained-prefix wave | 0.808 seconds; 2 HTTP 200 responses; every completion token 13; one restore observed |
| Concurrency 8 retained-prefix wave | 1.506 seconds; 8 HTTP 200 responses; every completion token 13; one restore observed |
| Concurrency 16 retained-prefix wave | 1.781 seconds; 16 HTTP 200 responses; every completion token 13; one restore observed |

## Conclusion

The exact four-rank image completed target fastsafetensors loading and
separate DFlash safetensors loading, stayed healthy, produced continued raw
completions, and restored the exact flat, reconstructed page-delta, persistent
128K-class, and retained-prefix cases recorded above. These observations
qualify those bounded cases only. The 703.826 ms page-delta observation shows
functional reconstruction without the former `TypeError`; it does not support
a page-delta performance claim.

## Limitations

- A 7,168-token base followed by a 12,032-token request can fail page
reconstruction because the base geometry is incompatible. The correction
exists only in
[draft SparkCache pull request 28](https://github.com/FujitsuPolycom/sparkcache/pull/28)
and is absent from this image.
- Null-block publication failures at 6,912 tokens remain under investigation.
- Page-delta read and reassembly are too slow for a performance qualification.
- No DFlash quality benchmark was run on this artifact. The two-token raw
completions prove continued generation, not answer quality.
- One wave at each concurrency does not establish latency variability,
throughput, soak behavior, or behavior beyond 16 concurrent requests.
- The retained evidence does not include a complete command transcript,
collection timestamp, or independent clock audit.
Loading
Loading