diff --git a/docs/GLM53_DFLASH7_PYTHON_OVERLAY_SPARKCACHE_TP4_QUICKSTART.md b/docs/GLM53_DFLASH7_PYTHON_OVERLAY_SPARKCACHE_TP4_QUICKSTART.md index 03d26527..48622405 100644 --- a/docs/GLM53_DFLASH7_PYTHON_OVERLAY_SPARKCACHE_TP4_QUICKSTART.md +++ b/docs/GLM53_DFLASH7_PYTHON_OVERLAY_SPARKCACHE_TP4_QUICKSTART.md @@ -1,8 +1,35 @@ # Serve GLM-5.3 with external DFlash7 and the exact Python-overlay runtime -Status: **implemented, not qualified** for the selected source contract. The -image builder, profile resolver, and four-rank dry-run contract pass without -GPUs. Historical local image ID +Status: **implemented** for reproducible image construction, profile +resolution, and four-rank dry-run planning from the selected source contract. +Exact local image +`sparkring-glm53-sparkcache:dflash7-pr39-reaching-d93cb3d-arm64`, image ID +`sha256:ed60be066d6d9eadea267bc4597a0687869f3ddb95a3e5c6f86649893a838eb8`, +is **qualified** only for the bounded cases in the +[exact-artifact validation record](../performance/records/glm53-flash/pr146-recurrent-publication-live-validation.md) +from [pull request #147](https://github.com/FujitsuPolycom/sparkring/pull/147). +It binds SparkRing `d93cb3d98305041081cf572521602625185112ae` and +SparkCache `65b6642df1afc64366430d3aef9aca01f5c5e1c3`. + +The exact image completed the fixed semantic canary, a clean-restart +8,192-token persistent restore, a 131,072-token persistent restore, tail-only +copy-on-write publication from 131,072 to 262,144 tokens, a verified +262,144-token persistent restore, and one C16 cohort in which 16 distinct +request tails shared one restored 131,072-token segment. Restore timings are +**research-only**. The fixed semantic canary is not a DFlash response-quality +benchmark, so response quality is **unsupported**. Public OCI publication is +also **unsupported**; the image has no published digest. No rebuild inherits +these observations without its own exact evidence. + +The `ed60...` artifact uses `block_pages_v1`. Its C16 case proves that one +exact 131,072-token prefix was restored once for requests with 16 distinct +tails. It does not exercise `per_token_rows` different-root descriptor-segment +coalescing. That behavior is implemented with GPU-free coverage only and is +absent from this artifact's live qualification. + +### Separate historical artifact + +Historical local image ID `sha256:eef863d8bc578815a80b0e2d9f0d745102b6363415225101fd92171a2e5a55cb` is **qualified** only for the TP4/DCP1 startup, health, semantic generation, arbitrary page-boundary replay, and 131,072- and 262,144-token restore cases @@ -20,7 +47,7 @@ identity and requires its own live checks. | vLLM native extensions and wheel metadata | `da4d7be6c97434f6942292ed8abbf4b32dc44355` | | vLLM Python source | `0b67266a0f37d6146a8403fb8482403c62f412d5`, tree `ba9484ccb33aa56e90ff2f447f15ca9b9da97639` | | B12X | `b1d541f9e71a35f030d45fae437630fff7507c2a`, tree `c69cdec1c59a08e8e0e549f930fa8abcfb5134ae` | -| SparkCache shared-segment restore, tail-only copy-on-write publication, canonical CUDA configuration, and bounded page-delta reads | `65b6642df1afc64366430d3aef9aca01f5c5e1c3`, tree `41ad0a119ba109fd28900a2dcc9f9b4d8c293809`, clean source SHA-256 `a2add45a9f97446f6c2a843355161da9a5499ff7501b4750d2163591785d7345` | +| SparkCache exact prefix restore, GPU-free row-descriptor coalescing, tail-only copy-on-write publication, canonical CUDA configuration, and bounded page-delta reads | `65b6642df1afc64366430d3aef9aca01f5c5e1c3`, tree `41ad0a119ba109fd28900a2dcc9f9b4d8c293809`, clean source SHA-256 `a2add45a9f97446f6c2a843355161da9a5499ff7501b4750d2163591785d7345` | | Recurrent replay-boundary producer | Patch SHA-256 `5a6561a5bbab990dcd03bfd6a485ea26c3b5a578c2fd61b76305767b16dbfba0`; produces the four postimages accepted by SparkCache lease contract SHA-256 `8adbdfa3fd4b06b213c3aab45255a0b039f1c9940a4b1fad0efd004d263227c9` | | DFlash draft-loader separation | Patch SHA-256 `39b567013ee7aed79f63200ed460129587933dc77fb430decdf19f78178de279`, postimage SHA-256 `98acbae2b3bb4482d83f9637c163ce7c92707ccdf6561b7e431f23337f151cf4` | | Unused DeepEP removal | Distribution `deep_ep==2.0.0+local`, removal receipt SHA-256 `65514f44829e7d176b0b2cacc9559ed22724e525b7041a8bcd4d2e02d1f372e3` | @@ -68,7 +95,7 @@ Two profiles share the same image and DFlash7 cache identity: | Profile | Status | Loader behavior | |---|---|---| | `glm53-flash-dflash7-python-overlay-safetensors-sparkcache-tp4-dcp1.example.json` | **implemented**, not qualified | Uses global safetensors for target and draft. This follows the qualified-compatible loader shape but still requires live qualification on the composed 0b image. | -| `glm53-flash-dflash7-python-overlay-fastsafetensors-sparkcache-tp4-dcp1.example.json` | **implemented**, not qualified for the selected source contract | Uses global fastsafetensors with queue size one for the target and `draft_load_config={"load_format":"safetensors"}` for DFlash. The historical receipt remains evidence only for its recorded image. | +| `glm53-flash-dflash7-python-overlay-fastsafetensors-sparkcache-tp4-dcp1.example.json` | **qualified** only for exact image `sha256:ed60be066d6d9eadea267bc4597a0687869f3ddb95a3e5c6f86649893a838eb8` and the bounded cases above; rebuilds are not qualified | Uses global fastsafetensors with queue size one for the target and `draft_load_config={"load_format":"safetensors"}` for DFlash. | The image applies an exact-input vLLM patch that passes `SpeculativeConfig.draft_load_config` to the DFlash model loader. The image @@ -115,7 +142,8 @@ python scripts/sparkring_generic_launcher.py \ plan ``` -`plan` is offline. Inspect every rank action before a lifecycle command. +`plan` is offline. Inspect every rank action before a command that changes +containers. ## Start and observe the four-rank service @@ -179,8 +207,9 @@ different cache root and one-shot clear token so loader observations remain isolated. The pinned SparkCache source combines canonical CUDA configuration names, -authenticated shared-segment restore, an eight-worker ordered page-delta -reader, and tail-only copy-on-write publication. Cache identities, digest +authenticated exact prefix restore, GPU-free row-descriptor coalescing, an +eight-worker ordered page-delta reader, and tail-only copy-on-write +publication. Cache identities, digest salts, 256-token chunk geometry, page-delta wire bytes, and the CUDA placement ABI are unchanged. The source change does not change the namespace, so compatible `page-tail-cow-v1` entries remain eligible. diff --git a/performance/receipts/README.md b/performance/receipts/README.md index 22cf9aa6..f355f819 100644 --- a/performance/receipts/README.md +++ b/performance/receipts/README.md @@ -14,6 +14,7 @@ qualified functional claim or a research-only observation. | [`glm-3.5bpw/temp1/`](glm-3.5bpw/temp1/) | 10 four-Spark TP4/DCP4 sustained-decode and Coding Peak receipts | | [`glm53-flash/sparkcache-dflash2-bf16-tp4-20260828/`](glm53-flash/sparkcache-dflash2-bf16-tp4-20260828/) | Sanitized post-restore semantic canary for the TP4/DCP1 SparkCache validation | | [`glm53-flash/sparkcache-dflash2-bf16-tp4-20260829/`](glm53-flash/sparkcache-dflash2-bf16-tp4-20260829/) | One accepted 16K prefill and C1 decode observation; capacity-limited C4 and C8 cells are excluded | +| [`glm53-flash/adaptive-mtp-vs-dflash7-20260829/`](glm53-flash/adaptive-mtp-vs-dflash7-20260829/) | Research-only server-log diagnostic; prompt and output receipts are absent | | [`glm53-flash/sparkcache-dflash2-bf16-tp4-20g-20260829/`](glm53-flash/sparkcache-dflash2-bf16-tp4-20g-20260829/) | One accepted functional observation with 20 GiB of GPU KV memory per rank | | [`qwen38-27b/temp1/`](qwen38-27b/temp1/) | 13 two-Spark and 16 four-Spark accepted prefill, decode, and Coding Peak receipts | diff --git a/performance/receipts/glm53-flash/adaptive-mtp-vs-dflash7-20260829/diagnostic.json b/performance/receipts/glm53-flash/adaptive-mtp-vs-dflash7-20260829/diagnostic.json new file mode 100644 index 00000000..cc20da83 --- /dev/null +++ b/performance/receipts/glm53-flash/adaptive-mtp-vs-dflash7-20260829/diagnostic.json @@ -0,0 +1,96 @@ +{ + "schema": "sparkring-glm53-decode-log-diagnostic/v1", + "status": "research-only", + "target_model": { + "repository": "local-inference-lab/GLM-5.3-Flash-NVFP4", + "revision": "520de24eabf507659eaef7c70f14fd584527facc", + "cache_identity_sha256": "a35e6bf2875c1875609b8deaec404c07c6cc80259e4222fc0b51e649498bd6b9" + }, + "topology": { + "systems": 4, + "tensor_parallel_size": 4, + "decode_context_parallel_size": 1, + "pipeline_parallel_size": 1 + }, + "adaptive_mtp": { + "container": "glm53-flash-public-python-overlay-mtp5-adaptive-fastsafetensors-sparkcache-tp4-r0", + "image_id": "sha256:d209cf986c1f14f320e53d8b425c5e3a255eef9320f25b632af50f5b5c977314", + "served_model": "glm-5.3-flash-nvfp4-python-overlay-0b67266-on-da4d7be-b12x-b1d541f-mtp5-adaptive-tp4", + "vllm_compiled_commit": "da4d7be6c97434f6942292ed8abbf4b32dc44355", + "vllm_python_commit": "0b67266a0f37d6146a8403fb8482403c62f412d5", + "b12x_commit": "b1d541f9e71a35f030d45fae437630fff7507c2a", + "sparkcache_commit": "20838ace3ebda570ca039cb7f1976c29da554b39", + "sparkring_revision": "914c94d084d6881e90660305dedaa410ef02b167", + "draft_identity_sha256": "2e06d909ce5bb71c0c0e3e8be74a70e3b41d92ba4c30196cfb0957fb812acef6", + "maximum_depth": 5, + "initial_depth": 3, + "observation_window_steps": 32, + "log_interval_utc": { + "start": "2026-08-30T03:23:25Z", + "end": "2026-08-30T03:25:15Z" + }, + "observations": [ + {"timestamp": "2026-08-30T03:23:25Z", "generation_tokens_per_second": 38.7, "depth": 4, "draft_acceptance_percent": 74.2}, + {"timestamp": "2026-08-30T03:23:35Z", "generation_tokens_per_second": 30.7, "depth": 3, "draft_acceptance_percent": 65.9}, + {"timestamp": "2026-08-30T03:23:45Z", "generation_tokens_per_second": 34.1, "depth": 3, "draft_acceptance_percent": 62.2}, + {"timestamp": "2026-08-30T03:23:55Z", "generation_tokens_per_second": 29.5, "depth": 4, "draft_acceptance_percent": 77.0}, + {"timestamp": "2026-08-30T03:24:05Z", "generation_tokens_per_second": 34.6, "depth": 4, "draft_acceptance_percent": 72.4}, + {"timestamp": "2026-08-30T03:24:15Z", "generation_tokens_per_second": 31.4, "depth": 3, "draft_acceptance_percent": 59.4}, + {"timestamp": "2026-08-30T03:24:25Z", "generation_tokens_per_second": 36.3, "depth": 3, "draft_acceptance_percent": 67.5}, + {"timestamp": "2026-08-30T03:24:35Z", "generation_tokens_per_second": 36.8, "depth": 4, "draft_acceptance_percent": 72.0}, + {"timestamp": "2026-08-30T03:24:45Z", "generation_tokens_per_second": 28.0, "depth": 3, "draft_acceptance_percent": 58.2}, + {"timestamp": "2026-08-30T03:24:55Z", "generation_tokens_per_second": 24.5, "depth": 2, "draft_acceptance_percent": 48.9}, + {"timestamp": "2026-08-30T03:25:05Z", "generation_tokens_per_second": 31.9, "depth": 3, "draft_acceptance_percent": 75.1}, + {"timestamp": "2026-08-30T03:25:15Z", "generation_tokens_per_second": 37.1, "depth": 4, "draft_acceptance_percent": 81.2} + ], + "sparkcache_events_in_interval": 0 + }, + "dflash7": { + "container": "glm53-flash-dflash2-bf16-sparkcache-public-tp4-r0-dflash7-stopped-20260829", + "rank_image_ids": [ + "sha256:80b407904d21211d9a6b435c8ee4292ce66bfebcf802d94deb6605cdbc94d648", + "sha256:de4e219db4bdc2dfe4a2bcf2d5adb11655da22b5d40bf42494028c5ed128ea83", + "sha256:b5c29a1fc97fe7e086b62cffd832d90b292dc4f60cbf10c5593ef2f3debea365", + "sha256:9aeaf31b13af8a940734e31ec2963ce2714db535430ce7953a8baa5cf6dd4401" + ], + "served_model": "glm-5.3-flash-nvfp4-dflash7-bf16-tp4", + "vllm_commit": "da4d7be6c97434f6942292ed8abbf4b32dc44355", + "b12x_commit": "2fcf23a0ce269be27b2e03fece73d46e90e6aeea", + "sparkcache_commit": "2b86fb9d02fa3595cca5caa864b81aedce44b8bb", + "draft_repository": "incoai/GLM-5.3-Flash-DFlash2", + "draft_revision": "dc77ff1c99eeb2df044ee3d4f0094eb033fee410", + "draft_weights_sha256": "b33c03475ba7322cf398828f2d8d1be376df30dc05c6b40c28c8ea8da23e410b", + "depth": 7, + "log_interval_utc": { + "start": "2026-08-29T21:13:57Z", + "end": "2026-08-29T21:15:07Z" + }, + "generation_tokens_per_second": [40.6, 48.2, 52.6, 94.9, 111.8, 69.8, 133.6, 136.0], + "sparkcache_events_in_interval": 0, + "post_switch_validation": { + "log_interval_utc": { + "start": "2026-08-30T04:02:49Z", + "end": "2026-08-30T04:04:09Z" + }, + "observations": [ + {"timestamp": "2026-08-30T04:02:49Z", "generation_tokens_per_second": 48.8, "mean_acceptance_length": 4.82, "draft_acceptance_percent": 54.6}, + {"timestamp": "2026-08-30T04:02:59Z", "generation_tokens_per_second": 61.3, "mean_acceptance_length": 4.61, "draft_acceptance_percent": 51.6}, + {"timestamp": "2026-08-30T04:03:09Z", "generation_tokens_per_second": 74.5, "mean_acceptance_length": 5.69, "draft_acceptance_percent": 67.0}, + {"timestamp": "2026-08-30T04:03:19Z", "generation_tokens_per_second": 60.9, "mean_acceptance_length": 4.54, "draft_acceptance_percent": 50.5}, + {"timestamp": "2026-08-30T04:03:29Z", "generation_tokens_per_second": 61.3, "mean_acceptance_length": 4.51, "draft_acceptance_percent": 50.1}, + {"timestamp": "2026-08-30T04:03:39Z", "generation_tokens_per_second": 56.3, "mean_acceptance_length": 4.33, "draft_acceptance_percent": 47.6}, + {"timestamp": "2026-08-30T04:03:49Z", "generation_tokens_per_second": 64.4, "mean_acceptance_length": 5.07, "draft_acceptance_percent": 58.1}, + {"timestamp": "2026-08-30T04:03:59Z", "generation_tokens_per_second": 72.1, "mean_acceptance_length": 5.30, "draft_acceptance_percent": 61.4}, + {"timestamp": "2026-08-30T04:04:09Z", "generation_tokens_per_second": 72.2, "mean_acceptance_length": 5.47, "draft_acceptance_percent": 63.9} + ], + "sparkcache_events_in_interval": 0 + } + }, + "user_reported_matched_coding_peak": { + "dflash7_tokens_per_second": 70.0, + "adaptive_mtp_tokens_per_second": 33.5, + "receipt_present": false + }, + "prompt_receipt_present": false, + "output_receipt_present": false +} diff --git a/performance/receipts/glm53-flash/dflash7-python-overlay-pr25/validation.json b/performance/receipts/glm53-flash/dflash7-python-overlay-pr25/validation.json new file mode 100644 index 00000000..e31e641f --- /dev/null +++ b/performance/receipts/glm53-flash/dflash7-python-overlay-pr25/validation.json @@ -0,0 +1,135 @@ +{ + "schema": "sparkring-glm53-dflash7-pr25-live-validation/v1", + "status": { + "artifact_construction": "implemented", + "startup_health": "qualified", + "semantic_smoke": "qualified", + "flat_restore": "qualified", + "page_delta_restore_correctness": "qualified", + "page_delta_restore_performance": "research-only", + "persistent_128k_restore": "qualified", + "shared_prefix_waves": "qualified", + "dflash_quality": "unsupported" + }, + "artifact": { + "image": "sparkring-glm53-sparkcache:dflash7-vllm-python-0b67266-native-da4d7be-b12x-b1d541f-arm64", + "image_id": "sha256:9faa36a9f37aee16d97ab9214ef3153b4d200121126e6b2dee5ebb63109fea18", + "oci_revision_label": "e2d92fdc7d0306d664d6fd9f296dc2adcaf0fe05", + "sparkring_revision": "e2d92fdc7d0306d664d6fd9f296dc2adcaf0fe05", + "vllm_compiled_revision": "da4d7be6c97434f6942292ed8abbf4b32dc44355", + "vllm_python_revision": "0b67266a0f37d6146a8403fb8482403c62f412d5", + "b12x_revision": "b1d541f9e71a35f030d45fae437630fff7507c2a", + "sparkcache_revision": "5d571018de5b63a9a90e5c11e6d6e86bbff4a957", + "sparkcache_tree": "e864ed9ad64f771188fdb59aa9738e348134d636", + "sparkcache_source_sha256": "f7c0565521fddeff7085e4cc08043cb8d1e2bde33abc67f83b8608a162d05b88", + "cuda_placement_library_sha256": "a2e495162bf3d58b01613cd82ac15c8e15031dd7d6de7299700d2c58d905ada8", + "dflash_loader_patch_sha256": "39b567013ee7aed79f63200ed460129587933dc77fb430decdf19f78178de279", + "dflash_loader_postimage_sha256": "98acbae2b3bb4482d83f9637c163ce7c92707ccdf6561b7e431f23337f151cf4" + }, + "conditions": { + "hardware": "four NVIDIA DGX Spark systems", + "topology": { + "tensor_parallel_size": 4, + "decode_context_parallel_size": 1, + "pipeline_parallel_size": 1 + }, + "target_loader": "fastsafetensors", + "draft_loader": "safetensors", + "draft_method": "dflash", + "draft_tokens": 7, + "draft_tensor_parallel_size": 4, + "kv_cache_dtype": "fp8", + "vllm_block_size": 256, + "max_num_seqs": 32, + "sparkcache_publication_schema": "tail-cow-v1", + "sparkcache_effective_publication_schema": "page-tail-cow-v1", + "sparkcache_config_surface": "PR25 legacy-key compatibility", + "sparkcache_accepted_legacy_keys": [ + "spark_cache_native_restore", + "spark_cache_native_library", + "spark_cache_native_library_sha256", + "spark_cache_native_arena_bytes", + "spark_cache_native_io_workers" + ] + }, + "measurements": { + "startup": { + "target_fastsafetensors_seconds": 69.52, + "draft_safetensors_seconds": 4.18, + "total_model_load_seconds": 79.27, + "warm_ready_seconds_approx": 190.0, + "healthy_ranks": 4, + "restart_count_by_rank": [0, 0, 0, 0], + "oom_killed_by_rank": [false, false, false, false] + }, + "semantic_smoke": { + "raw_completion_tokens": [2, 2], + "scope": "continued raw completion only; no quality score" + }, + "flat_restore": { + "restored_tokens": 11520, + "rank": 0, + "restore_milliseconds": 29.258 + }, + "page_delta_restore": { + "restored_tokens": 17152, + "restore_milliseconds": 703.826, + "restore_read_milliseconds": 579.259, + "chunk_count": 67, + "type_error_observed": false + }, + "prime_128k": { + "request_seconds": 55.522, + "completion_token": 13, + "snapshot_milliseconds": 2440.7, + "commit_milliseconds": 1687.7 + }, + "unrelated_sentinel": { + "completion_token": 271 + }, + "persistent_restore_128k": { + "restored_tokens": 131072, + "bytes_per_rank": 813068464, + "rank_restore_milliseconds": { + "minimum": 123.69, + "maximum": 153.253 + }, + "logical_tokens_per_second_at_slowest_rank": 855265.476 + }, + "shared_prefix_waves": [ + { + "concurrency": 2, + "wall_seconds": 0.808, + "http_200": 2, + "completion_token": 13, + "restore_events_observed": 1, + "retention_observed": true + }, + { + "concurrency": 8, + "wall_seconds": 1.506, + "http_200": 8, + "completion_token": 13, + "restore_events_observed": 1, + "retention_observed": true + }, + { + "concurrency": 16, + "wall_seconds": 1.781, + "http_200": 16, + "completion_token": 13, + "restore_events_observed": 1, + "retention_observed": true + } + ] + }, + "limitations": [ + "The 7168-to-12032 base-geometry incompatibility is addressed only in draft SparkCache PR28 and remains present in this artifact.", + "Null-block publication rejection at 6912 tokens remains under investigation.", + "Page-delta read and reconstruction latency is too high for a performance claim.", + "No DFlash quality benchmark was run on this artifact.", + "One observed wave per concurrency does not establish latency variability, throughput, or soak behavior.", + "The collection timestamp, complete command transcript, and independent clock audit are not part of this bounded receipt.", + "This receipt qualifies only image sha256:9faa36a9f37aee16d97ab9214ef3153b4d200121126e6b2dee5ebb63109fea18; it does not qualify image sha256:eef863d8bc578815a80b0e2d9f0d745102b6363415225101fd92171a2e5a55cb or image sha256:ed60be066d6d9eadea267bc4597a0687869f3ddb95a3e5c6f86649893a838eb8." + ] +} diff --git a/performance/receipts/glm53-flash/pr146-recurrent-publication/validation.json b/performance/receipts/glm53-flash/pr146-recurrent-publication/validation.json new file mode 100644 index 00000000..46f67892 --- /dev/null +++ b/performance/receipts/glm53-flash/pr146-recurrent-publication/validation.json @@ -0,0 +1,337 @@ +{ + "schema": "sparkring-glm53-pr146-live-validation/v2", + "normalization": { + "supersedes_schema": "sparkring-glm53-pr146-live-validation/v1", + "reason": "The C16 observation is classified as shared exact prefix reuse for block_pages_v1; it does not demonstrate per_token_rows different-root descriptor coalescing.", + "raw_measurements_changed": false + }, + "status": { + "artifact_construction": "implemented", + "construction_verification_before_container_creation": "qualified", + "four_rank_process_health": "qualified", + "semantic_canary": "qualified", + "persistent_8k_restore_after_restart": "qualified", + "persistent_128k_restore": "qualified", + "tail_publication_128k_to_256k": "qualified", + "persistent_256k_restore_correctness": "qualified", + "shared_exact_prefix_c16": "qualified", + "different_root_row_descriptor_coalescing_live_validation": "unsupported", + "restore_performance": "research-only", + "continuous_availability_during_replacement": "unsupported", + "dflash_response_quality": "unsupported" + }, + "artifact": { + "image": "sparkring-glm53-sparkcache:dflash7-pr39-reaching-d93cb3d-arm64", + "image_id": "sha256:ed60be066d6d9eadea267bc4597a0687869f3ddb95a3e5c6f86649893a838eb8", + "published_digest": null, + "image_receipt_sha256": "2c4a02efe91df5de21c5e3c92f65710b7d41680f25c22baed43ca96c1e5a51d3", + "oci_revision_label": "d93cb3d98305041081cf572521602625185112ae", + "sparkring_revision": "d93cb3d98305041081cf572521602625185112ae", + "sparkring_tree": "867c43d0107856c3ba43500912462008ba149cc8", + "sparkcache_pull_request": 39, + "sparkcache_revision": "65b6642df1afc64366430d3aef9aca01f5c5e1c3", + "sparkcache_tree": "41ad0a119ba109fd28900a2dcc9f9b4d8c293809", + "sparkcache_source_sha256": "a2add45a9f97446f6c2a843355161da9a5499ff7501b4750d2163591785d7345", + "sparkcache_vllm_contract_sha256": "8adbdfa3fd4b06b213c3aab45255a0b039f1c9940a4b1fad0efd004d263227c9", + "sparkcache_cuda_placement_library_sha256": "d57509052b73853bcc8e3c3f47bb81748d87b9cbd8d908fc20d4c79a09aa400c", + "vllm_compiled_revision": "da4d7be6c97434f6942292ed8abbf4b32dc44355", + "vllm_python_revision": "0b67266a0f37d6146a8403fb8482403c62f412d5", + "vllm_python_tree": "ba9484ccb33aa56e90ff2f447f15ca9b9da97639", + "b12x_revision": "b1d541f9e71a35f030d45fae437630fff7507c2a", + "b12x_tree": "c69cdec1c59a08e8e0e549f930fa8abcfb5134ae", + "recurrent_boundary_patch_sha256": "5a6561a5bbab990dcd03bfd6a485ea26c3b5a578c2fd61b76305767b16dbfba0", + "recurrent_publication_patch_sha256": "587fc332917a8ffd5a29712dc5253d51e6051eca1166ed4a165e576a84f2e300", + "source_receipt_sha256": "611c88a48d30aae933828c6938dea2790f841ceeb05adbb80721d738cc029085" + }, + "predecessor_artifact": { + "purpose": "Corroborating observations for the same connector publication-target API before the reaching-allocation correction; results do not qualify the final image.", + "image": "sparkring-glm53-sparkcache:dflash7-pr39-boundary-8a887be-arm64", + "image_id": "sha256:1b4e58dc0999292da34d7418688b2b7f745a5b4d06e048ceb19f06f9d63a1185", + "sparkring_revision": "8a887bebefa4bfcc0b47fc24de34a986b042fb29", + "sparkcache_revision": "b830c84d93a80db869e0cfeed433f330bd611a7b", + "sparkcache_source_sha256": "09b74cf425c5a4f6149cc9e9c518a50a679996c9abe2e6d176a3fd95ff66250a", + "sparkcache_vllm_contract_sha256": "9d5c9a4c4d4efdc56560d63135f5e85a1e083c2bf635b52a8ba1ad2ab86d4da8", + "recurrent_publication_patch_sha256": "5c8102866b18ea7cb411fa3e5611700eb2d8020b6a3392d3d6053afaa04ace22" + }, + "conditions": { + "observed_at_utc": "2026-08-30T09:59:15Z/2026-08-30T11:06:44Z", + "hardware": "four NVIDIA DGX Spark systems", + "topology": { + "tensor_parallel_size": 4, + "decode_context_parallel_size": 1, + "pipeline_parallel_size": 1 + }, + "served_model": "glm-5.3-flash-nvfp4-dflash7-python-overlay-0b67266-on-da4d7be-b12x-b1d541f-tp4", + "target_repository": "local-inference-lab/GLM-5.3-Flash-NVFP4", + "target_revision": "520de24eabf507659eaef7c70f14fd584527facc", + "target_cache_identity_sha256": "a35e6bf2875c1875609b8deaec404c07c6cc80259e4222fc0b51e649498bd6b9", + "draft_repository": "incoai/GLM-5.3-Flash-DFlash2", + "draft_revision": "dc77ff1c99eeb2df044ee3d4f0094eb033fee410", + "draft_weights_sha256": "b33c03475ba7322cf398828f2d8d1be376df30dc05c6b40c28c8ea8da23e410b", + "target_loader": "fastsafetensors", + "target_loader_queue_size": 1, + "draft_loader": "safetensors", + "draft_method": "dflash", + "draft_tokens": 7, + "draft_tensor_parallel_size": 4, + "kv_cache_dtype": "fp8", + "kv_cache_bytes_per_rank": 21474836480, + "vllm_block_size_tokens": 256, + "max_num_seqs": 32, + "max_num_batched_tokens": 8192, + "sparkcache_publication_schema": "tail-cow-v1", + "sparkcache_effective_publication_schema": "page-tail-cow-v1", + "sparkcache_config_schema": "canonical-v1", + "sparkcache_cuda_restore_io_workers": 8, + "sparkcache_load_threads": 2, + "restore_storage_mode": "block_pages_v1", + "different_root_row_descriptor_coalescing": "implemented with GPU-free coverage; not exercised by this artifact" + }, + "measurements": { + "construction_verification_before_container_creation": { + "image_created_at": "2026-08-30T05:23:13.09226879-05:00", + "image_id": "sha256:ed60be066d6d9eadea267bc4597a0687869f3ddb95a3e5c6f86649893a838eb8", + "image_receipt_modified_at": "2026-08-30T05:23:18.699254136-05:00", + "image_receipt_bytes": 8690, + "image_receipt_sha256": "2c4a02efe91df5de21c5e3c92f65710b7d41680f25c22baed43ca96c1e5a51d3", + "container_created_at_by_rank": { + "0": "2026-08-30T10:46:32.100779355Z", + "1": "2026-08-30T10:46:32.552667746Z", + "2": "2026-08-30T10:46:32.57152575Z", + "3": "2026-08-30T10:46:32.079344046Z" + }, + "container_observed_started_at_by_rank": { + "0": "2026-08-30T11:02:10.85447775Z", + "1": "2026-08-30T11:02:22.761844485Z", + "2": "2026-08-30T11:02:22.561316447Z", + "3": "2026-08-30T11:02:22.135979337Z" + }, + "result": "The exact construction and source-contract receipt existed before every observed replacement container was created.", + "continuous_availability": "unsupported" + }, + "final_image_process_state_after_clean_relaunch": { + "image_id_on_all_ranks": "sha256:ed60be066d6d9eadea267bc4597a0687869f3ddb95a3e5c6f86649893a838eb8", + "container_state_by_rank": ["running", "running", "running", "running"], + "docker_healthcheck": "not configured", + "restart_count_by_rank": [0, 0, 0, 0], + "oom_killed_by_rank": [false, false, false, false] + }, + "semantic_canary": { + "prompt_sha256": "2db5eec56761ff3e65d2ffac7831166072631322bae9f6535c20766cc518f70c", + "content_sha256": "b3247575fe963313f8b2f298a0210161c66049bbdd7f3a69a7c3f4f62b8992c6", + "elapsed_seconds": 1.411, + "prompt_tokens": 27, + "completion_tokens": 45, + "finish_reason": "stop", + "semantic_match": true, + "validation_passed": true + }, + "persistent_restore_8k_after_clean_relaunch": { + "prompt_sha256": "a8569c46a6cbf22bae4736c897023f7c552952440bf75e1ef6ebabe594f513cf", + "request_id": "chatcmpl-9c60514089aed6f5-b033fd19", + "client_elapsed_seconds": 1.573, + "prompt_tokens": 8215, + "completion_tokens": 64, + "restored_tokens": 8192, + "page_bytes_per_rank": 103841965, + "rank_restore": { + "0": { + "end_to_end_ms": 66.550, + "service_ms": 66.397, + "restore_read_ms": 26.756, + "h2d_submit_ms": 0.869, + "cuda_sync_ms": 7.521, + "outcome": "verified" + }, + "1": { + "end_to_end_ms": 59.759, + "service_ms": 59.648, + "restore_read_ms": 26.125, + "h2d_submit_ms": 0.840, + "cuda_sync_ms": 7.257, + "outcome": "verified" + }, + "2": { + "end_to_end_ms": 60.924, + "service_ms": 60.792, + "restore_read_ms": 26.813, + "h2d_submit_ms": 0.909, + "cuda_sync_ms": 7.050, + "outcome": "verified" + }, + "3": { + "end_to_end_ms": 63.868, + "service_ms": 63.589, + "restore_read_ms": 29.189, + "h2d_submit_ms": 1.320, + "cuda_sync_ms": 7.485, + "outcome": "verified" + } + } + }, + "persistent_restore_128k": { + "request_id": "cmpl-a0a54340ddee4a20-0-922b327d", + "context_digest_prefix": "2010bc9db8fe", + "restored_tokens": 131072, + "page_bytes_per_rank": 813068464, + "logical_page_objects": 512, + "rank_restore": { + "0": {"end_to_end_ms": 285.283, "service_ms": 285.130, "restore_read_ms": 114.632, "outcome": "verified"}, + "1": {"end_to_end_ms": 160.515, "service_ms": 160.367, "restore_read_ms": 117.423, "outcome": "verified"}, + "2": {"end_to_end_ms": 154.233, "service_ms": 154.087, "restore_read_ms": 105.059, "outcome": "verified"}, + "3": {"end_to_end_ms": 216.350, "service_ms": 216.119, "restore_read_ms": 137.881, "outcome": "verified"} + } + }, + "tail_publication_128k_to_256k": { + "manifest_schema": "sparkcache-page-delta-manifest/v2", + "manifest_sha256": "f21221318441d809c7393582741ff896cec0cf82e5c35c34a6dabd3d5b77eba6", + "context_digest": "e532f048762ffec5bcc0c69283a3d5afacb652efbbf4267d34afab99737460b9", + "base_context_digest": "2010bc9db8fee25489d1580d1105cdcc819fa7a04c8edeffaf23d298e52452cd", + "base_committed_tokens": 131072, + "result_committed_tokens": 262144, + "base_root_objects": 512, + "delta_encoded_bytes": 826457677, + "delta_object_target_bytes": 67108864, + "delta_object_count": 13, + "last_delta_object_bytes": 21151309, + "delta_sha256": "3976f30ef3575e6f8e80b1f90f742d22c2cac30c7d428b96e69d4ca98bf5a41c", + "layout_sha256": "0d0b44eeb515963cf262f5d4d5b345caf794b85b1fa0cf11ad18576c1e8e7331", + "logical_chunk_tokens": 256 + }, + "persistent_restore_256k": { + "request_id": "cmpl-96421d674c60905b-0-8b3016dc", + "context_digest_prefix": "e532f048762f", + "restored_tokens": 262144, + "page_bytes_per_rank": 1575821491, + "logical_chunk_count": 1024, + "rank_restore": { + "0": {"end_to_end_ms": 7149.323, "service_ms": 7149.171, "restore_read_ms": 5642.991, "h2d_submit_ms": 1021.598, "cuda_sync_ms": 311.630, "outcome": "verified"}, + "1": {"end_to_end_ms": 6688.887, "service_ms": 6688.648, "restore_read_ms": 5205.596, "h2d_submit_ms": 999.986, "cuda_sync_ms": 309.115, "outcome": "verified"}, + "2": {"end_to_end_ms": 7206.548, "service_ms": 7206.445, "restore_read_ms": 5693.261, "h2d_submit_ms": 1094.960, "cuda_sync_ms": 310.021, "outcome": "verified"}, + "3": {"end_to_end_ms": 6738.472, "service_ms": 6738.307, "restore_read_ms": 5369.241, "h2d_submit_ms": 983.740, "cuda_sync_ms": 314.508, "outcome": "verified"} + } + }, + "shared_exact_prefix_c16": { + "scenario": "shared_exact_prefix", + "storage_mode": "block_pages_v1", + "cache_state": "hot", + "input_mode": "pretokenized", + "concurrency": 16, + "shared_prefix_tokens": 131072, + "prompt_tokens_per_request": 131198, + "distinct_prompt_sha256_count": 16, + "prompt_sha256": [ + "d5f1b3ab33b2f2f15edeedc2cc2eba72e5383a90643b45d63424cb327da66482", + "d815fcfce3972dce322b12b4834103c098f3edb0792b40af2aea5b50975e36f4", + "b743a059dbd6f40efc1d49436595d304b5bce4fa9ee9cc4a162e8bb4ce70771e", + "4efc76064841085a037719d26665fa2f2bdcdc8138193aa8c23fcb2a537e09a1", + "2e9c159903bf85804405f936f86e209f17c0b09feaacbb61c97cdce56a054efe", + "b44f9d32533c4695851124f26227e75db40c2dbcb8acfffd6ed8bed6ff57ea76", + "a0b7280567c81425abe52561810bf87f24a009edf5360e3b7729593c98b43a60", + "6a557ba1e08903be2e4a01b28603196df1d3cccfd4c28f8b19ada630dacec2eb", + "26b067c27b6b655e139bf23458482e83bfad1ae42feba0031daf7a5cb6ef4bca", + "1b587fce54df6a28d716f78a41dce3487992f59309b8cdda17682a0fa44b1cc9", + "8b7d00b764e3ebac0c441d06209bbf42037c814f9cef41569d1b790b6fc0810f", + "2bd40c188874f684846083ea51e5a7a72b332ae40d953b5c2990d795fdb446d5", + "2fc76ad6f02962844e34b08fa36ca51e1ea3551368567ffe7011e7548a7f2976", + "fafe33fdca12bea5739cf2a35a9a9007d9af1e4b68535cd6c12b6547fd7c518e", + "c2ccd604267c338f388572e00e496eb30d8b177209aa7709df97a1a8c4a638fd", + "0f0bde228ec8fc00651e7a8808c5afcfa594debfbc3d637ee4b2ddc8ff995b5a" + ], + "http_200": 16, + "succeeded": 16, + "unsuccessful": 0, + "request_elapsed_seconds": [ + 4.240340, + 4.243560, + 2.932669, + 4.243596, + 4.239915, + 4.242356, + 4.241536, + 4.240078, + 4.244709, + 4.243628, + 4.240034, + 4.242638, + 4.243659, + 4.243318, + 4.243736, + 4.244259 + ], + "aggregate": { + "minimum_seconds": 2.932669, + "p50_seconds": 4.242638, + "p95_seconds": 4.244709, + "maximum_seconds": 4.244709 + }, + "external_restore_events_per_rank": 1, + "external_restore_request_id": "cmpl-9ac4e6b523215233-0-9c41ccbc", + "external_restore": { + "restored_tokens": 131072, + "page_bytes_per_rank": 813068464, + "rank_restore": { + "0": {"end_to_end_ms": 400.689, "service_ms": 400.345, "restore_read_ms": 196.672, "h2d_submit_ms": 9.367, "cuda_sync_ms": 3.413, "outcome": "verified"}, + "1": {"end_to_end_ms": 447.798, "service_ms": 447.384, "restore_read_ms": 181.070, "h2d_submit_ms": 9.699, "cuda_sync_ms": 3.467, "outcome": "verified"}, + "2": {"end_to_end_ms": 455.789, "service_ms": 455.542, "restore_read_ms": 193.841, "h2d_submit_ms": 10.823, "cuda_sync_ms": 3.394, "outcome": "verified"}, + "3": {"end_to_end_ms": 360.578, "service_ms": 360.305, "restore_read_ms": 205.863, "h2d_submit_ms": 11.200, "cuda_sync_ms": 3.414, "outcome": "verified"} + } + }, + "recurrent_warning_count": 0 + }, + "predecessor_same_boundary_api": { + "scope": "Observations from image sha256:1b4e58dc0999292da34d7418688b2b7f745a5b4d06e048ceb19f06f9d63a1185; not transferred to the final image.", + "persistent_8k": { + "prime_client_seconds": 7.694, + "clean_postrestart_client_seconds": 1.744, + "prompt_tokens": 8215, + "publication_target_tokens": 8192, + "rank_restore_milliseconds_rounded_range": {"minimum": 59, "maximum": 67} + }, + "persistent_128k_cold_flat": { + "logical_page_objects": 512, + "rank_restore_seconds_observed_range": {"minimum": 3.39, "maximum": 4.15} + }, + "identical_prefix_waves": [ + { + "concurrency": 2, + "http_200": 2, + "request_elapsed_seconds": [4.860512, 4.667638], + "minimum_seconds": 4.667638, + "p50_seconds": 4.667638, + "p95_seconds": 4.860512, + "maximum_seconds": 4.860512 + }, + { + "concurrency": 8, + "http_200": 8, + "request_elapsed_seconds": [4.924467, 4.924104, 4.925342, 4.152364, 4.923964, 4.925845, 4.924800, 4.925012], + "minimum_seconds": 4.152364, + "p50_seconds": 4.924467, + "p95_seconds": 4.925845, + "maximum_seconds": 4.925845 + }, + { + "concurrency": 16, + "http_200": 16, + "request_elapsed_seconds": [6.205471, 6.205773, 6.207588, 4.393610, 6.205077, 6.204673, 6.205487, 6.207900, 6.206825, 6.202956, 6.208569, 6.208230, 6.204779, 6.205961, 6.207079, 6.204510], + "minimum_seconds": 4.393610, + "p50_seconds": 6.205487, + "p95_seconds": 6.208569, + "maximum_seconds": 6.208569 + } + ] + } + }, + "limitations": [ + "Restore timing values are single observations or single waves and are research-only; they do not establish throughput, variability, or soak behavior.", + "The 131072-token opaque base contains 512 per-page objects. The 131072-to-262144 tail is macro-grouped into 13 objects, but flat opaque snapshots are not macro-grouped.", + "The 262144-token restore reads 1024 logical chunks and took 6688.887 to 7206.548 milliseconds across ranks, so it is a correctness result rather than a speed claim.", + "The final image has one C16 shared exact prefix wave. It does not exercise per_token_rows different-root descriptor coalescing. C2, C8, and C16 identical-prefix waves were observed only on the predecessor image and do not qualify the final image.", + "The semantic canary verifies one fixed expected response. No scored DFlash response-quality benchmark was run.", + "The image has no published OCI digest and exists only on the observed deployment hosts.", + "The retained timestamps prove construction and source-contract verification preceded replacement-container creation. They do not prove uninterrupted availability of the previous service before or during replacement.", + "The prefill schedule interval remained at the profile default; --prefill-schedule-interval 8 was not tested.", + "A complete command transcript and sanitized raw HTTP response bodies were not retained with this record." + ] +} diff --git a/performance/records/glm53-flash/adaptive-mtp-vs-dflash7-decode-diagnostic-20260829.md b/performance/records/glm53-flash/adaptive-mtp-vs-dflash7-decode-diagnostic-20260829.md new file mode 100644 index 00000000..813f0194 --- /dev/null +++ b/performance/records/glm53-flash/adaptive-mtp-vs-dflash7-decode-diagnostic-20260829.md @@ -0,0 +1,109 @@ +# GLM-5.3 adaptive-MTP and DFlash7 decode-log diagnostic + +Status: **research-only**. This record preserves bounded server-log evidence. +It is not a benchmark or a controlled A/B comparison because the prompt, +generated output, and client timing receipt were not retained. +It does not qualify either runtime, any image, or any SparkCache behavior. + +## Conditions + +Both observations used four NVIDIA DGX Spark systems at TP4/DCP1/PP1 and the +target checkpoint +`local-inference-lab/GLM-5.3-Flash-NVFP4@520de24eabf507659eaef7c70f14fd584527facc`. + +The adaptive-MTP service used image ID +`sha256:d209cf986c1f14f320e53d8b425c5e3a255eef9320f25b632af50f5b5c977314`. +Its immutable composition identified: + +- retained compiled vLLM commit `da4d7be6c97434f6942292ed8abbf4b32dc44355`; +- Python vLLM commit `0b67266a0f37d6146a8403fb8482403c62f412d5`; +- B12X commit `b1d541f9e71a35f030d45fae437630fff7507c2a`; +- SparkCache commit `20838ace3ebda570ca039cb7f1976c29da554b39`; +- SparkRing revision `914c94d084d6881e90660305dedaa410ef02b167`; + and +- embedded-MTP draft identity + `2e06d909ce5bb71c0c0e3e8be74a70e3b41d92ba4c30196cfb0957fb812acef6`. + +Adaptive MTP used maximum depth five, initial depth three, and a 32-step +observation window. The observed rank-0 log interval was +`2026-08-30T03:23:25Z` through `2026-08-30T03:25:15Z`. + +The retained DFlash7 service used vLLM commit +`da4d7be6c97434f6942292ed8abbf4b32dc44355`, B12X commit +`2fcf23a0ce269be27b2e03fece73d46e90e6aeea`, and SparkCache commit +`2b86fb9d02fa3595cca5caa864b81aedce44b8bb`. Its external BF16 drafter was +`incoai/GLM-5.3-Flash-DFlash2@dc77ff1c99eeb2df044ee3d4f0094eb033fee410` +at fixed depth seven. Per-rank image IDs are preserved in the linked receipt. +The observed rank-0 log interval was `2026-08-29T21:13:57Z` through +`2026-08-29T21:15:07Z`. + +After the DFlash7 service was restored, a second rank-0 interval from +`2026-08-30T04:02:49Z` through `2026-08-30T04:04:09Z` recorded the same +fixed-depth-seven runtime under operator traffic. The prompt and output were +again not retained, so this interval is diagnostic evidence rather than a +controlled replay. + +The two compositions differ in vLLM Python, B12X, SparkCache, model loading, +and draft implementation. The observations therefore do not isolate one +changed variable. + +## Measurement + +The source is vLLM's rank-0 ten-second `Avg generation throughput` gauge and +the speculative-decoding metrics emitted at the same timestamps. The +machine-readable fixture preserves all 12 adaptive-MTP observations, the +eight earlier DFlash7 throughput observations, and nine post-switch DFlash7 +observations used here. + +Neither interval contains a `spark-context-cache` store, restore, or +maintenance event. SparkCache was enabled, but it was idle during the measured +decode windows. The cumulative cache-hit percentages printed beside the gauge +do not identify work performed during an individual interval. + +No client request shape, prompt digest, output-token sequence, or quality +result was retained. The displayed ranges are minima and maxima, not means or +confidence intervals. + +## Result + +| Runtime observation | Depth seen | Draft acceptance | Generation throughput | +|---|---:|---:|---:| +| Adaptive embedded MTP | 2–4 | 48.9–81.2% | 24.5–38.7 tok/s | +| Retained DFlash7, earlier interval | fixed 7 | not summarized for this record | 40.6–136.0 tok/s | +| Retained DFlash7, post-switch interval | fixed 7 | 47.6–67.0% | 48.8–74.5 tok/s | + +The operator also reported a matched coding-peak observation of approximately +70 tok/s for DFlash7 and 33.5 tok/s for adaptive MTP. No prompt or output +receipt accompanies those two values, so they are recorded as user-reported +context rather than measured evidence. + +## Conclusion + +The adaptive runtime changed draft depth between two and four while serving, +which confirms that acceptance-based control executed. During the recorded +windows, its server generation gauge remained below both retained DFlash7 +intervals. The post-switch interval independently reproduced the operator's +approximately 70 tok/s DFlash7 performance class. The evidence does not +establish a general DFlash7 advantage because the requests and outputs were +not preserved and the runtime compositions were not otherwise matched. + +SparkCache performed no logged work in either decode interval. These numbers +do not measure or implicate SparkCache restore, placement, publication, or +maintenance. + +## Limitations + +- The prompt, output token IDs, response text, sampling settings, and client + timing receipt are absent. +- Server gauges are periodic windows, not request-level benchmark results. +- The two runtime compositions differ in more than draft implementation. +- The observations have no repetitions or uncertainty calculation. +- High DFlash7 throughput could reflect an easy or repetitive output; output + quality cannot be reconstructed from throughput logs. +- The user-reported coding-peak pair is not independently auditable. + +## Provenance + +The [machine-readable diagnostic fixture](../../receipts/glm53-flash/adaptive-mtp-vs-dflash7-20260829/diagnostic.json) +contains exact identities, intervals, and transcribed observations. It marks +the missing prompt and output receipts explicitly. diff --git a/performance/records/glm53-flash/dflash7-python-overlay-pr25-live-validation.md b/performance/records/glm53-flash/dflash7-python-overlay-pr25-live-validation.md new file mode 100644 index 00000000..5eabed85 --- /dev/null +++ b/performance/records/glm53-flash/dflash7-python-overlay-pr25-live-validation.md @@ -0,0 +1,105 @@ +# GLM-5.3 DFlash7 Python-overlay bounded live validation + +Status: **qualified** only for the startup, health, semantic smoke, and exact +SparkCache restore cases described below. Page-delta restore performance is +**research-only**. DFlash response quality is **unsupported** by this record. +This record applies only to image +`sha256:9faa36a9f37aee16d97ab9214ef3153b4d200121126e6b2dee5ebb63109fea18`. +It does not qualify image +`sha256:eef863d8bc578815a80b0e2d9f0d745102b6363415225101fd92171a2e5a55cb` +or image +`sha256:ed60be066d6d9eadea267bc4597a0687869f3ddb95a3e5c6f86649893a838eb8`. + +## Conditions + +- Four NVIDIA DGX Spark systems; TP4, DCP1, and PP1. +- Local image ID + `sha256:9faa36a9f37aee16d97ab9214ef3153b4d200121126e6b2dee5ebb63109fea18` + on all four ranks. Its `org.opencontainers.image.revision` label and the + SparkRing source commit are + `e2d92fdc7d0306d664d6fd9f296dc2adcaf0fe05`. +- retained compiled vLLM components from + `da4d7be6c97434f6942292ed8abbf4b32dc44355`, vLLM Python source from + `0b67266a0f37d6146a8403fb8482403c62f412d5`, and B12X from + `b1d541f9e71a35f030d45fae437630fff7507c2a`. +- [SparkCache pull request 25](https://github.com/FujitsuPolycom/sparkcache/pull/25) + commit + `5d571018de5b63a9a90e5c11e6d6e86bbff4a957`, Git tree + `e864ed9ad64f771188fdb59aa9738e348134d636`, and clean deployable-source + SHA-256 + `f7c0565521fddeff7085e4cc08043cb8d1e2bde33abc67f83b8608a162d05b88`. +- GLM-5.3 NVFP4 target loaded with fastsafetensors. The external BF16 DFlash + checkpoint loaded with safetensors through the exact draft-loader patch. + Serving used seven speculative tokens, draft TP4, FP8 KV, 32 sequences, + and 256-token vLLM blocks. +- SparkCache publication schema `tail-cow-v1`, resolved internally to + `page-tail-cow-v1` for opaque GLM pages. +- The corrected launch translated the canonical CUDA restore settings to the + names accepted by the recorded SparkCache commit. The machine-readable + receipt retains the exact five accepted compatibility keys. + +The machine-readable identities, conditions, measurements, and scoped status +are retained in +[`validation.json`](../../receipts/glm53-flash/dflash7-python-overlay-pr25/validation.json). + +## Measurement + +Startup timings came from loader and readiness logs. Health checks examined all +four containers for HTTP health, restart count, and the runtime OOM-killed +flag. Two raw-completion requests checked that generation continued; they were +not scored for response quality. + +Restore timings came from SparkCache logs for one flat restore, one +reconstructed page-delta restore, and one persistent 128K-class restore. The +persistent case first primed the cache, issued an unrelated sentinel request, +and then restored 813,068,464 bytes per rank. One retained-prefix wave was +observed at each of concurrency 2, 8, and 16. Each wave recorded one restore +event followed by retained-prefix followers; it does not assert one restore +per request. Wall-clock and component timing values are single observations, +so no variability estimate is available. + +## Result + +| Check or measurement | Observed result | +|---|---| +| Target fastsafetensors load | 69.52 seconds | +| Draft safetensors load | 4.18 seconds | +| Total model load | 79.27 seconds | +| Warm readiness | approximately 190 seconds | +| Four-rank health | 4 healthy; restart counts `0,0,0,0`; OOM-killed flags `false,false,false,false` | +| Raw semantic smoke | completion tokens `2,2` | +| Flat restore | 11,520 tokens; rank 0; 29.258 ms | +| Reconstructed page-delta restore | 17,152 tokens; 703.826 ms total; 579.259 ms read; 67 chunks; no `TypeError` | +| 128K-class prime | 55.522 seconds; completion token 13; snapshot 2,440.7 ms; commit 1,687.7 ms | +| Unrelated sentinel | completion token 271 | +| Persistent restore | 813,068,464 bytes per rank; rank range 123.690-153.253 ms | +| Slowest-rank logical restore rate | 855,265 tokens/s (`131,072 / 0.153253`); ranks are shards of one context and are not summed | +| Concurrency 2 retained-prefix wave | 0.808 seconds; 2 HTTP 200 responses; every completion token 13; one restore observed | +| Concurrency 8 retained-prefix wave | 1.506 seconds; 8 HTTP 200 responses; every completion token 13; one restore observed | +| Concurrency 16 retained-prefix wave | 1.781 seconds; 16 HTTP 200 responses; every completion token 13; one restore observed | + +## Conclusion + +The exact four-rank image completed target fastsafetensors loading and +separate DFlash safetensors loading, stayed healthy, produced continued raw +completions, and restored the exact flat, reconstructed page-delta, persistent +128K-class, and retained-prefix cases recorded above. These observations +qualify those bounded cases only. The 703.826 ms page-delta observation shows +functional reconstruction without the former `TypeError`; it does not support +a page-delta performance claim. + +## Limitations + +- A 7,168-token base followed by a 12,032-token request can be rejected during + page reconstruction because the base geometry is incompatible. The correction + exists only in + [draft SparkCache pull request 28](https://github.com/FujitsuPolycom/sparkcache/pull/28) + and is absent from this image. +- Null-block publication rejection at 6,912 tokens remains under investigation. +- Page-delta read and reassembly are too slow for a performance qualification. +- No DFlash quality benchmark was run on this artifact. The two-token raw + completions prove continued generation, not answer quality. +- One wave at each concurrency does not establish latency variability, + throughput, soak behavior, or behavior beyond 16 concurrent requests. +- The retained evidence does not include a complete command transcript, + collection timestamp, or independent clock audit. diff --git a/performance/records/glm53-flash/pr146-recurrent-publication-live-validation.md b/performance/records/glm53-flash/pr146-recurrent-publication-live-validation.md new file mode 100644 index 00000000..e9619345 --- /dev/null +++ b/performance/records/glm53-flash/pr146-recurrent-publication-live-validation.md @@ -0,0 +1,180 @@ +# GLM-5.3 recurrent publication and exact prefix restore validation + +Status: **qualified** for the bounded functional checks recorded below on the +exact local image. Restore timing is **research-only**. DFlash response quality +is **unsupported** by this record. + +## Conditions + +The final artifact was the local ARM64 image +`sparkring-glm53-sparkcache:dflash7-pr39-reaching-d93cb3d-arm64`, image ID +`sha256:ed60be066d6d9eadea267bc4597a0687869f3ddb95a3e5c6f86649893a838eb8`. +It had no published OCI digest. The image construction receipt had SHA-256 +`2c4a02efe91df5de21c5e3c92f65710b7d41680f25c22baed43ca96c1e5a51d3`. +The builder created the image at +`2026-08-30T05:23:13.09226879-05:00` and completed that 8,690-byte receipt at +`2026-08-30T05:23:18.699254136-05:00`. + +The artifact bound these source contracts: + +| Component | Exact identity | +|---|---| +| SparkRing pull request | [#146](https://github.com/FujitsuPolycom/sparkring/pull/146) | +| SparkRing commit | `d93cb3d98305041081cf572521602625185112ae` | +| SparkRing tree | `867c43d0107856c3ba43500912462008ba149cc8` | +| SparkCache pull request | [#39](https://github.com/FujitsuPolycom/sparkcache/pull/39) | +| SparkCache commit | `65b6642df1afc64366430d3aef9aca01f5c5e1c3` | +| SparkCache tree | `41ad0a119ba109fd28900a2dcc9f9b4d8c293809` | +| SparkCache deployable-source SHA-256 | `a2add45a9f97446f6c2a843355161da9a5499ff7501b4750d2163591785d7345` | +| SparkCache vLLM contract SHA-256 | `8adbdfa3fd4b06b213c3aab45255a0b039f1c9940a4b1fad0efd004d263227c9` | +| SparkCache CUDA placement library SHA-256 | `d57509052b73853bcc8e3c3f47bb81748d87b9cbd8d908fc20d4c79a09aa400c` | +| vLLM retained compiled commit | `da4d7be6c97434f6942292ed8abbf4b32dc44355` | +| vLLM Python commit and tree | `0b67266a0f37d6146a8403fb8482403c62f412d5`, `ba9484ccb33aa56e90ff2f447f15ca9b9da97639` | +| B12X commit and tree | `b1d541f9e71a35f030d45fae437630fff7507c2a`, `c69cdec1c59a08e8e0e549f930fa8abcfb5134ae` | +| Recurrent-boundary patch SHA-256 | `5a6561a5bbab990dcd03bfd6a485ea26c3b5a578c2fd61b76305767b16dbfba0` | +| Recurrent-publication patch SHA-256 | `587fc332917a8ffd5a29712dc5253d51e6051eca1166ed4a165e576a84f2e300` | + +Serving used four NVIDIA DGX Spark systems at TP4, DCP1, and PP1. The target +checkpoint was `local-inference-lab/GLM-5.3-Flash-NVFP4` revision +`520de24eabf507659eaef7c70f14fd584527facc`. The external DFlash checkpoint was +`incoai/GLM-5.3-Flash-DFlash2` revision +`dc77ff1c99eeb2df044ee3d4f0094eb033fee410`. The target used +fastsafetensors with queue size one; the draft used safetensors. Serving used +seven speculative tokens, FP8 KV, 256-token vLLM blocks, an 8,192-token +prefill batch limit, and 32 maximum sequences. + +SparkCache used the requested `tail-cow-v1` publication schema, resolved to +`page-tail-cow-v1` for opaque GLM pages, canonical CUDA configuration, eight +CUDA restore I/O workers, and two load threads. The observation window was +2026-08-30 09:59:15 through 11:06:44 UTC. + +The live restore observations used `block_pages_v1`. The C16 case restored one +exact 131,072-token prefix for requests with distinct tails. It did not use +`per_token_rows` or exercise different-root row-descriptor coalescing. That +coalescing behavior is implemented with GPU-free coverage only and receives no +live qualification from this artifact. + +A second local artifact supplied corroborating evidence for the connector's +publication-target API before the scheduler explicitly limited publication +to the allocation that reaches the target. That image was +`sparkring-glm53-sparkcache:dflash7-pr39-boundary-8a887be-arm64`, image ID +`sha256:1b4e58dc0999292da34d7418688b2b7f745a5b4d06e048ceb19f06f9d63a1185`, +at SparkRing commit `8a887bebefa4bfcc0b47fc24de34a986b042fb29`. Its observations are labeled +as predecessor evidence below and do not qualify the final image. + +The exact identities, raw per-rank fields, request observations, manifest +fields, scoped status, and limitations are retained in +[`validation.json`](../../receipts/glm53-flash/pr146-recurrent-publication/validation.json). + +## Measurement + +Image labels and construction fields came from the image receipt. A read-only +container-state snapshot checked the exact image ID, process state, restart +count, and OOM-killed flag on each rank after the clean relaunch. The +containers had no Docker healthcheck, so this record calls that check process +health rather than HTTP health. + +The earliest observed replacement-container creation time was +`2026-08-30T10:46:32.079344046Z`; all four creation times and observed start +times are retained in the machine receipt. The exact construction and +source-contract receipt therefore existed before every observed replacement +container was created. No retained service-availability timeline proves that +the previous service remained continuously available, so this record does +not make that claim. + +The request harness retained prompt and response digests, token counts, HTTP +outcomes, and client elapsed time. The semantic canary compared one fixed +response with its expected content. Client time is included only as a +single-run observation because the retained output does not specify an +independently audited clock or warm-up policy. + +SparkCache emitted `sparkcache-restore-timing/v1` records on every rank. A +restore qualified only when every rank named the same request and span and +reported `outcome=verified`. The clean-restart case required a scheduler hit +for the persistent 8,192-token entry after all four containers restarted. + +The 131,072-to-262,144 publication was inspected from the committed rank-zero +manifest. Its file SHA-256 was checked before parsing. The v2 manifest named +the committed 131,072-token context as its base and the 262,144-token context +as its result. It represented the appended 128K tail as 826,457,677 encoded bytes in 13 +objects with a 64 MiB target size. The base remained a 512-object opaque page +root. + +The shared exact prefix case sent 16 concurrent requests with one identical +131,072-token prefix and 16 distinct tails. All 16 prompt SHA-256 values were +different. During that cohort, each rank emitted exactly one restore shard +for request `cmpl-9ac4e6b523215233-0-9c41ccbc`, and no second external restore +or recurrent-publication warning was observed. One logical external restore +therefore supplied the shared exact prefix; its four log entries were the TP4 rank +shards of that operation, not four independent restores. + +The predecessor artifact also ran an 8K prime and clean persistent replay, +one cold 128K flat restore, and identical-prefix waves at C2, C8, and C16. +Those measurements show the behavior of that exact artifact only. + +## Result + +| Check or observation | Result | +|---|---| +| Construction verification before container creation | exact receipt completed at `2026-08-30T05:23:18.699254136-05:00`; earliest replacement container created at `2026-08-30T10:46:32.079344046Z` | +| Final-image process state after clean relaunch | four ranks running the exact image; restart counts `0,0,0,0`; OOM-killed flags `false,false,false,false` | +| Semantic canary | semantic match; 27 prompt tokens; 45 completion tokens; 1.411 s client time | +| Clean-restart persistent 8K restore | 8,192 tokens and 103,841,965 bytes per rank; all ranks verified; 59.759-66.550 ms rank end-to-end; 1.573 s client time | +| Persistent 128K restore | 131,072 tokens and 813,068,464 bytes per rank; all ranks verified; 154.233-285.283 ms rank end-to-end | +| Tail-only 128K-to-256K publication | base 131,072; result 262,144; 826,457,677 delta bytes; 13 delta objects; 64 MiB target object size | +| Persistent 256K restore | 262,144 tokens; 1,575,821,491 page bytes; 1,024 logical chunks; all ranks verified; 6,688.887-7,206.548 ms rank end-to-end | +| Final-image C16 shared exact prefix | 16 distinct prompts; 16 HTTP 200 responses; one logical external 128K restore; 2.932669 s minimum, 4.242638 s p50, 4.244709 s p95/maximum client time | +| Predecessor clean-restart 8K | 8,192-token publication target; 59-67 ms rounded rank range; 1.744 s client time | +| Predecessor cold flat 128K | 512 opaque page objects; 3.39-4.15 s observed rank range | +| Predecessor identical-prefix C2 | 2 HTTP 200 responses; 4.667638-4.860512 s client range | +| Predecessor identical-prefix C8 | 8 HTTP 200 responses; 4.152364-4.925845 s client range | +| Predecessor identical-prefix C16 | 16 HTTP 200 responses; 4.393610-6.208569 s client range | + +## Conclusion + +The exact final image preserved semantic generation and completed verified +persistent restores at 8K after a clean restart, at 128K, and at 256K. The +committed 256K manifest referenced the existing 128K base and represented the +appended 128K tail with 13 delta objects, which is the expected tail-only copy-on-write +publication shape. + +The exact construction and source-contract receipt preceded creation of all +four observed replacement containers. This establishes construction +verification before the observed replacement, but it does not establish +continuous availability of the previous service. + +The final C16 shared exact prefix cohort completed 16 distinct requests after +one rank-sharded external restore of the common 128K prefix. This qualifies +exact prefix reuse for that one bounded cohort. It does not qualify +`per_token_rows` different-root row-descriptor coalescing and does not +establish a throughput or latency claim. + +The predecessor C2, C8, and C16 waves corroborate the connector API behavior +before the reaching-allocation correction. They are not evidence for the +final image and are not used to widen its qualification. + +## Limitations + +- Every restore timing and concurrency value is a single observation or one + wave. The values are research-only and do not establish variability, + throughput, or soak behavior. +- The 128K opaque base still consists of 512 per-page objects. The appended + 128K delta is grouped into 13 objects, but flat opaque snapshots are not + macro-grouped. +- The 256K restore read 1,024 logical chunks and took 6.689-7.207 seconds + across ranks. This is a correctness result, not a speed claim. +- The final image has one C16 shared exact prefix wave. Different-root + row-descriptor coalescing is implemented with GPU-free coverage only and was + not exercised. The C2, C8, and C16 + identical-prefix matrix belongs to the predecessor artifact and cannot be + transferred to the final image. +- The semantic canary checks one fixed expected response. No scored DFlash + response-quality benchmark was run. +- The image has no published OCI digest and exists only on the observed + deployment hosts. +- The retained timestamps do not establish that the previous service remained + continuously available before or during replacement. +- `--prefill-schedule-interval 8` was not tested; the profile default remained + in use. +- A complete command transcript and sanitized raw HTTP response bodies were + not retained with this record. diff --git a/performance/records/glm53-flash/test_dflash7_pr25_live_validation.py b/performance/records/glm53-flash/test_dflash7_pr25_live_validation.py new file mode 100644 index 00000000..31abc103 --- /dev/null +++ b/performance/records/glm53-flash/test_dflash7_pr25_live_validation.py @@ -0,0 +1,154 @@ +"""Validate the bounded GLM-5.3 DFlash7 live-evidence record.""" + +from __future__ import annotations + +import json +from pathlib import Path + + +ROOT = Path(__file__).resolve().parents[3] +RECEIPT_PATH = ( + ROOT + / "performance" + / "receipts" + / "glm53-flash" + / "dflash7-python-overlay-pr25" + / "validation.json" +) +RECORD_PATH = ( + ROOT + / "performance" + / "records" + / "glm53-flash" + / "dflash7-python-overlay-pr25-live-validation.md" +) +def _receipt() -> dict[str, object]: + return json.loads(RECEIPT_PATH.read_text(encoding="utf-8")) + + +def test_receipt_binds_exact_artifact_and_scoped_status() -> None: + receipt = _receipt() + assert receipt["schema"] == "sparkring-glm53-dflash7-pr25-live-validation/v1" + assert receipt["artifact"] == { + "image": "sparkring-glm53-sparkcache:dflash7-vllm-python-0b67266-native-da4d7be-b12x-b1d541f-arm64", + "image_id": "sha256:9faa36a9f37aee16d97ab9214ef3153b4d200121126e6b2dee5ebb63109fea18", + "oci_revision_label": "e2d92fdc7d0306d664d6fd9f296dc2adcaf0fe05", + "sparkring_revision": "e2d92fdc7d0306d664d6fd9f296dc2adcaf0fe05", + "vllm_compiled_revision": "da4d7be6c97434f6942292ed8abbf4b32dc44355", + "vllm_python_revision": "0b67266a0f37d6146a8403fb8482403c62f412d5", + "b12x_revision": "b1d541f9e71a35f030d45fae437630fff7507c2a", + "sparkcache_revision": "5d571018de5b63a9a90e5c11e6d6e86bbff4a957", + "sparkcache_tree": "e864ed9ad64f771188fdb59aa9738e348134d636", + "sparkcache_source_sha256": "f7c0565521fddeff7085e4cc08043cb8d1e2bde33abc67f83b8608a162d05b88", + "cuda_placement_library_sha256": "a2e495162bf3d58b01613cd82ac15c8e15031dd7d6de7299700d2c58d905ada8", + "dflash_loader_patch_sha256": "39b567013ee7aed79f63200ed460129587933dc77fb430decdf19f78178de279", + "dflash_loader_postimage_sha256": "98acbae2b3bb4482d83f9637c163ce7c92707ccdf6561b7e431f23337f151cf4", + } + assert receipt["status"] == { + "artifact_construction": "implemented", + "startup_health": "qualified", + "semantic_smoke": "qualified", + "flat_restore": "qualified", + "page_delta_restore_correctness": "qualified", + "page_delta_restore_performance": "research-only", + "persistent_128k_restore": "qualified", + "shared_prefix_waves": "qualified", + "dflash_quality": "unsupported", + } + assert receipt["conditions"]["sparkcache_config_surface"] == ( + "PR25 legacy-key compatibility" + ) + assert receipt["conditions"]["sparkcache_accepted_legacy_keys"] == [ + "spark_cache_native_restore", + "spark_cache_native_library", + "spark_cache_native_library_sha256", + "spark_cache_native_arena_bytes", + "spark_cache_native_io_workers", + ] + + +def test_receipt_preserves_startup_and_restore_observations() -> None: + measurements = _receipt()["measurements"] + assert measurements["startup"] == { + "target_fastsafetensors_seconds": 69.52, + "draft_safetensors_seconds": 4.18, + "total_model_load_seconds": 79.27, + "warm_ready_seconds_approx": 190.0, + "healthy_ranks": 4, + "restart_count_by_rank": [0, 0, 0, 0], + "oom_killed_by_rank": [False, False, False, False], + } + assert measurements["semantic_smoke"]["raw_completion_tokens"] == [2, 2] + assert measurements["flat_restore"] == { + "restored_tokens": 11520, + "rank": 0, + "restore_milliseconds": 29.258, + } + assert measurements["page_delta_restore"] == { + "restored_tokens": 17152, + "restore_milliseconds": 703.826, + "restore_read_milliseconds": 579.259, + "chunk_count": 67, + "type_error_observed": False, + } + assert measurements["persistent_restore_128k"] == { + "restored_tokens": 131072, + "bytes_per_rank": 813068464, + "rank_restore_milliseconds": {"minimum": 123.69, "maximum": 153.253}, + "logical_tokens_per_second_at_slowest_rank": 855265.476, + } + + +def test_each_retained_prefix_wave_records_one_restore() -> None: + waves = _receipt()["measurements"]["shared_prefix_waves"] + assert [(wave["concurrency"], wave["wall_seconds"]) for wave in waves] == [ + (2, 0.808), + (8, 1.506), + (16, 1.781), + ] + for wave in waves: + assert wave["http_200"] == wave["concurrency"] + assert wave["completion_token"] == 13 + assert wave["restore_events_observed"] == 1 + assert wave["retention_observed"] is True + + +def test_receipt_retains_required_limitations() -> None: + limitations = "\n".join(_receipt()["limitations"]) + for required in ( + "7168-to-12032", + "SparkCache PR28", + "6912 tokens", + "too high for a performance claim", + "No DFlash quality benchmark", + ): + assert required in limitations + + +def test_record_follows_evidence_method_and_links_receipt() -> None: + record = RECORD_PATH.read_text(encoding="utf-8") + for heading in ( + "## Conditions", + "## Measurement", + "## Result", + "## Conclusion", + "## Limitations", + ): + assert heading in record + assert ( + "../../receipts/glm53-flash/dflash7-python-overlay-pr25/validation.json" + in record + ) + assert "one restore\nevent followed by retained-prefix followers" in record + + +def test_receipt_does_not_transfer_qualification_to_other_images() -> None: + limitations = "\n".join(_receipt()["limitations"]) + assert "qualifies only image sha256:9faa36" in limitations + assert "sha256:eef863d8" in limitations + assert "sha256:ed60be06" in limitations + record = RECORD_PATH.read_text(encoding="utf-8") + assert "This record applies only to image" in record + assert "sha256:9faa36a9" in record + assert "sha256:eef863d8" in record + assert "sha256:ed60be06" in record diff --git a/performance/records/glm53-flash/test_pr146_recurrent_publication_live_validation.py b/performance/records/glm53-flash/test_pr146_recurrent_publication_live_validation.py new file mode 100644 index 00000000..dfaa0f74 --- /dev/null +++ b/performance/records/glm53-flash/test_pr146_recurrent_publication_live_validation.py @@ -0,0 +1,243 @@ +"""Validate the bounded PR146 recurrent-publication evidence record.""" + +from __future__ import annotations + +import json +from pathlib import Path + + +ROOT = Path(__file__).resolve().parents[3] +RECEIPT_PATH = ( + ROOT + / "performance" + / "receipts" + / "glm53-flash" + / "pr146-recurrent-publication" + / "validation.json" +) +RECORD_PATH = ( + ROOT + / "performance" + / "records" + / "glm53-flash" + / "pr146-recurrent-publication-live-validation.md" +) +QUICKSTART_PATH = ( + ROOT / "docs" / "GLM53_DFLASH7_PYTHON_OVERLAY_SPARKCACHE_TP4_QUICKSTART.md" +) + + +def _receipt() -> dict[str, object]: + return json.loads(RECEIPT_PATH.read_text(encoding="utf-8")) + + +def test_receipt_binds_exact_final_artifact_and_source_contracts() -> None: + receipt = _receipt() + assert receipt["schema"] == "sparkring-glm53-pr146-live-validation/v2" + assert receipt["normalization"] == { + "supersedes_schema": "sparkring-glm53-pr146-live-validation/v1", + "reason": ( + "The C16 observation is classified as shared exact prefix reuse for " + "block_pages_v1; it does not demonstrate per_token_rows different-root " + "descriptor coalescing." + ), + "raw_measurements_changed": False, + } + artifact = receipt["artifact"] + assert artifact["image_id"] == ( + "sha256:ed60be066d6d9eadea267bc4597a0687869f3ddb95a3e5c6f86649893a838eb8" + ) + assert artifact["published_digest"] is None + assert artifact["sparkring_revision"] == ( + "d93cb3d98305041081cf572521602625185112ae" + ) + assert artifact["sparkcache_revision"] == ( + "65b6642df1afc64366430d3aef9aca01f5c5e1c3" + ) + assert artifact["sparkcache_source_sha256"] == ( + "a2add45a9f97446f6c2a843355161da9a5499ff7501b4750d2163591785d7345" + ) + assert artifact["sparkcache_vllm_contract_sha256"] == ( + "8adbdfa3fd4b06b213c3aab45255a0b039f1c9940a4b1fad0efd004d263227c9" + ) + + +def test_status_scopes_functional_and_performance_claims() -> None: + status = _receipt()["status"] + for status_name in ( + "semantic_canary", + "persistent_8k_restore_after_restart", + "persistent_128k_restore", + "tail_publication_128k_to_256k", + "persistent_256k_restore_correctness", + "shared_exact_prefix_c16", + ): + assert status[status_name] == "qualified" + assert status["restore_performance"] == "research-only" + assert ( + status["different_root_row_descriptor_coalescing_live_validation"] + == "unsupported" + ) + assert status["continuous_availability_during_replacement"] == "unsupported" + assert status["dflash_response_quality"] == "unsupported" + + +def test_construction_receipt_precedes_replacement_container_creation() -> None: + ordering = _receipt()["measurements"][ + "construction_verification_before_container_creation" + ] + assert ordering["image_id"] == ( + "sha256:ed60be066d6d9eadea267bc4597a0687869f3ddb95a3e5c6f86649893a838eb8" + ) + assert ordering["image_receipt_sha256"] == ( + "2c4a02efe91df5de21c5e3c92f65710b7d41680f25c22baed43ca96c1e5a51d3" + ) + receipt_time = ordering["image_receipt_modified_at"] + assert receipt_time == "2026-08-30T05:23:18.699254136-05:00" + assert ordering["image_receipt_bytes"] == 8690 + assert all( + created > "2026-08-30T10:23:18.699254136Z" + for created in ordering["container_created_at_by_rank"].values() + ) + assert ordering["continuous_availability"] == "unsupported" + + +def test_clean_restart_8k_restore_is_verified_on_every_rank() -> None: + restore = _receipt()["measurements"]["persistent_restore_8k_after_clean_relaunch"] + assert restore["request_id"] == "chatcmpl-9c60514089aed6f5-b033fd19" + assert restore["restored_tokens"] == 8192 + assert restore["page_bytes_per_rank"] == 103841965 + assert restore["client_elapsed_seconds"] == 1.573 + assert { + rank: values["end_to_end_ms"] + for rank, values in restore["rank_restore"].items() + } == { + "0": 66.550, + "1": 59.759, + "2": 60.924, + "3": 63.868, + } + assert all( + values["outcome"] == "verified" + for values in restore["rank_restore"].values() + ) + + +def test_tail_manifest_proves_bounded_copy_on_write_shape() -> None: + manifest = _receipt()["measurements"]["tail_publication_128k_to_256k"] + assert manifest == { + "manifest_schema": "sparkcache-page-delta-manifest/v2", + "manifest_sha256": "f21221318441d809c7393582741ff896cec0cf82e5c35c34a6dabd3d5b77eba6", + "context_digest": "e532f048762ffec5bcc0c69283a3d5afacb652efbbf4267d34afab99737460b9", + "base_context_digest": "2010bc9db8fee25489d1580d1105cdcc819fa7a04c8edeffaf23d298e52452cd", + "base_committed_tokens": 131072, + "result_committed_tokens": 262144, + "base_root_objects": 512, + "delta_encoded_bytes": 826457677, + "delta_object_target_bytes": 67108864, + "delta_object_count": 13, + "last_delta_object_bytes": 21151309, + "delta_sha256": "3976f30ef3575e6f8e80b1f90f742d22c2cac30c7d428b96e69d4ca98bf5a41c", + "layout_sha256": "0d0b44eeb515963cf262f5d4d5b345caf794b85b1fa0cf11ad18576c1e8e7331", + "logical_chunk_tokens": 256, + } + + +def test_256k_restore_retains_all_rank_correctness_evidence() -> None: + restore = _receipt()["measurements"]["persistent_restore_256k"] + assert restore["restored_tokens"] == 262144 + assert restore["page_bytes_per_rank"] == 1575821491 + assert restore["logical_chunk_count"] == 1024 + assert { + rank: values["end_to_end_ms"] + for rank, values in restore["rank_restore"].items() + } == { + "0": 7149.323, + "1": 6688.887, + "2": 7206.548, + "3": 6738.472, + } + assert all( + values["outcome"] == "verified" + for values in restore["rank_restore"].values() + ) + + +def test_shared_exact_prefix_cohort_records_one_logical_restore() -> None: + receipt = _receipt() + shared = receipt["measurements"]["shared_exact_prefix_c16"] + assert shared["scenario"] == "shared_exact_prefix" + assert shared["storage_mode"] == "block_pages_v1" + assert receipt["conditions"]["restore_storage_mode"] == "block_pages_v1" + assert receipt["conditions"]["different_root_row_descriptor_coalescing"] == ( + "implemented with GPU-free coverage; not exercised by this artifact" + ) + assert shared["concurrency"] == 16 + assert shared["distinct_prompt_sha256_count"] == 16 + assert len(shared["prompt_sha256"]) == len(set(shared["prompt_sha256"])) == 16 + assert shared["http_200"] == 16 + assert shared["unsuccessful"] == 0 + assert len(shared["request_elapsed_seconds"]) == 16 + assert shared["external_restore_events_per_rank"] == 1 + assert shared["external_restore_request_id"] == ( + "cmpl-9ac4e6b523215233-0-9c41ccbc" + ) + assert set(shared["external_restore"]["rank_restore"]) == {"0", "1", "2", "3"} + assert shared["recurrent_warning_count"] == 0 + + +def test_predecessor_evidence_is_not_transferred_to_final_image() -> None: + receipt = _receipt() + assert receipt["predecessor_artifact"]["image_id"] == ( + "sha256:1b4e58dc0999292da34d7418688b2b7f745a5b4d06e048ceb19f06f9d63a1185" + ) + predecessor = receipt["measurements"]["predecessor_same_boundary_api"] + assert [wave["concurrency"] for wave in predecessor["identical_prefix_waves"]] == [ + 2, + 8, + 16, + ] + assert "not transferred" in predecessor["scope"] + limitations = "\n".join(receipt["limitations"]) + assert "do not qualify the final image" in limitations + + +def test_record_follows_evidence_method_and_rejects_speed_claim() -> None: + record = RECORD_PATH.read_text(encoding="utf-8") + for heading in ( + "## Conditions", + "## Measurement", + "## Result", + "## Conclusion", + "## Limitations", + ): + assert heading in record + assert "One logical external restore" in record + assert "not a speed claim" in record + assert "shared exact prefix cohort" in record + assert "does not qualify\n`per_token_rows` different-root" in record + assert "do not qualify the final image" in record + assert "`--prefill-schedule-interval 8` was not tested" in record + assert ( + "../../receipts/glm53-flash/pr146-recurrent-publication/validation.json" + in record + ) + + +def test_quickstart_links_the_exact_artifact_evidence_separately() -> None: + quickstart = QUICKSTART_PATH.read_text(encoding="utf-8") + assert ( + "sha256:ed60be066d6d9eadea267bc4597a0687869f3ddb95a3e5c6f86649893a838eb8" + in quickstart + ) + assert "d93cb3d98305041081cf572521602625185112ae" in quickstart + assert "65b6642df1afc64366430d3aef9aca01f5c5e1c3" in quickstart + assert "pr146-recurrent-publication-live-validation.md" in quickstart + assert "`block_pages_v1`" in quickstart + assert "`per_token_rows` different-root descriptor-segment" in quickstart + assert "rebuilds are not qualified" in quickstart + assert "### Separate historical artifact" in quickstart + assert ( + "sha256:eef863d8bc578815a80b0e2d9f0d745102b6363415225101fd92171a2e5a55cb" + in quickstart + ) diff --git a/performance/test_glm53_decode_log_diagnostic.py b/performance/test_glm53_decode_log_diagnostic.py new file mode 100644 index 00000000..afed8be6 --- /dev/null +++ b/performance/test_glm53_decode_log_diagnostic.py @@ -0,0 +1,82 @@ +import json +from pathlib import Path + + +RECEIPT = ( + Path(__file__).parent + / "receipts/glm53-flash/adaptive-mtp-vs-dflash7-20260829/diagnostic.json" +) +RECORD = ( + Path(__file__).parent + / "records/glm53-flash/adaptive-mtp-vs-dflash7-decode-diagnostic-20260829.md" +) + + +def test_glm53_decode_log_diagnostic_preserves_bounded_claims() -> None: + document = json.loads(RECEIPT.read_text(encoding="utf-8")) + assert document["schema"] == "sparkring-glm53-decode-log-diagnostic/v1" + assert document["status"] == "research-only" + adaptive = document["adaptive_mtp"] + observations = adaptive["observations"] + assert adaptive["log_interval_utc"] == { + "start": observations[0]["timestamp"], + "end": observations[-1]["timestamp"], + } + assert {item["depth"] for item in observations} == {2, 3, 4} + assert min(item["draft_acceptance_percent"] for item in observations) == 48.9 + assert max(item["draft_acceptance_percent"] for item in observations) == 81.2 + assert min(item["generation_tokens_per_second"] for item in observations) == 24.5 + assert max(item["generation_tokens_per_second"] for item in observations) == 38.7 + assert min(document["dflash7"]["generation_tokens_per_second"]) == 40.6 + assert max(document["dflash7"]["generation_tokens_per_second"]) == 136.0 + post_switch = document["dflash7"]["post_switch_validation"] + post_switch_observations = post_switch["observations"] + assert post_switch["log_interval_utc"] == { + "start": post_switch_observations[0]["timestamp"], + "end": post_switch_observations[-1]["timestamp"], + } + assert min( + item["generation_tokens_per_second"] for item in post_switch_observations + ) == 48.8 + assert max( + item["generation_tokens_per_second"] for item in post_switch_observations + ) == 74.5 + assert min( + item["draft_acceptance_percent"] for item in post_switch_observations + ) == 47.6 + assert max( + item["draft_acceptance_percent"] for item in post_switch_observations + ) == 67.0 + assert post_switch["sparkcache_events_in_interval"] == 0 + assert adaptive["sparkcache_events_in_interval"] == 0 + assert document["dflash7"]["sparkcache_events_in_interval"] == 0 + assert document["prompt_receipt_present"] is False + assert document["output_receipt_present"] is False + assert ( + document["user_reported_matched_coding_peak"]["receipt_present"] is False + ) + + +def test_glm53_decode_record_does_not_claim_qualification() -> None: + text = RECORD.read_text(encoding="utf-8") + assert "Status: **research-only**" in text + assert "does not qualify either runtime" in text + + +def test_glm53_decode_record_has_required_sections_and_live_receipt_link() -> None: + text = RECORD.read_text(encoding="utf-8") + for heading in ( + "## Conditions", + "## Measurement", + "## Result", + "## Conclusion", + "## Limitations", + "## Provenance", + ): + assert heading in text + linked_receipt = ( + RECORD.parent + / "../../receipts/glm53-flash/adaptive-mtp-vs-dflash7-20260829/diagnostic.json" + ).resolve() + assert linked_receipt == RECEIPT.resolve() + assert linked_receipt.is_file() diff --git a/scripts/test_prepare_glm53_dflash7_python_overlay_profile.py b/scripts/test_prepare_glm53_dflash7_python_overlay_profile.py index 3ed326fb..ed431a8f 100644 --- a/scripts/test_prepare_glm53_dflash7_python_overlay_profile.py +++ b/scripts/test_prepare_glm53_dflash7_python_overlay_profile.py @@ -39,6 +39,10 @@ ) SITE = CONFIG / "glm53-flash-tp4-site.example.yaml" GUIDE = ROOT / "docs/GLM53_DFLASH7_PYTHON_OVERLAY_SPARKCACHE_TP4_QUICKSTART.md" +QUALIFIED_IMAGE_ID = ( + "sha256:ed60be066d6d9eadea267bc4597a0687869f3ddb95a3e5c6f86649893a838eb8" +) +QUALIFIED_SPARKRING_COMMIT = "d93cb3d98305041081cf572521602625185112ae" DIGESTS = { "cuda_placement_library_sha256": "1a" * 32, "native_elf_manifest_sha256": "2b" * 32, @@ -273,6 +277,24 @@ def test_quickstart_names_both_loader_statuses_and_exact_builder() -> None: assert "runtime/glm53-flash-dflash7-python-overlay/build-image.sh" in guide assert FAST.name in guide and SAFE.name in guide assert "implemented" in guide and "qualified" in guide + assert QUALIFIED_IMAGE_ID in guide + assert QUALIFIED_SPARKRING_COMMIT in guide + assert SPARKCACHE_COMMIT in guide + assert "pr146-recurrent-publication-live-validation.md" in guide + assert "pull request #147" in guide + assert "clean-restart" in guide + assert "131,072 to 262,144 tokens" in guide + assert "16 distinct" in guide + assert "`block_pages_v1`" in guide + assert "`per_token_rows` different-root descriptor-segment" in guide + assert "absent from this artifact's live qualification" in guide + assert "rebuilds are not qualified" in guide + assert "twelve-file lease contract" in guide + assert "command that changes\ncontainers" in guide + assert "Restore timings are" in guide and "research-only" in guide + assert "response quality is **unsupported**" in guide + assert "Public OCI publication is" in guide + assert "Separate historical artifact" in guide assert "sha256:eef863d8bc578815a80b0e2d9f0d745102b6363415225101fd92171a2e5a55cb" in guide assert DFLASH_WEIGHTS_SHA256 in guide assert DFLASH_LOADER_PATCH_SHA256 in guide