FujitsuPolycom · sethforprivacy · Sep 11, 2026 · Sep 11, 2026 · Sep 12, 2026
@@ -39,7 +39,8 @@ per-request Context limit.
 | GLM-5.3 Flash NVFP4-Spark · native MTP3, switched | [Shared SparkRing source image](runtime/sparkring/source_image/README.md) | Operator-selected NCCL links | TP4/DCP1 | 1M | 16 | — | [Switched quickstart](docs/GLM53_SWITCHED_TP4_QUICKSTART.md) |
 | GLM-5.2 EXL3 3.5-bpw | [SparkRing vLLM/ExLlamaV3 build](runtime/exl3-r7/README.md) | [SIRCL + NCCL](docs/SIRCL.md) | TP4/DCP4 | 1M | 16 | ~1.2M | [Quickstart](docs/GLM52_35BPW_QUICKSTART.md) |
 | DeepSeek-V4-Flash-0731 | [SparkRing vLLM/B12X image](runtime/deepseek0731-gb10/README.md) | [Patched NCCL](spark_transport/nccl/README.md) | TP4/DCP1 | 1M | 32 | ~1M | [Quickstart](docs/DEEPSEEK_V4_FLASH_QUICKSTART.md) |
-| DeepSeek-V4.1-Flash · Engram on NVMe · DSpark k=5 | [Self-built stock vLLM `dsv41-feat` image](runtime/deepseek-v41-gb10/README.md) | [Patched NCCL](spark_transport/nccl/README.md) | TP4/DCP1 | 430,080 | 8 | ~2.18M | [Quickstart](docs/DEEPSEEK_V41_FLASH_QUICKSTART.md) |
+| DeepSeek-V4.1-Flash · SGLang decoder-tail replay | [Source-pinned SGLang image](runtime/deepseek-v41-sglang/README.md) | [Patched NCCL](spark_transport/nccl/README.md), in-place library mount | TP4/EP4 | 262,144 | 8 | ~1.50M | [Operator guide](runtime/deepseek-v41-sglang/README.md) |
+| DeepSeek-V4.1-Flash · vLLM fallback · DSpark k=5 | [Self-built stock vLLM `dsv41-feat` image](runtime/deepseek-v41-gb10/README.md) | [Patched NCCL](spark_transport/nccl/README.md) | TP4/DCP1 | 430,080 | 8 | ~2.18M | [Quickstart](docs/DEEPSEEK_V41_FLASH_QUICKSTART.md) |
 | Qwen3.8-27B EXL3 K5/K6 | [SparkRing vLLM/ExLlamaV3 build](runtime/qwen38/README.md) | [Patched NCCL](spark_transport/nccl/README.md) | TP4/DCP1 | 1M | 64 | — | [Quickstart](docs/QWEN38_27B_EXL3_K5K6_QUICKSTART.md) |
 | DeepSeek-V4-Flash-Vision-Exp with DSpark (research-only) | [Anemll image / MiaAI-Lab recipe](runtime/deepseek-vision-exp/profile.json) | [SparkRing patched NCCL](spark_transport/nccl/README.md) | TP4 | 1M | 48 | — | [Quickstart](docs/DEEPSEEK_V4_FLASH_VISION_EXP_TP4_QUICKSTART.md) |
 

@@ -418,3 +418,15 @@ with its pinned CUTLASS, CCCL and spdlog submodules. No FlashInfer source is inc
 `deepseek-ai/DeepSeek-V4.1-Flash` is distributed by DeepSeek under the MIT License. No weights or
 model files are included; operators fetch the checkpoint themselves.
 
+
+### 16. Mia DeepSeek V4.1 SGLang adapter (external build input)
+
+`runtime/deepseek-v41-sglang/build-image.sh` clones
+[MiaAI-Lab/DeepSeek-v4.1-Flash-DGX-Sparks](https://github.com/MiaAI-Lab/DeepSeek-v4.1-Flash-DGX-Sparks)
+at commit `e59e6eb67479aa68f6fa700c600dc90a0729b5ec` (AGPL-3.0). The adapter,
+`boot.py`, row store, and Engram packer remain in that external repository and
+the operator-built image; none of those files are vendored here. The external
+Dockerfile also carries SGLang's Apache-2.0 FlashMLA wrapper. SparkRing's auth
+patch targets the Apache-2.0 SGLang auth module in the pinned base image and
+preserves its surrounding source. Operators retain the external components'
+licenses and notices with their builds.
@@ -1,5 +1,10 @@
 # DeepSeek-V4.1-Flash four-Spark cycle quickstart
 
+The [SGLang cycle runtime](../runtime/deepseek-v41-sglang/README.md) adds
+decoder-tail prefill and file-backed multi-key authentication. Its broader
+deployment qualification is still in progress. This page documents the vLLM
+fallback and its existing measured settings.
+
 Serve `deepseek-ai/DeepSeek-V4.1-Flash` (the stock checkpoint) as four tensor-parallel
 ranks on a directly cabled four-Spark cycle, with the model's two Engram lookup tables
 left on each rank's NVMe.

@@ -1,5 +1,9 @@
 # DeepSeek-V4.1-Flash four-Spark cycle profile
 
+See the [SGLang cycle runtime](../../runtime/deepseek-v41-sglang/README.md) and
+[controlled decoder-replay comparison](../../performance/records/deepseek-v41-flash/sglang-decoder-replay-20260911.md)
+for the SGLang profile. The measurements below belong to the vLLM fallback.
+
 ## Status
 
 **Implemented and live-benchmarked on one private four-Spark cycle; not qualified.** The image

@@ -0,0 +1,89 @@
+# SGLang decoder-tail replay on a four-Spark cycle
+
+Status: **implemented**. This controlled comparison attributes the measured
+prefill improvement to decoder-tail replay. It does not qualify a larger
+context setting or unattended production deployment.
+
+## Conditions
+
+Four GB10 DGX Sparks, directly cabled as a four-node cycle, TP4/EP4, local NVMe
+checkpoint `deepseek-ai/DeepSeek-V4.1-Flash` at
+`dba1be0a40aa45a94ad051997016db3960a90277`, SparkRing patched NCCL 2.30.7 mounted
+in place over SGLang's pip NCCL. Both arms used the same locally built image,
+Mia commit `e59e6eb67479aa68f6fa700c600dc90a0729b5ec`, image ID
+`sha256:4252984d1cd642a51bd0cde06665a4f6a73726141dec760392a73cf968bce744`.
+The source/base identity is recorded in the [runtime pins](../../../runtime/deepseek-v41-sglang/pins.json).
+
+The four ranks were rebooted before each arm. Context 262144, prefill chunk
+4096, maximum eight requests, memory fraction 0.90, requested total-token pool
+1500000, DSpark block five without SPS/STS tables, b12x MXFP8 backend, MoE fused
+finalize disabled, expandable segments disabled, Engram NVMe packed layout,
+96 I/O threads and zero GiB row cache. Only the decoder replay flag differed.
+
+## Measurement
+
+Client wall-clock prompt throughput is server-reported prompt tokens divided
+by request wall time for a one-token response. Target prompt sizes are labels;
+actual token counts are recorded in the raw samples. Decode uses fixed-length
+256-token generation at C1 and C4. The prompt distribution and decode timing
+are separate from the multi-category vLLM benchmark; do not compare their
+absolute rates as though the workloads were identical.
+
+There was one observation per size/concurrency in each arm. A warm-up request
+preceded the ladder, but the first nominal 16K prefill still included additional
+initialization and is retained without being represented as steady-state
+performance. There is no repeated-run confidence interval. Quality capture
+used 20 fixed prompts, including four roughly 12–15K prompts, first-token top-20
+log probabilities, and 48-token greedy continuations. Truncated-distribution
+KL is a diagnostic, not a full-vocabulary divergence measure.
+
+Raw observations: [replay-off prefill](sglang-decoder-replay-20260911/prefill-sgl-replay0.json),
+[replay-on prefill](sglang-decoder-replay-20260911/prefill-sgl-replay1.json),
+[replay-off decode](sglang-decoder-replay-20260911/decode-sgl-replay0.json),
+[replay-on decode](sglang-decoder-replay-20260911/decode-sgl-replay1.json),
+[off quality](sglang-decoder-replay-20260911/sgl-replay0.json),
+[on quality](sglang-decoder-replay-20260911/sgl-replay1.json),
+[off needle](sglang-decoder-replay-20260911/needle-sgl-replay0.json), and
+[on needle](sglang-decoder-replay-20260911/needle-sgl-replay1.json).
+Endpoint addresses are sanitized; numerical observations are retained.
+
+## Result
+
+| Measurement | Replay off | Replay on |
+|---|---:|---:|
+| Nominal 64K prefill, tok/s | 2006 | 3144 |
+| Roughly 131K needle prefill, tok/s | 1861 | 2934 |
+| Needle TTFT, seconds | 70.0 | 44.4 |
+| C1 decode aggregate, tok/s | 35.4 | 37.4 |
+| C4 decode aggregate, tok/s | 80.5 | 76.3 |
+
+Both needle requests returned the exact passphrase. All 20 first-token choices
+and all 20 greedy continuations matched. Top-20 diagnostic KL mean was 0.006,
+maximum 0.12. These are the handoff's rounded summaries; raw quality distributions
+are retained for independent recomputation.
+
+## Conclusion
+
+Enabling decoder-tail replay increased nominal 64K prefill throughput by about
+57% in this comparison. With replay disabled, SGLang prefill was near the
+separately measured vLLM implementation's approximately 2K tok/s. The matched
+SGLang comparison isolates the replay flag; it does not attribute this gain to
+two-batch overlap or generic engine scheduling.
+
+The optimization retains all rows through layer 20, then executes layers 21–39
+on the final 128 extend tokens per request. For a full 4096-token chunk, the
+layer-token row count falls from 163840 to 88448, approximately 54% of the full
+path. This is a work-count illustration, not a FLOP or measured communication
+ratio. Smaller late-layer tensors reduce computation and collective payload.
+
+## Limitations
+
+The shortcut retains global KV/indexer memory but changes late-layer local SWA
+visibility. A finite matching prompt set does not prove mathematical equivalence
+to full prefill. Prompt-token log probabilities and arbitrary full-prompt hidden
+state capture are not supported by the shortcut. The A/B does not cover mixed
+batches, exact tail/chunk-boundary placement, sampled decoding, vision/tools,
+400K retrieval, or long soaks. Those gates belong to the ongoing deployment
+qualification and are not implied by this result. No communication profile was
+captured in this experiment. Absolute results apply to this image, hardware,
+transport, and workload only.
@@ -0,0 +1,54 @@
+{
+  "api": "http://rank0:8000",
+  "model": "deepseek-v4.1-flash",
+  "mode": "decode",
+  "label": "sgl-replay0",
+  "gen": 256,
+  "thinking": false,
+  "started": "2026-09-11T18:49:19Z",
+  "samples": [
+    {
+      "c": 1,
+      "wall": 7.234400033950806,
+      "completion_tokens": 256,
+      "prompt_tokens": 248,
+      "agg_tok_s": 35.386486619291205,
+      "per_session_tok_s": 36.88715282912961,
+      "ttft_p50": 0.29383206367492676,
+      "ttft_max": 0.29383206367492676,
+      "steps_s": 0.0,
+      "acceptance": null,
+      "tok_per_step": null,
+      "preemptions": 0,
+      "errors": 0,
+      "finish": [
+        "length"
+      ],
+      "err_samples": [],
+      "max_wall": 7.233917713165283,
+      "run": 0
+    },
+    {
+      "c": 4,
+      "wall": 12.71714186668396,
+      "completion_tokens": 1024,
+      "prompt_tokens": 1056,
+      "agg_tok_s": 80.5212374553003,
+      "per_session_tok_s": 21.858063466301363,
+      "ttft_p50": 0.6920806169509888,
+      "ttft_max": 0.6926991939544678,
+      "steps_s": 0.0,
+      "acceptance": null,
+      "tok_per_step": null,
+      "preemptions": 0,
+      "errors": 0,
+      "finish": [
+        "length"
+      ],
+      "err_samples": [],
+      "max_wall": 12.71572470664978,
+      "run": 0
+    }
+  ],
+  "finished": "2026-09-11T18:49:39Z"
+}
@@ -0,0 +1,54 @@
+{
+  "api": "http://rank0:8000",
+  "model": "deepseek-v4.1-flash",
+  "mode": "decode",
+  "label": "sgl-replay1",
+  "gen": 256,
+  "thinking": false,
+  "started": "2026-09-11T18:35:54Z",
+  "samples": [
+    {
+      "c": 1,
+      "wall": 6.840591192245483,
+      "completion_tokens": 256,
+      "prompt_tokens": 263,
+      "agg_tok_s": 37.423665996910096,
+      "per_session_tok_s": 38.92745617985822,
+      "ttft_p50": 0.26373791694641113,
+      "ttft_max": 0.26373791694641113,
+      "steps_s": 0.0,
+      "acceptance": null,
+      "tok_per_step": null,
+      "preemptions": 0,
+      "errors": 0,
+      "finish": [
+        "length"
+      ],
+      "err_samples": [],
+      "max_wall": 6.840073108673096,
+      "run": 0
+    },
+    {
+      "c": 4,
+      "wall": 13.420984506607056,
+      "completion_tokens": 1024,
+      "prompt_tokens": 1052,
+      "agg_tok_s": 76.29842650484338,
+      "per_session_tok_s": 21.759539600812793,
+      "ttft_p50": 1.1422911882400513,
+      "ttft_max": 1.1427288055419922,
+      "steps_s": 0.0,
+      "acceptance": null,
+      "tok_per_step": null,
+      "preemptions": 0,
+      "errors": 0,
+      "finish": [
+        "length"
+      ],
+      "err_samples": [],
+      "max_wall": 13.420344352722168,
+      "run": 0
+    }
+  ],
+  "finished": "2026-09-11T18:36:14Z"
+}
@@ -0,0 +1,9 @@
+{
+  "target": 131072,
+  "prompt_tokens": 130258,
+  "depth": 0.5,
+  "ttft_s": 70.0,
+  "prefill_tok_s": 1860.9,
+  "answer": "COPPER-LANTERN-8315",
+  "pass": true
+}
@@ -0,0 +1,9 @@
+{
+  "target": 131072,
+  "prompt_tokens": 130258,
+  "depth": 0.5,
+  "ttft_s": 44.4,
+  "prefill_tok_s": 2933.5,
+  "answer": "COPPER-LANTERN-8315",
+  "pass": true
+}
@@ -0,0 +1,58 @@
+{
+  "api": "http://rank0:8000",
+  "model": "deepseek-v4.1-flash",
+  "mode": "prefill",
+  "label": "sgl-replay0",
+  "gen": 256,
+  "thinking": false,
+  "started": "2026-09-11T18:48:37Z",
+  "samples": [
+    {
+      "c": 1,
+      "wall": 12.965696096420288,
+      "completion_tokens": 1,
+      "prompt_tokens": 14763,
+      "agg_tok_s": 0.07712659563847798,
+      "per_session_tok_s": 1000000.0,
+      "ttft_p50": 12.964907169342041,
+      "ttft_max": 12.964907169342041,
+      "steps_s": 0.0,
+      "acceptance": null,
+      "tok_per_step": null,
+      "preemptions": 0,
+      "errors": 0,
+      "finish": [
+        "length"
+      ],
+      "err_samples": [],
+      "max_wall": 12.964907169342041,
+      "run": 0,
+      "target": 16384,
+      "prefill_tok_s": 1138.6199314108505
+    },
+    {
+      "c": 1,
+      "wall": 29.41055202484131,
+      "completion_tokens": 1,
+      "prompt_tokens": 59007,
+      "agg_tok_s": 0.03400140191708611,
+      "per_session_tok_s": 1000000.0,
+      "ttft_p50": 29.40910029411316,
+      "ttft_max": 29.40910029411316,
+      "steps_s": 0.0,
+      "acceptance": null,
+      "tok_per_step": null,
+      "preemptions": 0,
+      "errors": 0,
+      "finish": [
+        "length"
+      ],
+      "err_samples": [],
+      "max_wall": 29.40910029411316,
+      "run": 0,
+      "target": 65536,
+      "prefill_tok_s": 2006.3207229215
+    }
+  ],
+  "finished": "2026-09-11T18:49:19Z"
+}
@@ -0,0 +1,58 @@
+{
+  "api": "http://rank0:8000",
+  "model": "deepseek-v4.1-flash",
+  "mode": "prefill",
+  "label": "sgl-replay1",
+  "gen": 256,
+  "thinking": false,
+  "started": "2026-09-11T18:35:24Z",
+  "samples": [
+    {
+      "c": 1,
+      "wall": 10.706095218658447,
+      "completion_tokens": 1,
+      "prompt_tokens": 14758,
+      "agg_tok_s": 0.09340473623447816,
+      "per_session_tok_s": 1000000.0,
+      "ttft_p50": 10.70548939704895,
+      "ttft_max": 10.70548939704895,
+      "steps_s": 0.0,
+      "acceptance": null,
+      "tok_per_step": null,
+      "preemptions": 0,
+      "errors": 0,
+      "finish": [
+        "length"
+      ],
+      "err_samples": [],
+      "max_wall": 10.70548939704895,
+      "run": 0,
+      "target": 16384,
+      "prefill_tok_s": 1378.4670973484285
+    },
+    {
+      "c": 1,
+      "wall": 18.76260209083557,
+      "completion_tokens": 1,
+      "prompt_tokens": 58999,
+      "agg_tok_s": 0.053297511462359545,
+      "per_session_tok_s": 1000000.0,
+      "ttft_p50": 18.761067152023315,
+      "ttft_max": 18.761067152023315,
+      "steps_s": 0.0,
+      "acceptance": null,
+      "tok_per_step": null,
+      "preemptions": 0,
+      "errors": 0,
+      "finish": [
+        "length"
+      ],
+      "err_samples": [],
+      "max_wall": 18.761067152023315,
+      "run": 0,
+      "target": 65536,
+      "prefill_tok_s": 3144.4998787677505
+    }
+  ],
+  "finished": "2026-09-11T18:35:54Z"
+}