Add DeepSeek V4.1 SGLang cycle serving with decoder-tail prefill

[sethforprivacy]

DeepSeek V4.1 prefill currently executes all 40 backbone layers across the whole prompt in the vLLM recipe. This adds a separate SGLang cycle recipe using decoder-tail replay: after the full global-KV source layers, the late decoder processes only the final 128 extend tokens per request. A controlled same-image replay-off/on comparison measured nominal 64K prefill at 2,006 → 3,144 tok/s, with 20/20 first-token choices and 20/20 greedy continuations matching on the bounded gate.

The runtime builds Mia's external adapter at a pinned commit over a pinned ARM64 SGLang base digest; no AGPL adapter/launcher/packer source is vendored. The operator launcher renders an offline plan, checks image and NCCL identities, refuses busy GPUs, packs Mia-layout Engram files, and supports one bearer key per line without exposing keys in Docker arguments. NCCL replaces the pip library in place, avoiding a second loaded runtime. The vLLM recipe from #261 remains available as a fallback.

The separate 430080-context site deployment has completed its bounded functional gates and a six-hour C8 streaming soak. The soak handled 13,224 requests with zero failures or hangs, median per-wave aggregate throughput 106.7 tok/s, and a sampled available-memory floor of 28.47 GiB. All four memory guards were healthy and an authenticated completion through the public router passed afterward. Raw observations, exact harness inputs, timing definitions, and limitations are included. The preceding 20-minute gate also passed (464 requests, zero failures/hangs).

The public defaults retain context 262144. The site uses chunk4096, context430080, and requested KV1500000 (actual1499904), with at most eight requests. Seven-key authentication, bounded replay-off quality comparison, approximately 407K-token exact retrieval, vision/tool, mixed-prefill/decode, and chunk/tail-boundary checks passed in the deployment qualification. Further chunk/context/KV expansion is deferred after an 8K-chunk trial triggered a memory guard. The soaks use repeated short prompts with thinking disabled and 256-token output caps; they do not establish long-context stress tolerance, semantic correctness, or reasoning performance.

The deployment is qualified for the recorded bounded functional and streaming gates. Finite output agreement is not proof of mathematical equivalence: replay changes late-layer local SWA visibility. No unattended recovery or general high-availability claim is made.

Validation:

• 32 targeted launcher and recipe-index tests passed; pinned upstream auth-module behavior also checked CPU-only against seven keys, internal calls, negative credentials, health/metrics, and admin isolation.
• Repository Ruff, Markdown links, release-safety scan, and git diff --check passed.
• GitHub CI passed at fa77ac48: tests, lint, pinned LIL bridge, documentation links, and release safety.
• Raw controlled replay observations and quality distributions are included with scope and limitations. The separate bounded soak qualification is recorded above; no general production-readiness or public image publication is claimed.

Additional upstream kernel, SPS scheduling, speculative block-size, and Engram tuning experiments are excluded from this PR. Validated improvements will be submitted in a separate follow-up PR.

Follow-up to #261 and the engine discussion in #260.

[sethforprivacy]

Follow-up: we completed guarded context/KV qualification and brought the selected profile back into service on our four-DGX-Spark GB10 switchless ring.

Validated deployment profile

Setting	Value
Context length	655360 tokens (640 Ki tokens)
Requested total KV pool	1500000 tokens
Actual allocated KV pool	1499904 tokens
Chunked prefill	4096 tokens
Maximum running requests	8
DSPARK block size	5
Minimum sampled host MemAvailable during final qualification	22.849 GiB

This is TP4/EP4 with node-local packed NVMe Engram, I/O threads96 and row cache0. Software: SGLang e087e662ba1ac4ef7747537e2a9141085efd4561, Mia adapter e59e6eb67479aa68f6fa700c600dc90a0729b5ec, model revision dba1be0a40aa45a94ad051997016db3960a90277, SparkRing patched NCCL2.30.7, plus the SGLang #39187 memory backport and #39068 overlay. These later overlays/results are follow-up evidence, not a claim that the original PR image alone reproduces this capacity.

A final fresh boot passed the strict20/20 first-token and20/20 exact greedy-continuation reference checks, auth, vision/tools, boundary/mixed workloads, and112 structured-output requests covering nonstreaming chat, streaming chat and raw generation. Retrieval passed at647412 /647211 /647775 actual input tokens with the answer at50% /10% /90% depth. Four-rank source/configuration/boot audits matched before and after qualification.

The final20-minute C8 soak completed736 requests across92 waves with zero failures or hangs;168 responses reached the configured output cap. Median aggregate throughput was106.35tok/s for that soak workload. Authenticated router readiness passed afterward. This is a bounded stability result, not a new six-hour soak or a patch speedup claim. Client configs now advertise655360 context and32768 max output, retaining reasoning and text/image support.

Limits of the capacity conclusion: requested1.75M KV (actual1749760) also passed functional checks, including eight distinct262144-token exact-answer/normal-stop requests beyond the pool and recent-prefix reuse. It was not promoted by our sampled performance filter;2M KV was therefore skipped. C8 throughput also varied substantially across1.5M boots, so we cannot attribute the nonpromotion confidently to KV size. The pressure test validates retained-prefix eviction/reuse, not simultaneous full-pool live contexts. Thus640K/1.5M is our qualified deployed selection, not a statistically demonstrated optimum or a universal default.

Keeping these findings in this comment preserves the current ready-for-review PR scope; subsequent runtime/transport changes can be a separate follow-up.