Skip to content

Add R33 TP4/DCP4 SparkCache profile and qualify it on the four-Spark ring - #265

Merged
FujitsuPolycom merged 12 commits into
mainfrom
feat/mesh-reparent-r33
Sep 11, 2026
Merged

Add R33 TP4/DCP4 SparkCache profile and qualify it on the four-Spark ring#265
FujitsuPolycom merged 12 commits into
mainfrom
feat/mesh-reparent-r33

Conversation

@FujitsuPolycom

Copy link
Copy Markdown
Owner

What

Adds and qualifies the TP4/DCP4 SparkCache profile on the unchanged published R33 image.

  • Profiles: tp4-dcp4 and tp4-dcp4-sparkcache templates + contract entries (inherit tp4-dcp1, decode_context_parallel_size=4, same 24 GiB KV/rank and graph rows).
  • Launcher: tp4-dcp4(-sparkcache) case arms with a profile-name/DCP consistency guard; lease, restore-only, and bounded-capacity guards extended; overlay env passthrough.
  • Contract overlay: R33_PROFILE_CONTRACT_HOST_ROOT bind-mounts an updated profile-contract dir and a parallel-path overlay entrypoint (sparkring-r33-overlay) over the published image, so DCP4 serves from image 1328a4f6… without rebuilding it; the overlay entrypoint re-verifies every non-contract installed file against the in-image source lock.
  • Mesh renderer: DCP4 in the site allowlist, render allowlist, and cache diagnostics; DCP-aware cache namespace.
  • Qualification (four-Spark ring, image 1328a4f6…): exact answers ×4; two 37,032-token cold prefills; prefix reuse 36,352 cached tokens (15.8 s → 5.0 s); mHC rows=8192 owner_rows=2048 on every rank; SparkCache healthy on 4 ranks with committed page snapshots; 8K restore after planned restart (50.6–54.8 ms) and after SIGKILL fault injection (51.9–57.7 ms); KV capacity 8,364,901 tokens; 1M admission. Record: performance/records/glm53-flash/r33-image020-tp4-dcp4-sparkcache-20260911.md; activation receipt passes verify_profile.py validate-activation.
  • Docs: README gains the DCP4 flagship row (~8.36M KV) and a KV-per-layout footnote; the retired mesh row's replacement now points at the R33 DCP4 record; profile index gains the DCP4 row. Notes: the RoCEnante overlay arms at DCP4, so DCP4 activations require the managed fabric install (routes, tc, RDMA-TX markers) before launch — DCP1 disarms it.

Not in this change

Three-cold-start timing evidence, INFO-NCCL RouteFinal diagnostics, and decode/prefill throughput windows (research-only) — follow-ups in the DCP1 record format. No 1M-token request was run.

Validation

  • Affected suites: R33 profile contract tests, mesh profile tests, mesh launcher tests (183 passed, 1 platform skip); full local mesh+profiles+launcher set 323 passed earlier on the pre-record tree.
  • Live qualification on the DGX4 ring per the record above; planned and fault-injection restarts both restore correctly.

Add tp4-dcp4 and tp4-dcp4-sparkcache profile templates, contract entries,
launcher case arms with a profile-name/DCP consistency guard, mesh renderer
and cache-diagnostics allowlist entries, and a DCP-aware cache namespace.
mHC owner rows stay 2,048 (rows split across the TP4 group; DCP does not
re-split) per the R33 patch source. Templates and managed rendering are
implemented; bounded GPU qualification on the four-Spark ring is pending.
The published R33 image bakes its entrypoint allowlist and profile contract.
Add R33_PROFILE_CONTRACT_HOST_ROOT: when set with an R33 profile, the launcher
bind-mounts the host's jovian-r33/profiles directory (and the matching
entrypoint.py) read-only over the image copies, so DCP4 profiles can serve
from the unchanged published image. The mounted contract keeps the image's
artifact lock and mesh pins identities.
Mounting over /opt/sparkring/bin/sparkring-r33 breaks the in-image
verify-candidate installed-file identity check. Mount the overlay
entrypoint at /opt/sparkring/bin/sparkring-r33-overlay instead and select
it as serving_entrypoint only when the contract overlay is active, so the
baked entrypoint and its verified payload stay intact.
…rypoint

The baked verify-candidate hashes the baked profile-contract files, which a
mounted overlay supersedes. The overlay entrypoint now verifies every
non-contract installed file against the in-image source lock itself when the
overlay is active, and runs the unchanged baked verifier otherwise.
Record the bounded TP4/DCP4 qualification on the four-Spark ring serving
from the unchanged published R33 image: exact answers, two 37K cold
prefills, prefix reuse (36,352 cached tokens), mHC rows=8192
owner_rows=2048 on every rank, SparkCache page snapshots, 8K restore after
a planned restart (50.6-54.8 ms) and after SIGKILL fault injection
(51.9-57.7 ms), and 8,364,901-token KV capacity. Add the activation
receipt, the performance record, the README DCP4 row with its KV footnote,
the retired-mesh replacement pointer, and the profile-index row. The
RoCEnante overlay arms at decode_context_parallel_size=4, so DCP4
activations require the managed fabric installation before launch.
…evidence, six-case prefix regression, cache-disabled observation, publication entry, reproduction guide
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant