Expose cache memory budgets, isolate Docker polling, and validate MTP3 reuse - #227
Expose cache memory budgets, isolate Docker polling, and validate MTP3 reuse#227FujitsuPolycom wants to merge 5 commits into
Conversation
Resolve capture modes, report payload allocation budgets, and keep managed mesh supervision responsive during Docker status delays. Package source-attested MTP3 checkpoint and lease transforms with offline composition checks and request-level reuse analysis. Preserve published image pins, source verification boundaries, and merged startup memory gates. Record bounded four-GB10 text-only cache-pressure evidence without claiming 40 GiB or multimodal qualification. Validation: 2,748 offline tests passed, 42 skipped; numeric evidence verification and Ruff passed.
Exercise transport-error and receipt redaction with an unmistakable test value that does not match the release credential scanner. Production authentication behavior is unchanged. All nine conversation harness tests pass.
eaae03b to
320e9a5
Compare
Keep service ownership checks intact while simulating root-owned fixture metadata on unprivileged Linux runners. Validation: all 37 managed-service tests pass under WSL, including both formerly failing monitor cases. No runtime or cache identity changes.
Document incompatible indexer postimage hashes and the need for one attested combined image. Source branch integration does not repin images or cache namespaces. Documentation-only change; no stack activity.
Recommend an optional pre-installation reboot after stopping workloads on hosts that have run large GPU jobs. Explain contiguous-memory requirements and retain the managed startup gate as the readiness check. Documentation-only change; relative links and whitespace verified.
|
Status: implemented in SparkRing main through merged #236. The cache-memory controls, Docker polling changes, runtime prefix-reuse corrections, and bounded validation records in this PR are incorporated; this PR is closed as superseded by that integration. The component head The native-MTP3 cache/checkpoint guide selects the integrated image and deployment inputs. SparkCache #60 and #61 remain open: bounded maintenance work, exact reuse attribution, and original-workload validation are distinct from incorporating this PR's code. Closing this component PR does not assert that those performance reports are resolved. |
1 participant
Restore-only configuration should disable publication buffers, restore-memory estimates should count every load lane, and slow Docker status queries should not stall fabric supervision. This change implements those controls and adds a reproducible GLM-5.3 native-MTP3 source experiment for recurrent prefix reuse and cache accounting.
Status: controls and offline validation tools are implemented; native-MTP3 performance qualification remains research-only. Integration preserves the startup memory and readiness controls in base revision
474d992.Implemented behavior
SPARKCACHE_ASYNC_PAGE_CAPTURE=auto: enabled read-write/store-only modes select asynchronous capture; restore-only, disabled access, and a disabled connector select zero capture slots. Explicit incompatible settings still fail. The launcher retains its zero-capture default when the setting is absent.SPARKCACHE_BUFFER_BUDGET_BYTEScan reject that payload allocation before host/image checks. It excludes KV, model weights, retained Python objects, control arrays and other process memory.reported_cachedrather than being interpreted as local hits or misses.Research-only MTP3 source composition
runtime/glm53-spark-mtp3-mesh/experiments/cache-reuse/contains five exact source transforms, compact source fixtures, a hash-verifying composition entrypoint, and CPU tests of the extracted runtime algorithms. The transforms synchronize the fused indexer's histogram publication, account for attached GPU leases, prefer a strictly longer converged local prefix, pair speculative replay checkpoint production with retention, and derive recurrent partial-tail eligibility from recurrent page geometry.The input image is
sha256:2e41b1e934a85ff7c21b780532db2f0a0e978df081e52f4ae2bf11f8992fb24f. The output scheduler and manager hashes are respectively75efa57e7ff5a77c76714b85e2e4d8e1d7f456d9a9eec6c67ebb11ca382942f9andd2e35b012e0cf45ab3771f545c35ca48f2a5858549c574a352975607369124e2. The fixture manifest identifies every accepted input and script checksum.The source composer does not deploy an image. Serving requires a selected SparkCache commit, native libraries and a runtime contract attesting the composed source. Published runtime pins and production Dockerfiles are unchanged. The supported experiment geometry is native-MTP3 TP4/DCP4 with 512-token recurrent/hash pages and 2,048-token scheduling alignment.
Validation and limits
The repository-wide offline command from
AGENTS.mdpassed 2,748 tests with 42 skips on Windows. Ruff passed acrossspark_transport,runtime,scripts, andperformance. The added numeric evidence verifier also passed. The self-contained record and raw numeric observations are inperformance/records/glm53-flash/mtp3-cache-history-validation.mdand its linked compressed artifact.The combined offline command below passed 542 tests, with five skips for Windows filesystem restrictions, an optional native bundle, and optional companion-source integration. It covers launcher configuration, managed supervision, source composition, extracted lease/checkpoint rules, and strict trace joins. The allocator checker executes 672 fresh/resumed schedules against actual extracted allocation and block-registration methods, with zero stale/unwritten state registrations. Some resumed schedules retain fewer reusable checkpoints and still miss safely.
Bounded hardware evidence uses the custom MTP3 composition with SparkCache
607ccef061d0f511f45a2a8a93f74514c955d3a3, imagesha256:75050f7b4dd7287f1ecb3e7e34226d24aa6c5b3e012a8bd6c8399e411cfbd908, and four GB10 ranks. A one-hour admission window completed 551 responses, including 545 growing-conversation requests and six probes, with zero request errors or absent-base publication failures. It used C4, approximately 100K starting prompts, 2K growth, a 512-token output ceiling, and a 2 GiB/1.5 GiB per-rank cache policy. All 525 continuations reported 97.29–98.34% cached tokens; no explicit restore/lease traces belonged to that long-run population. Separate semantic canaries verified advancing reuse and explicit restore/lease paths.Before/after median client TTFT was 2.944/2.980 seconds; the streamed-delta decode estimate was 51.20/47.80 tokens/s. Three probes per side and no matched original-image/restore-only control limit interpretation. This custom composition includes companion SparkCache changes; the measurements do not isolate SparkRing's contribution or establish the original 40 GiB multimodal scenario. The observed startup failure before one successful retry also leaves unattended-startup availability unqualified.
Related investigations: SparkCache #60 and SparkCache #61. This draft does not close either issue.
Companion implementation drafts: bounded cache restoration and publication and periodic captures and backlog telemetry. The source composer requires an explicitly selected compatible SparkCache runtime; merging this PR does not repin or deploy those changes automatically.