Retain immutable cache pages through connector-job reads - #66
Merged
Conversation
Use generic scheduler block-state offers and connector-owned block references for exact recurrent checkpoints and complete attention pages. Queue native capture behind a post-MTP producer event, report distinct physical-rank read completion, and retain uncertain sources across cleanup. Keep native submission and recovery off the model thread. Reject retired or reordered job replays with bounded epoch/sequence state after checking active ownership. Require explicit reviewed JJ or R27 source contracts for connector-job mode; preserve request-finish defaults. Cache identity wire values remain unchanged. Use separate storage roots for unqualified arithmetic compositions. Validation: 1157 CPU tests passed, 8 skipped; Ruff and diff checks passed. Actual JJ/R27 scheduler, block-pool and connector callback conformance passed. CUDA ordering and full-model restore qualification remain required.
Job-leased captures release sources through per-job rank acknowledgements. Exclude them from request-owned completion tracking so finished request IDs do not accumulate after their reads retire (D22). Preserve delayed completion for request-owned captures and refresh the deployment profile's source hash. Cache identity, on-disk namespace and native code are unchanged. Validation: 41 targeted tests passed; full GPU-free suite passed 1160 tests with 8 skips. Ruff and git diff --check passed. No GPU or model validation was run for this isolated source change.
…ntegration Decline external whole-prefix placement when a request already has local computed tokens, keep allocated restore flights from admitting a second writer, and exclude shared null padding from failed-load block reports. These guards preserve unrelated requests and let unproven destinations recompute. Package exact source contracts for the measured hybrid-recovery composition and the standalone Jovian recovery port. Include source-bound TP4/DCP1 capture retirement and persistent-restore evidence, with CPU-only limits for the added admission guards. Require the job-ownership modules and contracts in the wheel. Cache identity values, stored formats, and native ABIs are unchanged. Refusing whole-prefix writes over a local prefix can increase recomputation. Validation: 1305 CPU tests passed, 9 environment/source-fixture skips; Ruff, wheel/sdist build, Twine, isolated wheel installation, and ten-file committed Jovian source-contract verification passed. The new restore guards are not GPU-qualified by the separate SparkCache 2bc05bc live record.
The ten connector source files retain the reviewed recovery and ownership semantics while the serving composition adds TP2 prefill admission. Record exact source fingerprints and include the contract in verified wheels. Cache identities and native ABIs are unchanged. Validation: 1305 CPU tests passed, 9 platform/source-fixture skips, package build and isolated installation passed; integrated GPU qualification remains separate.
Preserve independent capture-read ownership and whole-prefix restore guards alongside maintenance inventory reuse and optional request attribution. Recompute the DeepSeek and GLM-5.2 deployment source hashes for the combined package; cache identity, persisted formats, native ABI, and serving-image pins remain unchanged. CPU validation: 1353 tests passed with 9 documented platform or fixture skips; Ruff, wheel and sdist build, Twine checks, isolated installation, and distribution validation passed. Existing live evidence remains bound to its recorded source revisions.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Whole-prefix restore and asynchronous capture need independent ownership boundaries. Connector-job mode retains each exact immutable attention page and recurrent checkpoint until every distinct physical rank acknowledges that its native read has retired. Request completion, cancellation, and disk publication do not release these references early. Submission and error recovery run outside the inference callback; uncertain reads retain bounded references and disable optional capture.
The connector declines whole-prefix restore when a request already has local computed tokens, because those pages may be shared and the placement operation has no suffix write mask. Allocated or dispatched restores continue to protect their destinations. Failed loads report only nonzero destination IDs so shared null padding cannot invalidate unrelated requests. Source contracts bind the generic connector API, exact boundary offers, post-forward producer readiness, and rank-completion semantics to reviewed runtime files.
Status: implemented, with GPU-free source and ownership tests. Legacy request-finish mode remains the default. Cache identity, namespace, persisted formats, and native ABI are unchanged. Deployments with different arithmetic compositions still require separate cache roots.
Validation at head
d89fc74ae37e58a1f1e6cfbbf8fda1540a63ea75, including the maintenance and attribution source already merged through #65:python -m pytest sparkcache research deploy -q -rs: 1,353 passed, 9 skipped on Windows/Python 3.12. The skips require POSIX locks, symlink privileges, a retained-open-inode replacement, a C++17 compiler, or an external stock scheduler fixture.python -m ruff check .andgit diff --check: passed.Four-GB10 TP4/DCP1 evidence qualifies four cold 8K/16K capture completions, two active cancellations, and exact-answer persistent restores after all model processes restarted. That evidence names SparkCache
2bc05bc9, vLLMdf62335d, and imagesha256:cd92adc4436c61290dbecc35362db447c7ae69e1d03bdb99704af0ed6c517b38. The additional private-restore admission and null-block guards ind0cf7296retain CPU coverage and require their own integrated GPU qualification. The source contract for vLLM17bd2580is compatibility evidence, not a serving result.The DeepSeek and GLM-5.2 deployment source receipts bind the complete combined package digest
ce0cacb42427d6fdebb20f613b67fda883973c5522ef5d4d049707e934ba9fe5. The publishedd0cf7296commit and its image-source references remain immutable ancestors. Merging this source does not update an installed image, qualify TP2 SparkCache, or close the near-full storage and long-conversation investigations in #60 and #61.