Skip to content

Bound restore memory and reduce cache publication work - #62

Merged
FujitsuPolycom merged 2 commits into
mainfrom
perf/verified-cache-io
Sep 6, 2026
Merged

Bound restore memory and reduce cache publication work#62
FujitsuPolycom merged 2 commits into
mainfrom
perf/verified-cache-io

Conversation

@FujitsuPolycom

@FujitsuPolycom FujitsuPolycom commented Sep 6, 2026

Copy link
Copy Markdown
Owner

Large page restores can allocate outside the shared-base budget, repeat object reads and prefix hashing, and leave a single CUDA block copying an entire span. Under eviction pressure, an asynchronous tail capture can also lose its publication base before the saver reads it. This change bounds those paths and preserves verified-or-recompute behavior.

Status: implemented; performance observations are research-only and limited to the stated fixtures and combined runtime.

The implementation:

  • Adds spark_cache_cuda_restore_arena_budget_bytes. Zero preserves the configured lane count; a positive budget caps complete pairs of restore arenas. It does not claim to cap total process memory.
  • Uses selective object reads when base sharing is declined, bounds authenticated header/object retention, and releases source buffers after their final use. Immutable publication authenticates existing objects before staging while preserving data/directory barriers and corruption repair.
  • Shares exact prefix analysis across scheduler callbacks and tracks inventory mutations so unchanged reports avoid copying the complete held set. Page-cohort filesystem work runs on load workers, with cancellation and shutdown draining covered by regression tests.
  • Tiles CUDA page scatter into bounded 64 KiB tasks, with aligned vector copies and byte handling for unaligned fragments. Slot, range, coverage, error, and completion checks remain intact.
  • Reconstructs flat histories in private layer buffers while checking every base, tail, and intermediate result digest. Only the final immutable snapshot escapes.
  • Protects the base and result of one admitted publication through capacity reconciliation. Missing or unavailable bases select bounded full capture before GPU state is discarded; protected bytes remain counted against capacity.
  • Adds opt-in per-request evidence distinguishing restore offers, verified worker completion, and GPU-lease attachment.

Cache identities, digest salts, persisted formats, and the native ABI are unchanged. Default arena concurrency remains unchanged. Source-bound deployment receipts are refreshed to match the tree.

Validation:

  • python -m pytest sparkcache deploy -q: 1,202 passed, 8 skipped for the core changes; Ruff clean.
  • GPU-free tests cover work counts, corruption, publication races, bounded retention, shutdown/cancellation, and allocator remapping. A serial C++ harness compiles the actual page-scatter body and checks byte/guard coverage across span, page, alignment, slot, and capped-grid boundaries. It passes under WSL without CUDA; it does not prove device concurrency or CUDA compilation. A 257 MiB GPU benchmark fixture is included but has not been executed as part of this review.
  • Four-GB10 CUDA checks cover shuffled slots, rejected sharing, multi-stage histories, corruption rejection, and restoration after abort. The page-copy benchmark checks complete bytes and guard regions in mapped, managed, and staged modes.
  • In the GB10 component benchmark, the mapped 64 MiB single-span case changes from about 154 to 1.28 ms, and the 1,024-span case from 1.25 to 0.79 ms. These are placement-ABI timings, not serving-speedup claims.
  • A Windows CPU fixture reconstructing a roughly 3 MiB result through 128 stages measures 348.39 ms / 12.57 MB traced peak with source revision 5c3bd3e versus 147.95 ms / 6.39 MB with private-layer reconstruction; output bytes match.

The serving evidence uses a source-attested SparkRing MTP3 composition with additional runtime changes. It does not establish an isolated speedup from this PR or resolve the original 40 GiB workload without a matched control.

Related: #60 and #61. Neither issue is automatically closed by this draft.

Dependent capture policy and telemetry: #63. Companion runtime integration and the repository-contained sustained-run evidence: SparkRing #227.

Limit restore allocations and metadata work, reuse authenticated prefix and object analysis, and tile native page placement. Protect asynchronous publication dependencies through capacity reconciliation and verify every reconstructed history boundary.

Cache identities, digest salts, persisted formats, and native ABI remain unchanged; no namespace migration is required. GPU-free validation: 1,202 tests passed and eight skipped across sparkcache and deploy. CUDA and combined MTP3 measurements retain their workload-specific qualification limits.
Compile the actual page-scatter body as serial C++ to check spans, odd pages, alignment, shuffled slots, guards, rejection, and the 4096-block continuation. Add a 257 MiB GPU benchmark fixture, not executed here. Refresh source-bound deployment receipts. Cache identity and placement ABI are unchanged. Validation: CPU C++ traversal passes under WSL; 1202 tests pass with 9 platform/optional skips on Windows; Ruff and diff checks pass. This does not qualify CUDA execution.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant