Group flat page snapshots into authenticated macro objects

[FujitsuPolycom]

Resulting behavior

• Publishes flat block_pages_v1 snapshots as sparkcache-page-snapshot-manifest/v2 roots over content-addressed objects of at most 64 MiB.
• Exposes SparkCache CUDA macro restore through the canonical sparkcache.spark_context_cache_cuda_hybrid_restore module and CudaHybridRestore* / execute_cuda_hybrid_* API. spark_context_cache_native_hybrid_restore and NativeHybridRestoreError remain compatibility aliases.
• Validates CUDA placement source_bytes as the complete authenticated arena bytes, including the encoded snapshot header, while page-copy spans continue to cover payload bytes only.
• Keeps the logical cache identity, digest, admission boundary, and logical_chunk_tokens=256 geometry independent of physical object grouping.
• Publishes at most two object payloads per durable batch and reads at most four objects per Python reconstruction batch.
• Publishes directly from the owned opaque snapshot rather than encoding or retaining one .spcc payload per logical chunk. An 813,068,464-byte flat snapshot is represented by 13 payload objects instead of 512 logical-chunk files.
• Uses the same v2 flat root for page-delta compaction. Capacity accounting, invalidation, orphan collection, and embedded page-delta base graphs retain authenticated shared objects correctly.
• SparkCache CUDA restore re-authenticates the persisted manifest identity, reads each object into a bounded mapped arena, verifies the complete object before submitting its page-copy spans, and verifies the complete snapshot digest before the parked request may resume.

Authenticated abort behavior

A missing, truncated, replaced, or checksum-mismatched object becomes recomputation. If object corruption is detected after earlier slabs were submitted, the parked placement transaction aborts and cannot resume. A root whose objects each authenticate but whose declared complete-snapshot digest is wrong also aborts before completion. A forged in-memory lookup whose manifest digest does not match the persisted root is rejected before placement begins.

Compatibility and cache namespace

Version 1 flat manifests remain readable byte-for-byte. Row-oriented per_token_rows publication is unchanged.

This change does not alter CacheIdentity wire values, digest salts, 256-token logical geometry, the default flat namespace, or the opt-in page-tail-cow-v1 namespace. The physical representation changes under a strict manifest schema. Readers without sparkcache-page-snapshot-manifest/v2 support cannot reinterpret v2 objects as v1 chunks; they reject the root as a cache miss. A mixed-version rollback can therefore lose reuse and recompute, but cannot serve the entry under the wrong storage contract.

Evidence and qualification boundary

Implementation base: SparkCache runtime commit 65b6642df1afc64366430d3aef9aca01f5c5e1c3.

Source commit: 229d7d6158261e9510ab99d7e82d532abb9ade01

SparkCache source-tree SHA-256: 3f0d9b0aca8fb5cbb82dae3aa9daa2ed384e9edbced854930c0099e3d169f4bf

GPU-free validation:

• python -m pytest sparkcache -q: 773 passed, 7 skipped
• python -m pytest deploy -q: 108 passed, 1 skipped
• python -m ruff check sparkcache deploy: passed
• git diff --check: passed

Flat macro publication and its SparkCache CUDA restore path are implemented and GPU-free tested, not live qualified. This PR does not build or deploy an image, alter the active service, or claim a measured latency improvement.

Historical review note

This draft is stacked on SparkCache pull request #39 for review.

[FujitsuPolycom]

The resulting behavior and evidence are consolidated in retained draft stack #39 → #42 → #44. Independent heat and SSD research remains in #36. Closing this superseded draft and deleting only its remote head branch.