You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
racer: replace segment allocator with an extent volume - #948
This replaces the Racer dataplane's segment allocator and in-memory page index with a per-worker extent volume. Each volume splits the disk into 16 MiB slots that hold either one large page or many small pages in size classes from 4 KiB, and indexes them in an on-disk copy-on-write B+tree committed through A/B superblocks with fsync, so there is no write-ahead log. Pages have two tiers: cache pages are committed lazily and may be lost on crash, while held pages use the storage tier and are durable before the hold is acknowledged. Checkpoints now carry only object metadata, existing slab files are reformatted on first start, and disk eviction now frees whole slots. Tests that covered only the removed segment, freeze, and checkpoint pause code were deleted; the new volume has unit, io_uring, and crash and bit-flip simulation tests.
The reason will be displayed to describe this comment to others. Learn more.
🔵 Needs a closer look
Unresolved hold protection, capacity enforcement, allocation progress, and checkpoint failures affect storage reliability.
No open issues 🎉
1 previously missed issue
In code that hasn't changed since last review
Failed hold commits can strand in-memory reservations
cmd/racer-dataplane/src/read/dispatch.rs:897
A commit failure here leaves the in-memory reservation behind. StoreWriter::reserve_hold reserves capacity before awaiting the commit (src/store.rs:1142-1143), but the error prevents reserve_warm from adding this worker to its cleanup list (src/read/dispatch.rs:467-473). Failed admission therefore omits it when releasing unused holds (src/read/warm.rs:615-620). A refused HEAD can strand its requested capacity until expiry and cause later warm requests to be refused. Track reservations that succeeded even when their commit failed, and release their unused capacity without dropping published holds or interfering with another active job.
The reason will be displayed to describe this comment to others. Learn more.
🔵 Needs a closer look
Unresolved retention, reclamation, and checkpoint-boundary issues affect held-page guarantees and recovery.
No open issues 🎉
1 previously missed issue
In code that hasn't changed since last review
Deferred storage reclamation stalls without a second commit
cmd/racer-dataplane/src/app.rs:1680
Deferred storage frees need a second successful commit even when the index is clean. After a held full-page record is invalidated, the first commit clears is_dirty(), but its slot stays charged until the next successful epoch (cmd/racer-dataplane/alloc/src/volume/slots.rs:549-561). This gate schedules no follow-up, and an explicit clean commit also returns immediately (cmd/racer-dataplane/alloc/src/volume/store.rs:1205-1212). At the storage-slot quota, repair can therefore remain blocked with Busy until unrelated mutations or hold expiry. Treat pending reclamation as commit work and allow a superblock-only rotation without new index changes. Cover repair at the storage quota with no unrelated writes.
Open cache slots block allocation for new size classes
cmd/racer-dataplane/src/config.rs:473
The accepted four-slot geometry can leave cache writes for a new size class returning Busy indefinitely. Three small records in different size classes occupy all three data slots, which remain open while partly filled (alloc/src/volume/slots.rs:625-645,1083-1092). Eviction skips every open slot (alloc/src/volume/slots.rs:818-821), so a fourth class, including a full page, cannot allocate even after leases are released and with no holds. Add pressure reclamation that safely closes and evicts eligible open cache slots, with a regression test for this minimum-size volume. Raising the minimum alone would only hide the allocator problem.
The reason will be displayed to describe this comment to others. Learn more.
🔵 Needs a closer look
Human assessment is needed because metadata eviction can break pinned reads when the origin is offline, alongside cancellation-cleanup and scan-progress risks.
This replaces the Racer dataplane's segment allocator and in-memory
page index with a per-worker extent volume. Each volume splits the disk
into 16 MiB slots that hold either one large page or many small pages
in size classes from 4 KiB, and indexes them in an on-disk
copy-on-write B+tree committed through A/B superblocks with fsync, so
there is no write-ahead log. Pages have two tiers: cache pages are
committed lazily and may be lost on crash, while held pages use the
storage tier and are durable before the hold is acknowledged.
Checkpoints now carry only object metadata, existing slab files are
reformatted on first start, and disk eviction now frees whole slots.
Tests that covered only the removed segment, freeze, and checkpoint
pause code were deleted; the new volume has unit, io_uring, and crash
and bit-flip simulation tests.
…l returns
allocate_evicting and reserve_free moved each eviction's removals into a
local vector, so a later error or a dropped future lost them and the caller's
take_evicted cleanup could not release their bookkeeping. They now stay
queued on the volume until the call returns them.
…g it
Eviction yields between batches, and reads can lease the slot meanwhile.
Finishing the eviction freed the slot regardless, so allocation could reuse
it while a disk or GPU read still used it. Eviction now gives the slot back
when it is leased at the end; its removed entries stay removed, and a later
eviction frees it once the lease is gone.
A hold stays live until both its wall and monotonic deadlines pass, but
storage eviction compared retain_until with the wall clock alone, so a
forward wall-clock jump could evict pages the worker still reported as held.
Eviction now uses a time that follows the wall clock but never runs ahead of
monotonic time elapsed since its last reading.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This replaces the Racer dataplane's segment allocator and in-memory page index with a per-worker extent volume. Each volume splits the disk into 16 MiB slots that hold either one large page or many small pages in size classes from 4 KiB, and indexes them in an on-disk copy-on-write B+tree committed through A/B superblocks with fsync, so there is no write-ahead log. Pages have two tiers: cache pages are committed lazily and may be lost on crash, while held pages use the storage tier and are durable before the hold is acknowledged. Checkpoints now carry only object metadata, existing slab files are reformatted on first start, and disk eviction now frees whole slots. Tests that covered only the removed segment, freeze, and checkpoint pause code were deleted; the new volume has unit, io_uring, and crash and bit-flip simulation tests.