21460 cursor bti - #5154
Open
rustyrazorblade wants to merge 11 commits into
Open
Conversation
Cursor compaction's format-specific CursorIndexWriter gains a BTI implementation: BtiCursorIndexWriter builds the trie index directly from cursor-emitted partition/row boundaries, using ClusteringDescriptorPrefixView to expose a clustering prefix view over the reusable descriptor without allocating a full Clustering per row. A format now declares its own support and builds its own index writer. SSTableFormat.supportsCursorCompaction() replaces the format checks that CursorCompactor, SSTableCursorWriter and the differential harness each kept separately, and SortedTableWriter.newCursorIndexWriter() replaces the instanceof dispatch in the SSTableCursorWriter constructor. Adding a format no longer means editing four format lists, and the compactor's supportability check and the writer's dispatch can no longer disagree. ClusteringDescriptorPrefixView separates its two lifetimes. snapshotOf returns a view that owns a copy of the bytes; reset aliases the descriptor's live array and throws if called on a snapshot. retainable() returns a snapshot instead of throwing, which is what RowIndexWriter needs, since it holds prevMax and prevSep lazily across add() calls. Bugs fixed: - A stale 4-arg writePartitionEnd call in the SSTableCursorPipeUtil microbench helper, which only surfaced after a full clean rebuild once BTI's writer signature change landed. - assertCursorPathWillRun's format guard only accepted the BIG format, so the differential harness could not run BTI scenarios through the actual cursor path. - EdgeCaseDifferentialCompactionTest.blockCount hard-cast every sstable reader to BigTableReader, so partitionCrossingOneIndexBlock threw ClassCastException the moment BtiDifferentialCompactionTest inherited the scenario. SSTableReader gains a format-agnostic getRowIndexEntry(key, op) convenience method (BigTableReader already narrows it via a covariant override; BTI now gets it for free), and the test uses that instead of casting. New BTI-specific differential and allocation test classes (BtiCursorReadTest, BtiDifferentialCompactionTest, BtiRandomDifferentialCompactionTest, BtiCursorCompactionAllocationGateTest) extend the existing harness to pin BTI output against the iterator path, with BTI-appropriate allocation ceilings accounting for its inherent ~2KB/partition trie/key-snapshot overhead. Patch by Jon Haddad; reviewed by for CASSANDRA-21460
The BTI support added in the previous commit was exercised only through full scans, which read Partitions.db and Data.db and never open the row trie. These tests drive the trie itself. Coverage: - ClusteringDescriptorPrefixViewTest: a property test comparing the view against ClusteringDescriptor.toClusteringPrefix over generated clustering types, 1 to 40 components. The view previously ran in one shape only, a single fixed-width bigint, leaving the vint, null, empty, multi-component and second-header branches unexecuted. - DifferentialCompactionTester asserts the output descriptor's format and reads every row back through a slice, forward and reversed, so the trie is routed through rather than skipped. - EdgeCaseDifferentialCompactionTest gains block-boundary scenarios: a row landing on a cut, a partition carrying a live partition deletion, a range tombstone boundary marker on a cut, empty clustering components crossing a block, and a run at BTI's own 16KiB granularity. - BtiMultiOutputDifferentialCompactionTest covers the per-output index writer and openFinalEarly, which no test reached. - RandomDifferentialCompactionTest builds partitions that span index blocks, varies column_index_size per example and generates past 32 clustering columns, so the soak writes and reads a row trie instead of taking trieRoot -1 everywhere. Also here: - BtiCursorCompactionAllocationGateTest sets the large-file ceiling from measurement, 0.248 B/B under BTI against 0.178 under BIG. - runOneExample and three harness methods are split up; draw order is unchanged, so seeds still reproduce. - The two fluent-builder log calls in CursorCompactor use LazyToString.lazy.
- the guard runs before the curr/prev swap, so a rejected re-entry no longer rotates the descriptors onto content nothing wrote - add StatefulCursorPartialRangeTest, covering bounded reads, segment hops and both DONE routes; the re-entry test pins both slots and the swap counter
…iter switch - Set the output sstable's last key per partition, so an sstable opened early at a switch carries real bounds - Fire the preemptive reopen from the cursor path and record the readable boundary it needs - Count the partition deletion before the partition_tombstones guardrail - Migrate the key cache on the BIG format - Report merged partition counts, not row counts - Refresh compaction progress inside a partition Adds UCS and LCS coverage against the cursor writer, and compares more sstable metadata between the two paths.
- Move the accord purge timestamp, the scanner byte sums and the static-column scan into private statics - Allocate the strict-liveness probe scratch unconditionally instead of conditionally, dropping the null markers
The extended index walk debug-logs every index block. Ant's junit formatter buffers all test output in memory, and the volume exhausted the 1G fork heap.
- setLast returns the copy it already builds; endPartition takes it - drops the redundant copies in the BTI index writer and the key cache - cover early open, key cache migration, TWCS, disk boundaries, reversed clustering, compaction progress, and the pipeline on a node
- fold shouldMigrateKeyCache into the originals array it gates - drop an assertion implied by the two before it, and its field - one constant for the padded partition key
- detachKey copies straight from the byte array, one buffer not two - collapse the disk boundary test's two fields into the object they build - count progress samples instead of collecting them - one note on TaskFactory for the keepOriginals trap, was four copies
rustyrazorblade
force-pushed
the
21460-cursor-bti
branch
from
September 11, 2026 19:44
9cb6b05 to
d802b1d
Compare
- Reuse one key for the writer's last-key tracking instead of copying per partition - Deep-copy only where the key outlives the partition - Pin the retainable contract with a test
rustyrazorblade
force-pushed
the
21460-cursor-bti
branch
from
September 11, 2026 23:15
d802b1d to
8bd75ae
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds BTI support to cursor compaction.