Skip to content

fix(compaction): persist keys in lookup SST files while first-row merge function - #250

Merged
lxy-9602 merged 1 commit into
apache:mainfrom
lszskye:fix_first_row_lookup_empty_sst
Aug 26, 2026
Merged

fix(compaction): persist keys in lookup SST files while first-row merge function#250
lxy-9602 merged 1 commit into
apache:mainfrom
lszskye:fix_first_row_lookup_empty_sst

Conversation

@lszskye

@lszskye lszskye commented Aug 26, 2026

Copy link
Copy Markdown
Collaborator

Purpose

Linked issue: close #251

First-row lookup uses LookupLevels<bool> because it only needs to determine whether a primary key already exists.

The previous implementation special-cased bool by creating and immediately closing the lookup writer. This produced a valid but empty SST file. As a result, a compaction could incorrectly retain a newer row even though the first row for that key already existed.

This change:

  • removes the bool-specific empty SST shortcut;
  • writes serialized primary keys with empty value payloads;
  • uses a key-only read schema for first-row lookup, avoiding unnecessary value-column reads.

Tests

Added LookupMergeTreeCompactRewriterTest.TestFirstRowLooksUpExistingKeys, covering:

  • an existing key=1 in L2;
  • key=1 and key=2 in L0;
  • compaction of only L0 into L1;
  • verification that the output contains only key=2.

API and Format

No public API or protocol changes.

Documentation

No documentation changes are required.

Generative AI tooling

Generated-by: Codex (GPT-5)

@lszskye
lszskye requested a review from lxy-9602 August 26, 2026 11:11

@lxy-9602 lxy-9602 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@lxy-9602
lxy-9602 merged commit d76c2a6 into apache:main Aug 26, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] persist keys in SST files while first-row merge function

2 participants