Skip to content

CASSSIDECAR-486: Prefer exact-range CDC checkpoint on load to avoid restart rewind - #372

Open
Klose6 wants to merge 1 commit into
apache:trunkfrom
Klose6:CASSSIDECAR-486
Open

Klose6 wants to merge 1 commit into
apache:trunkfrom
Klose6:CASSSIDECAR-486

Conversation

@Klose6

@Klose6 Klose6 commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

On sidecar restart, CDC consumers reload checkpoint state from sidecar_internal.cdc_state_v2. Today loadStateForRange returns all rows whose (start, end) overlaps the owned range, then analytics merges them with Marker.min (earliest segment/offset wins), which can rewind the offset to days ago. However when an exact match exists, we can ignore stale overlapping historical rows in cdc_state_v2, and fall back to the existing overlap-based load (needed for cluster shrink) when no exact match is found.

@Klose6

Klose6 commented Aug 25, 2026

Copy link
Copy Markdown
Contributor Author

could anyone take a look of this PR? thanks! cc: @yifan-c @bbotella @jberragan

@yifan-c
yifan-c requested a review from jyothsnakonisa August 27, 2026 05:21
@yifan-c

yifan-c commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

Requested review from @jyothsnakonisa

@bbotella

Copy link
Copy Markdown
Contributor

Mmmm, I have a doubt here. Let's say:

  1. Job runs on range R1=(s1,e1) for a while, checkpoint row1 advances normally.
  2. Topology change reassigns it to R2=(s2,e2) (overlaps R1, different boundaries). First write for R2 correctly falls back to the full overlap set (no exact match yet), inherits row1's progress via min-merge, and starts writing row2. Row2 keeps advancing past where row1 ever got.
  3. Another topology change reassigns the job back to R1's exact boundaries.
  4. On load, exactRangeMatch finds row1 — an exact boundary match — and per the fix's logic, that wins outright, discarding row2 entirely, even though row2 is overlapping and strictly newer/further along than row1.

I know this event is rarer than the one this patch is addressing, but still possible, right?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants