CASSANDRA-19597: Isolate SystemKeyspace post-flush tasks on dedicated executor - #5142
Open
cheeeee wants to merge 1 commit into
Open
CASSANDRA-19597: Isolate SystemKeyspace post-flush tasks on dedicated executor#5142cheeeee wants to merge 1 commit into
cheeeee wants to merge 1 commit into
Conversation
…DRA-19597) MemtablePostFlush previously serialized post-flush tasks across all keyspaces and tables using a single static single-threaded sequential executor. When a user table experiences heavy flushes with slow IntervalTree construction and compactions, it blocks MemtablePostFlush. Subsequent flushes for SystemKeyspace (e.g., system.local, system.peers, Gossip node removal, cluster metadata) get queued behind user tables. This caused Gossip to pause, heartbeats to be missed, and nodes to falsely mark each other down. This patch: 1. Adds a dedicated systemPostFlushExecutor (SystemMemtablePostFlush) for local system keyspaces (SchemaConstants.isLocalSystemKeyspace), ensuring system and gossip tasks are never blocked by slow user table flushes. 2. Preserves strict FIFO sequential post-flush ordering within each keyspace domain. 3. Shuts down cleanly and in parallel using ExecutorUtils.shutdownAndWait, avoiding sequential timeout accumulation. Fixes: CASSANDRA-19597
cheeeee
force-pushed
the
CASSANDRA-19597-trunk
branch
from
September 11, 2026 01:06
f90d380 to
3a32907
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Status: changes requested — not ready to merge
Source review found no introduced production defect. Isolation and shutdown behavior still require clean, discriminating execution; this does not isolate all system I/O from user flush writers.
Verification correction
Earlier descriptions overstated correctness and/or test coverage. Those claims are withdrawn. AI-assisted source review has been performed; this is not maintainer approval. Previously mixed build artifacts are not accepted as verification evidence. Corrective changes and clean, targeted verification are in progress; the published head has not yet been replaced.
Published head under review:
3a3290709b9376060b67d378662a38f55bb98ec8.