fix(core/txpool): stop delivering special tx events under the pool lock - #2519
Open
gzliudan wants to merge 1 commit into
Open
fix(core/txpool): stop delivering special tx events under the pool lock#2519gzliudan wants to merge 1 commit into
gzliudan wants to merge 1 commit into
Conversation
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
gzliudan
requested review from
AnilChinchawale,
anunay-xin,
benjamin202410,
liam-lai and
wanwiset25
August 12, 2026 23:09
benjamin202410
approved these changes
Aug 13, 2026
gzliudan
force-pushed
the
fix-special-tx-feed-under-lock
branch
5 times, most recently
from
August 16, 2026 15:35
d7614e7 to
18dd2d4
Compare
There was a problem hiding this comment.
Pull request overview
Moves special-transaction event delivery outside the txpool lock to prevent stalled subscribers from freezing pool operations.
Changes:
- Queues special transaction events through the reorg path.
- Adds announcement limits and revised shutdown behavior.
- Adds regression tests for stalled subscribers and reset handling.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 6 comments.
| File | Description |
|---|---|
core/txpool/legacypool/legacypool.go |
Revises event delivery, reorg scheduling, and shutdown handling. |
core/txpool/legacypool/legacypool_test.go |
Adds regression tests for event delivery and queue behavior. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+1268
to
+1269
| queuedEvents = make(map[common.Address]*SortedMap) | ||
| queuedEventCount = 0 |
Comment on lines
+1311
to
+1319
| if queuedEventCount >= maxQueuedEvents && !tx.IsSpecialTransaction() { | ||
| // The set was created above only for this event; remove it | ||
| // again to keep the map free of entries holding no events. | ||
| if set.Len() == 0 { | ||
| delete(queuedEvents, addr) | ||
| } | ||
| announcementDropMeter.Mark(1) | ||
| log.Trace("Transaction event queue full, dropping announcement", "hash", tx.Hash()) | ||
| continue |
Comment on lines
+1372
to
+1376
| select { | ||
| case <-pool.reorgShutdownCh: | ||
| pool.mu.Unlock() | ||
| close(done) | ||
| return |
Comment on lines
+1420
to
+1421
| for addr, set := range events { | ||
| set.Filter(func(tx *types.Transaction) bool { return staleEvent(addr, tx) }) |
| // that stopped draining (e.g. a peer whose transaction broadcaster died), | ||
| // and waiters such as Reset or a synchronous Add must not be pinned behind | ||
| // such a subscriber. | ||
| close(done) |
Comment on lines
+3970
to
+3971
| pool.queueTxEvent(specialTx) | ||
| <-pool.requestPromoteExecutables(newAccountSet(pool.signer)) |
gzliudan
force-pushed
the
fix-special-tx-feed-under-lock
branch
from
August 16, 2026 16:44
18dd2d4 to
ff3fbb0
Compare
promoteSpecialTx called txFeed.Send while holding pool.mu, so a subscriber that stopped draining froze the whole pool. On mainnet a peer lost its transaction broadcaster, AsyncSendTransactions blocked, txBroadcastLoop stopped draining pm.txsCh, and the resulting Feed.Send pinned pool.mu for hours: 131 goroutines piled up in Add, 20 in Pending, runReorg never ran and the node stopped importing blocks while RPC stayed responsive. Route the event through queueTxEvent like every other path in add(), so runReorg delivers it after releasing the lock. Add TestSpecialTxPromotionDoesNotBlockOnTxFeed, which adds a special tx while a subscriber refuses to read and asserts both the add and a later pool read complete.
gzliudan
force-pushed
the
fix-special-tx-feed-under-lock
branch
from
August 16, 2026 16:48
ff3fbb0 to
b70f81c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Proposed changes
Problem
A mainnet node froze for ~2h40m while RPC stayed responsive: no new blocks were imported.
debug.stacks()showed the full failure chain:broadcastTransactionsexited while the peer stayed alive).eth/peer.goAsyncSendTransactionsblocked forever onp.txBroadcast, which no longer had a reader.txBroadcastLoopstopped drainingpm.txsChand became a stalledtxFeedsubscriber.LegacyPool.promoteSpecialTxcalledpool.txFeed.Sendwhile holdingpool.mu, soFeed.Sendpinned the pool write lock for hours.Result: 131 goroutines piled up in
LegacyPool.Add, 20 inPending,runReorgnever ran, and the node stopped importing blocks.Fix
Route the special tx event through
queueTxEvent, the same path every other transaction inadd()uses. The event is delivered byrunReorgafterpool.muis released, so a stalled subscriber can no longer pin the pool lock. This also aligns the XDPoS-specific path with the upstream geth design.Behavior change
Remote adds (
sync=false) now deliver theNewTxsEventfor special txs after the next reorg run instead of synchronously insideAdd— identical to normal transactions. Local adds (sync=true) still deliver beforeAddreturns. No consensus, RPC, or API surface changes.Scope / follow-up
This fixes the pool-side amplifier only. The peer-side defect that starts the chain (
eth/peer.go:broadcastTransactionscan exit while the peer stays alive, leavingAsyncSendTransactionsblocked forever) is tracked separately and needs its own fix. Until then, a stalled subscriber plus a fullpm.txsChcan block the reorg goroutine, but no longerAdd/Pending.Types of changes
What types of changes does your code introduce to XDC network?
Put an
✅in the boxes that applyImpacted Components
Which parts of the codebase does this PR touch?
Put an
✅in the boxes that applyChecklist
Put an
✅in the boxes once you have confirmed below actions (or provide reasons on not doing so) that