Skip to content

fix(eth): keep the peer transaction broadcaster draining after a send error - #2520

Open
gzliudan wants to merge 3 commits into
XinFinOrg:dev-upgradefrom
gzliudan:fix-peer-broadcast-drain
Open

fix(eth): keep the peer transaction broadcaster draining after a send error#2520
gzliudan wants to merge 3 commits into
XinFinOrg:dev-upgradefrom
gzliudan:fix-peer-broadcast-drain

Conversation

@gzliudan

@gzliudan gzliudan commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator

Proposed changes

Fixes a production hang on mainnet where a peer with a broken connection stopped servicing its transaction broadcast queues, eventually stalling the transaction pool for hours.

Root cause

broadcastTransactions and announceTransactions returned on the first network send error, leaving p.txBroadcast / p.txAnnounce without a reader. p.term is only ever closed once the peer handler unwinds through removePeer — and in production it was never closed at all, because peerSet.Unregister was missing the p.close() call. A peer that was already tearing down could therefore sit with no reader and an open term, and every later AsyncSendTransactions / AsyncSendPooledTransactionHashes blocked forever.

On mainnet this stalled txBroadcastLoop, filled pm.txsCh and pinned the transaction pool lock for hours. Six peers were found in this state during inspection: they still had a live broadcastBlocks goroutine but no transaction broadcaster.

Changes

eth/peer.go

  • broadcastTransactions / announceTransactions no longer return on a send error. After the first failure they set failed, drop the queued work and keep consuming the queue until p.term, so the channels always have a reader for the lifetime of the peer and senders can never block forever.
  • The fail channel is now buffered (capacity 1), so a send goroutine can report its failure even after the loop has already exited through p.term. A plain default case was not an option: both channels are unbuffered and would drop batches whenever a send is in flight.
  • peerSet.Unregister now calls p.close(), terminating the broadcast goroutines when the peer is removed. This restores the upstream go-ethereum lifecycle (upstream closes the peer via defer peer.Close() in the protocol Run). The close happens under the peer set lock, so concurrent removePeer calls cannot close term twice.
  • peer.close is now idempotent via sync.Once, hardening the exactly-once invariant against repeated or concurrent calls.

eth/peer_test.go

  • TestBroadcastTransactionsKeepsDrainingAfterSendFailure
  • TestAnnounceTransactionsKeepsDrainingAfterSendFailure
  • TestPeerSetUnregisterTerminatesBroadcasters
  • TestPeerCloseIsIdempotent

The drain tests reproduce the mainnet freeze: with the connection broken, they assert that repeated AsyncSend* calls never block, that exactly one write attempt is made, and that no further writes happen after the failure.

Compatibility / impact

  • No protocol, consensus, RPC or database changes; purely a peer lifecycle and goroutine drain fix.
  • Matches the behavior already present in upstream go-ethereum (verified against v1.10.26, eth/protocols/eth/broadcast.go); reduces divergence from upstream.
  • No migration or node-operator coordination required.

Types of changes

What types of changes does your code introduce to XDC network?
Put an in the boxes that apply

  • build: Changes that affect the build system or external dependencies
  • ci: Changes to CI configuration files and scripts
  • chore: Changes that don't change source code or tests
  • docs: Documentation only changes
  • feat: A new feature
  • fix: A bug fix
  • perf: A code change that improves performance
  • refactor: A code change that neither fixes a bug nor adds a feature
  • revert: Revert something
  • style: Changes that do not affect the meaning of the code
  • test: Adding missing tests or correcting existing tests

Impacted Components

Which parts of the codebase does this PR touch?
Put an in the boxes that apply

  • Consensus
  • Account
  • Network
  • Geth
  • Smart Contract
  • External components
  • Not sure (Please specify below)

Checklist

Put an in the boxes once you have confirmed below actions (or provide reasons on not doing so) that

  • This PR has sufficient test coverage (unit/integration test) OR I have provided reason in the PR description for not having test coverage
  • Tested on a private network from the genesis block and monitored the chain operating correctly for multiple epochs.
  • Provide an end-to-end test plan in the PR description on how to manually test it on the devnet/testnet.
  • Tested the backwards compatibility.
  • Tested with XDC nodes running this version co-exist with those running the previous version.
  • Relevant documentation has been updated as part of this PR
  • N/A

@coderabbitai

coderabbitai Bot commented Aug 12, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: e2424fc5-e92c-434a-8641-f9d07d9c6c89

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Prevents transaction propagation from blocking when peer network sends fail.

Changes:

  • Keeps transaction broadcast and announcement queues draining after failures.
  • Buffers failure signals.
  • Adds regression tests for both paths.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
eth/peer.go Updates broadcaster failure handling.
eth/peer_test.go Adds send-failure draining tests.
Suppressed comments (1)

eth/peer.go:287

  • This loop has the same lifecycle leak as the full-transaction broadcaster: no production path closes p.term when handle returns (removePeer only unregisters and disconnects). Once failed is set, this goroutine therefore survives permanently holding the peer. Ensure teardown closes term exactly once (including concurrent removePeer calls) so the drain state has a real endpoint.
			// p.term is only closed once the peer handler unwinds, so this loop has to
			// stay around as a reader or AsyncSendPooledTransactionHashes would block
			// forever.
			failed, queue, done = true, nil, nil

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread eth/peer.go
@gzliudan
gzliudan force-pushed the fix-peer-broadcast-drain branch 3 times, most recently from 51483d0 to 92b4e9c Compare August 13, 2026 02:04
@gzliudan
gzliudan requested a balanced review from Copilot August 13, 2026 02:33

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

@gzliudan
gzliudan force-pushed the fix-peer-broadcast-drain branch 2 times, most recently from 2a7e8e0 to 47e53a3 Compare August 13, 2026 03:02
@gzliudan
gzliudan requested a balanced review from Copilot August 13, 2026 05:03

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

@gzliudan
gzliudan force-pushed the fix-peer-broadcast-drain branch from 47e53a3 to 53ee99f Compare August 16, 2026 15:36
peerSet.Unregister never closed p.term, so the broadcast goroutines of
every removed peer leaked for the lifetime of the process. Close term
under the peer set lock so the draining loops wind down on removal.
broadcastTransactions and announceTransactions returned on the first
network send error, leaving p.txBroadcast and p.txAnnounce without a
reader while the peer was still registered; every later
AsyncSendTransactions blocked forever. On mainnet that stalled
txBroadcastLoop, filled pm.txsCh and pinned the transaction pool lock
for hours.

Stop sending after a failure but keep servicing the queue until p.term,
discarding queued events once sending is hopeless. Buffer the fail
channel so an in-flight sender whose error races the loop's term exit
cannot block forever on an unbuffered send.
Guard the close of term with a sync.Once so repeated or concurrent calls
cannot panic on a double close. Ownership of term stays with
peerSet.Unregister; the once hardens the exactly-once invariant.
@gzliudan
gzliudan force-pushed the fix-peer-broadcast-drain branch from 53ee99f to dca69c9 Compare August 16, 2026 16:12
@gzliudan
gzliudan requested a balanced review from Copilot August 16, 2026 22:05

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants