SOLR-17707: release the content-writing thread when an async request fails#4639
Merged
dsmiley merged 2 commits intoJul 16, 2026
Merged
Conversation
…fails HttpJdkSolrClient's async paths never cleaned up the content-writing task, so a connection failure while the body was being written left that thread blocked forever in PipedInputStream.awaitSpace(). Since the writer shares the client's executor, enough such failures exhaust the pool. The synchronous path already cancels the writer in its finally block; the async paths did not. Both async paths now release the writer on completion. Closing the pipe's sink is what unblocks a writer already stuck in awaitSpace(); cancelling the future alone does not, since the interrupt does not reliably reach the blocked thread.
dsmiley
reviewed
Jul 15, 2026
dsmiley
left a comment
dsmiley
left a comment
Contributor
There was a problem hiding this comment.
Nice! Very valuable fix.
Import CountDownLatch instead of fully-qualifying it, and wait for the request to settle via handle().get() rather than a catch that errorprone flags as MissingFail.
dsmiley
approved these changes
Jul 16, 2026
dsmiley
left a comment
dsmiley
left a comment
Contributor
There was a problem hiding this comment.
I'm so glad to see a fix here!
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Picking up the POC Jan left on SOLR-17707.
HttpJdkSolrClient's async paths (requestAsync,requestInputStreamAsync) never cleaned up the content-writing task. When a connection fails while the request body is still being written, that thread stays blocked forever inPipedInputStream.awaitSpace(). Because the writer runs on the client's own executor, enough of these exhaust the pool and the client stops working — the thread dump in the issue shows exactly this. The synchronousrequestWithBaseUrlalready cancels the writer in itsfinally; the async paths did not.Both async paths now release the writer via
whenComplete.One thing worth calling out for review: cancelling the
Futurealone does not fix it. In a repro I foundcancel(true)returnstruebut the interrupt does not reliably reach the thread blocked in the pipe (it staysTIMED_WAITING,interrupted=false). Closing the pipe's sink is what deterministically unblocks the writer (IOException: Pipe closed). The fix does both — close to release a stuck writer, cancel to stop one that hasn't started.Test:
testStuckContentWritingThreadIsReleasedOnFailuresends a body larger than the pipe/socket buffers to a server that accepts but never reads, then drops the connection, and asserts no writer thread is left blocked inawaitSpace(). It fails without the fix and passes with it. This is the reliable reproduction that was hard to pin down earlier — the key was a body big enough to actually block the writer before the failure.