feat: share a single OkHttpClient with injectable daemon threads across the ADK#1349
Open
copybara-service[bot] wants to merge 1 commit into
Open
feat: share a single OkHttpClient with injectable daemon threads across the ADK#1349copybara-service[bot] wants to merge 1 commit into
copybara-service[bot] wants to merge 1 commit into
Conversation
copybara-service
Bot
force-pushed
the
test_945640677
branch
2 times, most recently
from
5f02d1a to
67a4070
Compare
…ss the ADK Building a separate `OkHttpClient` per model or session leaks connection pools and threads. This routes `Gemini`, `ChatCompletionsHttpClient`, and the sessions `ApiClient` through `HttpClientFactory`, which caches one shared client per name so the dispatcher and connection pool are reused. Threading is injectable at construction rather than hard-coded: - `getOrCreateSharedHttpClient(name)`: the shared client, OkHttp's default threading. - `createHttpClient(executor)`: a new client whose dispatcher runs on the given executor; not cached, so the caller owns the executor and the returned client. - `daemonExecutor(name)`: an unbounded daemon-thread pool so a standalone or CLI JVM can exit once work is done. - `Gemini.builder().httpExecutorService(...)` and a new `ChatCompletionsHttpClient(HttpOptions, ExecutorService)` constructor accept the executor at construction. The sessions client issues synchronous requests, which do not use the dispatcher executor, so it shares the pool only. `Gemini` forks the shared client with zeroed connect/read/write timeouts to keep the prior infinite-timeout behavior. Adds `HttpClientFactoryTest` covering caching by name, executor injection, and daemon threads. PiperOrigin-RevId: 945640677
copybara-service
Bot
force-pushed
the
test_945640677
branch
from
67a4070 to
d3add2d
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
feat: share a single OkHttpClient with injectable daemon threads across the ADK
Building a separate
OkHttpClientper model or session leaks connection pools andthreads. This routes
Gemini,ChatCompletionsHttpClient, and the sessionsApiClientthroughHttpClientFactory, which caches one shared client per name sothe dispatcher and connection pool are reused.
Threading is injectable at construction rather than hard-coded:
getOrCreateSharedHttpClient(name): the shared client, OkHttp's defaultthreading.
createHttpClient(executor): a new client whose dispatcher runs on the givenexecutor; not cached, so the caller owns the executor and the returned client.
daemonExecutor(name): an unbounded daemon-thread pool so a standalone or CLIJVM can exit once work is done.
Gemini.builder().httpExecutorService(...)and a newChatCompletionsHttpClient(HttpOptions, ExecutorService)constructor accept theexecutor at construction. The sessions client issues synchronous requests, which
do not use the dispatcher executor, so it shares the pool only.
Geminiforks the shared client with zeroed connect/read/write timeouts to keepthe prior infinite-timeout behavior.
Adds
HttpClientFactoryTestcovering caching by name, executor injection, anddaemon threads.