fix(#3996): drop gateway SSE keepalive frames before the genai parser - #4022
Draft
aheritier wants to merge 2 commits into
Draft
fix(#3996): drop gateway SSE keepalive frames before the genai parser#4022aheritier wants to merge 2 commits into
aheritier wants to merge 2 commits into
Conversation
Exhaustively cover wantsImageResponseModalities (added by the preceding compatibility-guard commit): true only on supported Gemini surfaces, only when output_capabilities.image resolves true (explicit override taking precedence over models.dev output modalities), and only outside title-generation/compaction utility calls. Also pin catalogue resolution, override precedence, request construction, and the existing request-shape diagnostics.
During long generations (e.g. Gemini image output) the Docker AI Gateway
emits `event: keepalive` + `data: {}` frames. google.golang.org/genai
treats any `event:` line as a fatal invalid stream chunk, so a single
keepalive killed the whole stream.
Add an opt-in httpclient.WithSSEKeepaliveFilter that makes the shared SSE
filter transport also drop payload-free keepalive frames, and enable it
only on the Gemini gateway HTTP client. The shared default path is
unchanged: Anthropic-style named events with meaningful data still pass
through verbatim, and only the gateway surface — never direct
Gemini/Vertex — gets the new behavior.
aheritier
added
area/providers/gemini
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Filters gateway SSE keepalive frames before they reach the Gemini genai parser, while retaining normal response events.
Why
Gateway transport heartbeats are not model events and can otherwise cause parser failures or corrupt streaming behavior.
Validation
task test; HTTP SSE filter and Gemini transport tests.Test instructions
Run:
task testStream a Gemini response containing gateway SSE keepalive frames followed by content and a terminal event. Expected: keepalive frames are discarded before parsing, content remains intact, and the stream terminates normally.