Repository navigation
fix(proxy): fix the basics of the OpenAI-format request path - #266
Merged
Merged
Conversation
Part of #262. Each item below came with a test that failed before the change. Cache and request shape: - The proxy adds and signs the Claude Code billing line only for models whose name starts with `claude` (reusing attributionHeaderPolicy). The line changed on every request, so OpenAI and GLM could only reuse the cached tool definitions. Measured on OpenRouter, later turns now hit the cache for about 99% of the input. - Responses requests carry the CLI session id as `prompt_cache_key`; requests with tools send `parallel_tool_calls: true`; tool schemas drop the `$schema` URL and zod's safe-integer bounds. - Cache hits are also read from `prompt_cache_hit_tokens` and a top-level `cached_tokens`. Responses report the requested model name. Transport: - The proxy route turns off Bun's 60-second idle timeout per request. - A base URL that already ends in a version (`/v4`, `/v1beta/openai`) is used as the full prefix for requests, model lists and connection tests; others still get `/v1`. - The caller's abort signal reaches the upstream fetch, also before the upstream answers and for non-streaming requests. - With a manual HTTP proxy, the CLI env gets NO_PROXY for loopback, merged with the inherited value (an inherited `*` is kept as is). Scheduled tasks now get the network env too. - One stream supervisor times idle gaps (aiRequestTimeoutMs; only events sent to the CLI count, keep-alive comments become `ping`) and records how the stream ended. The CLI direct path gets the same limit through CLAUDE_ENABLE_STREAM_WATCHDOG and CLAUDE_STREAM_IDLE_TIMEOUT_MS. - Every upstream request is recorded as an `upstream_request` diagnostics event (provider, model, host, status, timings, request id, outcome); never the key or the body. diagnosticsService resolves its paths before awaiting, so a pending write cannot land in another config dir. - All three SSE readers share one spec framing function; a formatted JSON error body is surfaced instead of an empty reply. - OpenAI-format providers can carry extraHeaders and queryParams; configured headers go first, auth and Content-Type override them. They are rejected for anthropic-format providers. No desktop form yet. - The proxy answers count_tokens and other unknown paths with an Anthropic not_found_error. The server marks proxied providers with DREAMCODER_PROVIDER_PROXIED, and /context uses the local estimate for them instead of a Haiku request. Docs: CHANGELOG entries, status lines on the fixed items in docs/features/harness/proxy, measured cache results in the harness README.
This was referenced Nov 18, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of #262. Each item below came with a test that failed before the
change.
Cache and request shape:
whose name starts with
claude(reusing attributionHeaderPolicy). Theline changed on every request, so OpenAI and GLM could only reuse the
cached tool definitions. Measured on OpenRouter, later turns now hit the
cache for about 99% of the input.
prompt_cache_key;requests with tools send
parallel_tool_calls: true; tool schemas dropthe
$schemaURL and zod's safe-integer bounds.prompt_cache_hit_tokensand a top-levelcached_tokens. Responses report the requested model name.Transport:
/v4,/v1beta/openai) isused as the full prefix for requests, model lists and connection tests;
others still get
/v1.upstream answers and for non-streaming requests.
with the inherited value (an inherited
*is kept as is). Scheduledtasks now get the network env too.
sent to the CLI count, keep-alive comments become
ping) and records howthe stream ended. The CLI direct path gets the same limit through
CLAUDE_ENABLE_STREAM_WATCHDOG and CLAUDE_STREAM_IDLE_TIMEOUT_MS.
upstream_requestdiagnosticsevent (provider, model, host, status, timings, request id, outcome);
never the key or the body. diagnosticsService resolves its paths before
awaiting, so a pending write cannot land in another config dir.
error body is surfaced instead of an empty reply.
configured headers go first, auth and Content-Type override them. They
are rejected for anthropic-format providers. No desktop form yet.
not_found_error. The server marks proxied providers with
DREAMCODER_PROVIDER_PROXIED, and /context uses the local estimate for
them instead of a Haiku request.
Docs: CHANGELOG entries, status lines on the fixed items in
docs/features/harness/proxy, measured cache results in the harness README.
This PR also carries the two research documents the issue builds on:
docs/features/harnesscomparisons of the agent runtime and the proxylayer with Codex CLI and ZCode.
How I tested it
bun test --randomize: 341 pass, 0 fail.bun test: 2807 pass, 11 fail;mainfails the same 11 locally(ripgrep not on PATH and other local environment causes).
desktop:bun run lintpasses,bun run test -- --run3104 pass.sendToUpstreamagainst OpenRouter with atest key: gpt-4.1-mini hits about 99% of the input from turn 2, glm-4.6
hits 13312 of 13371 input tokens on turn 3.