Skip to content

Harness: context overflow, errors and stream robustness in the proxy #263

Closed

Description

@GoDiao

Part of #261. Second of three batches for the proxy layer between the CLI and third-party providers.

This batch is about what happens when a session grows or something goes wrong: sessions that get stuck at the context limit, errors that lose their meaning on the way to the CLI, retries that should not happen, and streams that end badly. Each item links to its finding, which has the current behavior, a concrete scenario, how Codex CLI and ZCode handle it, and a fix direction.

High

  • Context overflow is not recognized, and the session gets stuck (P-NET-02). The CLI only matches Anthropic's prompt is too long. OpenAI, DashScope, Moonshot and Zhipu (1261) overflow errors are not recognized, so every later turn fails and the "drop oldest messages and retry" path of /compact never runs; only /clear helps. Map known overflow errors (ZCode keeps a match table) to the Anthropic form, in the proxy and for the direct path in the CLI.
  • Missing usage is written as 0, so auto-compaction never triggers (P-RES-01). The CLI estimates context from the last usage; with a 0 baseline the conversation grows until the provider rejects it, which then hits the item above. Kimi puts usage in choices[0].usage, which the proxy does not read (inferred). Do not write 0 when usage is absent, read the alternative location, and treat all-zero usage as missing in the CLI.

Medium

  • Upstream errors lose their structure (P-NET-03). The proxy escapes the upstream body into one api_error string and drops response headers, so a 429 shows only Upstream returned HTTP 429: {\ and retry-after and the request id are lost. Parse the body into a structured message and type, and pass retry-after and the request id through.
  • Retry classification is built for Anthropic (P-NET-04). 401 is retried 10 times (about 160 s before the user sees it), quota and balance errors are retried, a 2xx error body becomes a retried 502, and with streaming plus non-streaming fallback one call can make up to 22 requests over about 25 minutes. Do not retry auth and quota errors for API-key providers; map status codes from the error content.
  • Finish reasons are mapped too coarsely (P-RES-05). content_filter, Zhipu sensitive, network_error, model_context_window_exceeded and DeepSeek insufficient_system_resource all become end_turn; a Responses incomplete caused by content filtering becomes max_tokens, so the CLI keeps asking the model to continue.
  • Mid-stream errors lose their code and force a non-streaming retry (P-RES-06). Non-streaming requests are capped at 120 seconds in total, so long answers cannot finish. Map the error code, and set CLAUDE_CODE_DISABLE_NONSTREAMING_FALLBACK so stream errors are retried as streams.
  • A stream that closes before its end marker counts as a normal end (P-RES-04). The answer is cut with no message and no retry. Emit an SSE error when no finish reason or completion event was seen.
  • One delta carrying text, reasoning and tool calls is only partly handled (P-RES-08). Process all three kinds in each delta.
  • Truncated or broken tool arguments become {} (P-RES-07). The model sees "missing parameter" instead of "your arguments were cut off"; the non-streaming path yields {raw} instead. Do not run the tool; return an error result saying the arguments were incomplete, the same on both paths.

Low

  • Responses stream conversion relies on deltas only (P-RES-09). Text is lost without content_part.added, completed items are not read, blocks are not closed on response.completed, and the model name is read from the wrong field.
  • A reply with valid chunks but no content ends silently (P-RES-12). Emit an error so the CLI can retry or tell the user; ZCode retries once.

Overview of all three batches: docs/features/harness.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingenhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions