Skip to content

Harness: per-model capabilities in the proxy #264

Open

Description

@GoDiao

Part of #261. Third of three batches for the proxy layer between the CLI and third-party providers.

This batch makes the request fit the model actually in use. Today DreamCoder infers capabilities from Claude model names, a custom provider declares every capability for every model, and per-model knowledge is spread across several functions. ZCode keeps one declarative rule file (model-name pattern plus endpoint) that decides context window, output limit, reasoning levels and which request field carries them; Codex keeps a model catalog with one fallback entry.

Design first

  • A per-model rule table that the proxy and the CLI both read: context window, output limit, reasoning levels and the field they map to, image and PDF support, reasoning round-trip format, native web search, extra body fields. Decide its schema, where defaults come from (presets, the provider's models endpoint, user overrides), and how a model not listed anywhere is handled. Write the design in docs/features/harness before the items below.

High

  • Effort and the thinking switch never reach the provider (P-REQ-02). On both OpenAI formats output_config.effort is dropped, adaptive and disabled produce no field, DeepSeek and opencode.ai hosts receive the Anthropic-only adaptive value, and the Qwen preset cannot turn thinking on.
  • Capabilities are declared per provider, not per model (P-NET-09, P-NET-14). A custom provider declares adaptive thinking and max effort for every model with no way to change it; presets disagree on which fields they send for the same setting.
  • Reasoning data is not sent back (P-RES-02, P-REQ-07, P-REQ-08). Gemini thought_signature, OpenRouter reasoning_details and Responses encrypted_content are dropped in the response direction (Gemini 3 is documented to reject the next tool call without the signature; inferred). reasoning_content is written back only for deepseek and opencode.ai hosts, so Kimi and DeepSeek on other hosts lose it. Record which field the reasoning came from and write it back the same way.
  • PDFs and images are dropped or sent blindly (P-REQ-03, P-REQ-04, P-REQ-11). PDF document blocks are dropped and the model only sees "PDF file read"; images in tool results (Read, browser screenshots, MCP) become [image omitted]; user images are sent even to text-only models. Carry them in the form each format supports, and replace them with a clear note when the model cannot take them.
  • Context windows for unlisted models (P-NET-10). The Qwen preset declares no window, windows are fetched only when the user clicks "Fetch models", and anything unlisted is treated as 200,000 tokens.

Medium

  • Output limit (P-REQ-05). max_tokens is not forwarded, so the provider's default applies; forwarding the CLI's 32,000 as is would exceed some models' limits. Send the model's limit in the right field, and nothing when it is unknown.
  • Native web search becomes an empty function (P-REQ-09). The search "succeeds" with no results and never falls back to a configured Tavily or Brave backend. Decide from the model's capability whether the native tool is offered.

Low

  • Title requests cannot turn thinking off or limit output on OpenAI formats (P-REQ-14); thinking models can exceed the 15-second timeout. Follows from the effort and output-limit mapping.
  • Structured output (output_config.format) is dropped (P-REQ-15). Map it to response_format or text.format.
  • <think> tags inside text are shown as the answer (P-RES-10), for example MiniMax M2 (inferred). At least strip them from title generation; how to display them is a product decision.
  • No per-model extra body fields (P-REQ-16). Express them in the rule table rather than as ad-hoc host checks.

Overview of all three batches: docs/features/harness.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions