diff --git a/CHANGELOG.md b/CHANGELOG.md index 22d8638..ce2c1ee 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -11,6 +11,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - npm publishes only when plugin or catalog files change since the last tag — CI, tests, and scripts-only merges skip a release. +### Added + +- Catalog models include vision vs text-only from the Command Code CLI `inputModalities` field on every SKU. models.dev only adds extra inputs (video/audio/pdf) on matches. + ## [0.6.1] - 2026-08-28 ### Added diff --git a/README.md b/README.md index dbe6e4e..368f560 100644 --- a/README.md +++ b/README.md @@ -19,6 +19,7 @@ This package is based on **[FanFan4204/opencode-commandcode-provider](https://gi - Bundled `models.json` is the default runtime catalog (no local CLI scrape). - CLI cost extraction can fail (as on `command-code@1.38.x`) without dropping models. - Official docs fill missing costs; remaining paid gaps use [models.dev](https://models.dev) as a reference. Command Code free SKUs stay `$0`. +- Vision vs text-only comes from the Command Code CLI catalog (`inputModalities` on every SKU). [models.dev](https://models.dev) only adds extra inputs (video/audio/pdf) when it matches. - Reasoning effort **variants** on models that declare `reasoningEfforts`. - Quiet OpenCode startup (diagnostics go to `startup.json`, not stdout). diff --git a/docs/2026-08-28-catalog-modalities/plan.md b/docs/2026-08-28-catalog-modalities/plan.md new file mode 100644 index 0000000..a4f95e1 --- /dev/null +++ b/docs/2026-08-28-catalog-modalities/plan.md @@ -0,0 +1,313 @@ +# Catalog Modalities Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Spec:** `docs/2026-08-28-catalog-modalities/spec.md` +**Branch:** `feature/2026-08-28-catalog-modalities` + +**Goal:** Bundled catalog models expose OpenCode `attachment` and `modalities` copied from models.dev, with unmatched SKUs defaulting to text-only. + +**Architecture:** Extend the existing models.dev parser and matching index. A new `applyModelsDevModalities` runs on every catalog model (unlike the cost skip set). `generateOpencodeModels` always emits the fields. Sync uses one models.dev fetch for both costs and modalities. + +**Tech Stack:** TypeScript, Bun (`bun test tests/unit/`), existing `src/costs-models-dev.ts` + `src/catalog.ts`. + +## Global Constraints + +- No vision allowlist. No session-prompt injection. No overwriting `limit` / `reasoning` / `tool_call`. +- Runtime does not fetch models.dev. +- Tests: `bun test tests/unit/` from `/home/cristhofer-pincetti/Documents/projects/personal/opencode-commandcode-provider`. +- Plugin imports stay `.js`; tests import `.ts`. +- Do not fold this package into workit. PRs against `BrainerVirus/opencode-commandcode` base `main`. +- Implementation on `feature/2026-08-28-catalog-modalities` (in-place checkout, no worktrees). + +--- + +### Task 1: Parse and apply modalities from models.dev + +**Files:** +- Modify: `src/costs-models-dev.ts` +- Modify: `tests/fixtures/models-dev/api.subset.json` +- Test: `tests/unit/costs-models-dev.test.ts` + +**Interfaces:** +- Consumes: existing `parseModelsDev`, `findRow` / `indexRows` matching +- Produces: + - `TEXT_ONLY_MODALITIES = { input: ["text"], output: ["text"] }` + - `ModelsDevRow` gains optional `attachment?: boolean` and `modalities?: { input: string[]; output: string[] }` + - `applyModelsDevModalities(models: ModelEntry[], rows: ModelsDevRow[]): number` — returns how many models matched a models.dev row with explicit attachment/modalities; every model is assigned fields + +- [ ] **Step 1: Write the failing test** + +Add vision + text-only rows to `tests/fixtures/models-dev/api.subset.json`: + +```json +{ + "google": { + "models": { + "google/gemini-3.5-flash": { + "id": "google/gemini-3.5-flash", + "name": "Gemini 3.5 Flash", + "attachment": true, + "modalities": { + "input": ["text", "image", "video", "audio", "pdf"], + "output": ["text"] + }, + "cost": { "input": 1.5, "output": 9, "cache_read": 0.15 } + } + } + }, + "tencent": { + "models": { + "tencent/hy4-preview": { + "id": "tencent/hy4-preview", + "name": "Tencent Hy4 Preview", + "attachment": false, + "modalities": { "input": ["text"], "output": ["text"] }, + "cost": { "input": 0.834, "output": 2.501, "cache_read": 0.042 } + } + } + } +} +``` + +Keep the existing `zai`, `vercel`, and `qwen` objects. Qwen stays cost-only (no modalities in the fixture) so the apply function must default it to text-only. + +In `tests/unit/costs-models-dev.test.ts`, add: + +```ts +import { applyModelsDevModalities, TEXT_ONLY_MODALITIES } from "../../src/costs-models-dev.ts"; + +describe("applyModelsDevModalities", () => { + const rows = parseModelsDev( + readFileSync(join(import.meta.dir, "../fixtures/models-dev/api.subset.json"), "utf-8"), + ); + + test("copies vision modalities and defaults unmatched plus cost-only rows to text-only", () => { + const models = [ + model({ id: "google/gemini-3.5-flash", name: "Gemini 3.5 Flash" }), + model({ id: "tencent/hy4-preview", name: "Tencent Hy4 Preview" }), + model({ id: "Qwen/Qwen3.6-Plus", name: "Qwen 3.6 Plus" }), + model({ id: "unknown/not-on-models-dev", name: "Unknown" }), + model({ + id: "inclusionai/ling-3.0-flash-free", + name: "Ling 3.0 Flash Free", + cost: { input: 0, output: 0 }, + }), + ]; + const n = applyModelsDevModalities(models, rows); + expect(n).toBe(2); + expect(models[0].attachment).toBe(true); + expect(models[0].modalities).toEqual({ + input: ["text", "image", "video", "audio", "pdf"], + output: ["text"], + }); + expect(models[1].attachment).toBe(false); + expect(models[1].modalities).toEqual(TEXT_ONLY_MODALITIES); + expect(models[2].attachment).toBe(false); + expect(models[2].modalities).toEqual(TEXT_ONLY_MODALITIES); + expect(models[3].attachment).toBe(false); + expect(models[3].modalities).toEqual(TEXT_ONLY_MODALITIES); + expect(models[4].attachment).toBe(false); + expect(models[4].modalities).toEqual(TEXT_ONLY_MODALITIES); + }); +}); +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `bun test tests/unit/costs-models-dev.test.ts` + +Expected: FAIL — `applyModelsDevModalities` / `TEXT_ONLY_MODALITIES` not exported. + +- [ ] **Step 3: Write minimal implementation** + +In `src/costs-models-dev.ts`: + +```ts +export const TEXT_ONLY_MODALITIES = { input: ["text"], output: ["text"] } as const; + +export type ModelsDevRow = { + id: string; + name: string; + cost: { input: number; output: number; cache_read?: number; cache_write?: number }; + attachment?: boolean; + modalities?: { input: string[]; output: string[] }; +}; +``` + +Extend `ModelsDevModel` with optional `attachment?: boolean` and `modalities?: { input?: string[]; output?: string[] }`. + +In `parseModelsDev`, after building `cost`, copy: + +```ts +if (typeof model.attachment === "boolean") row.attachment = model.attachment; +const input = model.modalities?.input?.filter((x) => typeof x === "string"); +const output = model.modalities?.output?.filter((x) => typeof x === "string"); +if (input?.length || output?.length) { + row.modalities = { + input: input?.length ? input : ["text"], + output: output?.length ? output : ["text"], + }; +} +``` + +Add `applyModelsDevModalities` using the existing `indexRows` / `findRow`. For each model: if the row has `modalities` or `attachment !== undefined`, copy them (`attachment` defaults to `modalities.input.includes("image")` when omitted); otherwise assign `attachment: false` and `{ ...TEXT_ONLY_MODALITIES }` (spread into a mutable `{ input: string[]; output: string[] }`). Return the count of models that used an explicit models.dev attachment/modalities (not the default). + +Extend `ModelEntry` in `src/catalog.ts`: + +```ts +attachment?: boolean; +modalities?: { input: string[]; output: string[] }; +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `bun test tests/unit/costs-models-dev.test.ts` + +Expected: PASS + +- [ ] **Step 5: Commit** (only if the user asked to commit) + +```bash +git add src/costs-models-dev.ts src/catalog.ts tests/unit/costs-models-dev.test.ts tests/fixtures/models-dev/api.subset.json +git commit -m "feat: copy vision and text-only modalities from models.dev" +``` + +--- + +### Task 2: Emit attachment and modalities on OpenCode models + +**Files:** +- Modify: `src/catalog.ts` (`generateOpencodeModels`) +- Test: `tests/unit/catalog.test.ts` + +**Interfaces:** +- Consumes: `ModelEntry.attachment` / `ModelEntry.modalities` from Task 1 +- Produces: each OpenCode model object includes `attachment: boolean` and `modalities: { input: string[]; output: string[] }`; missing fields emit text-only + +- [ ] **Step 1: Write the failing test** + +In `describe("generateOpencodeModels")` add: + +```ts +test("emits attachment and modalities, defaulting to text-only", () => { + const models = generateOpencodeModels([ + { + id: "google/gemini-3.5-flash", + name: "Gemini 3.5 Flash", + tier: "open-source", + reasoning: false, + tool_call: true, + cost: { input: 1.5, output: 9 }, + limit: { context: 1048576, output: 65536 }, + attachment: true, + modalities: { input: ["text", "image"], output: ["text"] }, + }, + { + id: "tencent/hy4-preview", + name: "Tencent Hy4 Preview", + tier: "open-source", + reasoning: true, + tool_call: true, + cost: { input: 0.834, output: 2.501 }, + limit: { context: 1048576, output: 64000 }, + }, + ]); + const gemini = models["gemini-3.5-flash"] as Record; + const hy4 = models["hy4-preview"] as Record; + expect(gemini.attachment).toBe(true); + expect(gemini.modalities).toEqual({ input: ["text", "image"], output: ["text"] }); + expect(hy4.attachment).toBe(false); + expect(hy4.modalities).toEqual({ input: ["text"], output: ["text"] }); +}); +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `bun test tests/unit/catalog.test.ts` + +Expected: FAIL — `attachment` / `modalities` undefined on emitted models. + +- [ ] **Step 3: Write minimal implementation** + +In `generateOpencodeModels`, after building `model`: + +```ts +model.attachment = entry.attachment ?? false; +model.modalities = entry.modalities ?? { input: ["text"], output: ["text"] }; +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `bun test tests/unit/catalog.test.ts` + +Expected: PASS + +- [ ] **Step 5: Commit** (only if the user asked to commit) + +```bash +git add src/catalog.ts tests/unit/catalog.test.ts +git commit -m "feat: emit OpenCode attachment and modalities" +``` + +--- + +### Task 3: Wire sync, refresh catalog, document + +**Files:** +- Modify: `scripts/sync-models.ts` +- Modify: `README.md` (What this package adds) +- Modify: `CHANGELOG.md` via `workit_changelog_apply` +- Modify: `models.json` (from `bun run sync`) + +**Interfaces:** +- Consumes: `parseModelsDev`, `applyModelsDevCosts`, `applyModelsDevModalities` +- Produces: one models.dev parse shared by costs and modalities; if fetch throws, still `applyModelsDevModalities(entries, [])` so every written model is text-only tagged + +- [ ] **Step 1: Write the failing test** + +No new unit test for the script. Assert the sync wiring by reading `scripts/sync-models.ts` in the step 4 check: `applyModelsDevModalities` is called with the same parsed rows, and the catch path still applies empty rows. + +- [ ] **Step 2: Run existing tests (still pass; wiring not covered)** + +Run: `bun test tests/unit/` + +Expected: PASS (Task 1–2 tests). + +- [ ] **Step 3: Write minimal implementation** + +In `scripts/sync-models.ts`, import `applyModelsDevModalities`. After the models.dev try/catch, always apply modalities: + +```ts +let modelsDevRows: ReturnType = []; +try { + const modelsDevJson = await fetchModelsDevJson(); + modelsDevRows = parseModelsDev(modelsDevJson); + const filled = applyModelsDevCosts(entries, modelsDevRows, skipAfterFree, thirdPartyIds); + console.log(` Applied models.dev costs to ${filled} models`); +} catch (err) { + const message = err instanceof Error ? err.message : String(err); + console.log(` models.dev fill skipped: ${message}`); +} +const modalityFilled = applyModelsDevModalities(entries, modelsDevRows); +console.log(` Applied models.dev modalities to ${modalityFilled} models`); +``` + +README, under “What this package adds”, add: vision vs text-only (`attachment` / `modalities`) is copied from models.dev during catalog sync; unmatched models are text-only. + +Changelog Unreleased / Added: catalog models include vision and text-only metadata from models.dev. + +Run: `bun run sync` from the repo root (needs network). Then `bun run check`. + +- [ ] **Step 4: Verify** + +- `bun run check` passes +- `models.json` contains `"attachment"` and `"modalities"` on `tencent/hy4-preview` with `attachment: false` and input `["text"]` +- A known vision id (e.g. `google/gemini-3.5-flash`) has `attachment: true` and `"image"` in `modalities.input` + +- [ ] **Step 5: Commit** (only if the user asked to commit) + +```bash +git add scripts/sync-models.ts README.md CHANGELOG.md models.json manifest.json +git commit -m "feat: sync OpenCode modalities from models.dev" +``` diff --git a/docs/2026-08-28-catalog-modalities/spec.md b/docs/2026-08-28-catalog-modalities/spec.md new file mode 100644 index 0000000..5eff6a5 --- /dev/null +++ b/docs/2026-08-28-catalog-modalities/spec.md @@ -0,0 +1,60 @@ +# Spec: Catalog modalities from Command Code CLI + +Status: approved (2026-08-28) +**Branch:** `feature/2026-08-28-catalog-modalities` + +Parent: [docs/2026-08-28-ci-catalog/spec.md](../2026-08-28-ci-catalog/spec.md) + +## Goals + +- G1: Every Command Code SKU in the bundled catalog carries OpenCode `attachment` and `modalities` from the CLI catalog `inputModalities` field (same extract as ids/reasoning). +- G2: models.dev only **enriches** extra input types (`video` / `audio` / `pdf`) on matches. It never overwrites CLI vision/text-only and never invents text-only for a SKU the CLI already classified. +- G3: Cost waterfall stays independent. + +## Locked (do not reopen) + +- Command Code CLI catalog is the source of truth for native vision vs text-only. The public GitHub repo is a stub; the field lives on every model object in the npm `command-code` bundle as `inputModalities: ["text"]` or `["text","image"]`. +- Provider API `GET /provider/v1/models` has `context_length` only — no vision flags. Official docs Capabilities column is icons, not structured data. +- models.dev may add extra modality strings; it must not flip a CLI text-only model to vision or wipe CLI vision on unmatched ids. +- Last-resort text-only applies only when a model has **no** CLI `inputModalities` and no models.dev match (e.g. hardcoded extras). +- Runtime still does not fetch models.dev. Sync writes the fields into `models.json`. +- Do not overwrite CLI `reasoning` / `reasoningEfforts` from models.dev. +- Do not inject capability text into the session prompt. + +## Context + +`extractModelCatalog` already evaluates full CLI model objects. `buildModelEntry` previously dropped `inputModalities` and `maxOutputTokens`. Hy4 Preview is `["text"]` in the CLI catalog (not a missing match). Gemini 3.5 Flash is `["text","image"]`. + +## Non-goals + +- Session/system-prompt injection +- Local OpenCode overlay `commandcode-modalities.ts` (redundant after this ships) +- Changing hybrid Provider API transport +- Parsing the HTML Capabilities column on commandcode.ai docs + +## Architecture + +```mermaid +flowchart TD + sync["bun run sync"] --> cli[Extract CLI catalog including inputModalities] + cli --> map["buildModelEntry: attachment + modalities per SKU"] + map --> fetch[Fetch models.dev] + fetch --> enrich[Union extra inputs on matches only] + enrich --> json[models.json] + json --> emit[generateOpencodeModels] +``` + +## Acceptance criteria + +- CA-01: `buildModelEntry` maps CLI `inputModalities` including `"image"` to `attachment: true` and maps `["text"]` to text-only, including Hy4 Preview. +- CA-02: `loadCatalogFromBundle` preserves those fields from a minified CLI-shaped catalog. +- CA-03: `applyModelsDevModalities` keeps CLI vision on ids models.dev does not list. +- CA-04: models.dev may append extra inputs (e.g. Gemini `video`/`audio`/`pdf`) without removing CLI `text`/`image`. +- CA-05: `generateOpencodeModels` always emits `attachment` and `modalities`. +- CA-06: README states vision/text-only comes from the CLI catalog. + +## Decisions + +- D-01: CLI `inputModalities` wins over models.dev for native vision vs text-only. +- D-02: Extra modality strings from models.dev are copied, not filtered to `text`/`image`. +- D-03: `maxOutputTokens` from the CLI sets `limit.output` when present. diff --git a/manifest.json b/manifest.json index 42caf7d..49be2eb 100644 --- a/manifest.json +++ b/manifest.json @@ -1,6 +1,6 @@ { "schemaVersion": 1, - "generatedAt": "2026-08-28T19:50:57.386Z", + "generatedAt": "2026-08-28T21:14:42.165Z", "pluginVersion": "0.6.1", "commandCodeVersion": "1.38.1", "commandCodeTarball": "https://registry.npmjs.org/command-code/-/command-code-1.38.1.tgz", diff --git a/models.json b/models.json index bc8195a..ac5a884 100644 --- a/models.json +++ b/models.json @@ -21,6 +21,16 @@ "limit": { "context": 1000000, "output": 65536 + }, + "attachment": true, + "modalities": { + "input": [ + "text", + "image" + ], + "output": [ + "text" + ] } }, { @@ -38,6 +48,17 @@ "limit": { "context": 200000, "output": 8192 + }, + "attachment": true, + "modalities": { + "input": [ + "text", + "image", + "pdf" + ], + "output": [ + "text" + ] } }, { @@ -62,6 +83,17 @@ "limit": { "context": 1000000, "output": 32000 + }, + "attachment": true, + "modalities": { + "input": [ + "text", + "image", + "pdf" + ], + "output": [ + "text" + ] } }, { @@ -86,6 +118,16 @@ "limit": { "context": 1000000, "output": 65536 + }, + "attachment": true, + "modalities": { + "input": [ + "text", + "image" + ], + "output": [ + "text" + ] } }, { @@ -110,6 +152,16 @@ "limit": { "context": 1000000, "output": 65536 + }, + "attachment": true, + "modalities": { + "input": [ + "text", + "image" + ], + "output": [ + "text" + ] } }, { @@ -134,6 +186,17 @@ "limit": { "context": 1000000, "output": 16000 + }, + "attachment": true, + "modalities": { + "input": [ + "text", + "image", + "pdf" + ], + "output": [ + "text" + ] } }, { @@ -158,6 +221,16 @@ "limit": { "context": 1000000, "output": 65536 + }, + "attachment": true, + "modalities": { + "input": [ + "text", + "image" + ], + "output": [ + "text" + ] } }, { @@ -180,6 +253,16 @@ "limit": { "context": 400000, "output": 128000 + }, + "attachment": true, + "modalities": { + "input": [ + "text", + "image" + ], + "output": [ + "text" + ] } }, { @@ -202,6 +285,17 @@ "limit": { "context": 400000, "output": 128000 + }, + "attachment": true, + "modalities": { + "input": [ + "text", + "image", + "pdf" + ], + "output": [ + "text" + ] } }, { @@ -223,6 +317,16 @@ "limit": { "context": 400000, "output": 128000 + }, + "attachment": true, + "modalities": { + "input": [ + "text", + "image" + ], + "output": [ + "text" + ] } }, { @@ -245,6 +349,16 @@ "limit": { "context": 400000, "output": 128000 + }, + "attachment": true, + "modalities": { + "input": [ + "text", + "image" + ], + "output": [ + "text" + ] } }, { @@ -265,6 +379,15 @@ "limit": { "context": 1000000, "output": 384000 + }, + "attachment": false, + "modalities": { + "input": [ + "text" + ], + "output": [ + "text" + ] } }, { @@ -285,6 +408,16 @@ "limit": { "context": 1000000, "output": 65536 + }, + "attachment": true, + "modalities": { + "input": [ + "text", + "image" + ], + "output": [ + "text" + ] } }, { @@ -305,6 +438,15 @@ "limit": { "context": 1000000, "output": 384000 + }, + "attachment": false, + "modalities": { + "input": [ + "text" + ], + "output": [ + "text" + ] } }, { @@ -325,6 +467,16 @@ "limit": { "context": 1000000, "output": 65536 + }, + "attachment": true, + "modalities": { + "input": [ + "text", + "image" + ], + "output": [ + "text" + ] } }, { @@ -346,6 +498,19 @@ "limit": { "context": 1000000, "output": 65536 + }, + "attachment": true, + "modalities": { + "input": [ + "text", + "image", + "video", + "audio", + "pdf" + ], + "output": [ + "text" + ] } }, { @@ -367,6 +532,19 @@ "limit": { "context": 1000000, "output": 65536 + }, + "attachment": true, + "modalities": { + "input": [ + "text", + "image", + "video", + "audio", + "pdf" + ], + "output": [ + "text" + ] } }, { @@ -388,6 +566,19 @@ "limit": { "context": 1000000, "output": 65536 + }, + "attachment": true, + "modalities": { + "input": [ + "text", + "image", + "video", + "audio", + "pdf" + ], + "output": [ + "text" + ] } }, { @@ -409,6 +600,19 @@ "limit": { "context": 1000000, "output": 65536 + }, + "attachment": true, + "modalities": { + "input": [ + "text", + "image", + "video", + "audio", + "pdf" + ], + "output": [ + "text" + ] } }, { @@ -431,6 +635,19 @@ "limit": { "context": 1048576, "output": 65536 + }, + "attachment": true, + "modalities": { + "input": [ + "text", + "image", + "video", + "audio", + "pdf" + ], + "output": [ + "text" + ] } }, { @@ -447,6 +664,15 @@ "limit": { "context": 200000, "output": 131072 + }, + "attachment": false, + "modalities": { + "input": [ + "text" + ], + "output": [ + "text" + ] } }, { @@ -463,6 +689,15 @@ "limit": { "context": 200000, "output": 131072 + }, + "attachment": false, + "modalities": { + "input": [ + "text" + ], + "output": [ + "text" + ] } }, { @@ -483,6 +718,15 @@ "limit": { "context": 1000000, "output": 65536 + }, + "attachment": false, + "modalities": { + "input": [ + "text" + ], + "output": [ + "text" + ] } }, { @@ -499,6 +743,16 @@ "limit": { "context": 1000000, "output": 65536 + }, + "attachment": true, + "modalities": { + "input": [ + "text", + "image" + ], + "output": [ + "text" + ] } }, { @@ -520,6 +774,15 @@ "limit": { "context": 1000000, "output": 65536 + }, + "attachment": false, + "modalities": { + "input": [ + "text" + ], + "output": [ + "text" + ] } }, { @@ -540,7 +803,18 @@ }, "limit": { "context": 1048576, - "output": 65536 + "output": 131072 + }, + "attachment": true, + "modalities": { + "input": [ + "text", + "image", + "video" + ], + "output": [ + "text" + ] } }, { @@ -565,6 +839,16 @@ "limit": { "context": 1050000, "output": 65536 + }, + "attachment": true, + "modalities": { + "input": [ + "text", + "image" + ], + "output": [ + "text" + ] } }, { @@ -589,6 +873,16 @@ "limit": { "context": 1050000, "output": 65536 + }, + "attachment": true, + "modalities": { + "input": [ + "text", + "image" + ], + "output": [ + "text" + ] } }, { @@ -613,6 +907,16 @@ "limit": { "context": 1050000, "output": 65536 + }, + "attachment": true, + "modalities": { + "input": [ + "text", + "image" + ], + "output": [ + "text" + ] } }, { @@ -634,6 +938,16 @@ "limit": { "context": 500000, "output": 65536 + }, + "attachment": true, + "modalities": { + "input": [ + "text", + "image" + ], + "output": [ + "text" + ] } }, { @@ -656,6 +970,16 @@ "limit": { "context": 500000, "output": 65536 + }, + "attachment": true, + "modalities": { + "input": [ + "text", + "image" + ], + "output": [ + "text" + ] } }, { @@ -672,6 +996,16 @@ "limit": { "context": 256000, "output": 65536 + }, + "attachment": true, + "modalities": { + "input": [ + "text", + "image" + ], + "output": [ + "text" + ] } }, { @@ -688,6 +1022,16 @@ "limit": { "context": 1000000, "output": 65536 + }, + "attachment": true, + "modalities": { + "input": [ + "text", + "image" + ], + "output": [ + "text" + ] } }, { @@ -704,6 +1048,16 @@ "limit": { "context": 256000, "output": 131072 + }, + "attachment": true, + "modalities": { + "input": [ + "text", + "image" + ], + "output": [ + "text" + ] } }, { @@ -720,6 +1074,17 @@ "limit": { "context": 256000, "output": 131072 + }, + "attachment": true, + "modalities": { + "input": [ + "text", + "image", + "video" + ], + "output": [ + "text" + ] } }, { @@ -736,6 +1101,17 @@ "limit": { "context": 256000, "output": 65536 + }, + "attachment": true, + "modalities": { + "input": [ + "text", + "image", + "video" + ], + "output": [ + "text" + ] } }, { @@ -752,6 +1128,16 @@ "limit": { "context": 262000, "output": 65536 + }, + "attachment": true, + "modalities": { + "input": [ + "text", + "image" + ], + "output": [ + "text" + ] } }, { @@ -768,6 +1154,17 @@ "limit": { "context": 1000000, "output": 65536 + }, + "attachment": true, + "modalities": { + "input": [ + "text", + "image", + "video" + ], + "output": [ + "text" + ] } }, { @@ -783,7 +1180,16 @@ }, "limit": { "context": 256000, - "output": 65536 + "output": 32768 + }, + "attachment": false, + "modalities": { + "input": [ + "text" + ], + "output": [ + "text" + ] } }, { @@ -798,7 +1204,16 @@ }, "limit": { "context": 256000, - "output": 65536 + "output": 32768 + }, + "attachment": false, + "modalities": { + "input": [ + "text" + ], + "output": [ + "text" + ] } }, { @@ -815,6 +1230,18 @@ "limit": { "context": 1000000, "output": 65536 + }, + "attachment": true, + "modalities": { + "input": [ + "text", + "image", + "audio", + "video" + ], + "output": [ + "text" + ] } }, { @@ -831,6 +1258,15 @@ "limit": { "context": 1000000, "output": 65536 + }, + "attachment": false, + "modalities": { + "input": [ + "text" + ], + "output": [ + "text" + ] } }, { @@ -847,6 +1283,15 @@ "limit": { "context": 200000, "output": 131072 + }, + "attachment": false, + "modalities": { + "input": [ + "text" + ], + "output": [ + "text" + ] } }, { @@ -863,6 +1308,15 @@ "limit": { "context": 1000000, "output": 131072 + }, + "attachment": false, + "modalities": { + "input": [ + "text" + ], + "output": [ + "text" + ] } }, { @@ -878,6 +1332,15 @@ "limit": { "context": 197000, "output": 65536 + }, + "attachment": false, + "modalities": { + "input": [ + "text" + ], + "output": [ + "text" + ] } }, { @@ -894,6 +1357,17 @@ "limit": { "context": 1000000, "output": 65536 + }, + "attachment": true, + "modalities": { + "input": [ + "text", + "image", + "video" + ], + "output": [ + "text" + ] } }, { @@ -909,6 +1383,17 @@ "limit": { "context": 1000000, "output": 65536 + }, + "attachment": true, + "modalities": { + "input": [ + "text", + "image", + "video" + ], + "output": [ + "text" + ] } }, { @@ -924,6 +1409,16 @@ "limit": { "context": 1000000, "output": 65536 + }, + "attachment": true, + "modalities": { + "input": [ + "text", + "image" + ], + "output": [ + "text" + ] } }, { @@ -940,6 +1435,19 @@ "limit": { "context": 1048576, "output": 65536 + }, + "attachment": true, + "modalities": { + "input": [ + "text", + "image", + "video", + "pdf", + "audio" + ], + "output": [ + "text" + ] } }, { @@ -956,6 +1464,19 @@ "limit": { "context": 1048576, "output": 65536 + }, + "attachment": true, + "modalities": { + "input": [ + "text", + "image", + "video", + "pdf", + "audio" + ], + "output": [ + "text" + ] } }, { @@ -972,6 +1493,19 @@ "limit": { "context": 1048576, "output": 65536 + }, + "attachment": true, + "modalities": { + "input": [ + "text", + "image", + "video", + "pdf", + "audio" + ], + "output": [ + "text" + ] } }, { @@ -988,6 +1522,15 @@ "limit": { "context": 1000000, "output": 65536 + }, + "attachment": false, + "modalities": { + "input": [ + "text" + ], + "output": [ + "text" + ] } }, { @@ -1005,6 +1548,15 @@ "limit": { "context": 1000000, "output": 131072 + }, + "attachment": false, + "modalities": { + "input": [ + "text" + ], + "output": [ + "text" + ] } }, { @@ -1021,6 +1573,17 @@ "limit": { "context": 1000000, "output": 131072 + }, + "attachment": true, + "modalities": { + "input": [ + "text", + "image", + "video" + ], + "output": [ + "text" + ] } }, { @@ -1038,6 +1601,17 @@ "limit": { "context": 1000000, "output": 65536 + }, + "attachment": true, + "modalities": { + "input": [ + "text", + "image", + "video" + ], + "output": [ + "text" + ] } }, { @@ -1055,6 +1629,15 @@ "limit": { "context": 1000000, "output": 131072 + }, + "attachment": false, + "modalities": { + "input": [ + "text" + ], + "output": [ + "text" + ] } }, { @@ -1072,6 +1655,17 @@ "limit": { "context": 1000000, "output": 65536 + }, + "attachment": true, + "modalities": { + "input": [ + "text", + "image", + "video" + ], + "output": [ + "text" + ] } }, { @@ -1092,7 +1686,17 @@ }, "limit": { "context": 262144, - "output": 65536 + "output": 32768 + }, + "attachment": true, + "modalities": { + "input": [ + "text", + "image" + ], + "output": [ + "text" + ] } }, { @@ -1114,6 +1718,17 @@ "limit": { "context": 1000000, "output": 65536 + }, + "attachment": true, + "modalities": { + "input": [ + "text", + "image", + "video" + ], + "output": [ + "text" + ] } }, { @@ -1136,6 +1751,17 @@ "limit": { "context": 1000000, "output": 65536 + }, + "attachment": true, + "modalities": { + "input": [ + "text", + "image", + "video" + ], + "output": [ + "text" + ] } }, { @@ -1152,6 +1778,15 @@ "limit": { "context": 1000000, "output": 131072 + }, + "attachment": false, + "modalities": { + "input": [ + "text" + ], + "output": [ + "text" + ] } }, { @@ -1168,6 +1803,17 @@ "limit": { "context": 256000, "output": 65536 + }, + "attachment": true, + "modalities": { + "input": [ + "text", + "image", + "video" + ], + "output": [ + "text" + ] } }, { @@ -1184,6 +1830,15 @@ "limit": { "context": 262144, "output": 65536 + }, + "attachment": false, + "modalities": { + "input": [ + "text" + ], + "output": [ + "text" + ] } }, { @@ -1199,6 +1854,15 @@ "limit": { "context": 262144, "output": 65536 + }, + "attachment": false, + "modalities": { + "input": [ + "text" + ], + "output": [ + "text" + ] } }, { @@ -1220,6 +1884,15 @@ "limit": { "context": 1048576, "output": 65536 + }, + "attachment": false, + "modalities": { + "input": [ + "text" + ], + "output": [ + "text" + ] } } ] diff --git a/scripts/sync-models.ts b/scripts/sync-models.ts index 364d290..ce97e4d 100644 --- a/scripts/sync-models.ts +++ b/scripts/sync-models.ts @@ -15,6 +15,7 @@ import { applyDocCosts, fetchOfficialModelsMarkdown, parseModelsTable } from ".. import { applyFreeCosts, applyModelsDevCosts, + applyModelsDevModalities, fetchModelsDevJson, parseModelsDev, } from "../src/costs-models-dev.js"; @@ -186,19 +187,18 @@ async function main() { const thirdPartyIds = new Set(); const skipAfterFree = new Set([...priced, ...freeIds]); + let modelsDevRows: ReturnType = []; try { const modelsDevJson = await fetchModelsDevJson(); - const filled = applyModelsDevCosts( - entries, - parseModelsDev(modelsDevJson), - skipAfterFree, - thirdPartyIds, - ); + modelsDevRows = parseModelsDev(modelsDevJson); + const filled = applyModelsDevCosts(entries, modelsDevRows, skipAfterFree, thirdPartyIds); console.log(` Applied models.dev costs to ${filled} models`); } catch (err) { const message = err instanceof Error ? err.message : String(err); console.log(` models.dev fill skipped: ${message}`); } + const modalityFilled = applyModelsDevModalities(entries, modelsDevRows); + console.log(` Applied models.dev modalities to ${modalityFilled} models`); console.log(`\nWriting ${MODELS_JSON} with ${entries.length} models from ${sourceLabel}...`); writeFileSync(MODELS_JSON, JSON.stringify(entries, null, 2) + "\n", "utf-8"); diff --git a/src/catalog.ts b/src/catalog.ts index 213c9ba..c350299 100644 --- a/src/catalog.ts +++ b/src/catalog.ts @@ -15,6 +15,8 @@ export interface ModelEntry { tool_call: boolean; cost: { input: number; output: number; cache_read?: number; cache_write?: number }; limit: { context: number; output: number }; + attachment?: boolean; + modalities?: { input: string[]; output: string[] }; } export interface CostEntry { @@ -38,6 +40,8 @@ export interface SnEntry { reasoning?: boolean; reasoningEfforts?: string[]; contextWindow?: number; + inputModalities?: string[]; + maxOutputTokens?: number; } export interface ResolvedCommandCodePackage { @@ -484,11 +488,21 @@ export function buildModelEntry( cost = { input: 0.5, output: 2 }; } - const limit = entry.contextWindow - ? { context: entry.contextWindow, output: FALLBACK_LIMITS[entry.id]?.output ?? 65536 } - : (FALLBACK_LIMITS[entry.id] ?? { context: 200000, output: 65536 }); + const fallback = FALLBACK_LIMITS[entry.id]; + const limit = { + context: entry.contextWindow ?? fallback?.context ?? 200000, + output: entry.maxOutputTokens ?? fallback?.output ?? 65536, + }; const efforts = entry.reasoningEfforts?.length ? [...entry.reasoningEfforts] : undefined; + const input = entry.inputModalities?.filter((x) => typeof x === "string") ?? []; + const fromCli = + input.length > 0 + ? { + attachment: input.includes("image"), + modalities: { input: [...input], output: ["text"] }, + } + : {}; return { id: entry.id, @@ -499,6 +513,7 @@ export function buildModelEntry( tool_call: true, cost, limit, + ...fromCli, }; } @@ -740,6 +755,8 @@ export function generateOpencodeModels(entries: ModelEntry[]): Record }; @@ -42,7 +47,17 @@ export function parseModelsDev(json: string): ModelsDevRow[] { const cost: ModelsDevRow["cost"] = { input: model.cost.input, output: model.cost.output }; if (model.cost.cache_read !== undefined) cost.cache_read = model.cost.cache_read; if (model.cost.cache_write !== undefined) cost.cache_write = model.cost.cache_write; - rows.push({ id: model.id, name: model.name ?? model.id, cost }); + const row: ModelsDevRow = { id: model.id, name: model.name ?? model.id, cost }; + if (typeof model.attachment === "boolean") row.attachment = model.attachment; + const input = model.modalities?.input?.filter((x) => typeof x === "string"); + const output = model.modalities?.output?.filter((x) => typeof x === "string"); + if (input?.length || output?.length) { + row.modalities = { + input: input?.length ? input : [...TEXT_ONLY_MODALITIES.input], + output: output?.length ? output : [...TEXT_ONLY_MODALITIES.output], + }; + } + rows.push(row); } } return rows; @@ -86,6 +101,43 @@ export function applyFreeCosts( return filled; } +function textOnly(): { input: string[]; output: string[] } { + return { input: [...TEXT_ONLY_MODALITIES.input], output: [...TEXT_ONLY_MODALITIES.output] }; +} + +export function applyModelsDevModalities(models: ModelEntry[], rows: ModelsDevRow[]): number { + const index = indexRows(rows); + let filled = 0; + for (const model of models) { + const row = findRow(model, index); + const current = model.modalities; + if (current && model.attachment !== undefined) { + const extra = row?.modalities?.input?.filter((x) => !current.input.includes(x)) ?? []; + if (extra.length > 0) { + model.modalities = { + input: [...current.input, ...extra], + output: [...current.output], + }; + if (model.modalities.input.includes("image")) model.attachment = true; + filled++; + } + continue; + } + if (row && (row.modalities || row.attachment !== undefined)) { + const modalities = row.modalities + ? { input: [...row.modalities.input], output: [...row.modalities.output] } + : textOnly(); + model.modalities = modalities; + model.attachment = row.attachment ?? modalities.input.includes("image"); + filled++; + } else { + model.attachment = false; + model.modalities = textOnly(); + } + } + return filled; +} + export function applyModelsDevCosts( models: ModelEntry[], rows: ModelsDevRow[], diff --git a/tests/fixtures/models-dev/api.subset.json b/tests/fixtures/models-dev/api.subset.json index 80fa860..cbc9dbd 100644 --- a/tests/fixtures/models-dev/api.subset.json +++ b/tests/fixtures/models-dev/api.subset.json @@ -4,10 +4,26 @@ "google/gemini-3.5-flash": { "id": "google/gemini-3.5-flash", "name": "Gemini 3.5 Flash", + "attachment": true, + "modalities": { + "input": ["text", "image", "video", "audio", "pdf"], + "output": ["text"] + }, "cost": { "input": 1.5, "output": 9, "cache_read": 0.15 } } } }, + "tencent": { + "models": { + "tencent/hy4-preview": { + "id": "tencent/hy4-preview", + "name": "Tencent Hy4 Preview", + "attachment": false, + "modalities": { "input": ["text"], "output": ["text"] }, + "cost": { "input": 0.834, "output": 2.501, "cache_read": 0.042 } + } + } + }, "zai": { "models": { "zai-org/GLM-5.1": { diff --git a/tests/unit/catalog.test.ts b/tests/unit/catalog.test.ts index 7c40eb6..c99ee56 100644 --- a/tests/unit/catalog.test.ts +++ b/tests/unit/catalog.test.ts @@ -95,6 +95,41 @@ describe("buildModelEntry", () => { expect(entry!.cost).toEqual({ input: 0.5, output: 2 }); }); + test("maps CLI inputModalities to attachment and modalities for every SKU", () => { + const vision = buildModelEntry( + { + id: "google/gemini-3.5-flash", + provider: "vercel-ai-gateway", + spec: "chatComplete", + label: "Gemini", + name: "Gemini 3.5 Flash", + description: "d", + inputModalities: ["text", "image"], + contextWindow: 1e6, + }, + new Map(), + ); + const text = buildModelEntry( + { + id: "tencent/hy4-preview", + provider: "vercel-ai-gateway", + spec: "chatComplete", + label: "Hy4", + name: "Tencent Hy4 Preview", + description: "d", + inputModalities: ["text"], + contextWindow: 1048576, + maxOutputTokens: 64000, + }, + new Map(), + ); + expect(vision!.attachment).toBe(true); + expect(vision!.modalities).toEqual({ input: ["text", "image"], output: ["text"] }); + expect(text!.attachment).toBe(false); + expect(text!.modalities).toEqual({ input: ["text"], output: ["text"] }); + expect(text!.limit).toEqual({ context: 1048576, output: 64000 }); + }); + test("does not invent a billed rate for models missing from the CLI cost map", () => { const sn: SnEntry = { id: "google/gemini-3.5-flash", @@ -158,6 +193,37 @@ describe("generateOpencodeModels", () => { high: { reasoningEffort: "high" }, }); }); + + test("emits attachment and modalities, defaulting to text-only", () => { + const models = generateOpencodeModels([ + { + id: "google/gemini-3.5-flash", + name: "Gemini 3.5 Flash", + tier: "open-source", + reasoning: false, + tool_call: true, + cost: { input: 1.5, output: 9 }, + limit: { context: 1048576, output: 65536 }, + attachment: true, + modalities: { input: ["text", "image"], output: ["text"] }, + }, + { + id: "tencent/hy4-preview", + name: "Tencent Hy4 Preview", + tier: "open-source", + reasoning: true, + tool_call: true, + cost: { input: 0.834, output: 2.501 }, + limit: { context: 1048576, output: 64000 }, + }, + ]); + const gemini = models["gemini-3.5-flash"] as Record; + const hy4 = models["hy4-preview"] as Record; + expect(gemini.attachment).toBe(true); + expect(gemini.modalities).toEqual({ input: ["text", "image"], output: ["text"] }); + expect(hy4.attachment).toBe(false); + expect(hy4.modalities).toEqual({ input: ["text"], output: ["text"] }); + }); }); describe("loadCatalogFromBundle", () => { @@ -167,8 +233,8 @@ describe("loadCatalogFromBundle", () => { '(Wt={ANTHROPIC:"anthropic",OPENAI:"openai",VERCEL_AI_GATEWAY:"vercel-ai-gateway"});', 'var Aa="chatComplete",Ba="responses",qt=Vt[0];', "var Sn=(Wt=>({", - 'SONNET_4_6:{id:"claude-sonnet-4-6",provider:Wt.ANTHROPIC,spec:Aa,label:"Sonnet",name:"Claude Sonnet 4.6",description:"d",reasoning:!0,reasoningEfforts:["low","medium","high"],contextWindow:2e5},', - 'GPT_X:{id:"gpt-5.5",provider:Wt.OPENAI,spec:Ba,label:"GPT",name:"GPT-5.5",description:"d",reasoningEfforts:["low","high"],contextWindow:256000}', + 'SONNET_4_6:{id:"claude-sonnet-4-6",provider:Wt.ANTHROPIC,spec:Aa,label:"Sonnet",name:"Claude Sonnet 4.6",description:"d",inputModalities:["text","image"],reasoning:!0,reasoningEfforts:["low","medium","high"],contextWindow:2e5},', + 'GPT_X:{id:"gpt-5.5",provider:Wt.OPENAI,spec:Ba,label:"GPT",name:"GPT-5.5",description:"d",inputModalities:["text"],reasoningEfforts:["low","high"],contextWindow:256000}', "}))(Wt);", 'var costs={anthropic:[{id:"anthropic:claude-sonnet-4-6",provider:"anthropic",category:"p",promptCost:3,completionCost:15,cacheWrite5mCost:3.75,cacheWrite1hCost:6,cacheHitCost:.3}],openai:[{id:"openai:gpt-5.5",provider:"openai",category:"p",promptCost:1,completionCost:2,cacheWrite5mCost:0,cacheWrite1hCost:0,cacheHitCost:0}]};', ].join(""); @@ -182,11 +248,15 @@ describe("loadCatalogFromBundle", () => { expect(sonnet!.reasoningEfforts).toEqual(["low", "medium", "high"]); expect(sonnet!.tier).toBe("premium"); expect(sonnet!.limit.context).toBe(200000); + expect(sonnet!.attachment).toBe(true); + expect(sonnet!.modalities).toEqual({ input: ["text", "image"], output: ["text"] }); const gpt = entries.find((e) => e.id === "gpt-5.5"); expect(gpt).toBeDefined(); expect(gpt!.reasoning).toBe(true); expect(gpt!.reasoningEfforts).toEqual(["low", "high"]); + expect(gpt!.attachment).toBe(false); + expect(gpt!.modalities).toEqual({ input: ["text"], output: ["text"] }); }); test("returns models when cost extraction fails", () => { @@ -194,8 +264,8 @@ describe("loadCatalogFromBundle", () => { '(Wt={ANTHROPIC:"anthropic",OPENAI:"openai",VERCEL_AI_GATEWAY:"vercel-ai-gateway"});', 'var Aa="chatComplete",Ba="responses",qt=Vt[0];', "var Sn=(Wt=>({", - 'SONNET_4_6:{id:"claude-sonnet-4-6",provider:Wt.ANTHROPIC,spec:Aa,label:"Sonnet",name:"Claude Sonnet 4.6",description:"d",reasoning:!0,reasoningEfforts:["low","medium","high"],contextWindow:2e5},', - 'GPT_X:{id:"gpt-5.5",provider:Wt.OPENAI,spec:Ba,label:"GPT",name:"GPT-5.5",description:"d",reasoningEfforts:["low","high"],contextWindow:256000}', + 'SONNET_4_6:{id:"claude-sonnet-4-6",provider:Wt.ANTHROPIC,spec:Aa,label:"Sonnet",name:"Claude Sonnet 4.6",description:"d",inputModalities:["text","image"],reasoning:!0,reasoningEfforts:["low","medium","high"],contextWindow:2e5},', + 'GPT_X:{id:"gpt-5.5",provider:Wt.OPENAI,spec:Ba,label:"GPT",name:"GPT-5.5",description:"d",inputModalities:["text"],reasoningEfforts:["low","high"],contextWindow:256000}', "}))(Wt);", ].join(""); diff --git a/tests/unit/costs-models-dev.test.ts b/tests/unit/costs-models-dev.test.ts index d0a1489..9490905 100644 --- a/tests/unit/costs-models-dev.test.ts +++ b/tests/unit/costs-models-dev.test.ts @@ -4,8 +4,10 @@ import { join } from "path"; import { applyFreeCosts, applyModelsDevCosts, + applyModelsDevModalities, isFreeSku, parseModelsDev, + TEXT_ONLY_MODALITIES, } from "../../src/costs-models-dev.ts"; import type { ModelEntry } from "../../src/catalog.ts"; @@ -75,6 +77,48 @@ describe("parseModelsDev + applyModelsDevCosts", () => { }); }); +describe("applyModelsDevModalities", () => { + const rows = parseModelsDev( + readFileSync(join(import.meta.dir, "../fixtures/models-dev/api.subset.json"), "utf-8"), + ); + + test("keeps CLI modalities on unmatched SKUs and only enriches extras from models.dev", () => { + const models = [ + model({ + id: "google/gemini-3.5-flash", + name: "Gemini 3.5 Flash", + attachment: true, + modalities: { input: ["text", "image"], output: ["text"] }, + }), + model({ + id: "unknown/cli-vision-only", + name: "CLI Vision Only", + attachment: true, + modalities: { input: ["text", "image"], output: ["text"] }, + }), + model({ + id: "tencent/hy4-preview", + name: "Tencent Hy4 Preview", + attachment: false, + modalities: { input: ["text"], output: ["text"] }, + }), + model({ id: "no-cli-no-dev", name: "Gap" }), + ]; + const n = applyModelsDevModalities(models, rows); + expect(n).toBe(1); + expect(models[0].modalities).toEqual({ + input: ["text", "image", "video", "audio", "pdf"], + output: ["text"], + }); + expect(models[1].attachment).toBe(true); + expect(models[1].modalities).toEqual({ input: ["text", "image"], output: ["text"] }); + expect(models[2].attachment).toBe(false); + expect(models[2].modalities).toEqual({ ...TEXT_ONLY_MODALITIES }); + expect(models[3].attachment).toBe(false); + expect(models[3].modalities).toEqual({ ...TEXT_ONLY_MODALITIES }); + }); +}); + describe("applyFreeCosts", () => { test("sets free SKUs to zero and leaves paid models alone", () => { const models = [