From 36ff653e8dc586f51506ed3b8597b8383511e522 Mon Sep 17 00:00:00 2001 From: Bruno Perez Date: Thu, 30 Jul 2026 15:13:14 +0200 Subject: [PATCH] feat: add an agent skill for looking up model params Last of four, on top of the MCP server. Skills are how coding agents pick up procedural knowledge without an MCP connection, so this covers the agents that can't or don't run the server. skills.sh format: one directory, one SKILL.md, installable with `npx skills add mnfst/modelparams.dev`. The skill teaches the thing agents get wrong: look parameters up rather than recalling them, because training data goes stale as providers drop knobs. It covers all three access paths and when to prefer each, gives a workflow for writing a call and one for debugging a provider 400, and lists the gotchas worth checking by hand. The description is deliberately loaded with the error strings people paste into a search box, since that is what has to match for the skill to fire. --- CONTRIBUTING.md | 1 + README.md | 5 +- skills/llm-model-parameters/SKILL.md | 157 +++++++++++++++++++++++++++ src/views/api.ejs | 4 + 4 files changed, 165 insertions(+), 2 deletions(-) create mode 100644 skills/llm-model-parameters/SKILL.md diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index e0c3f44..7ed76d8 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -189,6 +189,7 @@ Everything the catalog ships beyond the static site: so run `npm run codegen --workspace=modelparams` after changing the catalog. - `packages/modelparams-mcp/` — the MCP server. Ships in lockstep with `modelparams` and pins it exactly. +- `skills/` — agent skills, installable with `npx skills add mnfst/modelparams.dev`. Conventions: diff --git a/README.md b/README.md index 928841c..0e8a1e5 100644 --- a/README.md +++ b/README.md @@ -75,10 +75,11 @@ curl -s https://modelparams.dev/api/v1/validate \ ## Agents ```bash -npx -y modelparams-mcp # MCP server: 4 tools, stdio, no network needed +npx -y modelparams-mcp # MCP server: 4 tools, stdio, no network needed +npx skills add mnfst/modelparams.dev # the companion agent skill ``` -The server exposes `validate_model_params`, `get_model_params`, `list_models`, and `find_models_supporting`. Details in the [package README](packages/modelparams-mcp/README.md). There's also [llms.txt](https://modelparams.dev/llms.txt) if you'd rather just point an agent at a URL. +The MCP server exposes `validate_model_params`, `get_model_params`, `list_models`, and `find_models_supporting`. Details in the [package README](packages/modelparams-mcp/README.md). There's also [llms.txt](https://modelparams.dev/llms.txt) if you'd rather just point an agent at a URL. ## Adding a model diff --git a/skills/llm-model-parameters/SKILL.md b/skills/llm-model-parameters/SKILL.md new file mode 100644 index 0000000..66c301a --- /dev/null +++ b/skills/llm-model-parameters/SKILL.md @@ -0,0 +1,157 @@ +--- +name: llm-model-parameters +description: Look up and validate the parameters an LLM accepts before calling it — temperature, top_p, top_k, reasoning_effort, thinking budgets — including which combinations a provider rejects. Use when writing or reviewing code that calls an LLM API with non-default parameters, when picking a model by the knobs it exposes, when building a model settings UI or an LLM router, or when debugging a provider 400 such as "unsupported parameter", "unrecognized request argument", "unsupported value", or "temperature and top_p cannot both be specified". Backed by the open modelparams.dev catalog. +--- + +# LLM model parameters + +Model parameters are not uniform and not stable. `gpt-5.5` has no `temperature`. +Claude Opus 4.7 dropped it. Anthropic rejects `top_p` unless `temperature` is 1. +Reasoning models reject sampling knobs entirely. Training data goes stale on all +of this, so **look it up rather than recalling it.** + +## When to use this skill + +- Writing or editing code that passes parameters to an LLM API. +- Debugging a 400 from a provider, or output that ignores a parameter you set. +- Choosing a model based on a knob you need (a thinking budget, a seed, `top_k`). +- Building a model picker, settings UI, eval harness, gateway, or router. +- Reviewing a diff that hardcodes parameters across multiple models. + +## Pick an access path + +| Situation | Use | +| ------------------------------------ | ---------------------------------------------------- | +| An MCP server is available to you | The `modelparams` MCP tools — no network, no parsing | +| Any agent or shell, one-off question | The HTTP API (below) | +| You're writing TypeScript that ships | The `modelparams` npm package — compile-time safety | + +### MCP tools + +If the `modelparams` MCP server is connected, prefer these — they need no network access: + +- `validate_model_params` — the one to reach for. Give it a model and a params + object; it returns what's wrong and a corrected `safeParams` payload. +- `get_model_params` — every parameter for one model, with types, ranges, + defaults, and conditional rules. +- `list_models` — find the exact catalog id, filtered by provider or substring. +- `find_models_supporting` — which models expose a given parameter. + +Not connected? Install it: + +```bash +npx -y modelparams-mcp # stdio server; add to your MCP client config +``` + +### HTTP API + +CORS-enabled, no key, no rate limit. + +```bash +# Validate a request before you send it — the highest-value call +curl -s https://modelparams.dev/api/v1/validate \ + -H 'Content-Type: application/json' \ + -d '{"model":"claude-3-opus-20240229","params":{"temperature":0.5,"top_p":0.9}}' +``` + +```jsonc +{ + "model": "anthropic/claude-3-opus-20240229", + "valid": false, + "issues": [ + { + "path": "top_p", + "code": "not_applicable", // or unknown_parameter | invalid_value + "message": "top_p does not apply when temperature ≠ 1", + "conflictsWith": ["temperature"], + }, + ], + "safeParams": { "temperature": 0.5 }, // always safe to send as-is +} +``` + +Other endpoints: + +```bash +curl https://modelparams.dev/api/v1/params/gpt-5.5.json # one model's params +curl https://modelparams.dev/api/v1/models/anthropic/claude-opus-4-7.json +curl https://modelparams.dev/api/v1/models.json # full catalog +curl https://modelparams.dev/api/v1/index.json # endpoint map + live count +``` + +Ids are `provider/model`; subscription contracts append `-subscription`. The +validate endpoint also accepts a bare slug when only one provider publishes it. + +### npm package + +```bash +npm i modelparams +``` + +```ts +import { dropUnsupported, parseParams, type ParamsOf } from "modelparams"; + +// Compile-time: passing a parameter the model doesn't have won't build. +const params: ParamsOf<"openai/gpt-4.1"> = { max_tokens: 1024, temperature: 0.7 }; + +// Runtime, for untrusted input — enforces conflicts too. +const result = parseParams("openai/gpt-4.1", req.body.params); +if (!result.success) return res.status(422).json({ issues: result.issues }); + +// Or strip whatever won't fly and proceed (like LiteLLM's drop_params, +// extended to conditional conflicts). +const { params: safe, dropped } = dropUnsupported("openai/gpt-5.5", userParams); +``` + +## Workflow: writing code that calls an LLM + +1. Resolve the exact catalog id (`list_models`, or `/api/v1/models.json`). +2. Fetch the parameter list for that model — do not assume it matches a sibling + model or an earlier version. +3. Validate the params object you intend to send. +4. If invalid, use `safeParams` or fix the call. Do not silently drop the + parameter without telling the user which one went and why. + +## Workflow: debugging a provider 400 + +1. Extract the model id and the params object from the failing call. +2. Run `validate_model_params` (or POST to `/api/v1/validate`). +3. Match the `code`: + - `unknown_parameter` — the model has no such knob. Remove it. + - `invalid_value` — right knob, out of range or not in the enum. + - `not_applicable` — the knob exists but conflicts with another value in the + same request. Check `conflictsWith`; change one or drop the other. +4. If validation passes, the problem is not parameter shape — look at auth, + the endpoint, or the message payload instead. + +## Conditional rules + +The catalog's distinguishing data is _applicability_: when a parameter is +accepted, expressed in terms of the others in the same request. + +- `appliesOnlyWhen: { only: {...} }` — accepted only while that condition holds. +- `appliesOnlyWhen: { except: {...} }` — rejected while that condition holds. + +A parameter you never set still counts, because the provider applies its +default. Setting `thinking.budget_tokens` without `thinking.type: "enabled"` is +a silent no-op, and validation reports it. + +## Gotchas worth checking explicitly + +- Reasoning models (GPT-5.x, o-series, Claude with thinking on) reject or ignore + `temperature` and `top_p`. This is the single most common source of 400s. +- Anthropic rejects `top_p` alongside a non-default `temperature`. +- OpenAI reasoning models take `max_completion_tokens`, not `max_tokens`. +- Google nests everything under `generationConfig.*`. +- Some providers accept an unsupported parameter and ignore it silently — no + error, just drifting evals. Validate rather than trusting a 200. +- API-key and subscription contracts can expose different parameters for the + same model. Check the `-subscription` variant if that's how you authenticate. + +## Contributing a correction + +The catalog is open and community-maintained. If a model is missing or a +parameter is wrong, the data is YAML at +`models/{provider}/{model}.yaml` in +[github.com/mnfst/modelparams.dev](https://github.com/mnfst/modelparams.dev) — +one file, one PR, CI validates it against the published JSON Schema. diff --git a/src/views/api.ejs b/src/views/api.ejs index 0df5744..0f41f87 100644 --- a/src/views/api.ejs +++ b/src/views/api.ejs @@ -126,6 +126,10 @@ list_models, find_models_supporting.

+

+ Using a coding agent that supports skills? Install the companion skill with + npx skills add mnfst/modelparams.dev. +