[Feat] Automatic approvals: a decision model risk-assesses tool calls nobody has made a choice about

[daniel-lxs]

Part of the experimental per-tool approvals (integrationToolApprovals). Auto mode is a deployment-wide setting that lets a decision model risk-assess tool calls nobody has made a choice about, run the routine ones, and ask a person about the risky ones. A manual choice always wins.

Per-tool modes

Each tool row offers three stored choices: Always allow (Auto never looks), Always ask (always a person), Disable (hidden and refused). Auto is the default and shows as nothing pressed; pressing the selected choice again returns the tool to Auto. The group row carries the same buttons plus Auto, so a whole group can go back to it in one click. Personal policies still only tighten. The tool rows themselves are the toggle-button rows from #3103, unchanged apart from Auto no longer being a button of its own.

Auto mode (its own card on Settings → Integrations, admins only, shown while the experiment is on)

Customer-facing copy, no mechanics: "Let Roomote handle routine work and ask before anything risky. Your other tool choices stay the same." Off: "Keep each tool’s current choice." On: "Handle routine work automatically and ask before anything risky." The guidance field sits behind an "Additional instructions" disclosure, named like the other guidance fields in the app. When On can't be enabled, the card just says "Auto mode isn’t available yet."

• Off: tools run as they always have; Always ask still asks a person.
• On: every call to a tool left on Auto is assessed first. Routine → runs; risky → the approval card, with "Auto flagged this call as risky and asked you." Disable, Always allow, Always ask, and session overrides are never touched.
• Additional instructions (collapsed by default): what this deployment treats as routine or risky, in the admin's words. Given to the model as context, not as rules.
• On requires a hosted judgment model. The helper-model fallback would be an LLM call per tool call, so the On option is disabled until one is configured, and the control says why.
• Shadow is not an option: while Off, if a hosted judgment model is configured, every default tool call is still assessed in the background at the integration proxy and recorded in a new integration_tool_auto_evaluations table (migration 0114, additive), so the model's judgment can be reviewed against real traffic before turning On.

The assessment

Following the checked-in typesafe-ai skill: one Score question over five described situations (reads only → easy to undo → sends to people → spends or grants access → destroys data), plus Noul questions for "matches what the user asked" (only when a request is known), "steered by untrusted content", and "the guidance flags calls like this" (only when guidance is set). The guidance rides in the state, not the questions. The decision is in code: run only at the lowest risk level with confidence, matching the request, not steered, not flagged. Otherwise ask. The model never rejects; no model, an error, or a timeout all ask.

Enforcement

• Sessions: with Auto On, default tools compile to a native ask so the bridge can assess each call; a routine call runs through the reservation-and-claim path as auto_approved with no decider and the model's answers on the row.
• Tasks: default tools compile to <server>_*: ask; at start the worker lists each Auto-gated server's tools and maps every native key to the real tool name (a key two tools share, or a server that cannot be listed, gets its asks refused), and sends the user's latest request with the ask; the server decides who answers from the governing policies (never from what the worker says); a routine call is left approved for the integration proxy to claim for the exact arguments, once. The proxy gates every default tool call of a task while On.
• A tool asked under a rule compiled while Auto was on, after it went off, simply runs.

Testing

Unit and real-database tests across types, db, cloud-agents, sdk, api, worker, and web. Smoke-tested on a local stack with Jev via OpenRouter as the judgment model:

• Off + hosted model (shadow): a Session's read_wiki_structure call ran untouched and a shadow row was recorded (risk 0, confidence 1).
• On: the same call to a tool with no policy row was auto-approved by the model (risk 0, matches request 0.98, steered 0.08); no card.
• On + manual Always ask: the same tool set to Always ask showed the card with no model view on it.
• Earlier: guidance flagging a call made it ask (guidance 0.99).

Screenshots

The Auto mode card on Settings → Integrations, collapsed and with the additional instructions opened:

The per-tool choices in Manage tools: Auto is the default and shows as nothing pressed, the group row can return every tool to it, and pressing a choice again returns that tool to Auto:

An approval card after Auto judged a call risky and asked a person:

[roomote-community]

No new code issues found. See task

• Preserve an explicit session-level ask override instead of allowing Auto to bypass it.
• Expose and render the stored Auto evaluation on approval cards for calls that still need a person.
• Keep Caddy's local API and web proxy targets aligned with the development stack's published ports.
• Preserve the real MCP tool identity for task Auto wildcard asks so evaluation and approval claims cannot cross sanitized-name collisions, including collisions between different server/tool pairs.
• Hide the Auto mode section from non-admin members so it does not make unauthorized settings requests or render empty.

_{Reviewed 6ed52fd}