Skip to content

feat: select work tools per headless run #254

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
byapparov merged 5 commits into from
Oct 21, 2026
Merged

feat: select work tools per headless run #254

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
2db98b3
feat: select work tools per headless run
byapparov Oct 21, 2026
9088fa9
ci: provision ripgrep for guarded selector tests
byapparov Oct 21, 2026
ede414a
fix: preserve tool-selection failures and fixture cleanup
byapparov Oct 21, 2026
26c53b2
merge: integrate main contracts with tool selection
byapparov Oct 21, 2026
46f98d5
fix: align selection contracts and reject cleanup false positives
byapparov Oct 21, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
93 changes: 88 additions & 5 deletions packages/cli/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,89 @@ The headless agent harness for CI, cron jobs, and container workflows.
- **Declarative Permissions:** Per-tool allow/deny rules (`permission` config); tools denied everywhere are stripped from the model request to save tokens.
- **Skills & MCP:** Progressive skill loading (`--skill`) and first-class [Model Context Protocol](https://modelcontextprotocol.io) support.

## Select work tools for one run

Use exact registered names to choose an independent work-tool allowlist:

```bash
aictrl run --tools read,glob,grep "Find the configuration entry points"
aictrl run --tools '' "Answer using only the supplied prompt"
```

Omitting `--tools` uses the final config's optional `tool_allowlist` preference;
without either input, normal tools remain available. For example,
`"tool_allowlist": ["read", "grep"]` sets a default, and `[]` selects no work
tools. Later config sources replace the whole array, including an empty array.
The CLI replaces that preference for the current invocation. It does not change
the next invocation's default, including on reused or forked sessions.

Names are case-sensitive and exact: `read`, `write`, and `edit` are independent;
no aliases or automatic dependencies are added. Exact duplicates are removed in
first-seen order. Whitespace-only names, surrounding whitespace, empty comma
elements and unknown or model/flag-unavailable names fail before model inference.
A selected tool that permissions deny remains denied. Existing permission
aliases, path checks and last-match rules still apply.

Start with a toolset that matches the task and the model's available catalog:

| Task | `--tools` value |
| ------------------------------------- | --------------------------------- |
| Review code | `read,glob,grep` |
| Explain supplied code or a known file | `read` |
| Fix code with an edit/write catalog | `read,glob,grep,edit,write,bash` |
| Fix code with an apply_patch catalog | `read,glob,grep,apply_patch,bash` |

Use the fix variant supported by the selected model; unavailable names fail.
Include `bash` only when the task needs shell commands such as tests. The review
and explain examples select registered work tools; existing permission checks
still apply.

The saved user invocation retains its resolved selection for continuation and
compaction. When selection is active, delegated tasks (including resumed task
sessions) inherit a ceiling from the parent's effective tools; child config,
model and permissions can narrow it. Batch validates its whole group against the
same request catalog before executing any call. Recursive Batch and MCP tools
remain unsupported inside Batch. Custom tools retain their actual identity when
replacing a built-in name. Explicit caller-directed command subtasks keep their
existing Task authorization exception; an active selection still requires the
eligible native Task tool to be selected, without adding permission allow rules.

`StructuredOutput`, invalid-call repair and LiteLLM's compatibility placeholder
are protocol/control tools, outside the work allowlist. Schema mode still needs
permission to call `StructuredOutput`; selecting work tools never grants that
permission. An empty work allowlist can therefore coexist with an explicitly
permitted StructuredOutput protocol tool. Custom tools with protocol-like names
remain work tools, except that an active output schema reserves the
`StructuredOutput` wire name for the harness-created protocol tool. A custom
`StructuredOutput` is selectable as work only outside schema mode.

Custom tools can use the optional typed `context.selection` facade to inspect
`names`/`restricted`, assert availability, or call
`await context.selection.execute("read", { filePath: "/path/to/file" })`.
Dispatch is bound to the actual caller context, checks the same catalog and
permissions before hooks, and validates captured registry parameters. Omitting a
tool or denying it in permissions also prevents this supported nested dispatch.
The facade and names are immutable; arbitrary plugin host code remains trusted.

Selection applies to harness-dispatched model work. Caller `--file`/`@file`
attachments and configured command-template shell expansion are input preparation
and keep their existing behavior. This is not an operating-system sandbox for
selected Bash commands or arbitrary plugin host code. Independent debug-tool
commands do not inherit `run` selection.

**Attach:** this headless checkout has no owned server or verified server-side
selection capability. Any explicit or locally configured selection, including an
empty list, fails with `--attach` before receiver API calls, remote session
creation/fork/prompt/command or model inference. An explicit CLI selector fails
before configuration discovery. With the CLI selector omitted, read-only
configuration discovery preserves normal config precedence and can fetch
well-known configuration; this configuration HTTP is distinct from attach
receiver calls. It performs no plugin/dependency setup, config rewrites, legacy
TOML migration or token refresh. Invalid local configuration can therefore fail
before attach. Valid configuration without a selector retains the existing
attach transport and contents. Successful remote selection requires a future
supporting receiver and is not provided by this option.

## Automation Usage

Aictrl is built to be used in non-interactive environments.
Expand Down Expand Up @@ -143,12 +226,12 @@ Aictrl reads config from `.aictrl/` (project) or `~/.config/aictrl/` (global).

### Models

| Provider | Env Var |
|----------|---------|
| Anthropic | `ANTHROPIC_API_KEY` |
| OpenAI | `OPENAI_API_KEY` |
| Provider | Env Var |
| ---------------- | -------------------------------------------------------------------- |
| Anthropic | `ANTHROPIC_API_KEY` |
| OpenAI | `OPENAI_API_KEY` |
| Google AI Studio | `GOOGLE_GENERATIVE_AI_API_KEY`, `GEMINI_API_KEY` or `GOOGLE_API_KEY` |
| OpenRouter | `OPENROUTER_API_KEY` |
| OpenRouter | `OPENROUTER_API_KEY` |

Use `--variant` to select provider-specific reasoning effort. For OpenAI-compatible
Ollama models, `--variant none` sends `reasoning_effort: "none"` to disable
Expand Down
3 changes: 3 additions & 0 deletions packages/cli/src/cli/cmd/generate.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import type { CommandModule } from "yargs"
import z from "zod"
import { Config } from "@/config/config"
import { MessageV2 } from "@/session/message-v2"
import { Session } from "@/session"
import { SessionPrompt } from "@/session/prompt"
Expand All @@ -16,6 +17,8 @@ export const GenerateCommand = {
registry.add(ProviderTermination.Info, { id: "ProviderTermination" })
registry.add(MessageV2.Format, { id: "OutputFormat" })
registry.add(MessageV2.Assistant, { id: "AssistantMessage" })
registry.add(MessageV2.User, { id: "UserMessage" })
registry.add(Config.Info, { id: "Config" })
registry.add(
z.object({
type: z.literal(Session.Event.StructuredOutput.type),
Expand Down
41 changes: 41 additions & 0 deletions packages/cli/src/cli/cmd/run.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
import { outputSchema, outputResult, OUTPUT_SCHEMA_CONFIG, OUTPUT_CONFIG_EXIT, OUTPUT_FAILED_EXIT } from "./run.output"
import { ToolSelection } from "../../tool/selection"
import { Config } from "../../config/config"
import { OutputSchema } from "@/session/output-schema"
import type { Argv } from "yargs"
import path from "path"
Expand Down Expand Up @@ -375,6 +377,12 @@ export const RunCommand = cmd({
.strictOptions()
.strictCommands(false)
.parserConfiguration({ "populate--": true, "parse-positional-numbers": false })
.option("tools", {
type: "string",
nargs: 1,
describe:
"exact comma-separated work-tool names; use an empty string for no work tools (permissions still apply)",
})
.option("command", {
describe: "the command to run, use message for args",
type: "string",
Expand Down Expand Up @@ -488,6 +496,27 @@ export const RunCommand = cmd({
process.exit(exit)
}

const requested = args.tools
if (requested !== undefined && typeof requested !== "string") {
await fail(
"Use --tools once with one comma-separated argument (for example, --tools read,bash).",
"INVOCATION_INVALID_ARGUMENTS",
)
}
const toolAllowlist =
requested === undefined
? undefined
: await Promise.resolve()
.then(() => ToolSelection.parse(requested))
.catch((error) => fail(error.message, "INVOCATION_INVALID_ARGUMENTS"))

@aictrl-dev aictrl-dev Bot Oct 21, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚪ Parse error masks clearer --attach rejection.

🤖 Fix with your agent 
Fix this code review finding (aictrl-dev/cli PR #254, packages/cli/src/cli/cmd/run.ts:493-497):

Problem: Parse error masks clearer --attach rejection
Detail: ToolSelection.parse(requested) runs before the `args.attach && requested !== undefined` rejection, so `aictrl run --attach URL --tools "read ,bash"` reports the whitespace parse error instead of the more decisive "work-tool selection is not supported with --attach" message, which applies to any --tools value combined with --attach. Both fail before configuration discovery, so the documented ordering guarantee holds — only the reported message is suboptimal.
Suggested fix: Move the attach+requested check above the parse: a selector combined with --attach is invalid regardless of its syntax.

Implement the fix on the PR head branch and add a regression test that fails before the fix and passes after.
Why this matters

ToolSelection.parse(requested) runs before the args.attach && requested !== undefined rejection, so aictrl run --attach URL --tools "read ,bash" reports the whitespace parse error instead of the more decisive "work-tool selection is not supported with --attach" message, which applies to any --tools value combined with --attach. Both fail before configuration discovery, so the documented ordering guarantee holds — only the reported message is suboptimal.

    if (args.attach && requested !== undefined) {
      await fail(
        "Work-tool selection is not supported with --attach: this client cannot verify server-side enforcement. Remove --tools or run locally.",
        "INVOCATION_INVALID_ARGUMENTS",
      )
    }

@aictrl-dev aictrl-dev Bot Oct 21, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Invalid --tools prints raw ZodError JSON to user.

🤖 Fix with your agent 
Fix this code review finding (aictrl-dev/cli PR #254, packages/cli/src/cli/cmd/run.ts:493):

Problem: Invalid --tools prints raw ZodError JSON to user
Detail: toolAllowlist is computed via Promise.resolve().then(() => ToolSelection.parse(requested)).catch((error) => fail(error.message, …)). ToolSelection.parse throws a zod ZodError (Names.parse on the split array), and ZodError.message is the serialized JSON issues array, so `aictrl run --tools "read, bash"` (or "a,,b") prints a raw JSON blob on stderr instead of the authored "Tool names must not contain surrounding whitespace" text, unlike every neighbouring hand-written fail() message in this handler. The same raw ZodError JSON reaches users via Config.readToolSelection (Names.parse) on the --attach preflight path.
Suggested fix: Map zod issues to readable text before failing, e.g. .catch((error) => fail(error instanceof z.ZodError ? error.issues.map((i) => i.message).join("; ") : String(error), "INVOCATION_INVALID_ARGUMENTS")), or have ToolSelection.parse throw a plain Error carrying the first issue message. A plain try/catch would also drop the odd Promise.resolve().then wrapper.

Implement the fix on the PR head branch and add a regression test that fails before the fix and passes after.
Why this matters

toolAllowlist is computed via Promise.resolve().then(() => ToolSelection.parse(requested)).catch((error) => fail(error.message, …)). ToolSelection.parse throws a zod ZodError (Names.parse on the split array), and ZodError.message is the serialized JSON issues array, so aictrl run --tools "read, bash" (or "a,,b") prints a raw JSON blob on stderr instead of the authored "Tool names must not contain surrounding whitespace" text, unlike every neighbouring hand-written fail() message in this handler. The same raw ZodError JSON reaches users via Config.readToolSelection (Names.parse) on the --attach preflight path.

    const toolAllowlist =
      requested === undefined
        ? undefined
        : await Promise.resolve()
            .then(() => ToolSelection.parse(requested))
            .catch((error) => fail(error.message, "INVOCATION_INVALID_ARGUMENTS"))

    if (args.attach && requested !== undefined) {


if (args.attach && requested !== undefined) {
await fail(
"Work-tool selection is not supported with --attach: this client cannot verify server-side enforcement. Remove --tools or run locally.",
"INVOCATION_INVALID_ARGUMENTS",
)
}

if (args.share === true) {
await fail("--share was removed: session sharing is no longer supported", "INVOCATION_INVALID_ARGUMENTS")
}
Expand Down Expand Up @@ -1145,6 +1174,7 @@ export const RunCommand = cmd({
if (args.command) {
const opts = {
agent,
toolAllowlist,
model: args.model,
command: args.command,
arguments: { argv: normalized.argv, ...(stdin !== undefined ? { stdin } : {}) },
Expand All @@ -1157,6 +1187,7 @@ export const RunCommand = cmd({
const model = args.model ? Provider.parseModel(args.model) : undefined
const opts = {
agent,
toolAllowlist,
model,
variant: args.variant,
parts: [...files, { type: "text", text: message }],
Expand Down Expand Up @@ -1199,6 +1230,14 @@ export const RunCommand = cmd({

invocation.phase("bootstrap")
if (args.attach) {
// This checkout has no receiver/capability contract that can enforce selection.
const configured = await invocation.guard(() => Config.readToolSelection(process.cwd()))
if (configured !== undefined) {
await fail(
"Work-tool selection is not supported with --attach: this client cannot verify server-side enforcement. Remove --tools and local tool_allowlist or run locally.",
"INVOCATION_INVALID_ARGUMENTS",
)

@aictrl-dev aictrl-dev Bot Oct 21, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Attach preflight fetch failure crashes runs unguarded.

Suggested change
)
Wrap the readToolSelection preflight in try/catch and route failures through fail(...) with a clear message (or treat discovery errors as "selection unknown → fail closed"), and give the well-known fetch a timeout so offline hosts degrade gracefully instead of rejecting the await chain.
🤖 Fix with your agent 
Fix this code review finding (aictrl-dev/cli PR #254, packages/cli/src/cli/cmd/run.ts:1221-1224):

Problem: Attach preflight fetch failure crashes runs unguarded
Detail: Every `aictrl run --attach` now awaits Config.readToolSelection(process.cwd()) → resolve(false), which runs Auth.all(), filesystem walks, and an unconditional fetch() of each well-known auth URL, with no try/catch. A network failure of that fetch (or any malformed config making load() throw) rejects the preflight as an unstructured error even when the user configured no tool selection at all — previously attach worked without touching local config. README sanctions failing on "invalid local configuration", but an unreachable well-known host now also breaks attach runs that used to work, bypassing the structured fail(..., "INVOCATION_INVALID_ARGUMENTS") path.
Suggested fix: Wrap the readToolSelection preflight in try/catch and route failures through fail(...) with a clear message (or treat discovery errors as "selection unknown → fail closed"), and give the well-known fetch a timeout so offline hosts degrade gracefully instead of rejecting the await chain.

Implement the fix on the PR head branch and add a regression test that fails before the fix and passes after.
Why this matters

Every aictrl run --attach now awaits Config.readToolSelection(process.cwd()) → resolve(false), which runs Auth.all(), filesystem walks, and an unconditional fetch() of each well-known auth URL, with no try/catch. A network failure of that fetch (or any malformed config making load() throw) rejects the preflight as an unstructured error even when the user configured no tool selection at all — previously attach worked without touching local config. README sanctions failing on "invalid local configuration", but an unreachable well-known host now also breaks attach runs that used to work, bypassing the structured fail(..., "INVOCATION_INVALID_ARGUMENTS") path.

    invocation.phase("bootstrap")
    if (args.attach) {
      // This checkout has no receiver/capability contract that can enforce selection.
      const configured = await Config.readToolSelection(process.cwd())
      if (configured !== undefined) {

}
return await invocation.run(() => execute(createAictrlClient({ baseUrl: args.attach, directory })))
}

Expand Down Expand Up @@ -1234,6 +1273,7 @@ export const RunCommand = cmd({
sessionID: opts.sessionID,
parts: opts.parts,
agent: opts.agent,
toolAllowlist: opts.toolAllowlist,
model: opts.model,
variant: opts.variant,
...(opts.format ? { format: opts.format } : {}),
Expand All @@ -1245,6 +1285,7 @@ export const RunCommand = cmd({
command: opts.command,
arguments: opts.arguments ?? "",
agent: opts.agent,
toolAllowlist: opts.toolAllowlist,
model: opts.model,
variant: opts.variant,
...(opts.format ? { format: opts.format } : {}),
Expand Down
Loading
Loading