Add background mode for shell commands so long runs don't hold steers

docs/ARCHITECTURE.md

@@ -369,7 +369,7 @@ tool call
 - **Secret Guard** (`secret-guard-plugin.ts`) — Hard-denies path-keyed tool calls (`read_file`, `write_file`, …) that would put a sensitive file into (or write it from) the model context. Runs before the permission plugin, so the path-arg deny holds even under `--dangerously-skip-permissions`. Shell commands that _reference_ a sensitive path (tokenized so `cat .env`, `bun --env-file=.env run …`, and quote/env-assignment forms are detected) are not hard-denied here: they require operator approval via the permission gate, and auto mode forces an ask through the auto-shell policy (`sensitive-path` rule). Once the operator approves, the command runs. Shell detection is best-effort: token matching defeats quoting and env-assignment/redirection forms but not dynamic path construction (variable indirection, `printf` assembly). Tool-result secret scrub still redacts credential-shaped output.
 - **Authorization** (`run-shell-authz.ts`, wired by `authz-plugin.ts`) — Denies catastrophic shell command patterns by regex, and hard-blocks shell `find`, head-position `rg`, and recursive `grep -r` (they can walk huge trees and OOM the host). Bounded `grep`/`search_files` tools remain practical alternatives (timeout + output caps); the patterns match those three command shapes only — an `ls -R`, `fd`, or scripted `os.walk` is just as unbounded and is not caught, so the block message tells the model not to substitute one. The permission gate’s shell auto-allow path consults the same policy so it never pre-approves a command authz would reject.
 - **Permission** (`permission-plugin.ts`) — Delegates consequential calls to the permission gate.
-- **Shell Guard** (`shell-guard-plugin.ts`) — Corbits Code-only replacement for stock `run_shell` (interchange stays unpatched): no built-in default timeout (optional per-call or `settings.shell.timeoutMs`; `maxTimeoutMs` clamps only a resolved timeout), 512KB display cap with head+tail retention (the process keeps running when the cap is hit), process-group kill on timeout/abort only. Also applies a 10s wall-clock budget to `grep`/`search_files`.
+- **Shell Guard** (`shell-guard-plugin.ts`) — Corbits Code-only replacement for stock `run_shell` (interchange stays unpatched): no built-in default timeout (optional per-call or `settings.shell.timeoutMs`; `maxTimeoutMs` clamps only a resolved timeout), 512KB display cap with head+tail retention (the process keeps running when the cap is hit), process-group kill on timeout/abort only, and `background: true` — the call returns a `shell_id` at once (registry in `src/shell/background-shell.ts`), the process group keeps running past the turn, completion is delivered on a later turn via `buildShellBackgroundMessage`, and `shell_collect` retrieves or cancels (schema advertised by `advertiseShellGuardTimeout`; evaluated by the permission chain at start time like any shell call). Also applies a 10s wall-clock budget to `grep`/`search_files`.
 - **Read File Guard** (`read-file-guard-plugin.ts`) — Corbits Code-only short-circuit for `read_file` on real filesystem paths and configured `tool-output://` URIs (interchange stays unpatched): streaming reads that never decode the whole file in one pass, caps model-facing output at 50KB, defaults to 2000 lines, truncates long lines with recovery hints, samples the first chunk to reject binary, and stops at an 8MB scan ceiling. Emits `offset` continuation notices so the model can page without losing file or spill content on disk.
 - **Verify** (`verify-plugin.ts`) — Re-reads after `write_file` / `edit_file` and errors on mismatch. Per-path serialization (`file-mutation-lock.ts`) prevents parallel edits on one file from tripping verification.
 - **Edit file line range** (`edit-file-line-range-plugin.ts`) — Corbits Code-only short-circuit for `edit_file` mode B (`start_line`/`end_line`/`new_string`), same pattern as shell-guard; schema advertised via `advertiseEditFileLineRange`. Modes are mutually exclusive: a call supplying both `old_string` and `start_line`/`end_line` is rejected with a recoverable error naming which fields to omit (no file-content disambiguation).

docs/IMPLEMENTATION.md

@@ -118,6 +118,7 @@ src/
     permission-plugin.ts       Tiered operator approval
   shell/
     run-shell-authz.ts         Shared run_shell deny policy (authz + permission)
+    background-shell.ts        Background run_shell registry (start/collect/cancel/disposeAll)
     verify-plugin.ts           Write/edit verification (per-path lock)
     file-mutation-lock.ts      Serialize mutations per file for verify
     lsp-hint-plugin.ts         TS/JS LSP setup hint on unavailable server
@@ -185,7 +186,12 @@ Unmatched shell auto-allows, including contained non-force `git worktree add`/`r
 
 `ChatInputProps` carries `isProcessing?: boolean` and `onInterrupt?: (message: string) => void`. When `isProcessing` is true, drain timing is **parent-idle** vs **session-idle**:
 
-- **Enter** soft-steers while the parent is busy — enqueues kind `"steer"` and delivers at the next **parent** `tool.boundary` (the parent tool finishing, not a child). Does not interrupt. **Parent-idle** is when the primary Skywalker turn is not inside an in-flight parent tool; a long parent `run_shell` or awaiting `wait_agents` is parent-busy and holds steers.
+- **Enter** soft-steers while the parent is busy — enqueues kind `"steer"` and delivers at the next **parent** `tool.boundary` (the parent tool finishing, not a child). Does not interrupt. **Parent-idle** is when the primary Skywalker turn is not inside an in-flight parent tool; a long parent **foreground** `run_shell` or awaiting `wait_agents` is parent-busy and holds steers. A `run_shell` started with `background: true` returns at once and releases the boundary; its completion is delivered as a system message (`buildShellBackgroundMessage`, mailbox `system`, no operator-originated flag — it re-enters the reactor without counting as operator input) on a later turn.
+
+#### Background shell mode
+
+`run_shell` accepts `background: true` (shell-guard plugin, after the permission chain — a denied command spawns nothing). The starting call resolves `cwd`/`timeout` as usual, skips the pwd probe, spawns a detached process group via the registry in `src/shell/background-shell.ts`, and returns `{shell_id, status: "running"}` immediately; the retained shell cwd is never mutated by a background run. Limits: 8 running, 8 completed entries (ring; evicted ids collect as not-found — truncated output is spilled to a `tool-output:///bg-shell-<id>` blob named in the completion message). On process exit the host delivers `buildShellBackgroundMessage(exit)` (exit status, timed-out marker, ~2KB output preview, spill URI) through the same continuation channel as compaction — wired in all three loop hosts (TUI, exec, sub-agent). `shell_collect` (`{shell_id, action: "collect"|"cancel", wait_ms?}`, default non-blocking) retrieves status/output or kills the process group; it is ungated by design (cancel only kills the session's own child). Timeout keeps its meaning: expiry kills the group and reports exit code 124 with `timed_out: true`. The tool watchdog exempts background starts and `shell_collect` (same list as `spawn_agent`/`wait_agents`). Toolset dispose calls `disposeAll("session closed")` before the posix teardown, so `/clear`, interrupt, and reload kill every live background process group.
+
 - **Alt+Enter** queues a follow-up (kind `"queue"`) delivered only on **session-idle** — parent-idle **and** no live fleet lanes (`run` goes idle). Session-idle Alt+Enter is a no-op. **Ctrl+C** stops the run.
 
 Idle-with-fleet is shipped: after a non-blocking `spawn_agent` dispatch the parent turn can settle while workers keep running. The runner emits a `fleet` event carrying the live-lane count; the bridge holds the run busy on that count, so mid-hold Enter upgrades to a new primary turn (sent immediately) instead of queueing a steer, follow-ups keep waiting for true session-idle, and any steer left pending at the hold's engagement delivers immediately — the parent it was steering has already stopped.
@@ -250,7 +256,7 @@ Provider and model configuration lives in JSON settings files. The global file h
   }
   ```
 
-  - `timeoutMs` / `maxTimeoutMs` — outer execution watchdog around each tool `run()`. Unset leaves the watchdog unarmed; set these to arm it. `maxTimeoutMs` clamps non-shell tools when set and does not cap a longer requested `run_shell`. Fleet wait tools are exempt: a dispatched sub-agent is bounded by stall, opt-in `deadlineMs`, and operator cancel, not the generic per-tool budget.
+  - `timeoutMs` / `maxTimeoutMs` — outer execution watchdog around each tool `run()`. Unset leaves the watchdog unarmed; set these to arm it. `maxTimeoutMs` clamps non-shell tools when set and does not cap a longer requested `run_shell`. Fleet wait tools are exempt: a dispatched sub-agent is bounded by stall, opt-in `deadlineMs`, and operator cancel, not the generic per-tool budget. Background shell is exempt too: a `run_shell` with `background: true` arms nothing (the process's own timeout bounds it) and `shell_collect` never arms (a bounded poll over a process that outlives the turn).
   - `waitForApproval` (default **true** when unset) — freeze that budget while a permission prompt is open so a late approve still runs the tool. **Settings → Tools** toggles this live for the next tool call and persists it here. When **false**, the budget keeps ticking during the prompt; on expiry the tool is skipped and the modal is auto-dismissed. The freeze is bounded: after **30 minutes** with the prompt still unanswered the budget resumes ticking on its own, so a prompt that never becomes visible (overlay open, UI gone) cannot hang a tool run indefinitely.
 
   Optional `mcp` block bounds MCP tool calls (`mcp__*` names) specifically — unlike `tools.*`, this arms **unconditionally** even with no settings at all, defaulting to **5 minutes**, since a wedged MCP server otherwise hangs a call forever with nothing to bound it (CL-6895):

src/agent/background-shell-tool.test.ts

@@ -0,0 +1,138 @@
+import { describe, expect, test } from "bun:test";
+import { spawnSync } from "node:child_process";
+import { randomUUID } from "node:crypto";
+import { createPermissionGate } from "../permission/gate.js";
+import { createAgentToolset } from "./tools.js";
+import type { BackgroundShellExit } from "../shell/background-shell.js";
+import { buildShellBackgroundMessage } from "../session/runtime-assembly.js";
+import { OPERATOR_ORIGINATED_FLAG } from "../agent/message-provenance.js";
+
+function gate(cwd: string) {
+  return createPermissionGate({
+    approvals: [],
+    interactive: false,
+    skipPermissions: true,
+    reactorGated: false,
+    cwd,
+  });
+}
+
+describe("background shell through the agent toolset", () => {
+  test("run_shell background:true returns a handle and delivers the exit on exit", async () => {
+    const exits: BackgroundShellExit[] = [];
+    const toolset = await createAgentToolset({
+      cwd: process.cwd(),
+      permissionGate: gate(process.cwd()),
+      onOperatorGate: async () => ({ kind: "cancel" as const }),
+      onBackgroundShellExit: (exit) => exits.push(exit),
+    });
+    try {
+      const started = await toolset.dynamicRunner.run(
+        {
+          id: "bg-start",
+          name: "run_shell",
+          arguments: { command: "sleep 0.4; echo bg-done", background: true },
+        },
+        new AbortController().signal,
+      );
+      expect(started.isError).not.toBe(true);
+      const parsed = JSON.parse(String(started.content)) as { shell_id: string };
+      const snapshotNow = await toolset.dynamicRunner.run(
+        {
+          id: "bg-collect",
+          name: "shell_collect",
+          arguments: { shell_id: parsed.shell_id, action: "collect" },
+        },
+        new AbortController().signal,
+      );
+      expect(JSON.parse(String(snapshotNow.content))).toMatchObject({ status: "running" });
+      const final = await toolset.dynamicRunner.run(
+        {
+          id: "bg-collect2",
+          name: "shell_collect",
+          arguments: { shell_id: parsed.shell_id, action: "collect", wait_ms: 5_000 },
+        },
+        new AbortController().signal,
+      );
+      const result = JSON.parse(String(final.content)) as {
+        status: string;
+        exit_code: number;
+        output: string;
+      };
+      expect(result).toMatchObject({ status: "completed", exit_code: 0 });
+      expect(result.output).toContain("bg-done");
+      await new Promise((r) => setTimeout(r, 50));
+      expect(exits).toHaveLength(1);
+      expect(exits[0]!.id).toBe(parsed.shell_id);
+      const message = buildShellBackgroundMessage(exits[0]!);
+      expect(message.headers.messageId).toBe(`bg-shell-${parsed.shell_id}@local`);
+      expect(message.ref.mailbox).toBe("system");
+      expect(message.flags).not.toContain(OPERATOR_ORIGINATED_FLAG);
+      expect(message.content).toContain("exit code 0");
+      expect(message.content).toContain("bg-done");
+    } finally {
+      await toolset.dispose();
+    }
+  });
+
+  test("shell_collect cancel kills the session's own child process group", async () => {
+    const token = `ic_toolset_cancel_${randomUUID()}`;
+    const toolset = await createAgentToolset({
+      cwd: process.cwd(),
+      permissionGate: gate(process.cwd()),
+      onOperatorGate: async () => ({ kind: "cancel" as const }),
+    });
+    try {
+      const started = await toolset.dynamicRunner.run(
+        {
+          id: "c-start",
+          name: "run_shell",
+          arguments: {
+            command: `sleep 600 # ${token}`,
+            background: true,
+          },
+        },
+        new AbortController().signal,
+      );
+      const { shell_id } = JSON.parse(String(started.content)) as { shell_id: string };
+      const cancelled = await toolset.dynamicRunner.run(
+        {
+          id: "c-cancel",
+          name: "shell_collect",
+          arguments: { shell_id, action: "cancel" },
+        },
+        new AbortController().signal,
+      );
+      expect(JSON.parse(String(cancelled.content))).toMatchObject({ status: "cancelling" });
+      await new Promise((r) => setTimeout(r, 300));
+      const probe = spawnSync("pgrep", ["-f", token], { encoding: "utf8" });
+      expect(probe.stdout?.trim() ?? "").toBe("");
+      expect(probe.status).not.toBe(0);
+    } finally {
+      await toolset.dispose();
+    }
+  });
+
+  test("toolset dispose kills every live background process group", async () => {
+    const token = `ic_toolset_dispose_${randomUUID()}`;
+    const toolset = await createAgentToolset({
+      cwd: process.cwd(),
+      permissionGate: gate(process.cwd()),
+      onOperatorGate: async () => ({ kind: "cancel" as const }),
+    });
+    const started = await toolset.dynamicRunner.run(
+      {
+        id: "d-start",
+        name: "run_shell",
+        arguments: { command: `sleep 600 # ${token}`, background: true },
+      },
+      new AbortController().signal,
+    );
+    expect(started.isError).not.toBe(true);
+    await toolset.dispose();
+    await new Promise((r) => setTimeout(r, 300));
+    const probe = spawnSync("pgrep", ["-f", token], { encoding: "utf8" });
+    expect(probe.stdout?.trim() ?? "").toBe("");
+    expect(probe.status).not.toBe(0);
+  });
+});

src/agent/background-shell-tool.ts

@@ -0,0 +1,105 @@
+import { type } from "arktype";
+import type { ToolDefinition } from "@intx/types/runtime";
+import {
+  createBackgroundShellRegistry,
+  type BackgroundShellExit,
+  type BackgroundShellRegistry,
+} from "../shell/background-shell.js";
+import type { SpillBlobWriter } from "../plugins/result-truncation-plugin.js";
+
+const ShellCollectArgs = type({
+  shell_id: "string>0",
+  action: "'collect' | 'cancel'",
+  "wait_ms?": "number",
+});
+type ShellCollectArgs = typeof ShellCollectArgs.infer;
+
+export const shellCollectDefinition: ToolDefinition = {
+  name: "shell_collect",
+  description:
+    "Collect or cancel a background run_shell (started with background: true). " +
+    'action="collect" returns the result once finished (or status running); ' +
+    'action="cancel" kills the process group. Completion also arrives as a ' +
+    "system message on a later turn — collect is for polling or retrieving " +
+    "output again after eviction risk.",
+  inputSchema: {
+    type: "object",
+    properties: {
+      shell_id: { type: "string", description: "shell_id from the background run_shell start." },
+      action: {
+        type: "string",
+        enum: ["collect", "cancel"],
+        description: '"collect" retrieves status/output; "cancel" kills the process group.',
+      },
+      wait_ms: {
+        type: "number",
+        description:
+          'For action="collect": milliseconds to wait for completion before returning "running" (default 0, non-blocking).',
+      },
+    },
+    required: ["shell_id", "action"],
+  },
+};
+
+export function createSpillingBackgroundShellExitNotifier(args: {
+  getBlobWriter?: () => SpillBlobWriter | undefined;
+  notify: (exit: BackgroundShellExit) => void;
+}): (exit: BackgroundShellExit) => void {
+  return (exit) => {
+    void (async () => {
+      // Truncated output spills to the session blob store so the completion
+      // message can point at a readable tool-output:/// URI.
+      let spillUri: string | undefined;
+      if (exit.outputTruncated) {
+        const writeBlob = args.getBlobWriter?.();
+        if (writeBlob !== undefined) {
+          const key = `bg-shell-${exit.id}`;
+          await writeBlob(key, new TextEncoder().encode(exit.output), "text/plain");
+          spillUri = `tool-output:///${key}`;
+        }
+      }
+      args.notify(spillUri !== undefined ? { ...exit, spillUri } : exit);
+    })();
+  };
+}
+
+export function createShellCollectTool(
+  registry: BackgroundShellRegistry = createBackgroundShellRegistry(),
+) {
+  return {
+    definition: shellCollectDefinition,
+    handler: async (rawArgs: Record<string, unknown>): Promise<string> => {
+      const parsed = ShellCollectArgs(rawArgs);
+      if (parsed instanceof type.errors) {
+        return "Error: shell_collect requires shell_id (string) and action ('collect' | 'cancel').";
+      }
+      const { shell_id, action } = parsed;
+      if (action === "cancel") {
+        if (!registry.cancel(shell_id)) {
+          return `No running background shell with id ${shell_id}; it may have already finished or been collected.`;
+        }
+        return JSON.stringify({ shell_id, status: "cancelling" });
+      }
+      const snapshot = await registry.collect(shell_id, parsed.wait_ms ?? 0);
+      if (snapshot.state === "running") {
+        return JSON.stringify({ shell_id, status: "running" });
+      }
+      if (snapshot.state === "not-found") {
+        return (
+          `No background shell with id ${shell_id}. It may have been evicted from the ` +
+          "completed ring; if its output was truncated, the completion message carried " +
+          "a tool-output:/// URI for the full output."
+        );
+      }
+      const { exit } = snapshot;
+      return JSON.stringify({
+        shell_id,
+        status: "completed",
+        exit_code: exit.exitCode,
+        timed_out: exit.timedOut,
+        ...(exit.spillUri !== undefined ? { output_uri: exit.spillUri } : {}),
+        output: exit.output,
+      });
+    },
+  };
+}