diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 0c5204ea1..d70f796ee 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -116,7 +116,7 @@ Two directors, selected by role: Auto mode is toggled by CLI flags (`--auto` / `--no-auto`); there is currently no in-session key to toggle it (default on; constrained envelope — workspace writes and unconstrained shell auto-allow; installs, recursive rm, force/uncontained worktree changes, sensitive-path and opaque-wrapper shell still ask; contained non-force `git worktree add`/`remove`/`prune` and `list` auto-allow; shell file-mutation denied). It is not a separate edit/plan mode. -- **SubAgentDirector** (delegated work, `src/subagent/index.ts`) — Drives a dispatched worker until a turn arrives with no tool calls, then replies with the final assistant text and ends the run. A tool-less turn **after tools** completes only with the four-heading envelope (Summary, Findings, Blockers, Paths); a missing envelope nudges once (**incomplete-report**) and a second tool-less turn still without the envelope salvages as **incomplete-report-stop**. Explore/read-only workers that used tools then replied with findings remain normal completes; `requireEvidence` (off by default, set per director) additionally requires at least one read before a tool-less spawn-only reply can complete. Reads done through `run_shell` count as evidence too — `src/subagent/shell-evidence.ts` classifies shell reads (`cat`, `grep`, `sed` without `-i`, …) over the same subject expansion the auto-shell policy uses — but there is no corresponding shell-write evidence or file-write requirement: a run that never touches a file still completes normally once it replies with the envelope. There is no turn budget. Operator/parent cancel after any progress returns a **cancelled** salvage report (partial findings + tool activity) instead of a bare cancel string; cancel before progress still surfaces as cancelled-by-operator. There is no repetition/no-progress/never-acted/never-edited hard stop and no fingerprint-based re-dispatch block — a genuinely stuck worker runs until it completes, stalls, hits an opt-in wall-clock deadline, or is cancelled. +- **SubAgentDirector** (delegated work, `src/subagent/index.ts`) — Drives a dispatched worker until a turn arrives with no tool calls, then replies with the final assistant text and ends the run. A tool-less turn **after tools** completes only with the four-heading envelope (Summary, Findings, Blockers, Paths). Assistant text that prints explicit `` markup is treated as attempted tool use, not narration: one **verbatim-tool-call** nudge asks the worker to re-issue a real `tool_call` and does not count toward the tool-less spiral. A missing envelope otherwise nudges once (**incomplete-report**) and a second tool-less turn still without the envelope salvages as **incomplete-report-stop**. Explore/read-only workers that used tools then replied with findings remain normal completes; `requireEvidence` (off by default, set per director) additionally requires at least one read before a tool-less spawn-only reply can complete. Reads done through `run_shell` count as evidence too — `src/subagent/shell-evidence.ts` classifies shell reads (`cat`, `grep`, `sed` without `-i`, …) over the same subject expansion the auto-shell policy uses — but there is no corresponding shell-write evidence or file-write requirement: a run that never touches a file still completes normally once it replies with the envelope. There is no turn budget. Operator/parent cancel after any progress returns a **cancelled** salvage report (partial findings + tool activity) instead of a bare cancel string; cancel before progress still surfaces as cancelled-by-operator. There is no repetition/no-progress/never-acted/never-edited hard stop and no fingerprint-based re-dispatch block — a genuinely stuck worker runs until it completes, stalls, hits an opt-in wall-clock deadline, or is cancelled. `spawn_agent` starts each worker and records it in the caller's fleet mailbox. On the TUI primary, mailbox mail is the collect path: occupancy takes uncollected terminals and re-enters the parent as system inbound. Nested orchestrators still collect with `wait_agents`. TUI-primary `wait_agents` may yield as a timeout (workers untouched, no take) so occupancy can deliver mail or a queued Enter steer. Already-collected waits return status without a second report or error body. Wait JSON includes `stop_reason` from the session when present so a salvage that is wait-`done` is not mistaken for a clean complete, and so parent-initiated interrupt (`interrupted`) is not mistaken for operator-cancel (`cancelled`). Deadline salvage prepends an advisory parent hint suggesting continuation plus a longer deadline if more wall-clock time is warranted. Failed and incomplete-report salvage tell the parent to diagnose from the report or error and MAY spawn one successor with a changed brief. A parent-initiated interrupt is a resumable pause: wait unblocks with `stop_reason: interrupted` (often while the session is still running and has no report); the parent should `resume_agent` or re-wait, and must not spawn a successor against a still-live worker. Successor only if that session is no longer resumable. Operator-cancelled salvage asks the parent to synthesize Findings and Paths and wait for the operator instead of auto-starting another specialist. Identical re-dispatch of the same brief stays refused at the prompt / spawn-handoff layer; there is no fingerprint-based re-dispatch hard-block. Deadline hints are advisory only — an identical re-dispatch is still admitted at runtime. Parent hints are prepended on salvage reports returned to the parent. The runtime does not auto-spawn successors. #### Model-family policy (`src/agent/model-family-policy.ts`) diff --git a/docs/PRODUCT.md b/docs/PRODUCT.md index 1c67b3396..c3b5c3482 100644 --- a/docs/PRODUCT.md +++ b/docs/PRODUCT.md @@ -170,7 +170,7 @@ Corbits Code fans work out to short-lived **fleet agents** — workers with thei - **Tasks** are checklist items owned by one agent via `manage_tasks`. - **Fleet agents** are spawned with `spawn_agent`. On the TUI primary, mailbox mail arrives as inbound when a worker finishes or fails — do not poll `wait_agents`. Nested orchestrators still collect with `wait_agents`. Workers ask the parent with `ask_director`. That parks a question while the worker stays `running`. Nested `wait_agents` returns `awaiting_director` with a question payload — that is not terminal. The parent answers with `send_input` (`target` = the worker's session id). When the parent TUI is not blocked in `wait_agents`, a parked question arrives as a synthetic idle-send wake. Escalate to the human only with `ask_operator`. -Dispatch uses a structured brief (context / goal / optional goals seed) and returns a structured report. The TUI Agents strip and fleet board show who is running; live tool progress updates the status bar without dumping the child transcript into the parent chat. There is no turn budget. A tool-less final turn completes only with the four-heading report envelope; without it, one nudge is given and a second tool-less turn without the envelope salvages as `incomplete-report-stop`. A silent worker (no activity for `stallTimeoutMs`, opt-in) gets one continuation nudge, then salvages as `stalled` if a second consecutive check finds no activity. An opt-in `deadlineMs`, or an operator cancel, can also end a run early. Each of these returns a salvage report so a runaway or idle child cannot quietly burn a large token budget or look done after prose alone. +Dispatch uses a structured brief (context / goal / optional goals seed) and returns a structured report. The TUI Agents strip and fleet board show who is running; live tool progress updates the status bar without dumping the child transcript into the parent chat. There is no turn budget. A tool-less final turn completes only with the four-heading report envelope. Printed `` markup in assistant text gets one corrective nudge to issue a real tool call and does not count as the wrap-up; without the envelope, one incomplete-report nudge is given and a second tool-less turn without the envelope salvages as `incomplete-report-stop`. A silent worker (no activity for `stallTimeoutMs`, opt-in) gets one continuation nudge, then salvages as `stalled` if a second consecutive check finds no activity. An opt-in `deadlineMs`, or an operator cancel, can also end a run early. Each of these returns a salvage report so a runaway or idle child cannot quietly burn a large token budget or look done after prose alone. ## Roadmap (planned, not yet shipped) diff --git a/src/subagent/nudge-director.test.ts b/src/subagent/nudge-director.test.ts index 848553c62..19dfb2720 100644 --- a/src/subagent/nudge-director.test.ts +++ b/src/subagent/nudge-director.test.ts @@ -84,13 +84,20 @@ function inferenceDone( } function inferenceDoneText(text: string, inputTokens = 0): ReactorInboundEvent { + return inferenceDoneContent([{ type: "text", text }], inputTokens); +} + +function inferenceDoneContent( + content: readonly Record[], + inputTokens = 0, +): ReactorInboundEvent { return { type: "inference.done", turn: { role: "assistant", model: "test", timestamp: 0, - content: [{ type: "text", text }], + content, }, usage: { input: inputTokens, @@ -419,6 +426,155 @@ const REPORT_ENVELOPE = [ "src/gate.ts", ].join("\n"); +describe("SubAgentDirector verbatim tool markup recovery", () => { + const verbatimToolCall = + '{"path":"src/index.ts"}'; + + test("nudges once for explicit tool-call wrapper text before report policy", async () => { + const director = new SubAgentDirector("system", [], undefined, 30); + const caps = capabilities(); + + const correction = actions( + await director.decide(inferenceDoneText(verbatimToolCall), state, caps), + ); + expect(correction).toContainEqual({ + type: "checkpoint", + message: "subagent-verbatim-tool-call-nudge", + }); + expect(ephemeralTexts(inferAction(correction))?.[0]).toContain( + "real tool call", + ); + + const reportNudge = actions( + await director.decide(inferenceDoneText(verbatimToolCall), state, caps), + ); + expect(reportNudge).toContainEqual({ + type: "checkpoint", + message: "subagent-incomplete-report-nudge", + }); + + const stopped = actions( + await director.decide(inferenceDoneText(verbatimToolCall), state, caps), + ); + expect(stopped).toContainEqual({ + type: "checkpoint", + message: "subagent-incomplete-report", + }); + }); + + test("does not treat arbitrary XML or thinking as verbatim tool calls", async () => { + const caps = capabilities(); + const arbitraryXML = new SubAgentDirector("system", [], undefined, 30); + const arbitraryResult = actions( + await arbitraryXML.decide( + inferenceDoneText("src/index.ts"), + state, + caps, + ), + ); + expect(arbitraryResult).toContainEqual({ + type: "checkpoint", + message: "subagent-incomplete-report-nudge", + }); + + const thinkingOnly = new SubAgentDirector("system", [], undefined, 30); + const thinkingResult = actions( + await thinkingOnly.decide( + inferenceDoneContent([ + { type: "thinking", thinking: verbatimToolCall }, + ]), + state, + caps, + ), + ); + expect(thinkingResult).toContainEqual({ + type: "checkpoint", + message: "subagent-incomplete-report-nudge", + }); + }); + + test("resets correction only after genuine tool activity or parent follow-up", async () => { + const director = new SubAgentDirector("system", [], undefined, 30); + const caps = capabilities(); + + await director.decide(inferenceDoneText(verbatimToolCall), state, caps); + const narration = actions( + await director.decide(inferenceDoneText("Still working"), state, caps), + ); + expect(narration).toContainEqual({ + type: "checkpoint", + message: "subagent-incomplete-report-nudge", + }); + + await director.decide(inferenceDone(["read-1"]), state, caps); + await director.decide(toolDone("read-1"), state, caps); + const afterTool = actions( + await director.decide(inferenceDoneText(verbatimToolCall), state, caps), + ); + expect(afterTool).toContainEqual({ + type: "checkpoint", + message: "subagent-verbatim-tool-call-nudge", + }); + + await director.decide(messageReceived("Try again"), state, caps); + const afterFollowup = actions( + await director.decide(inferenceDoneText(verbatimToolCall), state, caps), + ); + expect(afterFollowup).toContainEqual({ + type: "checkpoint", + message: "subagent-verbatim-tool-call-nudge", + }); + }); + + test("after the verbatim nudge a real tool call executes", async () => { + const director = new SubAgentDirector("system", [], undefined, 30); + const caps = capabilities(); + + await director.decide(inferenceDoneText(verbatimToolCall), state, caps); + const result = actions( + await director.decide(inferenceDone(["read-1"]), state, caps), + ); + expect(result.some((action) => action.type === "execute_tools")).toBe(true); + expect(result.some((action) => action.type === "reply")).toBe(false); + }); + + test("after the verbatim nudge a four-heading envelope completes", async () => { + const director = new SubAgentDirector("system", [], undefined, 30); + const caps = capabilities(); + + await director.decide(inferenceDoneText(verbatimToolCall), state, caps); + const result = actions( + await director.decide(inferenceDoneText(REPORT_ENVELOPE), state, caps), + ); + expect(result).toContainEqual({ + type: "checkpoint", + message: "subagent-complete", + }); + }); + + test("a complete envelope that quotes tool-call markup still completes", async () => { + const director = new SubAgentDirector("system", [], undefined, 30); + const caps = capabilities(); + + const reportQuotingMarkup = `${REPORT_ENVELOPE}\n\nThe model emitted ${verbatimToolCall} as text.`; + const result = actions( + await director.decide( + inferenceDoneText(reportQuotingMarkup), + state, + caps, + ), + ); + expect(result).toContainEqual({ + type: "checkpoint", + message: "subagent-complete", + }); + expect(result).not.toContainEqual({ + type: "checkpoint", + message: "subagent-verbatim-tool-call-nudge", + }); + }); +}); + describe("SubAgentDirector incomplete-report wiring", () => { test("tool-less narration after tools gets one wrap-up nudge, not a complete", async () => { const director = new SubAgentDirector("system", [], undefined, 30); diff --git a/src/subagent/nudge-director.ts b/src/subagent/nudge-director.ts index fb1ec456e..3e9470f1b 100644 --- a/src/subagent/nudge-director.ts +++ b/src/subagent/nudge-director.ts @@ -47,6 +47,20 @@ const TOOL_FAILURE_RECOVERY_NUDGE = const INCOMPLETE_REPORT_NUDGE = "Write your final report now using ## Summary, ## Findings, ## Blockers, and ## Paths. Do not narrate status. No more tools unless one lookup is required to cite a line."; +const VERBATIM_TOOL_CALL_NUDGE = + "You wrote tool-call markup as assistant text. Invoke the real tool call instead of printing its markup, or write your final report if no tool is needed."; + +function hasVerbatimToolCallMarkup( + content: readonly { type: string; text?: string }[], +): boolean { + return content.some( + (block) => + block.type === "text" && + typeof block.text === "string" && + /\s*<(?:function|tool)=[A-Za-z_][\w.-]*>/.test(block.text), + ); +} + function ephemeralNudgeTurn(text: string): ConversationTurn { return { role: "user", @@ -128,6 +142,10 @@ export class SubAgentDirector extends DefaultDirector { // narration without the envelope salvages as incomplete-report // (MAX_TOOLLESS_NARRATION_CYCLES = 2). private toolLessNarrationCycles = 0; + // One corrective nudge per no-real-tool epoch when the assistant prints + // explicit tool-call markup as text instead of issuing a real tool_call. + // Cleared only by genuine tool activity or a non-empty parent follow-up. + private verbatimToolCallNudgeFired = false; // Once this leaf has replied with a terminal report (complete envelope or // salvage), empty continuations from idle-compact / stall must not fall @@ -227,6 +245,7 @@ export class SubAgentDirector extends DefaultDirector { // A real parent follow-up re-opens the brief; empty continuations do not. if (isNonEmptyParentMessage(event)) { this.reportReplied = false; + this.verbatimToolCallNudgeFired = false; } const afterCompact = this.compaction.resumeAfterCompact(event); @@ -288,6 +307,7 @@ export class SubAgentDirector extends DefaultDirector { this.lastAssistantText = lastText(content); const hasToolCalls = content.some((block) => block.type === "tool_call"); if (hasToolCalls) { + this.verbatimToolCallNudgeFired = false; this.thrashState = nextThrashState(this.thrashState, content); } @@ -314,6 +334,27 @@ export class SubAgentDirector extends DefaultDirector { if (compacted !== null) return compacted; return terminal; } + + // Below the stop policy so a finished report that merely quotes + // tool-call markup still completes; a markup turn with no envelope + // gets the corrective nudge instead of the generic wrap-up one. + if ( + !hasToolCalls && + !this.verbatimToolCallNudgeFired && + hasVerbatimToolCallMarkup(content) + ) { + this.verbatimToolCallNudgeFired = true; + this.interventions({ + id: "verbatim-tool-call", + class: "nudge", + state: this.interventionState(), + detail: "assistant emitted explicit tool-call markup as text", + }); + return [ + capabilities.checkpoint("subagent-verbatim-tool-call-nudge"), + inferWithSubAgentNudge(capabilities, VERBATIM_TOOL_CALL_NUDGE), + ]; + } if (stop === "incomplete-report") { // Tool-less turn after tools, no report envelope. Must not fall through // to super.decide — DefaultDirector completes any tool-less turn.