Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -116,7 +116,7 @@ Two directors, selected by role:

Auto mode is toggled by CLI flags (`--auto` / `--no-auto`); there is currently no in-session key to toggle it (default on; constrained envelope — workspace writes and unconstrained shell auto-allow; installs, recursive rm, force/uncontained worktree changes, sensitive-path and opaque-wrapper shell still ask; contained non-force `git worktree add`/`remove`/`prune` and `list` auto-allow; shell file-mutation denied). It is not a separate edit/plan mode.

- **SubAgentDirector** (delegated work, `src/subagent/index.ts`) — Drives a dispatched worker until a turn arrives with no tool calls, then replies with the final assistant text and ends the run. A tool-less turn **after tools** completes only with the four-heading envelope (Summary, Findings, Blockers, Paths); a missing envelope nudges once (**incomplete-report**) and a second tool-less turn still without the envelope salvages as **incomplete-report-stop**. Explore/read-only workers that used tools then replied with findings remain normal completes; `requireEvidence` (off by default, set per director) additionally requires at least one read before a tool-less spawn-only reply can complete. Reads done through `run_shell` count as evidence too — `src/subagent/shell-evidence.ts` classifies shell reads (`cat`, `grep`, `sed` without `-i`, …) over the same subject expansion the auto-shell policy uses — but there is no corresponding shell-write evidence or file-write requirement: a run that never touches a file still completes normally once it replies with the envelope. There is no turn budget. Operator/parent cancel after any progress returns a **cancelled** salvage report (partial findings + tool activity) instead of a bare cancel string; cancel before progress still surfaces as cancelled-by-operator. There is no repetition/no-progress/never-acted/never-edited hard stop and no fingerprint-based re-dispatch block — a genuinely stuck worker runs until it completes, stalls, hits an opt-in wall-clock deadline, or is cancelled.
- **SubAgentDirector** (delegated work, `src/subagent/index.ts`) — Drives a dispatched worker until a turn arrives with no tool calls, then replies with the final assistant text and ends the run. A tool-less turn **after tools** completes only with the four-heading envelope (Summary, Findings, Blockers, Paths). Assistant text that prints explicit `<tool_call>` markup is treated as attempted tool use, not narration: one **verbatim-tool-call** nudge asks the worker to re-issue a real `tool_call` and does not count toward the tool-less spiral. A missing envelope otherwise nudges once (**incomplete-report**) and a second tool-less turn still without the envelope salvages as **incomplete-report-stop**. Explore/read-only workers that used tools then replied with findings remain normal completes; `requireEvidence` (off by default, set per director) additionally requires at least one read before a tool-less spawn-only reply can complete. Reads done through `run_shell` count as evidence too — `src/subagent/shell-evidence.ts` classifies shell reads (`cat`, `grep`, `sed` without `-i`, …) over the same subject expansion the auto-shell policy uses — but there is no corresponding shell-write evidence or file-write requirement: a run that never touches a file still completes normally once it replies with the envelope. There is no turn budget. Operator/parent cancel after any progress returns a **cancelled** salvage report (partial findings + tool activity) instead of a bare cancel string; cancel before progress still surfaces as cancelled-by-operator. There is no repetition/no-progress/never-acted/never-edited hard stop and no fingerprint-based re-dispatch block — a genuinely stuck worker runs until it completes, stalls, hits an opt-in wall-clock deadline, or is cancelled.
`spawn_agent` starts each worker and records it in the caller's fleet mailbox. On the TUI primary, mailbox mail is the collect path: occupancy takes uncollected terminals and re-enters the parent as system inbound. Nested orchestrators still collect with `wait_agents`. TUI-primary `wait_agents` may yield as a timeout (workers untouched, no take) so occupancy can deliver mail or a queued Enter steer. Already-collected waits return status without a second report or error body. Wait JSON includes `stop_reason` from the session when present so a salvage that is wait-`done` is not mistaken for a clean complete, and so parent-initiated interrupt (`interrupted`) is not mistaken for operator-cancel (`cancelled`). Deadline salvage prepends an advisory parent hint suggesting continuation plus a longer deadline if more wall-clock time is warranted. Failed and incomplete-report salvage tell the parent to diagnose from the report or error and MAY spawn one successor with a changed brief. A parent-initiated interrupt is a resumable pause: wait unblocks with `stop_reason: interrupted` (often while the session is still running and has no report); the parent should `resume_agent` or re-wait, and must not spawn a successor against a still-live worker. Successor only if that session is no longer resumable. Operator-cancelled salvage asks the parent to synthesize Findings and Paths and wait for the operator instead of auto-starting another specialist. Identical re-dispatch of the same brief stays refused at the prompt / spawn-handoff layer; there is no fingerprint-based re-dispatch hard-block. Deadline hints are advisory only — an identical re-dispatch is still admitted at runtime. Parent hints are prepended on salvage reports returned to the parent. The runtime does not auto-spawn successors.

#### Model-family policy (`src/agent/model-family-policy.ts`)
Expand Down
2 changes: 1 addition & 1 deletion docs/PRODUCT.md
Original file line number Diff line number Diff line change
Expand Up @@ -170,7 +170,7 @@ Corbits Code fans work out to short-lived **fleet agents** — workers with thei
- **Tasks** are checklist items owned by one agent via `manage_tasks`.
- **Fleet agents** are spawned with `spawn_agent`. On the TUI primary, mailbox mail arrives as inbound when a worker finishes or fails — do not poll `wait_agents`. Nested orchestrators still collect with `wait_agents`. Workers ask the parent with `ask_director`. That parks a question while the worker stays `running`. Nested `wait_agents` returns `awaiting_director` with a question payload — that is not terminal. The parent answers with `send_input` (`target` = the worker's session id). When the parent TUI is not blocked in `wait_agents`, a parked question arrives as a synthetic idle-send wake. Escalate to the human only with `ask_operator`.

Dispatch uses a structured brief (context / goal / optional goals seed) and returns a structured report. The TUI Agents strip and fleet board show who is running; live tool progress updates the status bar without dumping the child transcript into the parent chat. There is no turn budget. A tool-less final turn completes only with the four-heading report envelope; without it, one nudge is given and a second tool-less turn without the envelope salvages as `incomplete-report-stop`. A silent worker (no activity for `stallTimeoutMs`, opt-in) gets one continuation nudge, then salvages as `stalled` if a second consecutive check finds no activity. An opt-in `deadlineMs`, or an operator cancel, can also end a run early. Each of these returns a salvage report so a runaway or idle child cannot quietly burn a large token budget or look done after prose alone.
Dispatch uses a structured brief (context / goal / optional goals seed) and returns a structured report. The TUI Agents strip and fleet board show who is running; live tool progress updates the status bar without dumping the child transcript into the parent chat. There is no turn budget. A tool-less final turn completes only with the four-heading report envelope. Printed `<tool_call>` markup in assistant text gets one corrective nudge to issue a real tool call and does not count as the wrap-up; without the envelope, one incomplete-report nudge is given and a second tool-less turn without the envelope salvages as `incomplete-report-stop`. A silent worker (no activity for `stallTimeoutMs`, opt-in) gets one continuation nudge, then salvages as `stalled` if a second consecutive check finds no activity. An opt-in `deadlineMs`, or an operator cancel, can also end a run early. Each of these returns a salvage report so a runaway or idle child cannot quietly burn a large token budget or look done after prose alone.

## Roadmap (planned, not yet shipped)

Expand Down
12 changes: 8 additions & 4 deletions scripts/intervention-forensics.ts
Original file line number Diff line number Diff line change
Expand Up @@ -151,15 +151,19 @@ for (const file of files) {
bucket = emptyBucket();
buckets.set(key, bucket);
}
bucket.count++;
const occurrence = record.count ?? 1;
bucket.count += occurrence;
const family = record.family ?? record.model ?? "unknown";
bucket.byFamily.set(family, (bucket.byFamily.get(family) ?? 0) + 1);
bucket.byFamily.set(
family,
(bucket.byFamily.get(family) ?? 0) + occurrence,
);
const model = record.model ?? "unknown";
bucket.byModel.set(model, (bucket.byModel.get(model) ?? 0) + 1);
bucket.byModel.set(model, (bucket.byModel.get(model) ?? 0) + occurrence);
if (record.class === "stop" || record.class === "nudge") {
interventionsByModel.set(
model,
(interventionsByModel.get(model) ?? 0) + 1,
(interventionsByModel.get(model) ?? 0) + occurrence,
);
}
if (record.measurement !== undefined) {
Expand Down
10 changes: 10 additions & 0 deletions src/subagent/intervention-log.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,16 @@ describe("intervention log", () => {
expect(records.map((r) => r.id)).toEqual(["report-forced", "turn-budget"]);
});

test("preserves an optional coalesced count on the record", async () => {
const dir = await mkdtemp(join(tmpdir(), "intervention-log-"));
const sink = createInterventionLog(dir, { role: "leaf" });
sink({ id: "tool-failure-recovery", class: "nudge", count: 3 });
await flush();

const [record] = await readRecords(dir);
expect(record?.count).toBe(3);
});

test("a write failure never throws into the caller", async () => {
const sink = createInterventionLog(
join(tmpdir(), "intervention-log-missing-dir-xyz"),
Expand Down
6 changes: 6 additions & 0 deletions src/subagent/intervention-log.ts
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,12 @@ export interface InterventionRecord {
};
/** Free-form specifics, kept short (a looped window, a refused fingerprint). */
detail?: string;
/**
* How many consecutive same-trigger audits this record represents. Present when
* the director coalesced a burst (e.g. several failed tool.done events before
* the pending recovery nudge was consumed) into one flush. Absent means one.
*/
count?: number;
}

/** Fields every record from one run shares, supplied once at construction. */
Expand Down
235 changes: 234 additions & 1 deletion src/subagent/nudge-director.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -84,13 +84,20 @@ function inferenceDone(
}

function inferenceDoneText(text: string, inputTokens = 0): ReactorInboundEvent {
return inferenceDoneContent([{ type: "text", text }], inputTokens);
}

function inferenceDoneContent(
content: readonly Record<string, unknown>[],
inputTokens = 0,
): ReactorInboundEvent {
return {
type: "inference.done",
turn: {
role: "assistant",
model: "test",
timestamp: 0,
content: [{ type: "text", text }],
content,
},
usage: {
input: inputTokens,
Expand Down Expand Up @@ -172,6 +179,83 @@ describe("SubAgentDirector tool failure recovery", () => {
expect(texts?.[0]).toContain("report the blocker");
});

test("coalesces consecutive failed tool audits into one counted intervention", async () => {
const director = new SubAgentDirector("system", [], undefined, 30);
const caps = capabilities();
const records: { id: string; count?: number }[] = [];
director.observeInterventions((event) => {
records.push(
event.count === undefined
? { id: event.id }
: { id: event.id, count: event.count },
);
});

await director.decide(
inferenceDone(["fail-a", "fail-b", "ok-c"]),
state,
caps,
);
await director.decide(toolDone("fail-a", true), state, caps);
expect(records).toEqual([]);
await director.decide(toolDone("fail-b", true), state, caps);
expect(records).toEqual([]);

const texts = ephemeralTexts(
inferAction(await director.decide(toolDone("ok-c"), state, caps)),
);
expect(texts).toHaveLength(1);
expect(texts?.[0]).toContain("A tool call failed");
expect(records).toEqual([{ id: "tool-failure-recovery", count: 2 }]);
});

test("a single failed tool audit omits the count field", async () => {
const director = new SubAgentDirector("system", [], undefined, 30);
const caps = capabilities();
const records: { id: string; count: number | null }[] = [];
director.observeInterventions((event) => {
records.push({ id: event.id, count: event.count ?? null });
});

await director.decide(inferenceDone(["fail-a"]), state, caps);
await director.decide(toolDone("fail-a", true), state, caps);
await director.decide(inferenceDoneText(REPORT_ENVELOPE), state, caps);

expect(records).toEqual([{ id: "tool-failure-recovery", count: null }]);
});

test("flushes an undelivered recovery burst when the run goes terminal", async () => {
const director = new SubAgentDirector("system", [], undefined, 30);
const caps = capabilities();
const records: { id: string; count?: number }[] = [];
director.observeInterventions((event) => {
records.push(
event.count === undefined
? { id: event.id }
: { id: event.id, count: event.count },
);
});

// ok-c stays pending so the armed recovery nudge never reaches an infer.
await director.decide(
inferenceDone(["fail-a", "fail-b", "ok-c"]),
state,
caps,
);
await director.decide(toolDone("fail-a", true), state, caps);
await director.decide(toolDone("fail-b", true), state, caps);
expect(records).toEqual([]);

const result = actions(
await director.decide(inferenceDoneText(REPORT_ENVELOPE), state, caps),
);
expect(result).toContainEqual({
type: "checkpoint",
message: "subagent-complete",
});
expect(records).toEqual([{ id: "tool-failure-recovery", count: 2 }]);
});

test("successful tool result has no ephemeral recovery turn", async () => {
const director = new SubAgentDirector("system", [], undefined, 30);
const caps = capabilities();
Expand Down Expand Up @@ -419,6 +503,155 @@ const REPORT_ENVELOPE = [
"src/gate.ts",
].join("\n");

describe("SubAgentDirector verbatim tool markup recovery", () => {
const verbatimToolCall =
'<tool_call><function=read_file>{"path":"src/index.ts"}</function></tool_call>';

test("nudges once for explicit tool-call wrapper text before report policy", async () => {
const director = new SubAgentDirector("system", [], undefined, 30);
const caps = capabilities();

const correction = actions(
await director.decide(inferenceDoneText(verbatimToolCall), state, caps),
);
expect(correction).toContainEqual({
type: "checkpoint",
message: "subagent-verbatim-tool-call-nudge",
});
expect(ephemeralTexts(inferAction(correction))?.[0]).toContain(
"real tool call",
);

const reportNudge = actions(
await director.decide(inferenceDoneText(verbatimToolCall), state, caps),
);
expect(reportNudge).toContainEqual({
type: "checkpoint",
message: "subagent-incomplete-report-nudge",
});

const stopped = actions(
await director.decide(inferenceDoneText(verbatimToolCall), state, caps),
);
expect(stopped).toContainEqual({
type: "checkpoint",
message: "subagent-incomplete-report",
});
});

test("does not treat arbitrary XML or thinking as verbatim tool calls", async () => {
const caps = capabilities();
const arbitraryXML = new SubAgentDirector("system", [], undefined, 30);
const arbitraryResult = actions(
await arbitraryXML.decide(
inferenceDoneText("<read_file>src/index.ts</read_file>"),
state,
caps,
),
);
expect(arbitraryResult).toContainEqual({
type: "checkpoint",
message: "subagent-incomplete-report-nudge",
});

const thinkingOnly = new SubAgentDirector("system", [], undefined, 30);
const thinkingResult = actions(
await thinkingOnly.decide(
inferenceDoneContent([
{ type: "thinking", thinking: verbatimToolCall },
]),
state,
caps,
),
);
expect(thinkingResult).toContainEqual({
type: "checkpoint",
message: "subagent-incomplete-report-nudge",
});
});

test("resets correction only after genuine tool activity or parent follow-up", async () => {
const director = new SubAgentDirector("system", [], undefined, 30);
const caps = capabilities();

await director.decide(inferenceDoneText(verbatimToolCall), state, caps);
const narration = actions(
await director.decide(inferenceDoneText("Still working"), state, caps),
);
expect(narration).toContainEqual({
type: "checkpoint",
message: "subagent-incomplete-report-nudge",
});

await director.decide(inferenceDone(["read-1"]), state, caps);
await director.decide(toolDone("read-1"), state, caps);
const afterTool = actions(
await director.decide(inferenceDoneText(verbatimToolCall), state, caps),
);
expect(afterTool).toContainEqual({
type: "checkpoint",
message: "subagent-verbatim-tool-call-nudge",
});

await director.decide(messageReceived("Try again"), state, caps);
const afterFollowup = actions(
await director.decide(inferenceDoneText(verbatimToolCall), state, caps),
);
expect(afterFollowup).toContainEqual({
type: "checkpoint",
message: "subagent-verbatim-tool-call-nudge",
});
});

test("after the verbatim nudge a real tool call executes", async () => {
const director = new SubAgentDirector("system", [], undefined, 30);
const caps = capabilities();

await director.decide(inferenceDoneText(verbatimToolCall), state, caps);
const result = actions(
await director.decide(inferenceDone(["read-1"]), state, caps),
);
expect(result.some((action) => action.type === "execute_tools")).toBe(true);
expect(result.some((action) => action.type === "reply")).toBe(false);
});

test("after the verbatim nudge a four-heading envelope completes", async () => {
const director = new SubAgentDirector("system", [], undefined, 30);
const caps = capabilities();

await director.decide(inferenceDoneText(verbatimToolCall), state, caps);
const result = actions(
await director.decide(inferenceDoneText(REPORT_ENVELOPE), state, caps),
);
expect(result).toContainEqual({
type: "checkpoint",
message: "subagent-complete",
});
});

test("a complete envelope that quotes tool-call markup still completes", async () => {
const director = new SubAgentDirector("system", [], undefined, 30);
const caps = capabilities();

const reportQuotingMarkup = `${REPORT_ENVELOPE}\n\nThe model emitted ${verbatimToolCall} as text.`;
const result = actions(
await director.decide(
inferenceDoneText(reportQuotingMarkup),
state,
caps,
),
);
expect(result).toContainEqual({
type: "checkpoint",
message: "subagent-complete",
});
expect(result).not.toContainEqual({
type: "checkpoint",
message: "subagent-verbatim-tool-call-nudge",
});
});
});

describe("SubAgentDirector incomplete-report wiring", () => {
test("tool-less narration after tools gets one wrap-up nudge, not a complete", async () => {
const director = new SubAgentDirector("system", [], undefined, 30);
Expand Down
Loading
Loading