Give stall recovery a full timeout grace after the first nudge (#887)

* Nudge once when assistants print tool-call markup as text

Models sometimes emit <tool_call><function=...> wrappers as
assistant text instead of real tool_call blocks. Catch that narrow
shape, give one corrective nudge per no-real-tool epoch without
counting it as incomplete-report narration, then fall through to the
existing report policy. Thinking blocks and arbitrary XML stay out of
scope; the epoch resets only on genuine tool activity or a parent
follow-up.

* Let a complete report envelope win over quoted tool-call markup

* Give stall recovery a full timeout grace after the first nudge

Queued empty stall pings that drain right after the first nudge used
to count as a second consecutive stall and stop the leaf immediately.
Record stallNudgeAt on the nudge, wait through pings inside
stallTimeoutMs without treating them as activity or restarting grace,
and stop only once the grace elapses with no tool.done or turn reset.

* Cover same-tick stall pings and document the grace window

Queued empty continuations with no clock advance must nudge then wait,
not salvage. Architecture now describes stallNudgeAt instead of a
consecutive-ping streak.

* Align product stall copy with the post-nudge grace window

* Document the post-nudge stall grace in the changelog

Author: TheGreatAxios

CHANGELOG.md

@@ -13,6 +13,12 @@ parallel copies under `docs/` or `scripts/notes/`. At cut time: rename
 
 ## [Unreleased]
 
+### Fixed
+
+- A stalled worker now gets a full `stallTimeoutMs` grace after the first
+  continuation nudge before salvage — stall pings queued inside that window
+  wait instead of counting toward escalation.
+
 ### Removed
 
 - `--force` is no longer accepted. It had no runtime effect; resume and the

docs/ARCHITECTURE.md

@@ -140,7 +140,7 @@ The ChatDirector counts consecutive assistant turns that contain tool calls and
 
 #### Sub-agent stall management
 
-`SubAgentDirector` tracks `lastActivityAt`, updated on every real `inference.done` and `tool.done`. Directors are pure `decide(event, ...)` functions with no timer of their own and the reactor has no proactive "idle" event, so a genuinely silent worker (e.g. parked on a long-running background command with nothing else to do) produces no event for the director to react to. `runSubAgent` (`src/subagent/index.ts`) arms an external interval, at `subAgentStallTimeoutMs`, that pings the same content-less continuation channel the compaction governor uses to re-enter an idle reactor (`requestContinuation`). The director only acts on a ping if the elapsed time since `lastActivityAt` has crossed the timeout — a ping delivered while a tool call is still executing simply queues until that cycle finishes, so "no pending harness-tracked work" falls out of when the check can run at all rather than needing separate bookkeeping. The first stall past the timeout gets one continuation nudge (asking the worker to check on the background work or report status); a second **consecutive** stall (no activity since that nudge) escalates to the existing salvage path, returning a `stalled` `forcedStopReport` with the same structured shape (summary/findings/blockers) as `turn-budget` and `cancelled`. Any real activity between pings resets the streak, so a worker that is genuinely working through a slow single turn is never penalized. After the worker has already replied with a terminal report (complete envelope or salvage), further empty continuations — idle-compact meter sync or stall pings — return `wait` instead of falling through to `DefaultDirector.infer`; only a non-empty parent message (`resume_agent` / `send_input`) re-opens the brief.
+`SubAgentDirector` tracks `lastActivityAt`, updated on every real `inference.done` and `tool.done`. Directors are pure `decide(event, ...)` functions with no timer of their own and the reactor has no proactive "idle" event, so a genuinely silent worker (e.g. parked on a long-running background command with nothing else to do) produces no event for the director to react to. `runSubAgent` (`src/subagent/index.ts`) arms an external interval, at `subAgentStallTimeoutMs`, that pings the same content-less continuation channel the compaction governor uses to re-enter an idle reactor (`requestContinuation`). The director only acts on a ping if the elapsed time since `lastActivityAt` has crossed the timeout — a ping delivered while a tool call is still executing simply queues until that cycle finishes, so "no pending harness-tracked work" falls out of when the check can run at all rather than needing separate bookkeeping. The first stall past the timeout records `stallNudgeAt` and issues one continuation nudge (asking the worker to check on the background work or report status). Later empty pings inside `subAgentStallTimeoutMs` of that instant wait without stopping or treating the ping as activity; salvage fires only once a ping arrives after that grace with still no `tool.done` / turn-boundary reset. That grace is what keeps two queued interval ticks from salvaging hundreds of milliseconds after the nudge. Any real activity clears `stallNudgeAt`, so a worker that is genuinely working through a slow single turn is never penalized. After the worker has already replied with a terminal report (complete envelope or salvage), further empty continuations — idle-compact meter sync or stall pings — return `wait` instead of falling through to `DefaultDirector.infer`; only a non-empty parent message (`resume_agent` / `send_input`) re-opens the brief.
 
 **Intervention log**: every stop and nudge is appended as one JSONL record to `interventions.jsonl` in the firing worker's trace dir (`src/subagent/intervention-log.ts`), carrying the trigger's measured value beside the threshold it crossed, the provider/model/family it fired on, and the run state at that moment (turns used vs budget, tool calls, read/edit counts). A refused parent re-dispatch is recorded on the parent side, where no worker run exists to record it. The parent also appends one `outcome` record per completed dispatch — the salvage kind `classifyBriefSalvage` assigned, or a clean-complete marker, plus the dispatch count — so the log carries dispatch outcomes as well as interventions, and a stop record can later be read alongside what the dispatch it touched actually produced. Writes are fire-and-forget and swallow their own errors — a diagnostic must not be able to fail a run. `scripts/intervention-forensics.ts` aggregates these across local sessions: per-intervention counts by model family, the measured-value distribution against the threshold, two context columns (stops that fired on runs which had already edited files; stops that fired before half the turn budget was spent — neither is a measured false-positive rate, since either is equally consistent with a correct stop or a wrong one), and outcome counts by kind. This exists because every threshold in this tree was set by judgment and four of those judgments were later reverted — a threshold change is expected to cite this data (CL-6938).
 

docs/PRODUCT.md

@@ -172,7 +172,7 @@ Corbits Code fans work out to short-lived **fleet agents** — workers with thei
 - **Tasks** are checklist items owned by one agent via `manage_tasks`.
 - **Fleet agents** are spawned with `spawn_agent`. On the TUI primary, mailbox mail arrives as inbound when a worker finishes or fails — do not poll `wait_agents`. Nested orchestrators still collect with `wait_agents`. Workers ask the parent with `ask_director`. That parks a question while the worker stays `running`. Nested `wait_agents` returns `awaiting_director` with a question payload — that is not terminal. The parent answers with `send_input` (`target` = the worker's session id). When the parent TUI is not blocked in `wait_agents`, a parked question arrives as a synthetic idle-send wake. Escalate to the human only with `ask_operator`.
 
-Dispatch uses a structured brief (context / goal / optional goals seed) and returns a structured report. The TUI Agents strip and fleet board show who is running; live tool progress updates the status bar without dumping the child transcript into the parent chat. There is no turn budget. A tool-less final turn completes only with the four-heading report envelope. Printed `<tool_call>` markup in assistant text gets one corrective nudge to issue a real tool call and does not count as the wrap-up; without the envelope, one incomplete-report nudge is given and a second tool-less turn without the envelope salvages as `incomplete-report-stop`. A silent worker (no activity for `stallTimeoutMs`, opt-in) gets one continuation nudge, then salvages as `stalled` if a second consecutive check finds no activity. An opt-in `deadlineMs`, or an operator cancel, can also end a run early. Each of these returns a salvage report so a runaway or idle child cannot quietly burn a large token budget or look done after prose alone.
+Dispatch uses a structured brief (context / goal / optional goals seed) and returns a structured report. The TUI Agents strip and fleet board show who is running; live tool progress updates the status bar without dumping the child transcript into the parent chat. There is no turn budget. A tool-less final turn completes only with the four-heading report envelope. Printed `<tool_call>` markup in assistant text gets one corrective nudge to issue a real tool call and does not count as the wrap-up; without the envelope, one incomplete-report nudge is given and a second tool-less turn without the envelope salvages as `incomplete-report-stop`. A silent worker (no activity for `stallTimeoutMs`, opt-in) gets one continuation nudge, then salvages as `stalled` only after a full `stallTimeoutMs` grace with still no activity — queued checks inside that window wait, they do not salvage. An opt-in `deadlineMs`, or an operator cancel, can also end a run early. Each of these returns a salvage report so a runaway or idle child cannot quietly burn a large token budget or look done after prose alone.
 
 ## Roadmap (planned, not yet shipped)
 

src/subagent/nudge-director.test.ts

@@ -1014,6 +1014,117 @@ describe("SubAgentDirector post-complete terminalization (CL-7068)", () => {
   });
 });
 
+describe("SubAgentDirector stall nudge grace", () => {
+  test("two queued empty pings in the same tick nudge then wait, not stop", async () => {
+    let now = 3_000_000;
+    const director = new SubAgentDirector(
+      "system",
+      [],
+      undefined,
+      1_000,
+      () => now,
+    );
+    const caps = capabilities();
+
+    await director.decide(inferenceDoneText("working"), state, caps);
+
+    now += 1_000;
+    const first = actions(
+      await director.decide(messageReceived(""), state, caps),
+    );
+    expect(first).toContainEqual({
+      type: "checkpoint",
+      message: "subagent-stall-nudge",
+    });
+    expect(first.some((action) => action.type === "reply")).toBe(false);
+
+    const second = actions(
+      await director.decide(messageReceived(""), state, caps),
+    );
+    expect(second).toEqual([{ type: "wait" }]);
+  });
+
+  test("queued pings inside grace wait; stop only after grace with no activity", async () => {
+    let now = 1_000_000;
+    const director = new SubAgentDirector(
+      "system",
+      [],
+      undefined,
+      1_000,
+      () => now,
+    );
+    const caps = capabilities();
+
+    await director.decide(inferenceDoneText("working"), state, caps);
+
+    now += 1_000;
+    const first = actions(
+      await director.decide(messageReceived(""), state, caps),
+    );
+    expect(first).toContainEqual({
+      type: "checkpoint",
+      message: "subagent-stall-nudge",
+    });
+
+    now += 200;
+    const midGrace = actions(
+      await director.decide(messageReceived(""), state, caps),
+    );
+    expect(midGrace).toEqual([{ type: "wait" }]);
+
+    now += 200;
+    const stillGrace = actions(
+      await director.decide(messageReceived(""), state, caps),
+    );
+    expect(stillGrace).toEqual([{ type: "wait" }]);
+
+    now += 600;
+    const stopped = actions(
+      await director.decide(messageReceived(""), state, caps),
+    );
+    expect(stopped).toContainEqual({
+      type: "checkpoint",
+      message: "subagent-stalled",
+    });
+    expect(stopped.some((action) => action.type === "reply")).toBe(true);
+  });
+
+  test("tool.done during grace clears stallNudgeAt so a later silence nudges again", async () => {
+    let now = 2_000_000;
+    const director = new SubAgentDirector(
+      "system",
+      [],
+      undefined,
+      1_000,
+      () => now,
+    );
+    const caps = capabilities();
+
+    await director.decide(inferenceDone(["read-1"]), state, caps);
+    now += 1_000;
+    const first = actions(
+      await director.decide(messageReceived(""), state, caps),
+    );
+    expect(first).toContainEqual({
+      type: "checkpoint",
+      message: "subagent-stall-nudge",
+    });
+
+    now += 100;
+    await director.decide(toolDone("read-1"), state, caps);
+
+    now += 1_000;
+    const afterActivity = actions(
+      await director.decide(messageReceived(""), state, caps),
+    );
+    expect(afterActivity).toContainEqual({
+      type: "checkpoint",
+      message: "subagent-stall-nudge",
+    });
+    expect(afterActivity.some((action) => action.type === "reply")).toBe(false);
+  });
+});
+
 function stubAdmission(
   notes: { provider: string; until: number }[],
 ): AdmissionQueue {

src/subagent/nudge-director.ts

@@ -176,7 +176,11 @@ export class SubAgentDirector extends DefaultDirector {
   private readonly stallTimeoutMs: number | undefined;
   private readonly now: () => number;
   private lastActivityAt: number;
-  private consecutiveStalls = 0;
+  // Wall clock when the first stall nudge was issued. Later empty pings inside
+  // stallTimeoutMs of this instant wait without stopping or restarting grace;
+  // stop only after the grace elapses with no activity. Cleared on real
+  // tool.done / turn-boundary activity.
+  private stallNudgeAt: number | undefined;
   private lastAssistantText = "";
   // Every stop and nudge is recorded with its measured value beside its
   // threshold, so a later threshold change can cite data instead of judgment
@@ -303,7 +307,7 @@ export class SubAgentDirector extends DefaultDirector {
     if (onTurnBoundary(event)) {
       this.lastConsumedNudgeText = null;
       this.lastActivityAt = this.now();
-      this.consecutiveStalls = 0;
+      this.stallNudgeAt = undefined;
       this.compaction.noteInferenceDone(event, state.turns);
       this.turnsCompleted++;
       const content = event.turn.content as readonly {
@@ -410,7 +414,7 @@ export class SubAgentDirector extends DefaultDirector {
     }
     if (event.type === "tool.done") {
       this.lastActivityAt = this.now();
-      this.consecutiveStalls = 0;
+      this.stallNudgeAt = undefined;
       if (event.result.isError === true) {
         // Failed-tool recovery guidance. Arm once; coalesce consecutive failure
         // audits until applyPendingNudge flushes a single counted record.
@@ -438,12 +442,12 @@ export class SubAgentDirector extends DefaultDirector {
    * "no pending harness-tracked work" falls out of when this method can run
    * at all rather than needing separate bookkeeping.
    *
-   * First stall past the timeout: one continuation nudge, asking the leaf to
-   * report status or keep going. A second consecutive stall (no activity
-   * since the nudge) escalates to the existing salvage path, same shape as
-   * the turn-boundary checks above. Returns null when this event is not
-   * a stall check the director should act on (let it fall through as an
-   * ordinary continuation).
+   * First silence past the timeout: one continuation nudge, and record
+   * stallNudgeAt. Queued pings that arrive inside the stallTimeoutMs grace
+   * after that nudge neither stop nor restart the grace (and do not count as
+   * activity). Stop only when a ping arrives after the grace with still no
+   * activity. Returns null when this event is not a stall check the director
+   * should act on (let it fall through as an ordinary continuation).
    */
   private checkStallPing(
     event: ReactorInboundEvent,
@@ -456,8 +460,8 @@ export class SubAgentDirector extends DefaultDirector {
     const elapsed = this.now() - this.lastActivityAt;
     if (elapsed < this.stallTimeoutMs) return null;
 
-    this.consecutiveStalls++;
-    if (this.consecutiveStalls === 1) {
+    if (this.stallNudgeAt === undefined) {
+      this.stallNudgeAt = this.now();
       this.interventions({
         id: "stall-nudge",
         class: "nudge",
@@ -473,6 +477,14 @@ export class SubAgentDirector extends DefaultDirector {
         inferWithSubAgentNudge(capabilities, SUBAGENT_STALL_NUDGE),
       ];
     }
+
+    const sinceNudge = this.now() - this.stallNudgeAt;
+    if (sinceNudge < this.stallTimeoutMs) {
+      // Still inside the post-nudge grace. Wait without faking activity or
+      // restarting the grace clock.
+      return [capabilities.wait()];
+    }
+
     this.interventions({
       id: "stalled",
       class: "stop",