From c78a55fb22bc35214a854d6117a825d3072236df Mon Sep 17 00:00:00 2001 From: Dmitry Polishuk Date: Sun, 1 Nov 2026 16:19:50 +0300 Subject: [PATCH 1/2] fix(bridge): keep threads alive after ZCode-side failures Two reliability fixes from the 2026-10-29/11-01 incidents: - Classify the two observed transient Z.ai account failures (inactive API-key status, account-service request failure) as typed internal provider errors before the failed boundary, so bb's Provider retry plugin can apply its bounded four-attempt backoff instead of leaving the turn terminal. - Self-heal threads whose ZCode process exits unexpectedly. session.ended does not clear bb's stored provider-thread claim, so every later turn.submit failed forever with "No ZCode session; start or resume the thread first." (observed on thr_8htvmki9xm and thr_d6pdiigxzx). The bridge now sends an unsolicited provider/recovery restartRecommended notification for restorable sessions - bb restarts the bridge worker and resumes the thread plus idle siblings from native checkpoints - and attaches the same recovery hint to the residual no-session error for submits that race the restart. Non-restorable sessions still just end; resume keeps failing loudly when the checkpoint cannot restore. 254 tests green; marketplace:check, typecheck, format clean. --- CHANGELOG.md | 12 +++ docs/configuration.md | 17 ++++ knowledge/architecture/session-invariants.md | 6 ++ knowledge/contracts/protocol-contracts.md | 21 +++++ knowledge/log.md | 25 +++++ knowledge/runbooks/diagnostics.md | 43 ++++++--- skills/zcode-provider/SKILL.md | 9 ++ src/bridge.ts | 49 +++++++++- tests/core-features.test.ts | 97 ++++++++++++++++++++ 9 files changed, 263 insertions(+), 16 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index b252328..8bf82ff 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,17 @@ # Changelog +## Unreleased + +- Classify the observed transient Z.ai account failures as typed internal + provider errors so bb's Provider retry plugin can retry the failed turn up to + four times with backoff. Retry remains owned by bb; other authentication and + configuration failures stay terminal. +- Self-heal threads whose ZCode process exits unexpectedly: the bridge now asks + bb (`provider/recovery` with `restartRecommended`) to restart it and resume + the thread from its native checkpoint, and the residual `No ZCode session` + turn error carries the same recovery hint. Previously every later submit on + such a thread failed permanently until the thread was manually stopped. + ## 0.14.1 — 2026-10-28 - Drop the `[Experimental]` label from the plugin's store name and overview diff --git a/docs/configuration.md b/docs/configuration.md index 4f21b95..1852e52 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -206,6 +206,23 @@ stale-turn recovery when no turn remains active. `/compact` uses the native compaction operation. Normal context reporting uses native estimates; billing cost is not fabricated. +Recognized transient Z.ai account failures (an inactive API-key status or an +account-service request/response failure) are reported to bb as retryable +internal provider errors. With bb's built-in Provider retry plugin enabled, a +failed turn is retried up to four times with backoff; bb decides whether to +continue accepted input or resend unaccepted input. Other authentication and +configuration failures remain terminal. `bb provider-retry explain` shows the +last scheduling or skip decision. + +When the ZCode process exits unexpectedly mid-thread, the bridge fails the +running turn, ends the session, and asks bb to restart the bridge and resume +the thread from its native checkpoint (`provider/recovery`, +`restartRecommended`). bb defers that restart while sibling threads on the same +process are mid-turn. If a submit still races the restart and fails with +`No ZCode session; start or resume the thread first.`, the error carries the +same recovery hint and the next submit lands on the resumed session; threads +that never started a turn have no checkpoint to restore and start fresh. + ## MCP and native runtime preferences Set `memoryEnabled` or `nativeSearchEnhancementsEnabled` through bb's plugin diff --git a/knowledge/architecture/session-invariants.md b/knowledge/architecture/session-invariants.md index 029de5e..3420952 100644 --- a/knowledge/architecture/session-invariants.md +++ b/knowledge/architecture/session-invariants.md @@ -57,6 +57,12 @@ matching tests and documentation updates (see invented, native Bash file edits are reported as ignored rather than claimed restorable, and the fallback "ZCode models unavailable" entry does not claim authentication. +- Losing the ZCode process is reported, not papered over: the running turn + fails, the session ends, and — when the native checkpoint can restore it — + the bridge asks bb to restart and resume (`provider/recovery` + `restartRecommended`). The bridge never rebuilds a dead session in place or + answers as if nothing happened; resume still goes through the strict + checkpoint path and fails loudly when the session is not restorable. ## Context selection lifetime diff --git a/knowledge/contracts/protocol-contracts.md b/knowledge/contracts/protocol-contracts.md index 31514ac..6257f5f 100644 --- a/knowledge/contracts/protocol-contracts.md +++ b/knowledge/contracts/protocol-contracts.md @@ -132,6 +132,27 @@ snapshots. - These are agent approval policies, not an OS sandbox; the CLI runs with the host account's permissions. +- **Transient failure classification**: the two observed temporary Z.ai account + failures (inactive API-key status and account-service request/response + failure) produce a `provider.error` delta with `errorInfo.category: +"internal"` before `turn.boundary(status: "failed")`. This lets bb's Provider + retry plugin apply its own bounded durable retry policy and accepted-input + continuation semantics. The bridge never replays the prompt itself and leaves + other authentication/configuration failures unclassified. +- **Crash recovery contract**: bb keeps the stored provider-thread claim after + the `session.ended` delta and only clears it through `thread/stop` semantics, + so a bridge that silently drops a session strands the thread (every later + `turn/start` fails with `No ZCode session`). The protocol escape hatch — + verified against the bb 0.46 daemon (`daemon-bundle.mjs`) — is the + unsolicited `provider/recovery` notification with + `kind: "restartRecommended"` plus the failing thread id: the daemon shuts the + bridge worker down, resumes the named thread via `thread/resume`, then + resumes idle sibling threads, deferring while a sibling is mid-turn. The + bridge sends it from the ZCode child-exit handler when the session is + restorable, and attaches the same hint as `data.recovery` on the residual + `No ZCode session` `-32602` error so a racing submit also triggers the + restart. The daemon auto-resumes on its own only when its own session map + was reset (daemon restart); an in-worker child exit is invisible to it. ## Marketplace catalog contract diff --git a/knowledge/log.md b/knowledge/log.md index 9675f79..9b4f9a0 100644 --- a/knowledge/log.md +++ b/knowledge/log.md @@ -1,5 +1,30 @@ # Knowledge log +## 2026-11-01 (crash self-healing) + +- **Fixed** threads stranded by an unexpected ZCode process exit. `session.ended` + does not clear bb's stored provider-thread claim, so every later `turn.submit` + failed forever with `No ZCode session; start or resume the thread first.` + (observed on `thr_8htvmki9xm` and `thr_d6pdiigxzx`; the bb 0.46 daemon keeps + its own session map and cannot see an in-worker child exit). The bridge now + sends an unsolicited `provider/recovery` `restartRecommended` notification + when a restorable session crashes — bb restarts the bridge worker and resumes + the thread and idle siblings from native checkpoints — and attaches the same + recovery hint to the residual no-session turn error for submits that race the + restart. Non-restorable sessions (no turn ever started) still just end. + Recorded the daemon-side contract in + [protocol contracts](contracts/protocol-contracts.md); three focused + regressions cover both paths; 254 tests green. + +## 2026-10-29 (transient provider retry) + +- **Fixed** ZCode turns ending permanently on the two observed temporary Z.ai + account failures. The bridge now emits typed `provider.error` before the + failed boundary, allowing bb's Provider retry plugin to schedule its normal + four-attempt backoff rather than adding a plugin-side replay loop. Regression + coverage includes both transient messages, delta ordering, and a terminal + reasoning-configuration error; full `npm run check` passes with 251 tests. + ## 2026-10-27 (v0.13.0, feat/workflow-visualization branch) - **Added** timeline rendering for dynamic workflows (roadmap stage 9 v1): diff --git a/knowledge/runbooks/diagnostics.md b/knowledge/runbooks/diagnostics.md index 8a57e6b..939be94 100644 --- a/knowledge/runbooks/diagnostics.md +++ b/knowledge/runbooks/diagnostics.md @@ -36,18 +36,19 @@ runbook adds the code-level view for contributors. Error messages are stable search keys; each is produced by exactly one module: -| Message / behavior | Produced by | Meaning | -| ------------------------------------------------------------------------------ | -------------------- | -------------------------------------------------------------------------------------- | -| `ZCode was not found. Install ZCode or set the plugin executable setting.` | `src/launch.ts` | Resolution order (setting → `BB_ZCODE_EXECUTABLE` → PATH → macOS bundle) found nothing | -| `Cannot launch ZCode: …` / `ZCode exited (signal/code)` | `src/client.ts` | Child spawn failure or early exit; check executable and provider configuration | -| `ZCode stdin closed.` | `src/client.ts` | Child closed stdin while requests were pending | -| `ZCode Coding Plan: cannot read the desktop configuration…` | `src/coding-plan.ts` | Desktop sign-in missing; open ZCode and sign in again | -| `The ZCode follow-up queue is full.` | `src/bridge.ts` | More than 32 pending follow-ups (bounded queue, by design) | -| `This ZCode CLI does not expose compatible v4 history controls. Update ZCode.` | `src/lifecycle.ts` | v4 row query failed schema check; CLI too old or protocol changed | -| `ZCode returned inconsistent history rows.` | `src/lifecycle.ts` | Rows not strictly descending / outside the requested range | -| `Invalid bb tool catalog: too many or duplicate tools.` | `src/tool-server.ts` | >256 tools or duplicate names | -| `bb skill roots must be absolute paths (maximum 128).` | `src/skills.ts` | bb sent relative skill paths or too many roots | -| Tool endpoint 401 / 404 / 405 / 413 | `src/tool-server.ts` | Bad bearer token or browser `Origin` / wrong path or inactive / non-POST / body >1 MiB | +| Message / behavior | Produced by | Meaning | +| ------------------------------------------------------------------------------ | -------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `ZCode was not found. Install ZCode or set the plugin executable setting.` | `src/launch.ts` | Resolution order (setting → `BB_ZCODE_EXECUTABLE` → PATH → macOS bundle) found nothing | +| `Cannot launch ZCode: …` / `ZCode exited (signal/code)` | `src/client.ts` | Child spawn failure or early exit; check executable and provider configuration | +| `ZCode stdin closed.` | `src/client.ts` | Child closed stdin while requests were pending | +| `ZCode Coding Plan: cannot read the desktop configuration…` | `src/coding-plan.ts` | Desktop sign-in missing; open ZCode and sign in again | +| `The ZCode follow-up queue is full.` | `src/bridge.ts` | More than 32 pending follow-ups (bounded queue, by design) | +| `No ZCode session; start or resume the thread first.` | `src/bridge.ts` | `turn/start`/`turn/steer` arrived after the ZCode process exited or the bridge worker restarted; carries `data.recovery` `restartRecommended` so bb restarts the bridge and resumes the thread | +| `This ZCode CLI does not expose compatible v4 history controls. Update ZCode.` | `src/lifecycle.ts` | v4 row query failed schema check; CLI too old or protocol changed | +| `ZCode returned inconsistent history rows.` | `src/lifecycle.ts` | Rows not strictly descending / outside the requested range | +| `Invalid bb tool catalog: too many or duplicate tools.` | `src/tool-server.ts` | >256 tools or duplicate names | +| `bb skill roots must be absolute paths (maximum 128).` | `src/skills.ts` | bb sent relative skill paths or too many roots | +| Tool endpoint 401 / 404 / 405 / 413 | `src/tool-server.ts` | Bad bearer token or browser `Origin` / wrong path or inactive / non-POST / body >1 MiB | Bridge-level errors are thrown as `ProtocolError` (default code `-32000`); native requests time out after 30 s by default (`src/client.ts`). @@ -92,6 +93,24 @@ From the provider skill's diagnostics section (kept authoritative there): [Change verification](change-verification.md)). - Follow-up failures after queue overflow are expected bounded behavior, not a bug; report them to the user instead of retrying unbounded. +- The two observed transient Z.ai account failures (`The API key status is not +active.` and `account request failed or returned an invalid response.`) emit + `provider.error` with `errorInfo.category: "internal"` before the failed turn + boundary. bb's Provider retry plugin owns the bounded retry and accepted-input + continuation; the bridge does not replay input itself. Inspect the recorded + decision with `bb provider-retry explain `. Other native failures, + including reasoning/configuration errors, stay unclassified and terminal. +- When the ZCode process exits unexpectedly, the bridge emits the + `session.ended` delta and then an unsolicited `provider/recovery` + (`restartRecommended`, `retryable: true`) notification for restorable sessions. + bb reacts by restarting the bridge worker and resuming the thread (and idle + sibling threads) from native checkpoints — look for `Restarting the "zcode" +bridge` in the host-daemon log to confirm it ran. This matters because + `session.ended` alone does not clear bb's stored provider-thread claim, which + is why submits used to fail forever with `No ZCode session` (observed + 2026-11-01: threads `thr_8htvmki9xm`, `thr_d6pdiigxzx`). The notification is + deliberately skipped when no turn ever started (`restorable: false`) — there is + no checkpoint to restore. ## Related knowledge diff --git a/skills/zcode-provider/SKILL.md b/skills/zcode-provider/SKILL.md index 792c0a8..9f5ee88 100644 --- a/skills/zcode-provider/SKILL.md +++ b/skills/zcode-provider/SKILL.md @@ -61,6 +61,15 @@ Use this skill for setup and diagnostics of the `zcode` provider in bb. - A model catalog or healthy protocol probe is not proof of authenticated model connectivity. `npm run test:live` uses a real CLI and synthetic loopback model with isolated data. Keep that evidence separate from a real provider run. +- Recognized transient Z.ai account failures emit typed `provider.error` deltas + so bb's Provider retry plugin can retry the turn up to four times with + backoff. Check `bb provider-retry explain ` for the decision; + authentication and configuration errors otherwise remain terminal. +- When the ZCode process exits unexpectedly, the bridge asks bb to restart it + and resume the thread from its native checkpoint; a submit racing that + restart fails once with `No ZCode session; start or resume the thread +first.` and works on the next attempt. `Restarting the "zcode" bridge` in + `~/.bb/logs/host-daemon.*.log` confirms the recovery ran. - Diagnose a stopped session by its native error. Never silently replace an unresumable session, replay a prompt after an uncertain timeout, or copy credentials out of ZCode to work around authentication. diff --git a/src/bridge.ts b/src/bridge.ts index f0ca60b..2395559 100644 --- a/src/bridge.ts +++ b/src/bridge.ts @@ -604,6 +604,24 @@ export function createBridge( session.timeline = undefined; session.activeTurnId = undefined; session.compacting = false; + if ( + status === "failed" && + (response?.includes("The API key status is not active.") || + response?.includes( + "account request failed or returned an invalid response.", + )) + ) + emit(session, [ + { + kind: "provider.error", + message: response, + errorInfo: { + category: "internal", + httpStatusCode: null, + providerCode: null, + }, + }, + ]); timeline?.finish(status, response, checkpoint); for (const [id, pending] of interactions) if (pending.session === session) { @@ -1678,6 +1696,18 @@ export function createBridge( settle(session, "failed", failure.message); sessions.delete(session.threadId); emit(session, [{ kind: "session.ended" }]); + // session.ended does not clear bb's stored provider-thread + // claim, so every later turn/start would fail with "No ZCode + // session" until this bridge restarts. Ask bb to restart and + // resume the thread while the native checkpoint still restores; + // threads that never started a turn have nothing to restore. + if (session.restorable) + notify("provider/recovery", { + threadId: session.threadId, + kind: "restartRecommended", + message: `The ZCode process exited unexpectedly: ${failure.message}`, + retryable: true, + }); } if (client) { clients.delete(client); @@ -2534,10 +2564,21 @@ export function createBridge( result(id, {}); return; } - throw new ProtocolError( - "No ZCode session; start or resume the thread first.", - -32602, - ); + // The session was lost without a thread/stop (a crashed ZCode process + // or a restarted bridge worker). bb keeps the provider-thread claim, so + // attach the protocol recovery hint: bb restarts this bridge, resumes + // the thread from its native checkpoint, and the next submit works. + const message = "No ZCode session; start or resume the thread first."; + error(id, -32602, message, { + recovery: { + kind: "restartRecommended", + message: + "The ZCode session is gone. Restart the bridge to resume the " + + "thread from its native checkpoint, then submit again.", + retryable: true, + }, + }); + return; } if (p.providerThreadId !== session.providerThreadId) throw new ProtocolError("Provider session id mismatch.", -32602); diff --git a/tests/core-features.test.ts b/tests/core-features.test.ts index ae38cfc..1fbfce2 100644 --- a/tests/core-features.test.ts +++ b/tests/core-features.test.ts @@ -328,6 +328,48 @@ it("settles native compaction from its state-update completion when no normal tu ]); }); +it.each([ + "The API key status is not active. Request id: request-id", + "ZCode Coding Plan: account request failed or returned an invalid response. Check ZCode sign-in and network access.", +])( + "classifies transient native account failure for bb automatic retry: %s", + async (message) => { + const h = harness(); + await h.start(); + await h.send("first"); + h.event("turn.failed", { error: { message } }); + await h.until(() => h.deltas().some((d) => d.kind === "turn.boundary")); + const deltas = h.deltas(); + const errorIndex = deltas.findIndex((d) => d.kind === "provider.error"); + const boundaryIndex = deltas.findIndex((d) => d.kind === "turn.boundary"); + expect(errorIndex).toBeGreaterThanOrEqual(0); + expect(deltas[errorIndex]).toMatchObject({ + kind: "provider.error", + message: `ZCode: ${message}`, + errorInfo: { + category: "internal", + httpStatusCode: null, + providerCode: null, + }, + }); + expect(errorIndex).toBeLessThan(boundaryIndex); + }, +); + +it("does not mark native configuration failures as automatically retryable", async () => { + const h = harness(); + await h.start(); + await h.send("first"); + h.event("turn.failed", { + error: { message: "Reasoning level is required for test/model" }, + }); + await h.until(() => h.deltas().some((d) => d.kind === "turn.boundary")); + expect(h.deltas().some((d) => d.kind === "provider.error")).toBe(false); + expect(h.deltas().find((d) => d.kind === "turn.boundary")).toMatchObject({ + status: "failed", + }); +}); + it("returns typed stale-turn recovery after the target turn has completed", async () => { const h = harness(); await h.start(); @@ -645,6 +687,61 @@ it("never drains queued work after a child exits during terminal usage refresh", ]); }); +it("asks bb to restart and resume after a restorable session crashes", async () => { + const h = harness(); + await h.start(); + await h.send("first"); + h.event("turn.completed", { resultType: "success" }); + await h.until(() => h.deltas().some((d) => d.kind === "turn.boundary")); + h.crash(); + await h.until(() => h.messages.some((m) => m.method === "provider/recovery")); + const endedAt = h.messages.findIndex((m) => + (m.params as { deltas?: { kind: string }[] } | undefined)?.deltas?.some( + (d) => d.kind === "session.ended", + ), + ); + const recoveryAt = h.messages.findIndex( + (m) => m.method === "provider/recovery", + ); + expect(endedAt).toBeGreaterThanOrEqual(0); + expect(recoveryAt).toBeGreaterThan(endedAt); + expect( + h.messages.find((m) => m.method === "provider/recovery")!.params, + ).toMatchObject({ + threadId: "thread", + kind: "restartRecommended", + retryable: true, + }); +}); + +it("does not request a bridge restart when nothing is restorable", async () => { + const h = harness(); + await h.start(); + h.crash(); + await h.until(() => h.deltas().some((d) => d.kind === "session.ended")); + await new Promise((r) => setTimeout(r, 15)); + expect(h.messages.some((m) => m.method === "provider/recovery")).toBe(false); +}); + +it("attaches a restart recovery hint to turns submitted without a session", async () => { + const h = harness(); + await h.start(); + await h.send("first"); + h.event("turn.completed", { resultType: "success" }); + await h.until(() => h.deltas().some((d) => d.kind === "turn.boundary")); + h.crash(); + await h.until(() => h.deltas().some((d) => d.kind === "session.ended")); + const reply = await h.send("after crash", "abcdefghik"); + expect(record(reply.error).code).toBe(-32602); + expect(record(reply.error).message).toBe( + "No ZCode session; start or resume the thread first.", + ); + expect(record(record(reply.error).data).recovery).toMatchObject({ + kind: "restartRecommended", + retryable: true, + }); +}); + it("does not replay an accepted turn/start client request", async () => { const h = harness(); await h.start(); From 6a43bc167b899a2ae2ce33b82986180b09be7900 Mon Sep 17 00:00:00 2001 From: Dmitry Polishuk Date: Sun, 1 Nov 2026 17:05:07 +0300 Subject: [PATCH 2/2] test: cover crash during an active turn and align recovery docs Address PR #6 review: - Crash the session while the turn is still in flight (the previous test always completed the turn first), and assert provider/recovery fires with restartRecommended + retryable from the terminal-usage-refresh crash path too; the settled-turn path now asserts the restart request precedes the racing no-session error. - docs/configuration.md and the zcode-provider skill no longer promise the next submit works: bb can defer the bridge restart while sibling threads are mid-turn, so a repeated 'No ZCode session' means recovery has not finished - wait for the restart, then resubmit. - session-invariants: record sources and a 2026-11-01 verification entry for the crash-recovery invariant, with the evidence boundary (synthetic regressions + statically verified bb 0.46 daemon contract; no live bb-smoke) stated in the body. --- docs/configuration.md | 11 +++++++---- knowledge/architecture/session-invariants.md | 16 ++++++++++++++++ skills/zcode-provider/SKILL.md | 9 +++++---- tests/core-features.test.ts | 8 +++++--- 4 files changed, 33 insertions(+), 11 deletions(-) diff --git a/docs/configuration.md b/docs/configuration.md index 1852e52..7d0baae 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -218,10 +218,13 @@ When the ZCode process exits unexpectedly mid-thread, the bridge fails the running turn, ends the session, and asks bb to restart the bridge and resume the thread from its native checkpoint (`provider/recovery`, `restartRecommended`). bb defers that restart while sibling threads on the same -process are mid-turn. If a submit still races the restart and fails with -`No ZCode session; start or resume the thread first.`, the error carries the -same recovery hint and the next submit lands on the resumed session; threads -that never started a turn have no checkpoint to restore and start fresh. +process are mid-turn, and the restart plus resume take a moment once begun — a +submit that races it fails with `No ZCode session; start or resume the thread +first.` and carries the same recovery hint. A repeated error means the +restart has not completed yet (the host-daemon log shows `Restarting the +"zcode" bridge` once it starts): wait for the recovery to finish, then +resubmit. Threads that never started a turn have no checkpoint to restore and +start fresh. ## MCP and native runtime preferences diff --git a/knowledge/architecture/session-invariants.md b/knowledge/architecture/session-invariants.md index 3420952..ed8edd0 100644 --- a/knowledge/architecture/session-invariants.md +++ b/knowledge/architecture/session-invariants.md @@ -18,8 +18,14 @@ sources: resource: https://github.com/dpolishuk/bb-zcode/blob/3dc639de8e7926b2460059dcf20156950fc7f2d3/src/tool-server.ts - title: Current accepted model selection and turn settlement resource: ../../src/bridge.ts + - title: Crash recovery notification and no-session recovery hint + resource: ../../src/bridge.ts - title: Native follow-up, resume, rejection, and model-switch regressions resource: ../../tests/core-features.test.ts + - title: Crash-recovery regressions (in-flight and settled turns) + resource: ../../tests/core-features.test.ts + - title: bb daemon crash-recovery contract, bb 0.46 daemon bundle + resource: ../contracts/protocol-contracts.md#crash-recovery-contract - title: Real CLI fixture for reporting budgets and background work resource: ../../scripts/smoke-live.ts generated: @@ -30,6 +36,8 @@ verified: at: 2026-10-20T11:15:00Z - by: Codex at: 2026-10-22T11:22:00Z + - by: zcode-agent/GLM-5.3 + at: 2026-11-01T16:35:00Z --- # Session invariants @@ -85,6 +93,14 @@ loopback fixture (native background completion and cold resume). Other pinned bounds retain the verification dates stated below; see [validation](../../docs/validation.md) for the evidence boundary. +The crash-recovery invariant was re-verified on 2026-11-01: synthetic +regressions cover both a crash while the turn is still in flight and a crash +after it settled, and the bb-side behavior (restart the bridge worker, resume +the thread and idle siblings, defer while a sibling is mid-turn) was verified +statically against the bb 0.46 daemon bundle — see +[Protocol contracts](../contracts/protocol-contracts.md#crash-recovery-contract). +Live bb-smoke on a disposable instance remains the unexecuted step. + ## Credential containment - Short-lived Coding Plan project tokens exist only in memory; desktop diff --git a/skills/zcode-provider/SKILL.md b/skills/zcode-provider/SKILL.md index 9f5ee88..3c8a1d5 100644 --- a/skills/zcode-provider/SKILL.md +++ b/skills/zcode-provider/SKILL.md @@ -66,10 +66,11 @@ Use this skill for setup and diagnostics of the `zcode` provider in bb. backoff. Check `bb provider-retry explain ` for the decision; authentication and configuration errors otherwise remain terminal. - When the ZCode process exits unexpectedly, the bridge asks bb to restart it - and resume the thread from its native checkpoint; a submit racing that - restart fails once with `No ZCode session; start or resume the thread -first.` and works on the next attempt. `Restarting the "zcode" bridge` in - `~/.bb/logs/host-daemon.*.log` confirms the recovery ran. + and resume the thread from its native checkpoint. A submit racing that + restart fails with `No ZCode session; start or resume the thread first.`; if + the error repeats, the restart is still deferred or in progress (bb defers + it while sibling threads are mid-turn) — wait for `Restarting the "zcode" +bridge` in `~/.bb/logs/host-daemon.*.log`, then resubmit. - Diagnose a stopped session by its native error. Never silently replace an unresumable session, replay a prompt after an uncertain timeout, or copy credentials out of ZCode to work around authentication. diff --git a/tests/core-features.test.ts b/tests/core-features.test.ts index 1fbfce2..d4138ad 100644 --- a/tests/core-features.test.ts +++ b/tests/core-features.test.ts @@ -685,14 +685,13 @@ it("never drains queued work after a child exits during terminal usage refresh", expect(h.deltas().filter((d) => d.kind === "turn.boundary")).toEqual([ { kind: "turn.boundary", status: "failed" }, ]); + await h.until(() => h.messages.some((m) => m.method === "provider/recovery")); }); -it("asks bb to restart and resume after a restorable session crashes", async () => { +it("asks bb to restart and resume when a crash interrupts the active turn", async () => { const h = harness(); await h.start(); await h.send("first"); - h.event("turn.completed", { resultType: "success" }); - await h.until(() => h.deltas().some((d) => d.kind === "turn.boundary")); h.crash(); await h.until(() => h.messages.some((m) => m.method === "provider/recovery")); const endedAt = h.messages.findIndex((m) => @@ -730,6 +729,9 @@ it("attaches a restart recovery hint to turns submitted without a session", asyn h.event("turn.completed", { resultType: "success" }); await h.until(() => h.deltas().some((d) => d.kind === "turn.boundary")); h.crash(); + // A settled restorable session must still trigger the restart request + // before the racing submit sees the typed no-session error. + await h.until(() => h.messages.some((m) => m.method === "provider/recovery")); await h.until(() => h.deltas().some((d) => d.kind === "session.ended")); const reply = await h.send("after crash", "abcdefghik"); expect(record(reply.error).code).toBe(-32602);