Skip to content

fix: fork-steer silently dropped the manager message (fleet send on idle worker was a no-op) - #7

Open
aaalliii wants to merge 2 commits into
mainfrom
fix/fork-steer-dropped-steer
Open

fix: fork-steer silently dropped the manager message (fleet send on idle worker was a no-op)#7
aaalliii wants to merge 2 commits into
mainfrom
fix/fork-steer-dropped-steer

Conversation

@aaalliii

Copy link
Copy Markdown
Contributor

Bug: fleet send on an idle worker was a silent no-op

Found while live-testing fleet workers against a LongCat-2.0 backend (isolated CLAUDE_CONFIG_DIR namespace, per docs/longcat-fleet-usage.md).

Symptom

fleet send <worker> "<msg>" on an idle worker did nothing. No error — fleet status showed turns incremented and status back to idle, and compose_prompt had already drained and audited (mail_drained) the message. Silent data loss.

Reproduced deterministically twice (steer #1 "reply PONG2", steer #2 "Ignore all previous instructions… reply BANANA" → worker replied PONG both times).

Root cause

dispatch_bg put a byte-identical pointer in argv on every dispatch:

tiny_prompt = f"Read {task_path.as_posix()} and follow it exactly."

On a fork-steer the session is resumed (--resume), so the carried transcript already contains a Read(<that exact path>) tool_use and its tool_result. The model satisfies the identical instruction from cache, never re-reads the rewritten task file, and the <MANAGER MESSAGE> never enters context.

Transcript proof (turn 2 has no Read call):

[user] 'Read .../lc-probe.md and follow it exactly.'
[assistant] TOOL_USE: Read -> {...}
[user] tool_result: '1  You are fleet worker `lc-probe`...'
[assistant] text: 'PONG'
[user] 'Read .../lc-probe.md and follow it exactly.'
[assistant] text: 'PONG'          <-- no Read; steer never seen

Delivery depended on the model choosing to re-issue a tool call it believed it had already made — a volition dependency, not a mechanism. (The mid-turn hook path works precisely because it inlines text via additionalContext.)

Secondary effect: each fork re-carries the growing transcript, so input tokens climbed 268 → 12,238 across dropped steers — burning budget to accomplish nothing.

Fix

New pure helper _dispatch_argv_prompt(task_path, prompt_body, resume_sid):

  • Fresh dispatch (resume_sid falsy) → unchanged pointer. Empty transcript, worker must Read and does. argv is byte-identical to before, so cmd_spawn/cmd_respawn are untouched.
  • Resumed dispatch → inline the body, leading with the real <MANAGER MESSAGE> block (the resumed transcript already carries the preamble).
  • Oversized body (> NATIVE_INLINE_PROMPT_MAX_BYTES = 8 KiB) → pointer + content digest + explicit "it has CHANGED, re-read it, do NOT answer from an earlier Read result". Still volition-dependent, but no longer byte-identical to the cached instruction. 8 KiB sits well under the 32,767-char Windows limit (SPEC §6 G8) and POSIX MAX_ARG_STRLEN.

The marker match is line-anchored (re.search(r"^<MANAGER MESSAGE>$", body, re.M)) on purpose: the preamble itself mentions `<MANAGER MESSAGE>` inline, so a plain find() slices the preamble mid-sentence — caught live, a worker received a prompt starting `; treat them as user instructions.

Blast radius

Only the two resume_sid call sites change behaviour: _cmd_send_native (fork-steer) and _resume_one_limited_native. cmd_sup_handoff_begin is annotated exempt (fresh session + per-incarnation unique task path). No signature changes; mail_drained / claim-bracket semantics untouched.

Verification

  • Live, LongCat-2.0: before the fix the steer text never appeared in the dispatched turn; after, it is present in the user message. ⚠️ Honest caveat: in one post-fix run LongCat-2.0 still answered PONG — it anchors on its PONG-heavy transcript. The same text delivered via the Stop-hook path in that session produced BANANA, so the model can follow it. This PR fixes delivery; it cannot make a weak model obey.
  • Tests: 2 regression tests in TestDispatchBg (fresh argv unchanged; resume inlines + leads with the real block; oversized falls back to the digest pointer). Suite: 1420 passed.
  • ⚠️ tests/test_receipts.py::test_receipts_reproduce[claim-nonce.md] fails — PRE-EXISTING, verified by stashing this change and re-running (fails identically on a clean tree).
  • ⚠️ Could not run the declared 3.10 floor (python3.10 not installed on this machine); only 3.13 was exercised. Please run py -3.10 -m pytest -q before merge.

Follow-ups (not in this PR)

  • docs/SPEC.md §6 G8 states "argv carries only the tiny fixed prompt" — now true only for the non-resume case; needs narrowing.
  • fleet peek reported "no transcript found" for a just-forked session whose .jsonl existed on disk (timing/lookup race).
  • fleet spawn printed worker name already exists while nonetheless creating and running the worker (non-atomic registration).

🤖 Generated with Claude Code

aaalliii and others added 2 commits July 23, 2026 19:24
`fleet send` on an IDLE worker was a silent no-op: turns incremented,
status returned to idle, and compose_prompt had already drained (and
audited) the mail -- but the worker never saw it.

Cause: dispatch_bg put a byte-identical pointer in argv on every dispatch
("Read <task> and follow it exactly."). On a fork-steer the resumed
transcript already contains a Read(<task>) tool_use AND its tool_result
for that exact path, so the model satisfied the pointer from cache, never
re-read the rewritten file, and the <MANAGER MESSAGE> was dropped.
Delivery depended on the model choosing to re-issue a tool call it
believed it had already made.

Fix: _dispatch_argv_prompt() -- fresh dispatches keep the pointer
(unchanged argv, empty transcript); RESUMED dispatches inline the body,
leading with the real <MANAGER MESSAGE> block. Oversized bodies (>8 KiB)
fall back to a pointer carrying a content digest + an explicit
'it has CHANGED, re-read it' directive, which at least is not
byte-identical to the cached instruction.

The marker match is line-anchored on purpose: the preamble itself mentions
`<MANAGER MESSAGE>` inline, so a plain find() slices the preamble
mid-sentence (caught live -- a worker received a prompt starting
"\`; treat them as user instructions.").

Only the two resume_sid call sites change behaviour (_cmd_send_native
fork-steer, _resume_one_limited_native). cmd_spawn/cmd_respawn are
untouched; sup_handoff_begin is annotated exempt (fresh session +
per-incarnation unique task path).

Verified live against a LongCat-2.0 worker: before, the steer text never
appeared in the dispatched turn at all; after, it is present in the user
message. (Whether a weak model then obeys is separate -- LongCat-2.0 still
anchored on its transcript in one run.)

Tests: 2 regression tests in TestDispatchBg. Suite 1418 passed; the single
failure (test_receipts[claim-nonce.md]) is PRE-EXISTING -- verified by
stashing this change and re-running.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Generalises longcat-fleet-usage.md from 'how to use LongCat' to 'how to
point fleet at any Anthropic-compatible endpoint', since the mechanism is
provider-agnostic: a worker's backend is fixed by the daemon that launches
it, and a daemon's backend is fixed at its boot, so one CLAUDE_CONFIG_DIR
= one daemon = one backend.

Also CORRECTS a claim in longcat-fleet-usage.md. It said mixing providers
simultaneously was 'Impossible without a CLI change'. That holds only
inside ONE config dir. Across config dirs it works today, because fleet's
registry is shared (fleet home is ~/fleet regardless of CLAUDE_CONFIG_DIR)
while the daemons are not. Verified live 2026-07-23: two workers spawned
from different namespaces ran concurrently in one fleet status --
  mix-lc -> model=LongCat-2.0
  mix-an -> model=claude-haiku-4-5-20251001
Documents the caveats that come with it (global worker names, the registry
does not record provider, re-dispatch paths must run from the worker's own
namespace).

Also corrects the stale verification recipe: grepping ~/.claude/projects
misses an isolated namespace's transcripts; 'fleet result' printing model=
is the ground truth.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@aaalliii

Copy link
Copy Markdown
Contributor Author

Added: provider docs + a correction to longcat-fleet-usage.md

Two docs commits pushed on top of the fix.

docs/any-provider-fleet-usage.md (new) — generalises the LongCat guide into a provider-agnostic one, since the mechanism has nothing to do with LongCat: a worker's backend is fixed by the daemon that launches it, and a daemon's backend is fixed at its boot → one CLAUDE_CONFIG_DIR = one daemon = one backend. Covers the recipe for any Anthropic-compatible endpoint, the "name the model in all four env slots" rule (Anthropic aliases don't exist on foreign providers), the mandatory thinking-off setting, verification, cost caveats, and troubleshooting.

Correction to longcat-fleet-usage.md — it claimed mixing providers simultaneously was "Impossible without a CLI change." That is true only inside one config dir. Across config dirs it works today, because fleet's registry is shared while the daemons are not (fleet home is ~/fleet regardless of CLAUDE_CONFIG_DIR).

Verified live 2026-07-23 — two workers spawned from different namespaces, running concurrently, one fleet status:

$ fleet result mix-lc     # spawned from ~/.claude-longcat
-- tokens in=253 out=5 model=LongCat-2.0

$ fleet result mix-an     # spawned from the default namespace
-- tokens in=10 out=103 model=claude-haiku-4-5-20251001

Documented with its caveats: worker names are global (registry is shared), the registry does not record which provider a worker is on (fleet result's model= is ground truth), and re-dispatch paths (fork-steer, resume-limited) must be run from the worker's own namespace so it re-lands on the right backend.

Also fixes a stale verification recipe: grepping ~/.claude/projects/**/*.jsonl misses an isolated namespace's transcripts (they live under $CLAUDE_CONFIG_DIR/projects/).

Scope note for the reviewer: this is docs-only on top of a bugfix PR — happy to split it into its own PR if you'd rather keep this one narrow. No code changed in these two commits; suite still 1420 passed with the same pre-existing test_receipts[claim-nonce.md] failure.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant