Skip to content

docs(review): adversarial review of fleet-index design - #5

Draft
Techn0Ninja27 wants to merge 5 commits into
fleet-implfrom
worktree-idx-adv2-review
Draft

docs(review): adversarial review of fleet-index design#5
Techn0Ninja27 wants to merge 5 commits into
fleet-implfrom
worktree-idx-adv2-review

Conversation

@Techn0Ninja27

Copy link
Copy Markdown
Contributor

Hostile design review of docs/superpowers/specs/2026-07-22-fleet-index-design.md (341 lines @ c991cd4) against the 8 named traps. Report only — no design file touched.

10 defects: 2 CRITICAL, 5 HIGH, 2 MED, 1 LOW. Verdict: restructure into 3 milestones; do not ship as one; do not reject the idea.

Trap Verdict Sev
1. §2 impossibility claim DEFECT — conclusion survives, mechanism is false MED
2. Index-in-git under parallel workers DEFECT ×2 CRITICAL + HIGH
3. Break-even arithmetic DEFECT — always-on map.md is net-negative here HIGH
4. Adoption DEFECT — fleet q is blocked, not merely unadopted CRITICAL
5. Staleness split DEFECT — no lock specified; §9 undoes §8 HIGH
6. Verified code claims DEFECT — 6 of 8 wrong HIGH
7. Receipt collision DEFECT — 2 executable receipts, both unfixable as written HIGH
8. Other 3 further defects + 1 unspecified rule MED/MED/LOW

The two CRITICALs

symbols.tsv's sort key conflicts at N = 2. §5.2 sorts by name, which scatters each file's symbols through the whole table, so a per-file change edits hunks everywhere. Reproduced with this repo's real 2090-symbol table: two branches, fully disjoint source files → CONFLICT (content) in symbols.tsv. A path-sorted control with the identical edits auto-merges clean. This retires the disjoint-file-parallelism rule at knowledge/lessons.md:24 and the 7-wide result at :103.

fleet q cannot execute for the default worker. bin/fleet.py:8478 sets spawn's default --mode to dontask, and worker-settings.template.json ships no permissions key at all. §6.2 asserts Bash(fleet q:*) is safe to allowlist but never adds it anywhere. Same failure class as the headless permission hang documented at bin/fleet.py:7390-7395, which is why --add-dir exists.

Also worth the manager's attention

  • The design's worked example is wrong. compose_prompt 887→917 (end 946→972), dispatch_bg 6621→7357 (end 6702→7573), _PREAMBLE_TEMPLATE 879→909, mailbox_dir end 71→70, file length 7832→8706, and both signatures. Those values are §5.2/§5.3's sample symbols.tsv rows — the demonstration of "exact derived facts, never summaries treated as truth" is a hand-written summary treated as truth.
  • §2's mechanism is backwards. bin/fleet.py:7381,7385,7407: the composed prompt is written to a task file and the argv prompt is Read <path> and follow it exactly. Name/task/journal are never in the prefix — they arrive as a tool result. The conclusion holds; the reason does not, and the wrong reason is what sends map.md into the one channel guaranteed never to be shared.
  • Two fenced blocks parse as executable receipts (§6.2 lines 161 and 168) and fail today unpinned and pinned. Line 170's expected output is the placeholder <actual source of those 60 lines> — unmatchable forever. docs/specs/*.md is a non-recursive glob, so the doc currently sits outside the harness entirely; moving it in fires test_every_spec_is_classified (reproduced in the review).

Suggested split

  1. M1 — indexer + .fleet-index/ gitignored + --context digest injection only. No map, no fleet q, no committed artifact, no concurrency question, no permissions migration.
  2. M2fleet q, gated on worker-settings.template.json gaining Bash(fleet q:*) and the fleet init migration that implies.
  3. M3map.md, opt-in per spawn, with the break-even measurement §12.1 currently cannot produce.

Committing the index is a separate decision needing a path-sorted table, an atomic-replace writer, and an answer for git worktree remove.

Every claim in the review carries a re-runnable witness (path:line or pasted command output). Worker journal: state/journals/idx-adv2.md.

🤖 Generated with Claude Code

Techn0Ninja27 and others added 5 commits July 22, 2026 02:47
…date

Operator closed all four §14 residuals 2026-07-20:

- R1 Phase-1 gate ships first behind an on/off feature flag (sdd.enabled,
  default off) so it can be exercised before it binds anything; the
  Phase-2 live Stop-fence follows once Phase-1 is proven on a real
  campaign. Flag lands as build slice 0 so every later slice is testable
  in isolation.
- R2 both scopes: a minor whole-spec scope (campaign floor) AND per-worker
  slices layered on it. Effective scope = slice INTERSECT whole-spec; a
  slice may only narrow, never widen.
- R3 both judge paths: build `spec verify --judge` auto-dispatch AND keep
  the campaign-template doctrine step. Advisory either way -- neither
  moves the exit code.
- R4 specs git-tracked from birth at docs/specs/campaigns/<campaign>.md,
  not runtime state/. Git adds a second tamper surface over
  accepted_digest. Once built, SDD folds into docs/SPEC.md fully as a
  first-class section, not a pointer.

Retargeted M-E -> M-F (M-E shipped 2026-07-21). Verify stamp stays runtime
at state/spec-verify/<campaign>.json (derived, regenerated).

New §15 records two teammate asks as out-of-scope-for-M-F, needing their
own specs: cross-platform agent runtimes (Hermes/OpenCode/Codex/Cursor --
an agent-runtime adapter, larger than the native pivot; note the Phase-1
gate is already runtime-agnostic while the Phase-2 hook is not), and a
standalone non-agentic oracle for agent-to-agent verification (partially
answered today by deterministic-tier-decides, accepted_digest, and git
tracking; the isolated-process trust boundary and loop detection are
unspeced).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Indexing is now opt-in per project: `fleet index init` is the only
command that creates .fleet-index/. build/update/q all refuse when
it is absent, so a project that never opts in is untouched and
spawn there behaves exactly as today.

Adds §6.4 index lifecycle. Two distinct writes, previously
conflated: query-time staleness refresh (working tree, uncommitted,
exists only so `fleet q` never serves a stale line number) vs the
gated update (post-review, post-merge, committed alongside the code
it describes). Manager owns the gated step, as it owns merges.
Rejects post-merge git hooks and PostToolUse for it -- both would
index unreviewed code into the shared artifact and defeat the gate.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ROADMAP speccing discipline requires all three before a spec is
proposable. The graveyard check found a direct ancestor:
IDEA-FORGE-REPORT §5 entry 5 "Knowledge-Aware Context Assembly at
Spawn/Respawn" -- right moat, wrong build.

Answered per cause of death. The load-bearing one is
"silently-rotting tag schema": this design IS a tag file, and the
difference is content-hash staleness detection (§8) -- it cannot rot
silently, it detects and repairs. The ancestor had none.

"grep-isn't-ranking" is dodged by construction everywhere except
`degrade = "relevance"`, which is ranking and re-imports the exact
flaw; demoted to opt-in and flagged for deletion if unproven.
"ahead of journal replay" answered by ordering -- journal keeps final
position; index material is per-project derived fact, not lore.

Flag re-vet: `--context` raw inlining is the flag-sized alternative
and delivers real value, but does not scale to orientation (7832-line
file costs more than it saves) and cannot slice. Records the cut
order if scope must shrink: relevance, then map, then `fleet q`,
keeping digest injection last.

Invariants touched: 4 (journal-injection) and 9 (one-state-many-views),
both argued preserved; 1, 2, 8 explicitly preserved.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Plan (docs/superpowers/plans/2026-07-22-sdd-mf-phase1.md): 10 TDD tasks
covering SDD slices 0-3 -- sdd_enabled() flag first so every later slice is
testable in isolation, JSON machine-block parser, accepted_digest over
executable fields only, pattern-domain scope resolution (slice may only
narrow, R2), the four-query touched-path union incl. the mandatory ignored
query, files+pytest criteria with zero-collect classed FAIL not INFRA,
spec new/accept with the durable-actor promotion guard, the verify gate
with tamper check and atomic stamp, binding with explicit respawn carry +
baseline re-stamp, and stamp-read-only surfacing. Deferred to a Phase-2
plan: --judge, the live Stop-fence, spec supersede, the SPEC.md fold.

Oracle design (docs/superpowers/specs/2026-07-22-oracle-design.md, DRAFT,
under review): a non-agentic auditor answering the teammate's
"agents can only see another agent's answer" problem. Isolation defined as
four testable properties (never runs in an agent session, verdicts only
from agent-unwritable evidence, no LLM in the verdict path, its own
evidence integrity-checked). Evidence is classed TRUSTED /
UNTRUSTED-measurable / UNTRUSTED-never-a-verdict-input; a check is
malformed if removing prose would change its verdict. Six checks O1-O6:
task integrity, contract stability, output-within-contract (deliberately
duplicating the SDD gate so disagreement is itself CRITICAL), criteria
actually ran, topology/spiral detection, evidence integrity. Gate teeth
(blocks spec verify) but no kill authority; fails CLOSED, unlike the SDD
fence which fails open.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Hostile review against the 8 named traps. 10 defects: 2 CRITICAL, 5 HIGH,
2 MED, 1 LOW. Verdict: restructure into 3 milestones, do not ship as one.

The two CRITICALs:

- symbols.tsv is sorted by name, which interleaves every file's symbols
  through the whole table, so a per-file change edits hunks everywhere.
  Two workers on disjoint files conflict at N=2. Reproduced with this
  repo's real 2090-symbol table; a path-sorted control merges clean.
  This retires the disjoint-file-parallelism rule in knowledge/lessons.md:24.

- fleet q cannot execute for the default worker. bin/fleet.py:8478 sets
  spawn's default mode to dontask, and worker-settings.template.json ships
  no permissions block, so the query half of the milestone is blocked
  rather than merely unadopted. Same failure class as the headless
  permission hang documented at bin/fleet.py:7390-7395.

Also: the design's own worked example is wrong. Six of eight cited code
facts are stale or incorrect, including every value in the sample
symbols.tsv rows -- compose_prompt 887->917, dispatch_bg 6621->7357,
file length 7832->8706, both signatures. And two fenced blocks parse as
executable receipts that fail today and cannot be made to pass as written.

Report-only; no design file touched.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant