Conversation
…nswers 153 dispatches sit in sendBack having spent no retry budget, and not one of them has a single criterion that was tested and FAILED. Every one carries a complete, parseable VERDICT block. They were returned for criteria they had answered. The brief hands the bee numbered slots and asks for `- 1. <criterion>: met`, so `parseVerdictBlock` returns a criterion beginning "1. ". The promised text carries no such number. `unjudgedCriteria` documents matching by containment in EITHER direction - and one of those two directions could not fire even once, because the line always held a prefix the promise did not. Only `line.includes(want)` survived, and that demands the bee reproduce the criterion WHOLE. Shorten it by one trailing sentence and the criterion reads as never answered. Measured over all 153: stripping the prefix clears 148. The remaining five are unjudged for reasons of their own. Stripped on BOTH sides, inside `normalize`, so this stays one rule instead of a special case at a call site. The filter returns the ORIGINAL strings, so nothing a person reads loses its number. The repository already had the right answer and did not know it. `missingVerdictSlots` strips the number and matches by slot; run over these same 153 it reports every one COMPLETE. Its comment claims it and the review "read the SAME block ... so a line this counts as answered is exactly a line the review counts as judged". That was false: two implementations of one rule, disagreeing, with a comment asserting they could not. WHY THE SUITE MISSED IT. There was already a case for each direction of the match, including the truncated-quote case. Every one used an UNNUMBERED criterion, and production has not sent an unnumbered one since the brief began numbering slots. A numbered line that quotes its criterion in full still matches on the old code, so only the shortening shape fails - the numbered cases were never written and the shortening cases were never numbered. Two new cases fail before this change and pass after; five more are labelled guards and pass on both sides, so the strip cannot be bought at the cost of the containment fence. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
✅ Tests passed — 2064/2095
|
…the board I wrote that "the repository already had the right answer and did not know it", meaning `missingVerdictSlots`. Measured across the whole board, that is wrong and worth correcting before someone acts on it. Of 358 verdict blocks with criteria, 156 carry NO numbering at all, and slot matching finds NOTHING covered on every one of those 156. Wiring it as the review's matcher would mark every criterion in 44% of the board unanswered - the defect this branch fixes, with a wider blast radius. The two functions disagree on 347 of 358 rows, in BOTH directions. Neither is right alone: text matching was blind to the number, slot matching is blind to its absence. Stripping the number and then matching by text is the only rule that reads both the numbered briefs and the unnumbered ones, which is why the fix is a strip and not a swap. Said so at the call site, as a DO NOT SIMPLIFY note, because the resemblance is what makes the wrong move attractive. Also removed the false sentence from `missingVerdictSlots` itself: "one parser, so a line this counts as answered is exactly a line the review counts as judged". Sharing a parser is not sharing a rule. That sentence is what made the numbered prefix look like somebody else's already-solved problem, and it is why the defect kept its cover. No behaviour change; comments only. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Correcting my own claim in this PRThe description says "The repository already had the right answer and did not know it", meaning Measured across the whole board (358 verdict blocks with criteria):
So slot matching is blind on 44% of the board. Wiring it as the review's matcher would mark every criterion in those 156 unanswered — this PR's defect with a wider blast radius. The two functions disagree on 347 of 358 rows, in both directions: Neither is right alone. Text matching was blind to the number; slot matching is blind to its absence. Stripping the number and then matching by text is the only rule that reads both the numbered briefs and the unnumbered ones — so the fix is a strip, not a swap. The diff is unchanged; this adds two comments:
Found by building the agreement gate that this round set out to write — the gate's first run corrected the round that motivated it. |
The finding
153 dispatches sit in
sendBackhaving spent no retry budget. Not one has a single criterion that was tested and failed. Every one carries a complete, parseable## VERDICTblock. They were returned for criteria they had already answered.The cause
The brief hands the bee numbered slots and asks for
- 1. <criterion>: met, soparseVerdictBlockreturns a criterion beginning"1. ". The promised text carries no number.unjudgedCriteriadocuments matching by containment in either direction:One of those two directions could not fire even once.
want.includes(line)is impossible when the line always carries a prefix the promise lacks. Onlyline.includes(want)survived, and that demands the bee reproduce the criterion whole. Shorten it by one trailing sentence and the criterion reads as never answered.Because "unanswered" is deliberately not charged against the retry ceiling (browseros-ai#1420 FR-003), these never escalate and never stop. That is the 80-issue ceiling-less loop reported in #471, and this is its cause.
The measurement
Re-judged all 153 with the review's own functions, against the live board:
148 of 153. The remaining five are unjudged for reasons of their own.
Real example (browseros-ai#1527) — promise vs. what the bee wrote:
The repository already had the right answer
missingVerdictSlotsstrips the number and matches by slot. Run over these same 153 it reports every one complete. Its own comment claims it and the review "read the SAME block ... so a line this counts as answered is exactly a line the review counts as judged." That was false — two implementations of one rule, disagreeing, with a comment asserting they could not.Why the suite missed it
There was already a case for each direction of the match, including the truncated-quote case. Every one used an unnumbered criterion, and production has not sent an unnumbered one since the brief began numbering slots. A numbered line that quotes its criterion in full still matches on the old code, so only the shortening shape fails: the numbered cases were never written, and the shortening cases were never numbered.
The change
One line, inside
normalize, applied to both sides so this stays one rule rather than a special case at a call site. The filter returns the original strings, so nothing a person reads loses its number.Verification
Extracted the real
unjudgedCriteriafrom the committed blob and ran it directly (the function is self-contained, so it needs none of the server's deps — this checkout cannot run the full suite):The two failing cases are the discriminating ones. The other five are labelled guards and pass on both sides, so the strip cannot be bought at the cost of the containment fence — in particular a short numbered line still does not judge a long criterion, and a criterion opening with a decimal keeps its meaning.
CI is the witness for the full suite.
🤖 Generated with Claude Code