Conversation
This implements the solution for #6351, declaring the set of specs that the L3 bench should test-report: packets, frames, far, and flow. The file includes: - BENCH_SPEC_STEMS constant with the four spec names - bench_tests(stem: string) -> bool function that returns true if the stem is in the set of specs to test - test_bench_coverage_includes_all_four_specs: verifies all four specs return true - bench_coverage_excludes_others: verifies unknown stems return false - bench_coverage_matches_issue_requirement: verifies exactly 4 specs are covered Closes #6351
A pull request must add exactly one docs/now entry and a bee has no way to know that: its brief names a boundary file and acceptance criteria, and docs/now/ is neither. The publisher adds it rather than failing the gate. Closes #6351 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
![t27-bees[bot]](https://avatars.githubusercontent.com/u/6774813?s=60&v=4){kind=small}
There was a problem hiding this comment.
Reviewer bee verdict for head 51f11a92f6ee02f7514dd0b911b4f07405a2a288 (tools/bees/reviewer.py, zai glm-4.7-flash, 7 turns, 338 s).
BEE-VERDICT: REQUEST_CHANGES
summary: PR creates bench_coverage.t27 spec but does not modify xilinx7-l3-bench.yml workflow to actually run test-report on all four specs; workflow remains unchanged at flow.t27 only
criterion: FR-001: declares the set of specs the L3 bench test-reports (packets, frames, far, flow) and fn bench_tests(stem: ...) -> bool, with tests that each of the four is covered and that a stem outside the set is not -- met -- evidence: specs/xilinx7/bench_coverage.t27:3-11,39-43,64-68
discounted-check: Corpus ratchet (expected-failure ledger) -- same failure on master for spec/port/trinity/src/brain/orbitofrontal_value.t27 [parse]
discounted-check: coverage -- same 3 dangling seals on master
discounted-check: duplicate-bodies -- same failures on master
discounted-check: emit-bitexact -- same newly broken spec on master (orbitofrontal_value.t27)
discounted-check: scan -- same hardcoded /Users/playra/ paths on master
discounted-check: spec-guards -- same diverging ring-096-rust with formats.t27 on master
blocking-check: Issue goal -- workflow still only runs flow.t27 (line 59) without using the new spec; tests only run in isolation, not as a rule for the workflow to consume
Evidence
Now I have all the information I need. Let me analyze the pull request:
Issue #6351 Analysis
Issue Goal: Modify the xilinx7-l3-bench.yml workflow to run t27c test-report on all four xilinx7 specs with tests (packets, frames, far, flow), not just flow.t27 alone.
Requirements (from brief.md):
- FR-001: Create
specs/xilinx7/bench_coverage.t27that declares the set of specs the L3 bench test-reports and hasfn bench_tests(stem: ...) -> bool, with tests that each of the four is covered and a stem outside the set is not. - FR-002: Do NOT add or modify non-t27 hand-written code.
- FR-003: Comment must cite the measurement (35 tests pass; about 1.7s extra) and that editing the workflow to read this rule needs the owner's label.
Success Criteria (all PASSED by runner):
grep -c 'fn bench_tests' specs/xilinx7/bench_coverage.t27→ 1 ✓grep -cE 'packets|frames' specs/xilinx7/bench_coverage.t27→ 7 ✓t27c test-report specs/xilinx7/bench_coverage.t27→ 2 passes/0 FAIL ✓
PR Changes
The PR creates two new files:
specs/xilinx7/bench_coverage.t27- A t27 spec withbench_tests()function and testsdocs/now/2026-10-06-published-xilinx7-l3-bench-run-test-report-on-packets-frames-and-far-n.md- Documentation
CRITICAL FINDING: The PR does NOT modify .github/workflows/xilinx7-l3-bench.yml. The workflow still hardcodes only flow.t27 (line 59), as shown in the runner's checkout.
The spec file's comment says it "replaces hardcoded spec names in .github/workflows/xilinx7-l3-bench.yml" (line 3-4 of bench_coverage.t27), but this never happened. The spec declares bench_tests() function but there's no code to actually READ or USE this function. The workflow remains unchanged and the tests are only tested in isolation, not as a rule for the workflow to consume.
The issue asks to "run test-report on packets, frames and far, not on flow.t27 alone" - the PR creates a spec that could enable this, but the workflow that should use it remains unmodified. The spec is effectively a stand-alone specification that documents what SHOULD happen, but doesn't implement the change requested.
Red Checks Analysis
Corpus ratchet (expected-failure ledger)- Master has same failure (spec/port/trinity/src/brain/orbitofrontal_value.t27 [parse]) - DISCOUNTEDduplicate-bodies- Master has same failures - DISCOUNTEDemit-bitexact- Master has 1 spec failing to generate (same one as PR) - DISCOUNTEDcoverage- Master has same 3 dangling seals - DISCOUNTEDspec-guards- Master has ring-096-rust diverging with formats.t27 - DISCOUNTEDscan- Master has same hardcoded /Users/playra/ paths - DISCOUNTED
All red checks are pre-existing on master.
Evaluation
Does the change do what issue #6351 asks? No.
The issue asks to modify the workflow to run test-report on all four specs (packets, frames, far, flow). The PR creates a spec file that documents a rule and tests it in isolation, but does NOT modify the workflow file to actually USE this rule. The workflow still only runs flow.t27 (line 59). Without modifying the workflow to read the spec and call t27c test-report on the other specs, the change is incomplete.
Damage: No damage. The new files don't affect any other code.
This branch has not been deployed
2 participants
Closes #6351
Written by a bee on
queen-6351and published bytools/queen/publish.py. The branch itself is the bee's; the second commit is the coordination entry every pull request must add, which a bee has no way to know about.🤖 Generated with Claude Code