Repository navigation
docs(#169): add AGENTS.md numbered-list consistency rules - #177
Conversation
Add a self-consistency section to AGENTS.md so agents that add or modify a numbered rule first read the sibling rules in that same list and reuse established terminology. Review agents must flag terminology drift within a list, not only cross-document keyword checks. Issues #58 and #86 already require sibling-read consistency for docs/ and .agents/skills/. Neither instruction covers AGENTS.md's own rule lists, which is how PR #168's first draft of rule 5 used "governed document" instead of the sibling term "specification document". The review check lives in this new section rather than in specification-document rule 4, whose scope is docs/ files. CLAUDE.md is a symlink to AGENTS.md and inherits the change. Note: pre-commit could not fetch remote hook repositories (HTTP 403). Equivalent hooks were run via python scripts/lint.py --files AGENTS.md and passed. Closes #169
|
Important Review skippedBot user detected. To trigger a single review, invoke the ⚙️ Run configurationConfiguration used: Repository: redhat-et/ProtoBot/.coderabbit.yaml Review profile: CHILL Plan: Advanced Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Comment |
|
🤖 Review · Commit: |
|
/fs-review |
|
🤖 Review · Commit: |
|
Risk Assessment: moderate (2/5) DetailsAGENTS.md is a low-risk, single-file, docs-only, bot-authored change with no protected/security/dependency signals in Tier 1 individually (aside from the protected-path hit itself), but the file shows heavy 30-day churn, frequent fix/revert commits, and multi-author contention in Tier 2, while the linked issue is fresh, unlabeled, well-scoped, and maintainer-authorized in Tier 3, combining to a moderate composite score of 2. |
ReviewFindingsMedium
|
fullsend-ai-review
Bot
added
the
requires-manual-review
|
🤖 Finished Review · ✅ Success · Started 4:52 PM UTC · Completed 5:02 PM UTC Commit: Runtime: pi · Model: sonnet → claude-sonnet-5 · Effort: high · Cost: $1.97 |
|
PR #177 (redhat-et/ProtoBot, merged) added a self-consistency section to AGENTS.md per issue #169. The pipeline itself ran cleanly: triage ($0.24) -> code (xai/grok-4.6, $2.35) -> review (claude-sonnet-5, $1.97) -> one human approval, no rework iterations. A dedicated check confirmed the code agent actually read the sibling rule lists before drafting (visible in its trace) and that the merged text reuses established terminology consistently with the two existing sibling sections -- no terminology drift, the class of bug issue #169 targeted did not recur. The review agent's single Medium finding (protected-path, human-approval-required) was correct and expected, and the human's uncommented approval reflects that there was nothing else to flag, not a rubber stamp. Two things worth flagging. First, this PR's own GitHub Actions history reproduced a known, already-tracked platform bug: the bot-created PR fired both an 'opened' and a 'labeled(ready-for-review)' event ~1 second apart, both routed to stage=review, causing duplicate concurrent review dispatches (runs 35890091175 and 35890093229). This is the exact race tracked as fullsend-ai/fullsend#7384 (open, high priority, a recurrence of #391/#393/#2679/#5782), so no new issue is filed for it. This run adds a new, previously undocumented harm worth noting on that issue: the quickly-cancelled duplicate posted an early '⚠ Cancelled' comment ~2 minutes after PR open, which appears to have led the human maintainer to manually invoke /fs-review ~10 minutes later -- and that manual invocation cancelled the other, still-healthy automatic review run that had already been executing for ~12 minutes and would likely have completed on its own, wasting that compute and delaying the real review by several minutes. Second, and more novel: this issue/PR is itself the third installment of a recurring pattern where a retro agent finds one more surface lacking a 'read sibling entries first, reuse established terminology, an undeclared alias is a defect' instruction and proposes adding a narrow copy of that same instruction for just that surface (#58 for docs/, #86 for .agents/skills/, #169/PR #177 for AGENTS.md's own rule lists). Each instance was well-evidenced and merged without complaint, but AGENTS.md now carries three near-duplicate copies of the same rule shape, and nothing has proposed generalizing it -- so the same cycle (a full retro -> issue -> code -> review -> merge loop, costing several dollars and 30-60 minutes each time) is set up to repeat on the next new list-like surface. A search of fullsend-ai/agents found no existing issue covering this; the closest, agents#1428, addresses a different failure mode (insufficient evidence causing proposal rejection) rather than consolidating already-accepted, structurally-repeated rules. Two proposals below address this at the process layer (agents repo) and the immediate cleanup (ProtoBot's AGENTS.md). Proposals filed |
|
🤖 Finished Retro · ✅ Success · Started 5:12 PM UTC · Completed 5:21 PM UTC Commit: Runtime: claude · Model: sonnet → claude-sonnet-5 · Effort: high · Cost: $1.58 |
1 participant
Summary
Add a self-consistency section to
AGENTS.mdso agents that add or modify a numbered rule first read the sibling rules in that same list and reuse established terminology.Issues #58 and #86 already require sibling-read consistency for
docs/files and.agents/skills/files. Neither instruction coversAGENTS.md's own numbered rule lists. That is how PR #168's first draft of specification-document rule 5 used "governed document" instead of the sibling term "specification document", forcing a fix-agent iteration and a second review pass.The new section has three rules, matching the existing pattern:
The review check lives in this new section rather than in specification-document rule 4, whose scope is
docs/files. Putting AGENTS.md self-review into that rule would mix surfaces.The section is placed before the specification-document hierarchy so it is visible when agents later add rules to either existing list.
CLAUDE.mdis a symlink toAGENTS.mdand inherits the change.AGENTS.mdis listed inREVIEW_PROTECTED_PATHS; extra review scrutiny is expected.Testing
python scripts/lint.py --files AGENTS.md(offline pre-commit equivalent;pre-commititself could not fetch hook repos in this sandbox — HTTP 403)AGENTS.mdand of staged contentCloses #169
Post-script verification
agent/169-rule-list-consistency)2d365e268c2cde40810448b8e8deeb1e6942cf52..HEAD)