docs(#163): add inward-staleness review rule - #168
Conversation
Add AGENTS.md specification-document rule 5 so review agents search all governed docs under docs/ for stale descriptions of a changed contract, lifecycle, or behavior — not only the files the diff touches. Rule 4 only checks that a modified document covers components.md/overview.md (outward coverage). PR #153 left components.md describing a now-prohibited same-state dependency write after other governed docs changed the contract; round 1 missed it. This rule is additive to #140 (hierarchy manifest completeness), not a substitute. Note: pre-commit could not fetch remote hook repositories (HTTP 403). Equivalent hooks were run via python scripts/lint.py --files AGENTS.md and passed. Closes #163
|
Important Review skippedBot user detected. To trigger a single review, invoke the ⚙️ Run configurationConfiguration used: Repository: redhat-et/ProtoBot/.coderabbit.yaml Review profile: CHILL Plan: Advanced Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Comment |
|
Risk Assessment: moderate (2/5) DetailsRe-review of an unchanged single-file, 8-line, bot-authored, docs-only AGENTS.md addition: Tier 1 is unchanged from the prior review, elevated Tier 2 churn/author-diversity/fix-revert history and untouched-doc coupling on AGENTS.md keep the git-history tier moderate-to-elevated, while a clean scope match and revertible nature keep the linked-issue tier low, yielding the same weighted composite of 2 (moderate) as the prior assessment since the intervening fix-agent edit was wording-only. Previous runRisk Assessment: moderate (2/5) DetailsA single-file, 7-line, bot-authored docs change with a clean scope match to its linked issue (#163) is offset by AGENTS.mds history of frequent recent edits, fix/revert commits, and tight coupling to several untouched governed spec files, yielding a moderate composite of 2. |
ReviewFindingsMedium
Previous runReviewFindingsMedium
Low
Next steps:
|
|
🤖 Finished Review · ✅ Success · Started 1:54 PM UTC · Completed 2:05 PM UTC Commit: Runtime: pi · Model: sonnet → claude-sonnet-5 · Effort: high · Cost: $1.81 |
Update rule 5 in AGENTS.md to refer to "a specification document" and "other governed Markdown documents under docs/" instead of "governed document", consistent with sibling rules 1, 2, and 4. Disagreed finding: AGENTS.md protected path is the explicit scope of PR #168 (#163); human maintainer approval is required before merge. Addresses #168
🔧 Fix agent — iteration 1 (bot-triggered)Addressed the naming-convention finding on AGENTS.md:102 by aligning Rule 5 terminology with sibling rules in the specification documents section. Disagreed with the protected-path finding as modifying AGENTS.md is the express objective of PR #168, and human approval is a merge-time governance gate rather than an automated code defect. Fixed (1):
Disagreed (1):
Tests: passed Decision points
Next steps:
|
|
🤖 Finished Fix · ✅ Success · Started 2:07 PM UTC · Completed 2:15 PM UTC Commit: Runtime: pi · Model: google-vertex/gemini-3.8-flash → gemini-3.8-flash · Effort: high · Cost: $0.44 |
Superseded by updated review
|
🤖 Finished Review · ✅ Success · Started 2:16 PM UTC · Completed 2:30 PM UTC Commit: Runtime: pi · Model: sonnet → claude-sonnet-5 · Effort: high · Cost: $2.49 |
|
PR #168 ( One new, unfiled gap surfaced: AGENTS.md already has explicit "read siblings first, reuse terminology" instructions for editing Two supporting data points for existing open issues, not filed as new proposals: (1) round 2's full 5-dimension + risk-assessment + challenger review pipeline ($2.49) to re-verify a single-line wording fix is another concrete instance of the pattern fullsend-ai/agents#343 ("scope re-review to finding verification when push only addresses prior findings") already targets — worth citing as additional evidence there. (2) In that same round-2 pass, the adversarial "challenger" sub-agent caught and corrected a real misattribution error (the intent-coherence sub-agent wrongly attributed an orchestrator-context note to the PR body) in what was otherwise a "clean" re-review — this is a data point cautioning against fullsend-ai/agents#841's proposal to skip the challenger pass specifically for clean re-reviews, since that's the exact condition under which it just caught a real error here. Also, review round 2 reconfirmed the still-open #140 (stale REVIEW_SPEC_HIERARCHY manifest); note current AGENTS.md's hierarchy also includes Proposals filed |
|
🤖 Finished Retro · ✅ Success · Started 4:54 PM UTC · Completed 5:02 PM UTC Commit: Runtime: claude · Model: sonnet → claude-sonnet-5 · Effort: high · Cost: $1.28 |
Add a self-consistency section to AGENTS.md so agents that add or modify a numbered rule first read the sibling rules in that same list and reuse established terminology. Review agents must flag terminology drift within a list, not only cross-document keyword checks. Issues redhat-et#58 and redhat-et#86 already require sibling-read consistency for docs/ and .agents/skills/. Neither instruction covers AGENTS.md's own rule lists, which is how PR redhat-et#168's first draft of rule 5 used "governed document" instead of the sibling term "specification document". The review check lives in this new section rather than in specification-document rule 4, whose scope is docs/ files. CLAUDE.md is a symlink to AGENTS.md and inherits the change. Note: pre-commit could not fetch remote hook repositories (HTTP 403). Equivalent hooks were run via python scripts/lint.py --files AGENTS.md and passed. Closes redhat-et#169
Summary
Add AGENTS.md specification-document rule 5 so review agents search all governed documents under
docs/for stale descriptions of a changed contract, lifecycle, or behavior — including documents the diff never touches.Rule 4 only requires an outward coverage check (does a modified document account for
components.md/overview.md). That missed the PR #153 case:validation-rules.mdanddrafting-table-wms.mdchanged the blocked-resolution contract whilecomponents.md's Job Site Escalations section still described the now-prohibited same-state dependency write.This is additive to #140 (hierarchy-manifest completeness). A complete manifest does not by itself require the reverse-direction staleness search.
CLAUDE.mdis a symlink toAGENTS.mdand inherits the change.AGENTS.mdis listed inREVIEW_PROTECTED_PATHS; extra review scrutiny is expected.Testing
python scripts/lint.py --files AGENTS.md(offline pre-commit equivalent;pre-commititself could not fetch hook repos in this sandbox)AGENTS.mdand of staged contentCloses #163
Post-script verification
agent/163-inward-staleness-check)747f135ffb8956d3b958ab2db97b70a984865104..HEAD)