Skip to content

docs(#163): add inward-staleness review rule - #168

Merged
JohnStrunk merged 2 commits into
mainfrom
agent/163-inward-staleness-check
Sep 22, 2026
Merged

JohnStrunk merged 2 commits into
mainfrom
agent/163-inward-staleness-check

Conversation

@fullsend-ai-coder

Copy link
Copy Markdown
Contributor

Summary

Add AGENTS.md specification-document rule 5 so review agents search all governed documents under docs/ for stale descriptions of a changed contract, lifecycle, or behavior — including documents the diff never touches.

Rule 4 only requires an outward coverage check (does a modified document account for components.md / overview.md). That missed the PR #153 case: validation-rules.md and drafting-table-wms.md changed the blocked-resolution contract while components.md's Job Site Escalations section still described the now-prohibited same-state dependency write.

This is additive to #140 (hierarchy-manifest completeness). A complete manifest does not by itself require the reverse-direction staleness search.

CLAUDE.md is a symlink to AGENTS.md and inherits the change.

AGENTS.md is listed in REVIEW_PROTECTED_PATHS; extra review scrutiny is expected.

Testing

  • python scripts/lint.py --files AGENTS.md (offline pre-commit equivalent; pre-commit itself could not fetch hook repos in this sandbox)
  • Direct secret scan of AGENTS.md and of staged content

Closes #163

Post-script verification

  • Branch is not main/master (agent/163-inward-staleness-check)
  • Secret scan passed (gitleaks — 747f135ffb8956d3b958ab2db97b70a984865104..HEAD)
  • PR body secret scan passed (gitleaks — no-git)

Add AGENTS.md specification-document rule 5 so review agents
search all governed docs under docs/ for stale descriptions of a
changed contract, lifecycle, or behavior — not only the files
the diff touches.

Rule 4 only checks that a modified document covers
components.md/overview.md (outward coverage). PR #153 left
components.md describing a now-prohibited same-state dependency
write after other governed docs changed the contract; round 1
missed it. This rule is additive to #140 (hierarchy manifest
completeness), not a substitute.

Note: pre-commit could not fetch remote hook repositories
(HTTP 403). Equivalent hooks were run via python scripts/lint.py
--files AGENTS.md and passed.

Closes #163
@fullsend-ai-coder
fullsend-ai-coder Bot requested a review from a team September 22, 2026 13:53
@fullsend-ai-coder fullsend-ai-coder Bot added the ready-for-review Triggers review agent dispatch label Sep 22, 2026
@coderabbitai

coderabbitai Bot commented Sep 22, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Bot user detected.

To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository: redhat-et/ProtoBot/.coderabbit.yaml

Review profile: CHILL

Plan: Advanced

Run ID: c9832f99-ef2b-423c-bedc-6304f18d2033

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Comment @coderabbitai help to get the list of available commands.

@fullsend-ai-review fullsend-ai-review Bot added the risk/moderate PR risk: moderate label Sep 22, 2026
@fullsend-ai-review

fullsend-ai-review Bot commented Sep 22, 2026 •

Copy link
Copy Markdown

Risk Assessment: moderate (2/5)

Details

Re-review of an unchanged single-file, 8-line, bot-authored, docs-only AGENTS.md addition: Tier 1 is unchanged from the prior review, elevated Tier 2 churn/author-diversity/fix-revert history and untouched-doc coupling on AGENTS.md keep the git-history tier moderate-to-elevated, while a clean scope match and revertible nature keep the linked-issue tier low, yielding the same weighted composite of 2 (moderate) as the prior assessment since the intervening fix-agent edit was wording-only.

Previous run

Risk Assessment: moderate (2/5)

Details

A single-file, 7-line, bot-authored docs change with a clean scope match to its linked issue (#163) is offset by AGENTS.mds history of frequent recent edits, fix/revert commits, and tight coupling to several untouched governed spec files, yielding a moderate composite of 2.

@fullsend-ai-review

fullsend-ai-review Bot commented Sep 22, 2026 •

Copy link
Copy Markdown

Review

Findings

Medium

Previous run

Review

Findings

Medium

Low

  • [naming-convention] AGENTS.md:102 — New rule 5 uses the phrase "governed document" ("in any governed document", "other governed documents under docs/"), which does not match the sibling terminology already established in the file: the section uses "specification document" (heading, rules 1, 2, 4) and "governed Markdown document under docs/" (rule 1) / "governed hierarchy member" (line 19). Rule 4 itself states that "an undeclared alias for a defined keyword is a defect."
    Remediation: Replace "in any governed document" with "in a specification document" and replace "other governed documents under docs/" with "other governed Markdown documents under docs/" (or "other specification documents under docs/") for consistency with sibling rules.

Next steps:

  • /fs-fix — agent addresses review findings automatically
  • /fs-fix <your instruction> — agent fixes with your specific guidance
  • Push commits directly — review re-runs automatically on push
  • /fs-fix-stop — disable automatic fix runs for this PR

fullsend-ai-review[bot]

This comment was marked as outdated.

@fullsend-ai-review

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 1:54 PM UTC · Completed 2:05 PM UTC

Commit: 3d23fcb · View workflow run →

Runtime: pi · Model: sonnet → claude-sonnet-5 · Effort: high · Cost: $1.81

Update rule 5 in AGENTS.md to refer to "a specification document"
and "other governed Markdown documents under docs/" instead of
"governed document", consistent with sibling rules 1, 2, and 4.

Disagreed finding: AGENTS.md protected path is the explicit scope
of PR #168 (#163); human maintainer approval is required before merge.

Addresses #168
@fullsend-ai-coder

Copy link
Copy Markdown
Contributor Author

🔧 Fix agent — iteration 1 (bot-triggered)

Addressed the naming-convention finding on AGENTS.md:102 by aligning Rule 5 terminology with sibling rules in the specification documents section. Disagreed with the protected-path finding as modifying AGENTS.md is the express objective of PR #168, and human approval is a merge-time governance gate rather than an automated code defect.

Fixed (1):

  1. [naming-convention] AGENTS.md:102 - Use established specification document terminology (AGENTS.md): Replaced 'in any governed document' with 'in a specification document' and 'other governed documents under docs/' with 'other governed Markdown documents under docs/' in rule 5, aligning with sibling rules 1, 2, and 4.

Disagreed (1):

  1. [protected-path] AGENTS.md requires human approval: Modifying AGENTS.md is the intentional purpose of PR docs(#163): add inward-staleness review rule #168 (resolving issue AGENTS.md's cross-document review rule only checks outward coverage, not inward staleness in untouched sibling docs #163). The requirement for human approval is a repository governance policy that cannot be satisfied via automated code changes; human maintainer approval must be obtained before merge.

Tests: passed

Decision points
  • Adopt 'governed Markdown documents under docs/' for Rule 5 staleness search target (alternatives: Use 'other specification documents under docs/', Use 'other governed Markdown documents under docs/'; rationale: Matches Rule 1's precise phrasing ('read every other governed Markdown document under docs/') while distinguishing the source specification document.)

Next steps:

  • /fs-review — request a re-review of the changes
  • /fs-fix <your instruction> — run another fix pass with specific guidance
  • Push commits directly — review re-runs automatically on push
    Updated by fullsend fix agent

@fullsend-ai-coder

Copy link
Copy Markdown
Contributor Author

🤖 Finished Fix · ✅ Success · Started 2:07 PM UTC · Completed 2:15 PM UTC

Commit: 3d23fcb · View workflow run →

Runtime: pi · Model: google-vertex/gemini-3.8-flash → gemini-3.8-flash · Effort: high · Cost: $0.44

@fullsend-ai-review
fullsend-ai-review Bot dismissed their stale review September 22, 2026 14:30

Superseded by updated review

@fullsend-ai-review fullsend-ai-review Bot added the requires-manual-review Review requires human judgment label Sep 22, 2026
@fullsend-ai-review

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 2:16 PM UTC · Completed 2:30 PM UTC

Commit: 454e241 · View workflow run →

Runtime: pi · Model: sonnet → claude-sonnet-5 · Effort: high · Cost: $2.49

@JohnStrunk
JohnStrunk added this pull request to the merge queue Sep 22, 2026
Merged via the queue into main with commit 5884a7d Sep 22, 2026
34 checks passed
@JohnStrunk
JohnStrunk deleted the agent/163-inward-staleness-check branch September 22, 2026 16:53
@fullsend-ai-retro

Copy link
Copy Markdown

PR #168 (docs(#163): add inward-staleness review rule, merged) was a clean, single-iteration run of the fullsend pipeline: triage → code → review round 1 → fix → review round 2 → human approval/merge. The code agent added a new Rule 5 to AGENTS.md; round-1 review correctly flagged (a) the expected protected-path gate on AGENTS.md and (b) a real naming-convention defect (the new rule used "governed document" instead of the established "specification document" / "governed Markdown document under docs/" terminology used by sibling rules 1/2/4). The fix agent corrected the wording and explicitly — and correctly — declined to "fix" the protected-path finding, reasoning it was a merge-time governance gate rather than a code defect, which matched the eventual human-approval outcome. No CI failures, no wasted iterations beyond the one legitimate rework round.

One new, unfiled gap surfaced: AGENTS.md already has explicit "read siblings first, reuse terminology" instructions for editing docs/ specification documents (added via issue #58) and for editing .agents/skills/ files (added via issue #86), but has no analogous instruction for agents adding/editing rules within AGENTS.md's own numbered rule lists. That gap is what let the terminology drift into Rule 5 in the first place, costing one avoidable fix+re-review cycle. I'm filing this as a new proposal below (confirmed non-duplicate via search of open/recently-closed issues in redhat-et/ProtoBot).

Two supporting data points for existing open issues, not filed as new proposals: (1) round 2's full 5-dimension + risk-assessment + challenger review pipeline ($2.49) to re-verify a single-line wording fix is another concrete instance of the pattern fullsend-ai/agents#343 ("scope re-review to finding verification when push only addresses prior findings") already targets — worth citing as additional evidence there. (2) In that same round-2 pass, the adversarial "challenger" sub-agent caught and corrected a real misattribution error (the intent-coherence sub-agent wrongly attributed an orchestrator-context note to the PR body) in what was otherwise a "clean" re-review — this is a data point cautioning against fullsend-ai/agents#841's proposal to skip the challenger pass specifically for clean re-reviews, since that's the exact condition under which it just caught a real error here. Also, review round 2 reconfirmed the still-open #140 (stale REVIEW_SPEC_HIERARCHY manifest); note current AGENTS.md's hierarchy also includes drafting-table-wms.md, which #140's current issue text does not mention — worth a scope-check on that issue but not a new filing.

Proposals filed

@fullsend-ai-retro

Copy link
Copy Markdown

🤖 Finished Retro · ✅ Success · Started 4:54 PM UTC · Completed 5:02 PM UTC

Commit: 454e241 · View workflow run →

Runtime: claude · Model: sonnet → claude-sonnet-5 · Effort: high · Cost: $1.28

jerry-ng2 pushed a commit to jerry-ng2/ProtoBot that referenced this pull request Sep 24, 2026
Add a self-consistency section to AGENTS.md so agents that add or
modify a numbered rule first read the sibling rules in that same
list and reuse established terminology. Review agents must flag
terminology drift within a list, not only cross-document keyword
checks.

Issues redhat-et#58 and redhat-et#86 already require sibling-read consistency for
docs/ and .agents/skills/. Neither instruction covers AGENTS.md's
own rule lists, which is how PR redhat-et#168's first draft of rule 5 used
"governed document" instead of the sibling term "specification
document".

The review check lives in this new section rather than in
specification-document rule 4, whose scope is docs/ files.

CLAUDE.md is a symlink to AGENTS.md and inherits the change.

Note: pre-commit could not fetch remote hook repositories
(HTTP 403). Equivalent hooks were run via python
scripts/lint.py --files AGENTS.md and passed.

Closes redhat-et#169
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-review Triggers review agent dispatch requires-manual-review Review requires human judgment risk/moderate PR risk: moderate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

AGENTS.md's cross-document review rule only checks outward coverage, not inward staleness in untouched sibling docs

1 participant