Skip to content

docs(agents): one Copilot review per PR, not one per push - #11216

Merged
MarkusNeusinger merged 4 commits into
mainfrom
docs/copilot-review-cadence
Sep 3, 2026
Merged

docs(agents): one Copilot review per PR, not one per push#11216
MarkusNeusinger merged 4 commits into
mainfrom
docs/copilot-review-cadence

Conversation

@MarkusNeusinger

Copy link
Copy Markdown
Owner

Markus turned review-on-push off in the "Automated Copilot Code Review" ruleset of both repositories today (2026-09-03): the bot now runs once when a PR opens or leaves draft, and a push triggers nothing. The instructions still read as though a review followed every push, so an agent would keep asking for one.

CLAUDE.md § "PR Follow-Through", step 3 now states the cadence and the reason not to force it:

Each request is a full re-read of the whole diff, and the bot then surfaces "previously missed" findings in files the push never touched — which draws another push, which draws another request (the sibling repo's PR #406 collected ~15 requests in a day over a one-line docstring fix).

A fresh review is requested explicitly only after a substantive rework (new behaviour, a reworked mechanism); a round that yields no new inline comments but only carried-over items is the end of it; a PR that is green with no open threads needs none at all.

agentic/commands/pull_request.md gains the same note as a new step 8 — placed there deliberately, because its step 7 (append the PR number to the changelog bullets and push) is exactly the push that used to look like a reason to re-request.

Not mirrored into .github/copilot-instructions.md: that file has no PR-follow-through block, and this is agent workflow rather than one of the rules the two guides share. If it should live there too, say so and it is one more bullet.

Carried over from kurrentschrift's CLAUDE.md, where the same guardrail has stood since 2026-08-23 (sibling-transfer rule).

Verification

pytest tests/unit/test_agent_instructions.py tests/unit/agentic — 189 passed, so the backticked paths still resolve and the mirrored-rule pins still hold.

🤖 Generated with Claude Code

https://claude.ai/code/session_01UEScQMZFvxxNNyNJYryfa3

Review-on-push was turned off in the "Automated Copilot Code Review" ruleset of
both repositories on 2026-09-03 (owner), so the bot runs once when a PR opens or
leaves draft and a push triggers nothing.

CLAUDE.md's PR follow-through step 3 now says that, and says not to re-request
one per push: each request is a full re-read of the whole diff, and the bot then
surfaces "previously missed" findings in files the push never touched — which
draws another push, and another request. A fresh review is requested only after
a substantive rework, and a PR that is green with no open threads needs none.

agentic/commands/pull_request.md gains the same note at its own step 7, whose
follow-up commit is exactly the push that used to look like a reason to
re-request. The rule is not mirrored into .github/copilot-instructions.md: that
file has no PR-follow-through block, and this is agent workflow rather than one
of the rules the two guides share.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UEScQMZFvxxNNyNJYryfa3
Copilot AI balanced review requested due to automatic review settings September 3, 2026 21:37
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UEScQMZFvxxNNyNJYryfa3

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The review-cadence wording and sibling PR references need correction.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Updates agent guidance to reflect Copilot’s one-automatic-review-per-PR cadence.

Changes:

  • Documents the substantive-rework exception.
  • Updates the pull-request workflow.
  • Records the change in the changelog.
File summaries
File Review
CLAUDE.md Nit (2 votes): Replace bare #406 with a repository-qualified link.
CHANGELOG.md Nit (1 vote): Clarify the automatic-versus-manual review distinction and link the sibling PR explicitly.
Nit (1 vote): Replace bare #406 with a qualified reference.
agentic/commands/pull_request.md Nit (1 vote): Clarify that only the automatic review runs once; substantive rework may warrant another request.
Review details

Suppressed comments (3)

CHANGELOG.md:232

  • The bare #406 links to this repository's unrelated issue rather than the sibling pull request being described. Qualify the reference so the changelog does not send readers to the wrong record.
  another push, and another request (the sibling repo's PR #406 collected ~15 in a day over a
  one-line docstring fix). A fresh review is requested only after a substantive rework, and a PR

CHANGELOG.md:233

  • This release note repeats the exact-one claim despite the documented substantive-rework exception, and bare #406 links readers to anyplot's unrelated issue rather than the sibling PR used as evidence. Preserve the automatic-versus-manual distinction and use an explicit cross-repository link.
  so, and say not to re-request one per push: each request is a full re-read of the whole diff, and
  the bot then surfaces "previously missed" findings in files the push never touched — which draws
  another push, and another request (the sibling repo's PR #406 collected ~15 in a day over a
  one-line docstring fix). A fresh review is requested only after a substantive rework, and a PR
  that is green with no open threads needs no further round. (#11216)

agentic/commands/pull_request.md:61

  • “Runs once” conflicts with the exception two lines later and can make an agent skip a warranted post-rework review. Clarify that the automatic review runs once, while an additional review is requested only for substantive rework.
8. **The Copilot review runs ONCE**, when the PR opens (or leaves draft) — review-on-push is off in
   the "Automated Copilot Code Review" ruleset (owner, 2026-09-03), so step 7's follow-up commit and
   every later push trigger nothing. Do not re-request a review per push; the rule and its one
   exception (a substantive rework) are in `CLAUDE.md` § "PR Follow-Through", step 3.
  • Files reviewed: 3/3 changed files
  • Comments generated: 1
  • Review effort level: Balanced

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread CLAUDE.md Outdated
MarkusNeusinger and others added 2 commits September 3, 2026 23:51
Copilot: a bare #406 resolves to this repository's own issue 406, not the
sibling PR the evidence lives in. Both mentions are now full links to
MarkusNeusinger/kurrentschrift#406.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UEScQMZFvxxNNyNJYryfa3
@MarkusNeusinger
MarkusNeusinger merged commit 5a25f7d into main Sep 3, 2026
9 checks passed
@MarkusNeusinger
MarkusNeusinger deleted the docs/copilot-review-cadence branch September 3, 2026 22:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants