bug: Maintainer Approval check fails and emails on every review comment #4559

User Story

I maintain OpenShell and review pull requests daily. Since #3176 landed, contributors and reviewers get a "Run failed: Maintainer Approval" email every time they comment on a PR that has no maintainer approval yet, so the gate meant to be silent has become one of our noisiest notifications.

Problem Statement

.github/workflows/maintainer-approval.yml runs on every pull_request_review submitted and dismissed event. Most of those events cannot change the outcome:

• Comment reviews. Every comment review, including each reply in a review thread, is a submitted review with state commented. check_maintainer_approval.py ignores COMMENTED, so the result never changes, but the job re-runs, fails again, and GitHub emails whoever posted the comment.
• Pushes. dismiss_stale_reviews_on_push dismisses approvals on push, which fires dismissed. The new head commit has no approvals, so the outcome is already known to be a failure.

Since the check is the job's own exit code, every one of these runs fails, and GitHub emails a failure notice to the actor who triggered it. Kris Hicks reported getting these emails while replying to review threads on their own PR (#4500). Their two failures at 17:44 and 17:45 UTC on 2026-10-14 were both their own COMMENTED reviews.

On 2026-10-14 (UTC) the workflow failed 35 times. For example, rhuss got 8 failures in 25 minutes commenting on one RFC PR, and alangou got 6 on their own PR.

Why the obvious fix is unsafe

Skipping comment reviews (or push dismissals) with a job-level if: would make the noise stop, but the job's name is the required status context. A skipped job reports skipped, and GitHub treats skipped as passing for a required check. #4496 relies on exactly that to clear the merge queue. Once the context is required:

• A comment on an unapproved PR would replace the earlier failure on that commit with skipped, which passes the gate.
• A skipped run after a push would put skipped on the new head, which passes the gate with zero reviews.

The fix therefore cannot be a skip as long as the job's own result is the gate.

Impact / Why This Matters

• Reviewers and authors get a failure email for every review comment until a maintainer approves. This trains people to ignore Actions failure emails, including real ones.
• feat(ci): gate merges on maintainer approval via a required status check #3176 chose a required status check specifically because it gates merges without notifying anyone. The current behavior undoes that.
• The context is not in ruleset 13332227 yet, so these failures block nothing today. They are pure noise until rollout step 4 of feat(ci): gate merges on maintainer approval via a required status check #3176 happens.

There is no per-workflow way to silence Actions failure emails. The only workaround is for each person to turn off Actions email notifications entirely.

Acceptance Criteria

• Submitting a comment-only review, or replying in a review thread, does not re-evaluate the gate and does not send anyone a failure email.
• Approval, request-changes, and manual dismissal still re-evaluate the gate.
• A push leaves the new head commit blocked until a maintainer approves again, without sending a failure email.
• No event (comment, skip, merge queue, fork PR) can turn a blocked PR green without a current maintainer approval.
• The gate works on fork PRs, where review-triggered runs get a read-only GITHUB_TOKEN.
• The merge queue is still satisfied, as it is today after fix(ci): skip maintainer approval in merge queue #4496.
• The required status context name to add to ruleset 13332227 is documented in CONTRIBUTING.md.

Reproduction Steps

1. Open a PR against NVIDIA/OpenShell with no maintainer approval.
2. As anyone, submit a comment review or reply to an existing review thread.
3. The Maintainer Approval workflow runs, OpenShell / Maintainer Approval fails, and the commenter receives a "Run failed" email.
4. Repeat step 2. Each comment produces another failed run and another email.

Environment

• OpenShell: main at 7225bae (maintainer-approval.yml as of fix(ci): skip maintainer approval in merge queue #4496)
• Runtime, deployment, or integration: GitHub Actions, pull_request_review and merge_group triggers

Suggested Approach

One approach that would meet the criteria is to stop using the job's own result as the gate:

• The review-triggered workflow would skip commented reviews and never fail, under a job name that is not the required context.
• A workflow_run workflow, which always runs main's definition with a write-capable token (fork PRs included), would evaluate approval and write an OpenShell / Maintainer Approval commit status on the PR head.
• A push would leave the new head without a status, so the required context stays pending.
• Merge-queue runs would write a passing status on the merge group commit.

Commit status failures do not send email. As a side effect, the decision would always run main's copy of the workflow, which closes the workflow-tampering path that #3176 currently covers with CODEOWNERS.

Logs

Example runs, all pull_request_review events on 2026-10-14:

• 17:44:12Z and 17:45:24Z, actor krishicks, feat(server): export gateway OCSF events as OTLP log records #4500: COMMENTED replies, failure
• 08:32:07Z through 08:57:56Z, actor rhuss, docs/4425-repository-migration-rfc/drew: 8 failures