fix(ci): scope the Bugbot gate's fast-fail to findings raised against the head (backend#2816) - #399
Open
LukasWodka wants to merge 1 commit into
Open
fix(ci): scope the Bugbot gate's fast-fail to findings raised against the head (backend#2816)#399LukasWodka wants to merge 1 commit into
LukasWodka wants to merge 1 commit into
Conversation
… the head (backend#2816)
The gate fast-failed on any open high finding while Bugbot was IN_PROGRESS,
so every fix cycle (push -> reply -> resolve) cost one guaranteed-red run: the
push fires the gate before the resolve lands, and the gate saw the
about-to-be-resolved finding -- which was raised against the PREVIOUS head.
Narrow the fast-fail to distinguish WHERE a finding was raised, read from each
finding comment's originalCommit.oid (the review's commit; commit fast-forwards
to the head and isOutdated is unreliable, so neither can be used):
* a finding against THIS head still fails fast, as before;
* a finding against an OLDER head while Bugbot is IN_PROGRESS on this one now
WAITS for the verdict, exactly as the gate already does with zero findings.
Narrowing only: a finding against the current head still blocks, a never-claimed
head still blocks any finding (no laundering across a dropped review), a stalled
Bugbot still blocks at the deadline, and a finding whose commit cannot be read
is treated as against-this-head (fail closed). A query-integrity guard refuses
the run if the query stops requesting originalCommit, mirroring the existing
totalCount / author.__typename guards.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes the guaranteed-red
bugbot / reviewrun on every Bugbot fix cycle (tracebloc/backend#2816).A fix cycle is push the fix → reply on the thread → resolve it. The push fires the gate before the resolve can land, so the gate saw an open high finding and fast-failed — even though that finding was raised against the previous head and Bugbot was already re-reviewing the new one. Measured 5×:
.github#383runs 33249286694 / 33251888807,e2e-test-agent#319run 33253828060; all cleared on a plain re-run.The logic lives in
scripts/bugbot-gate.py(the.ymlis only the wiring), so the fix is there.Before → after
The fast-fail discriminator changes from "are there open high findings" to "are there open high findings against this head":
Each finding's raised-against commit is read from its comment's
originalCommit.oid. Verified on.github#383:originalCommitstays pinned to the review's commit, whereascommitfast-forwards to the head andisOutdatedis unreliable (findings raised against older commits there showisOutdated=false,commit==head) — sooriginalCommitis the only correct field.This is a narrowing, never a weakening
.github#383stall cases still fail);query_lacks_finding_commit) refuses the run if the query stops requestingoriginalCommit, mirroring the existingtotalCount/author.__typenameguards, so the head-scope can't silently go inert.Test evidence (armed while green, mutation-proven)
scripts/tests/bugbot-gate-selftest.py: 121 assertions pass (was 109). New section 3b covers: (a) older-head + IN_PROGRESS waits, (b) this-head + IN_PROGRESS fails fast, (c) stalled Bugbot still exits 1 (viamain), (d) unknown-commit fails closed, plus the reviewed-path and never-claimed anti-launder cases and the query-integrity guard both directions.scripts/tests/bugbot-gate-mutations.py: 54 mutations, 0 stale, 0 uncaught. 3 existing anchors updated to the new fast-fail line; 7 new mutations added (the two fast-blocker collapses, fail-closed removal, inverted head match,findingsdropping the SHA, query droppingoriginalCommit, guard disarmed) — each reddens the intended section-3b case.Both run in the already-required
selftestscontext (make selftest-bugbot-gate/make mutation-bugbot-gate); no branch-protection edit needed.🤖 Generated with Claude Code
Note
Medium Risk
Changes merge-gating verdict timing for open Bugbot findings during IN_PROGRESS reviews; behavior is heavily self-tested and mutation-covered but wrong head matching could let stale findings slip or block valid fix pushes.
Overview
Fixes false FAIL on the Bugbot fix cycle (push fix → gate runs before thread resolve) by scoping the mid-review fast-fail to open findings raised against this head, not leftovers from an older commit.
The gate now loads each finding’s
originalCommit.oid, classifies blockers as against-this-head vs older-head, and onlyfast_blockerstrigger immediate FAIL while Bugbot is IN_PROGRESS; older-head-only open highs go to PENDING and wait for the new verdict (still exit 1 if Bugbot stalls). Never-claimed heads still fast-fail on any blocker (no #356 laundering). Missing commit data stays fail-closed (treated as this head). Aquery_lacks_finding_commitstartup guard mirrors existing query-integrity checks so droppingoriginalCommitcannot silently revert behavior.Selftest section 3b, updated mutation anchors, and seven new mutations pin the head-scope split, fail-closed, anti-launder, and guard behavior.
Reviewed by Cursor Bugbot for commit 6ea9faa. Bugbot is set up for automated code reviews on this repo. Configure here.
Closes tracebloc/backend#2816