Skip to content

fix(ci): scope the Bugbot gate's fast-fail to findings raised against the head (backend#2816) - #399

Open
LukasWodka wants to merge 1 commit into
developfrom
fix/2816-bugbot-gate-head-scoped-findings
Open

fix(ci): scope the Bugbot gate's fast-fail to findings raised against the head (backend#2816)#399
LukasWodka wants to merge 1 commit into
developfrom
fix/2816-bugbot-gate-head-scoped-findings

Conversation

@LukasWodka

@LukasWodka LukasWodka commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Summary

Fixes the guaranteed-red bugbot / review run on every Bugbot fix cycle (tracebloc/backend#2816).

A fix cycle is push the fix → reply on the thread → resolve it. The push fires the gate before the resolve can land, so the gate saw an open high finding and fast-failed — even though that finding was raised against the previous head and Bugbot was already re-reviewing the new one. Measured 5×: .github#383 runs 33249286694 / 33251888807, e2e-test-agent#319 run 33253828060; all cleared on a plain re-run.

The logic lives in scripts/bugbot-gate.py (the .yml is only the wiring), so the fix is there.

Before → after

The fast-fail discriminator changes from "are there open high findings" to "are there open high findings against this head":

  • finding raised against THIS head → fail fast, exactly as before;
  • finding raised against an OLDER head while Bugbot is IN_PROGRESS on this head → wait for the verdict, exactly as the gate already does with zero findings.

Each finding's raised-against commit is read from its comment's originalCommit.oid. Verified on .github#383: originalCommit stays pinned to the review's commit, whereas commit fast-forwards to the head and isOutdated is unreliable (findings raised against older commits there show isOutdated=false, commit==head) — so originalCommit is the only correct field.

This is a narrowing, never a weakening

  • a finding against the current head still blocks (fast-fail, and the completed-review path is head-agnostic);
  • a never-claimed head still fast-fails on any blocking finding — no laundering a finding across a dropped review (the fix(2284): the gate tolerates a review that never came, and says so #356 order bug stays closed);
  • a stalled Bugbot still blocks at the deadline (both .github#383 stall cases still fail);
  • fail closed: a finding whose commit cannot be read is treated as against-this-head and blocks;
  • a new query-integrity guard (query_lacks_finding_commit) refuses the run if the query stops requesting originalCommit, mirroring the existing totalCount / author.__typename guards, so the head-scope can't silently go inert.

Test evidence (armed while green, mutation-proven)

  • scripts/tests/bugbot-gate-selftest.py: 121 assertions pass (was 109). New section 3b covers: (a) older-head + IN_PROGRESS waits, (b) this-head + IN_PROGRESS fails fast, (c) stalled Bugbot still exits 1 (via main), (d) unknown-commit fails closed, plus the reviewed-path and never-claimed anti-launder cases and the query-integrity guard both directions.
  • scripts/tests/bugbot-gate-mutations.py: 54 mutations, 0 stale, 0 uncaught. 3 existing anchors updated to the new fast-fail line; 7 new mutations added (the two fast-blocker collapses, fail-closed removal, inverted head match, findings dropping the SHA, query dropping originalCommit, guard disarmed) — each reddens the intended section-3b case.

Both run in the already-required selftests context (make selftest-bugbot-gate / make mutation-bugbot-gate); no branch-protection edit needed.

🤖 Generated with Claude Code


Note

Medium Risk
Changes merge-gating verdict timing for open Bugbot findings during IN_PROGRESS reviews; behavior is heavily self-tested and mutation-covered but wrong head matching could let stale findings slip or block valid fix pushes.

Overview
Fixes false FAIL on the Bugbot fix cycle (push fix → gate runs before thread resolve) by scoping the mid-review fast-fail to open findings raised against this head, not leftovers from an older commit.

The gate now loads each finding’s originalCommit.oid, classifies blockers as against-this-head vs older-head, and only fast_blockers trigger immediate FAIL while Bugbot is IN_PROGRESS; older-head-only open highs go to PENDING and wait for the new verdict (still exit 1 if Bugbot stalls). Never-claimed heads still fast-fail on any blocker (no #356 laundering). Missing commit data stays fail-closed (treated as this head). A query_lacks_finding_commit startup guard mirrors existing query-integrity checks so dropping originalCommit cannot silently revert behavior.

Selftest section 3b, updated mutation anchors, and seven new mutations pin the head-scope split, fail-closed, anti-launder, and guard behavior.

Reviewed by Cursor Bugbot for commit 6ea9faa. Bugbot is set up for automated code reviews on this repo. Configure here.

Closes tracebloc/backend#2816

… the head (backend#2816)

The gate fast-failed on any open high finding while Bugbot was IN_PROGRESS,
so every fix cycle (push -> reply -> resolve) cost one guaranteed-red run: the
push fires the gate before the resolve lands, and the gate saw the
about-to-be-resolved finding -- which was raised against the PREVIOUS head.

Narrow the fast-fail to distinguish WHERE a finding was raised, read from each
finding comment's originalCommit.oid (the review's commit; commit fast-forwards
to the head and isOutdated is unreliable, so neither can be used):

  * a finding against THIS head still fails fast, as before;
  * a finding against an OLDER head while Bugbot is IN_PROGRESS on this one now
    WAITS for the verdict, exactly as the gate already does with zero findings.

Narrowing only: a finding against the current head still blocks, a never-claimed
head still blocks any finding (no laundering across a dropped review), a stalled
Bugbot still blocks at the deadline, and a finding whose commit cannot be read
is treated as against-this-head (fail closed). A query-integrity guard refuses
the run if the query stops requesting originalCommit, mirroring the existing
totalCount / author.__typename guards.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@LukasWodka LukasWodka self-assigned this Sep 1, 2026
@LukasWodka
LukasWodka requested a review from saadqbal September 1, 2026 15:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant