Repository navigation
test(chain): run the live triage chain from webhook to verdict - #642
Merged
TheGreatAxios merged 1 commit intoOct 26, 2026
Conversation
TheGreatAxios
force-pushed
the
cl-11528-add-an-end-to-end-test-of-the-live-triage-chain-from-webhook
branch
from
October 26, 2026 20:00
1d06dbc to
41b0093
Compare
TheGreatAxios
marked this pull request as ready for review
October 26, 2026 20:00
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes CL-11528
Why
Nothing tested the live triage chain as it really runs: a webhook reaches the hub, the listener deploys a manager, and the manager deploys the judge and posts. The unit tests stub every hop, which is how a manager that crashed on any judge or code review reached main unnoticed (fixed in CL-11532).
What changed
tests/chain/suite, run withbun run test:chainand as its own CI step. It starts the real hub on a fresh Postgres database, with each sidecar as a local process. It then sets the workspace up through the hub's routes and publishes and upgrades the workflows with the portal's own Upgrade code. Finally it sends signed GitHub webhooks.Unverified hypothesis
After the judge case, the pull request's claim was once left
live, naming a manager run that had already stopped. The run logs suggest the judge's wake-up mail started a second body after done had ended the claim; that body took the claim again, and the run's own stop then aborted it before its done. The design treats this as harmless, since the next take finds the run gone. The test therefore checks that the manager's run is gone, not that the claim readsended. I saw this once and have not reproduced it.Verified
bun run test:chain: 7 of 7 green in 2 runs in a row (172 and 262 seconds), on main with fix(manager): post the judge's verdict as the judge reported it #635, fix(manager): record the manager's verdict when it reports done #637, fix(manager): read the mail step's reply when waiting for the judge or code review #638 and fix(manager): mark a new pull request waiting while its first CI run is in progress #640.bun run check,bun run lint(no new warnings) and the policy check (7 of 7) all pass.