Repository navigation
ci(canary): Open one issue per failed canary test application - #25867
Closed
sentry-junior[bot] wants to merge 5 commits into
Closed
sentry-junior[bot] wants to merge 5 commits into
sentry-junior[bot] wants to merge 5 commits into
Conversation
A report job opens an issue for each test application with failed canary variants. If an issue for the application is still open, it comments on it with the link to the failed run instead. Co-Authored-By: Francesco Gringl-Novy <francesco.novy@sentry.io>
This was referenced Oct 29, 2026
Contributor
size-limit report 📦Compared: 330aa6b (PR test merge using develop at 6214fd3) Against: fddeafc (develop) · base run #64066
|
…ssues Co-Authored-By: Francesco Gringl-Novy <francesco.novy@sentry.io>
Contributor
Optional E2E testsSome optional checks were not successful. They do not block the merge. 1 not successful, 56 successful. See the run.
Successful checks
|
…ssues Co-Authored-By: Francesco Gringl-Novy <francesco.novy@sentry.io>
…ssues Co-Authored-By: Francesco Gringl-Novy <francesco.novy@sentry.io>
Contributor
Author
|
Closing: per-variant issues from the existing |
An error occurred while trying to automatically change base from
feat/canary-workflow-matrix
to
develop
October 29, 2026 09:34
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why: Each failed canary job ran
create-an-issuewithupdate_existing, which keyed issues by variant label and overwrote the issue body, so a repeat failure left no trace of when it failed again. Now one report job runs after all canary jobs, groups the failed jobs by test application, and per app either opens an issue (Type: Tests,Waiting for: Product Owner) or, if an issue for that app is still open, comments on it with the run link and the failed variant jobs. One job instead of one step per matrix job also avoids two failing variants of the same app racing to create duplicate issues.The logic lives in
scripts/report-canary-failures.mjs(run withactions/github-script, likeweekly-flaky-tests.mjs) so it has unit tests.issues: writemoved from the workflow to this job only.Stacked on #25866.
How to review:
scripts/report-canary-failures.mjs, then the new job incanary.yml.Unsure about:
E2E <label> Test) from the matrix, so the two have to stay in sync. Artifacts per failed job would avoid that, but need more moving parts.Canary tests failed: <app>. Open issues from the old per-variant titles won't be picked up, so the first failing run after this opens new ones.scheduleruns, not manual dispatches.via Francesco Gringl-Novy.
--
View Junior Session [Sentry]