Repository navigation
ci(canary): Build the canary test matrix from canaryVariants - #25866
Merged
Merged
Conversation
The canary workflow no longer has a hard-coded list of apps. It runs the `canaryVariants` of all test apps and uses the shared prepare-e2e-test action like the E2E jobs in build.yml. Co-Authored-By: Francesco Gringl-Novy <francesco.novy@sentry.io>
This was referenced Oct 29, 2026
Contributor
size-limit report 📦Compared: 6214fd3 (PR test merge using develop at a101064) Against: fddeafc (develop) · base run #64066
|
Co-Authored-By: Francesco Gringl-Novy <francesco.novy@sentry.io>
Contributor
Optional E2E testsSome optional checks were not successful. They do not block the merge. 1 not successful, 56 successful. See the run.
Successful checks
|
Co-Authored-By: Francesco Gringl-Novy <francesco.novy@sentry.io>
Co-Authored-By: Francesco Gringl-Novy <francesco.novy@sentry.io>
mydea
marked this pull request as ready for review
October 29, 2026 08:55
mydea
requested review from
isaacs,
logaretm,
msonnb,
mydea and
nicohrubec
and removed request for
a team
October 29, 2026 08:55
msonnb
approved these changes
Oct 29, 2026
mydea
added a commit
that referenced
this pull request
Oct 29, 2026
**Why:** Canary variants ran in two places: as `optionalVariants` on every PR and in the nightly `canary.yml`. This adds a `sentryTest.canaryVariants` field for variants that build against upstream prereleases (`canary`, `next`, nightly) and moves them there, so they stop running on PRs. `create-react-app`, `react-create-hash-router` and `react-router-6` only had canary builds in `canary.yml`, so they get `canaryVariants` entries. The `latest` builds that only ran in `canary.yml` (`tanstackstart-react (latest)`, `nestjs-12 (latest)`) are not canary builds, so they become `optionalVariants`, and `nextjs-14 (latest)` / `nextjs-15 (latest)` stay optional. `getTestMatrix.mjs` now takes `--variant=optional|canary` instead of the boolean `--optional=true`, because the matrices are mutually exclusive. `ci:build-matrix-canary` outputs the canary variants, and `run.mjs --variant` finds them for local runs. First of 2 stacked PRs (#25866). `canary.yml` keeps its hard-coded list until #25866, so the canary variants that were only optional before (`angular-21`, `nextjs-16-standalone`, `nuxt-4-static`, `vue-3`) run nowhere until that one lands. Merge them together. **How to review:** Start with `getTestMatrix.mjs`. The `package.json` changes only move entries. The angular canary labels are renamed to `angular-21 (canary)` / `angular-22 (canary)` so the two are not both called `angular (canary)`. **Unsure about:** `nextjs-app-dir (next@15)` was in `canary.yml`, but it is already a required variant on PRs, so I left it in `variants`. After #25866 it no longer also runs nightly. <!-- junior-request-attribution:start --> via **Francesco Gringl-Novy**. <!-- junior-request-attribution:end --> <!-- junior-session-footer:start --> <!-- junior-conversation-id:slack%3AD0CA8FXFXAS%3A1793261840.120889 --> -- [View Junior Session](https://junior-prod.sentry.dev/conversations/slack%3AD0CA8FXFXAS%3A1793261840.120889) [[Sentry]](https://sentry.sentry.io/explore/conversations/slack%3AD0CA8FXFXAS%3A1793261840.120889/?project=4510944073809921) <!-- junior-session-footer:end --> --------- Co-authored-by: sentry-junior[bot] <264270552+sentry-junior[bot]@users.noreply.github.com> Co-authored-by: Francesco Gringl-Novy <francesco.novy@sentry.io>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why:
canary.ymlhad a hard-coded list of apps and build commands that drifted from the variants in the test apps. It now gets its matrix fromci:build-matrix-canary, the same waybuild.ymlgets the optional E2E matrix, so adding acanaryVariantsentry is enough to run it nightly.The test jobs now use the shared
prepare-e2e-testaction like the E2E jobs inbuild.yml, so canary variants getruntime,deno-version,runnerand default build/assert commands for free. For that, the prepare job uploads the tarballs as artifacts instead of theactions/cacheit used before. The test jobs use their own cache volume (sentry-javascript-e2e-canary), so prerelease installs don't land in the volume the regular E2E jobs use. Like inbuild.yml, they run on the small runner unless the app setssentryTest.runner; slower nightly runs are fine.Stacked on #25865.
How to review: Only
.github/workflows/canary.yml. The issue step is unchanged and still opens one issue per failed variant.via Francesco Gringl-Novy.
--
View Junior Session [Sentry]