Repository navigation
test(e2e): Move canary variants into canaryVariants - #25865
Merged
Merged
Conversation
Canary variants no longer run as optional variants on every PR. This also adds the apps that so far only ran in canary.yml. Co-Authored-By: Francesco Gringl-Novy <francesco.novy@sentry.io>
Contributor
size-limit report 📦Compared: a101064 (PR test merge using develop at fddeafc) Against: fddeafc (develop) · base run #64066 ✅ Baseline matches: this diff contains only this PR's changes.
|
Contributor
Optional E2E testsSome optional checks were not successful. They do not block the merge. 1 not successful, 56 successful. See the run.
Successful checks
|
Only prerelease builds are canary variants. The `latest` variants that ran in canary.yml now run as optional variants on PRs. Co-Authored-By: Francesco Gringl-Novy <francesco.novy@sentry.io>
The optional and canary matrices are mutually exclusive, so one option replaces the two boolean flags. Co-Authored-By: Francesco Gringl-Novy <francesco.novy@sentry.io>
mydea
reviewed
Oct 29, 2026
mydea
marked this pull request as ready for review
October 29, 2026 08:55
mydea
requested review from
andreiborza,
logaretm,
msonnb,
mydea and
nicohrubec
and removed request for
a team
October 29, 2026 08:55
msonnb
approved these changes
Oct 29, 2026
mydea
added a commit
that referenced
this pull request
Oct 29, 2026
**Why:** `canary.yml` had a hard-coded list of apps and build commands that drifted from the variants in the test apps. It now gets its matrix from `ci:build-matrix-canary`, the same way `build.yml` gets the optional E2E matrix, so adding a `canaryVariants` entry is enough to run it nightly. The test jobs now use the shared `prepare-e2e-test` action like the E2E jobs in `build.yml`, so canary variants get `runtime`, `deno-version`, `runner` and default build/assert commands for free. For that, the prepare job uploads the tarballs as artifacts instead of the `actions/cache` it used before. The test jobs use their own cache volume (`sentry-javascript-e2e-canary`), so prerelease installs don't land in the volume the regular E2E jobs use. Like in `build.yml`, they run on the small runner unless the app sets `sentryTest.runner`; slower nightly runs are fine. Stacked on #25865. **How to review:** Only `.github/workflows/canary.yml`. The issue step is unchanged and still opens one issue per failed variant. <!-- junior-request-attribution:start --> via **Francesco Gringl-Novy**. <!-- junior-request-attribution:end --> <!-- junior-session-footer:start --> <!-- junior-conversation-id:slack%3AD0CA8FXFXAS%3A1793261840.120889 --> -- [View Junior Session](https://junior-prod.sentry.dev/conversations/slack%3AD0CA8FXFXAS%3A1793261840.120889) [[Sentry]](https://sentry.sentry.io/explore/conversations/slack%3AD0CA8FXFXAS%3A1793261840.120889/?project=4510944073809921) <!-- junior-session-footer:end --> --------- Co-authored-by: sentry-junior[bot] <264270552+sentry-junior[bot]@users.noreply.github.com> Co-authored-by: Francesco Gringl-Novy <francesco.novy@sentry.io>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why: Canary variants ran in two places: as
optionalVariantson every PR and in the nightlycanary.yml. This adds asentryTest.canaryVariantsfield for variants that build against upstream prereleases (canary,next, nightly) and moves them there, so they stop running on PRs.create-react-app,react-create-hash-routerandreact-router-6only had canary builds incanary.yml, so they getcanaryVariantsentries. Thelatestbuilds that only ran incanary.yml(tanstackstart-react (latest),nestjs-12 (latest)) are not canary builds, so they becomeoptionalVariants, andnextjs-14 (latest)/nextjs-15 (latest)stay optional.getTestMatrix.mjsnow takes--variant=optional|canaryinstead of the boolean--optional=true, because the matrices are mutually exclusive.ci:build-matrix-canaryoutputs the canary variants, andrun.mjs --variantfinds them for local runs.First of 2 stacked PRs (#25866).
canary.ymlkeeps its hard-coded list until #25866, so the canary variants that were only optional before (angular-21,nextjs-16-standalone,nuxt-4-static,vue-3) run nowhere until that one lands. Merge them together.How to review: Start with
getTestMatrix.mjs. Thepackage.jsonchanges only move entries. The angular canary labels are renamed toangular-21 (canary)/angular-22 (canary)so the two are not both calledangular (canary).Unsure about:
nextjs-app-dir (next@15)was incanary.yml, but it is already a required variant on PRs, so I left it invariants. After #25866 it no longer also runs nightly.via Francesco Gringl-Novy.
--
View Junior Session [Sentry]