Skip to content

Batch, merge and shard the package integration tests to cut PR time - #622

Merged
stanoddly merged 2 commits into
mainfrom
faster-package-tests
Oct 5, 2026
Merged

stanoddly merged 2 commits into
mainfrom
faster-package-tests

Conversation

@botoddly

@botoddly botoddly commented Oct 5, 2026 •

Copy link
Copy Markdown
Collaborator

The package integration job decided PR time: 9 m 24 s for 66 tests run one at a time, against under 4 minutes for the validation job.

  • The 24 BrowserBuildRejectsInvalidTargetPath builds are 5, one per PIXELY0011 check, because the first failing check stops the target. Each build declares its paths through an items file the fixture imports (BrowserContentConsumerExtraItems). The test now asserts that the error names every rejected path, not only the code.
  • The pattern build uses _framework/y.pak beside _Framework/x.pak: MSBuild batches ignoring case, so two spellings of one path would share a batch and only one would reach the pattern. The error must name each path as a whole word, so C:/abs.pak does not stand in for /abs.pak.
  • The changed-archive and deleted-file publish tests are merged with the Mono case of the content-archive publish: 5 publishes become 2. The CoreCLR case moves to its own method. The archive checks run without node; only the runs under node need it.
  • The implicit-restore publish test is folded into HostedConsumerPublishesABrowserBundle, which now publishes with --runtime and its implicit restore.
  • Consumer builds run one dotnet build or dotnet publish with its implicit restore instead of a separate dotnet restore. NU restore warnings still appear in that output.
  • The PR workflow runs the package tests in two jobs, package-integration (1/2) and (2/2), each packing Pixely. A [SetUp] reads PIXELY_PACKAGE_TEST_SHARD, orders the test methods by name and deals them out in turn, so every test runs in exactly one job. A test filter cannot split them: NUnit does not select an explicit test through a negated filter. Only pull-request-validation is a required check, so the renamed jobs do not block merging.
  • The PR workflow prints each test's duration.

Verification

  • Both shards pass locally with the wasm-tools workload: 20 and 24 tests, 2 m 11 s and 1 m 46 s including the pack. The 5 batched builds take 1 to 2 s each.
  • Not measured: the time on the ARM runners, which this PR's own run will show.

@botoddly
botoddly force-pushed the faster-package-tests branch from 44de3b8 to cd33ab4 Compare October 5, 2026 13:49
@stanoddly
stanoddly enabled auto-merge (squash) October 5, 2026 13:50
@stanoddly
stanoddly merged commit e24282f into main Oct 5, 2026
3 checks passed
@stanoddly
stanoddly deleted the faster-package-tests branch October 5, 2026 13:54
@github-actions

github-actions Bot commented Oct 5, 2026

Copy link
Copy Markdown

📦 Development package Pixely 0.0.114 is available.

Workflow run (attempt 1)

@github-actions

github-actions Bot commented Oct 5, 2026

Copy link
Copy Markdown

✅ Development package validation passed on all platforms.

Workflow run (attempt 1)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants