Skip to content

ci(qa): serialise runs sharing the integration test project - #65

Merged
ajimae merged 1 commit into
mainfrom
fix/qa-concurrency
Sep 24, 2026
Merged

ajimae merged 1 commit into
mainfrom
fix/qa-concurrency

Conversation

@ajimae

@ajimae ajimae commented Sep 24, 2026

Copy link
Copy Markdown
Member

Why

Every Qa run points at one commercetools project, and the integration suites seed and clear fixtures in it. Nothing serialised those runs.

This was latent while cleanup was broken. Once #63 made teardown reliable, one run's clear step began reliably wiping fixtures another run was mid-way through asserting on — the inverse of the old duplicate-key failures.

Observed directly on main when the #63 and #64 merges landed 41 seconds apart:

Run Trigger Result
36021926028 #63 merge all jobs green
36022011913 #64 merge 3 failed / 808 passed
● Resource Deleter › should delete resource › carts deleted
    // Check that resource exists
    expect(payload.body.results.length).toBeGreaterThanOrEqual(1);
    Expected: >= 1
    Received:    0

The fixture was seeded, then deleted by the other run before the assertion.

The change

concurrency:
  group: qa-integration
  cancel-in-progress: false

Two deliberate choices:

  • Constant group, not keyed on github.ref. The usual ${{ github.workflow }}-${{ github.ref }} only dedupes within a branch. The contended resource here is the shared project, not the branch, so runs must serialise across all branches and PRs.
  • cancel-in-progress: false. Cancelling is the default reflex, but a run killed mid-suite skips teardown and leaks fixtures into the project — exactly the failure this is meant to prevent.

max-parallel: 1 is already set on the regression matrix, so intra-run serialisation was never the gap. Cross-run was.

Trade-off worth knowing

Qa runs now queue globally instead of running in parallel, so concurrent PRs will wait on each other. With one shared project that is unavoidable without provisioning per-run projects.

Also note GitHub keeps only one pending run per concurrency group — if three runs queue, the middle one is cancelled. That is a real rough edge. It is much better than the current state, and #63's clear-before-seed means a cancelled run no longer poisons the project permanently, but it is not free. The durable fix is a dedicated project per run.

Verification

Workflow YAML parses; concurrency resolves to {"group":"qa-integration","cancel-in-progress":false} and all five jobs are intact.

Concurrency behaviour cannot be exercised from a single PR run — it only shows up when runs overlap.

🤖 Generated with Claude Code

Every Qa run points at one commercetools project, and the integration
suites seed and clear fixtures in it. Overlapping runs therefore
corrupt each other: after the teardown fix made cleanup reliable, one
run's clear step began reliably wiping fixtures another run was
asserting on.

Seen on main when the #63 and #64 merges landed 41 seconds apart --
the first run passed every job, the second failed three tests with

    ● Resource Deleter > should delete resource > carts deleted
      expect(payload.body.results.length).toBeGreaterThanOrEqual(1)
      Expected: >= 1
      Received:    0

Add a workflow-level concurrency group. It is deliberately constant
rather than keyed on github.ref: the contended resource is the shared
project, not the branch, so runs must serialise across all branches
and PRs. cancel-in-progress stays false because a run killed mid-suite
skips teardown and leaks fixtures -- the failure mode this is meant to
prevent.

The regression matrix already sets max-parallel: 1, so intra-run
serialisation was never the gap; cross-run was.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@ajimae
ajimae requested a review from a team as a code owner September 24, 2026 15:46
@changeset-bot

changeset-bot Bot commented Sep 24, 2026

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: 88113c6

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@ajimae
ajimae merged commit 566dbb7 into main Sep 24, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant