ci(qa): serialise runs sharing the integration test project - #65
Merged
Merged
Conversation
Every Qa run points at one commercetools project, and the integration suites seed and clear fixtures in it. Overlapping runs therefore corrupt each other: after the teardown fix made cleanup reliable, one run's clear step began reliably wiping fixtures another run was asserting on. Seen on main when the #63 and #64 merges landed 41 seconds apart -- the first run passed every job, the second failed three tests with ● Resource Deleter > should delete resource > carts deleted expect(payload.body.results.length).toBeGreaterThanOrEqual(1) Expected: >= 1 Received: 0 Add a workflow-level concurrency group. It is deliberately constant rather than keyed on github.ref: the contended resource is the shared project, not the branch, so runs must serialise across all branches and PRs. cancel-in-progress stays false because a run killed mid-suite skips teardown and leaks fixtures -- the failure mode this is meant to prevent. The regression matrix already sets max-parallel: 1, so intra-run serialisation was never the gap; cross-run was. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Every
Qarun points at one commercetools project, and the integration suites seed and clear fixtures in it. Nothing serialised those runs.This was latent while cleanup was broken. Once #63 made teardown reliable, one run's clear step began reliably wiping fixtures another run was mid-way through asserting on — the inverse of the old duplicate-key failures.
Observed directly on
mainwhen the #63 and #64 merges landed 41 seconds apart:The fixture was seeded, then deleted by the other run before the assertion.
The change
Two deliberate choices:
github.ref. The usual${{ github.workflow }}-${{ github.ref }}only dedupes within a branch. The contended resource here is the shared project, not the branch, so runs must serialise across all branches and PRs.cancel-in-progress: false. Cancelling is the default reflex, but a run killed mid-suite skips teardown and leaks fixtures into the project — exactly the failure this is meant to prevent.max-parallel: 1is already set on the regression matrix, so intra-run serialisation was never the gap. Cross-run was.Trade-off worth knowing
Qa runs now queue globally instead of running in parallel, so concurrent PRs will wait on each other. With one shared project that is unavoidable without provisioning per-run projects.
Also note GitHub keeps only one pending run per concurrency group — if three runs queue, the middle one is cancelled. That is a real rough edge. It is much better than the current state, and #63's clear-before-seed means a cancelled run no longer poisons the project permanently, but it is not free. The durable fix is a dedicated project per run.
Verification
Workflow YAML parses;
concurrencyresolves to{"group":"qa-integration","cancel-in-progress":false}and all five jobs are intact.Concurrency behaviour cannot be exercised from a single PR run — it only shows up when runs overlap.
🤖 Generated with Claude Code