Skip to content

Start production deploy runs only for main CI runs - #261

Merged
anth-volk merged 4 commits into
mainfrom
fix/deploy-trigger-main-only
Oct 22, 2026
Merged

anth-volk merged 4 commits into
mainfrom
fix/deploy-trigger-main-only

Conversation

@anth-volk

@anth-volk anth-volk commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #260

What changes

.github/workflows/deploy.yml ("Deploy production"):

  1. The workflow_run trigger now lists branches: [main], so only CI runs whose head branch is main create a deploy run. Pull-request CI runs from branches in this repository no longer create one.
  2. The calibration-diagnostics-production concurrency group (still cancel-in-progress: false) moves from the workflow to the deploy job. A workflow-level group admits a run as soon as GitHub creates it, before the job's if: is evaluated, so a run that would skip could replace a pending deploy of main. That is how Refuse a country with no configured release instead of serving the US one #248's pending deploy (run 39680251409) was cancelled on 2026-10-21; Pull-request CI runs can cancel a pending production deploy of main #260 lists the runs involved.
  3. The job's if: stays the same: a successful CI run, triggered by push, on main.

A newer main deploy still replaces an older pending main deploy. That is intended, because the newer commit contains the older one.

frontend/HOSTED_RUNTIME.md describes the trigger filter and where the concurrency group is declared.

Pull requests from a fork whose branch is named main

GitHub matches the workflow_run branches filter against the triggering run's head_branch. For a pull request from a fork, that is the fork's own branch name; fork pull-request runs in PolicyEngine/policyengine-us show this. A fork pull request from a branch named main therefore still passes the filter and creates a deploy run. Its workflow_run.event is pull_request, so the job's if: skips it, and the job never deploys. Because the concurrency group now belongs to the job, the skipped job should not enter the group or replace a pending deploy. GitHub's documentation does not say outright whether a job skipped by if: takes part in job-level concurrency. See "Not verified" below. This repository has had no fork pull-request runs in its last 100 pull-request runs.

The same applies to a CI run on main that fails or is cancelled, for example when a newer push cancels it under CI's cancel-in-progress: true. It still creates a deploy run, but the job skips and no longer occupies the group.

Tests

The workflow tests are in frontend/tests/test_deploy_workflow.py.

  • 63a2851 (tests only): 3 failed, 17 passed.
    • test_deployment_waits_for_successful_main_ci: the trigger has no branches: [main].
    • test_production_concurrency_group_belongs_to_the_deploy_job_not_the_workflow: the workflow declares concurrency, and the deploy job does not.
    • test_every_job_in_the_production_group_requires_a_successful_main_push: no job declares the group, so no job can be checked for the three required if: clauses (success, push, main, all joined by &&).
  • 4107145 (fix): test_deploy_workflow.py: 20 passed. With test_telemetry_service_workflow.py and test_deploy_script.py: 35 passed.
  • actionlint 1.7.12 on deploy.yml: no findings. CI does not run actionlint; this was a local check.

workflow_run workflows run from the default branch's copy of the file, so this pull request's own CI does not exercise the new trigger. The first real run happens when this pull request's merge commit passes CI on main. Merging this pull request deploys production.

Environment variables and secrets

Moving the concurrency group does not change environment: Production, permissions or any step's env: on the deploy job, and the workflow-level env: block is unchanged. GitHub resolves environment secrets and variables for a job from its environment: key, which is still Production. The Production environment has no protection rules or deployment branch policy. Providers below are from gh secret list / gh variable list (repository, Production environment, and the organization secrets shared with this repository). The last production deploy, run 39684343767 (2026-10-21), passed Require deployment credentials, google-github-actions/auth@v2 and Check collector read secret exists with this same wiring.

Name Read by Provider Reaches the job after this change
MODAL_ENVIRONMENT, VERCEL_CLI_VERSION, VERCEL_ORG_ID, VERCEL_PROJECT_ID Workflow env: (all steps); deploy-dashboard.sh Literal values in deploy.yml Yes. The workflow-level env: is unchanged
HF_RELEASE_READ_TOKEN Require deployment credentials; Build unaliased Vercel production candidate (passed to Vercel as HF_TOKEN) GitHub Actions repository secret Yes. Step env: is unchanged
MODAL_TOKEN_ID, MODAL_TOKEN_SECRET Require deployment credentials; Deploy Modal backend; Resolve Modal backend URL GitHub Actions organization secrets shared with this repository (not repository or Production secrets) Yes. Step env: is unchanged
MICROCOSM_MODAL_KEY, MICROCOSM_MODAL_SECRET Credentials check; Verify Modal backend metadata; Vercel build (--env) Production environment secrets Yes. environment: Production is unchanged
VERCEL_TOKEN Credentials check; Vercel build; Promote verified Vercel deployment Production environment secret Yes
VERCEL_AUTOMATION_BYPASS_SECRET Credentials check; staged metadata, calculation and calibration checks Production environment secret Yes
MICROCOSM_TELEMETRY_COLLECTOR_URL Credentials check; Vercel build (--env) Production environment variable Yes
TELEMETRY_GCP_WIF_PROVIDER, TELEMETRY_GCP_DEPLOY_SERVICE_ACCOUNT google-github-actions/auth@v2 inputs Production environment secrets Yes. The job keeps id-token: write
TELEMETRY_GCP_PROJECT_ID → PROJECT_ID, TELEMETRY_READ_SECRET_NAME → READ_SECRET_NAME Check collector read secret exists Production environment variables Yes
SOURCE_COMMIT, CALIBRATION_MODAL_APP_NAME, MICROCOSM_CALCULATION_URL, MICROCOSM_BACKEND_SOURCE_COMMIT Backend naming, Modal deploy, Vercel build github.event.workflow_run.head_sha and earlier step outputs in the same job Yes. The workflow_run payload is unchanged by the branch filter
GITHUB_OUTPUT, OIDC token request variables deploy-dashboard.sh; google-github-actions/auth@v2 GitHub runner (OIDC through the job's id-token: write) Yes

Modal runtime: backend/modal_app.py attaches no Modal secrets. It sets CALIBRATION_SOURCE_COMMIT, CALIBRATION_MODAL_APP_NAME and the source-tree hash in the image from the checked-out commit, so this change does not affect them. Vercel runtime: the frontend's server variables come from the --env arguments in deploy-dashboard.sh deploy-frontend, listed above. This change does not modify Vercel project settings.

Not verified

  • Whether a job skipped by if: stays out of its job-level concurrency group. GitHub's workflow syntax reference describes job-level concurrency and the pending-run replacement, but it does not say whether a skipped job enters the group. This matters only for runs that pass the branch filter but skip: a fork pull request from a branch named main, and a failed or cancelled main CI run. The branch filter alone removes every same-repository pull-request run, which caused the 2026-10-21 cancellation. It was not tested, because testing would mean dispatching or merging. If needed, the job's if: could also require github.event.workflow_run.head_repository.full_name == github.repository, but that would not change whether the skipped run enters the group.
  • Secret values. The table confirms names, scope and wiring, and that run 39684343767 passed the credentials and Google Cloud checks. It does not confirm that the current values are still valid.

Other workflows

No other workflow in this repository uses workflow_run. publish-calibration-tree.yml has a workflow-level concurrency group and a job-level if: on vars.CALIBRATION_TREE_BUILD_ENABLED, but it runs only on workflow_dispatch. Its comments describe the replacement of pending dispatches as intended. telemetry-service.yml uses separate groups for pull requests and dispatches. ci.yml cancels in-progress runs for the same ref. None has this pattern.

Review (2026-10-22)

The change is correct.

  • The branches: [main] filter matches the triggering CI run's head branch. Another repository with the identical trigger (openops-cloud/openops) created deploy runs only for its 29 main CI runs, out of 178.
  • Job-level concurrency still serializes deploys and never cancels a running one.
  • Moving the group lost nothing: environment: Production, permissions, the timeout and every env: block are unchanged.
  • Every environment variable and secret has a provider and reaches the job.

Not stated by GitHub's documentation: whether a job skipped by its if: stays out of a job-level group. This matters only for failed main CI runs and for a fork pull request from a branch named main, which the workflow-level group already admits today, so this is never worse than before.

Fixed, failing test first: the if: test was too strict. It rejected equally strict rewordings, such as the ${{ }} wrapper, reordered conditions, or an extra required condition.

Commit Content Result
0066f9e Tests only 3 fail, 24 pass
9792ede The fix: any &&-joined condition containing success, push and main is accepted; a missing condition, `

CI passes on 9792ede.

Merge order: merge this first, before #253, #255 and #259, while no deploy is running or waiting. Its own deploy is the first one protected. After it, check that the next pull-request CI completion creates no deploy run.

🤖 Generated with Claude Code

anth-volk and others added 2 commits October 22, 2026 02:27
Require the deploy workflow to filter its workflow_run trigger to main,
to declare the calibration-diagnostics-production concurrency group on
the deploy job instead of the workflow, and to keep requiring a
successful push run on main in every job that holds that group.

These fail against the current workflow: the trigger has no branch
filter, the group is declared at workflow level, and no job declares it.

Refs #260

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The deploy workflow ran after every CI run, pull requests included, and
declared its concurrency group at workflow level. Each new run joined
the group as soon as GitHub created it, before the deploy job's
condition skipped it, and replaced the pending run. On 2026-10-21 a
pull-request CI run's deploy run cancelled the pending deploy of #248.

Filter the workflow_run trigger to main and declare the concurrency
group on the deploy job, so a run whose job is skipped does not take
the pending position. The job keeps its condition requiring a
successful push run on main, which rejects a pull request from a fork
whose branch is named main; that run still passes the branch filter.

Fixes #260

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vercel

vercel Bot commented Oct 21, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
calibration-diagnostics Ready Ready Preview Oct 22, 2026 12:54pm UTC

Request Review

anth-volk and others added 2 commits October 22, 2026 16:52
The check that each job in the production concurrency group requires a
successful main push compared the job's `if:` with exactly three
`&&`-joined clauses. It rejected conditions that are just as strict:
the `${{ }}` expression wrapper, parenthesised or reordered clauses, and
an extra required clause such as a repository check.

Move the comparison into a helper and test it directly. Three cases fail
with the exact comparison: the expression wrapper, reordered and grouped
clauses, and an extra required clause. The cases for a missing clause,
an `||` alternative and a negation already pass and must keep passing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The check now passes when a job's `if:` is a conjunction that includes
the three required clauses, whatever the whitespace, clause order,
grouping parentheses, `${{ }}` wrapper or extra `&&`-joined clauses. It
still fails when a clause is missing, and it fails on any `||` or `!`
(other than `!=`), because either could let the job run without them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@anth-volk
anth-volk marked this pull request as ready for review October 22, 2026 13:10
@anth-volk
anth-volk merged commit 2f7cdcd into main Oct 22, 2026
5 checks passed
@anth-volk
anth-volk deleted the fix/deploy-trigger-main-only branch October 22, 2026 13:10

This branch was successfully deployed

1 active deployment
Preview — 9792ede2 Deployed Oct 22, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Pull-request CI runs can cancel a pending production deploy of main

1 participant