[anth-volk]

Fixes #4161

Warning

Deploying this PR changes the staging and production v2 database schemas and table privileges. The release's v2 migration job adds three tables, moves the execution columns off simulations, re-keys the result tables by run, adds the economy output's missing columns and drops 14 foreign keys. It needs the changed tables empty, which they are while no managed role can write them (see "Data"). It also declares two new database roles, which an operator creates (see "What this waits for before merging"), and changes two rules in the rules registry, so it needs a core developer's review.

Summary

API v2's simulation tables recorded one execution per simulation. This PR makes every run a row, with the versions it used, so re-running a simulation or report keeps the earlier results, and prepares the tables for the simulation API (PolicyEngine/policyengine-sim-api) and its Modal workers to write them.

• simulations keeps only what to calculate. Its execution columns (status, error_message, started_at, completed_at, household_result) move to runs; output_dataset_id is replaced by the run's output file.
• simulation_runs (new): one row per run of a simulation, with the catalog version (tax_benefit_model_version_id), country_package_version and policyengine_version it used, the dataset and region it resolved, its status (pending, running, succeeded, failed, reusing the v2_simulation_status type), Modal call ID, error, timings, household_result for a household run, and output_uri with output_sha256 for an economy run's output file. Checks: a finished run has completed_at; the output URI and digest are set together; the digest is 64 characters. Indexes for the newest successful run of a simulation on given versions, and for runs by status and start time.
• report_run_simulation_runs (new): the simulation runs a report run combines, each as its baseline or reform (new type v2_simulation_role), unique per report run, role and run. A budget window's run combines one pair per year.
• report_run_comparisons (new): one row per report run and compared pair of simulation runs, holding the parts of an economy comparison that have no result table: the labour supply response, cliff impact, SPM receipts and data release.
• report_runs (already one row per run with both versions) gains modal_call_id and an index by status and start time.
• Results keyed by run. poverty and inequality describe one simulation, so they belong to a simulation run (simulation_run_id, cascade). The other result tables (budget_summary, program_statistics, decile_impacts, intra_decile_impacts, congressional_district_impacts, constituency_impacts, local_authority_impacts) still belong to a report run and name the two simulation runs they compare (baseline_simulation_run_id, reform_simulation_run_id) instead of two simulations; each row is unique within its report run and pair, so a budget window stores one set per year. aggregates and change_aggregates are unchanged.
• Each result row says what it measures. decile_impacts.decile_variable names the variable that groups households when it isn't the income variable's own deciles (UK wealth deciles: household_wealth_decile), and poverty.filter_group names the group a filter variable selects (age: child, adult, senior), as PolicyEngine.py's outputs name them; both join their table's unique key, where missing values count as equal (NULLS NOT DISTINCT).
• The economy output's missing fields. congressional_district_impacts gains the winner, loser and no-change shares; constituency_impacts and local_authority_impacts gain the UK disclosure fields (disclosure_tier, checked to be reported or approximate, and the four _lower/_upper interval bounds); program_statistics.entity and is_tax become nullable, because the detailed budget names neither. Every field of the simulation API's single-year economy output now has a column.
• household_jobs calculates one household for one year under one policy: it gains household_id, year (1900–2200), the catalog and package versions, modal_call_id, updated_at and the same completion check; request_data now holds the calculation's options, such as axes. Null policy_id means current law.
• No foreign keys between the two services' tables. The simulation API and its workers write simulations, reports, household_jobs, the run tables and the results; the Public API writes users, saved items, policies, dynamics, households and the catalog. The 14 foreign keys from the first group to the second are dropped, and the columns stay as plain UUIDs that the writing service checks (reports.user_id, the owner the Public API names, is checked by the Public API). Foreign keys within each group stay; the ORM relationships across the groups are removed. Deleting a user no longer clears reports.user_id; a saved report's owner is its user association, which the deletion removes.
• simulation_runs is also a v1 Cloud SQL table name; the v2 metadata validation now treats it as a name the two databases share, as it does simulations.

No table is dropped. No route, request or response changes: nothing serves these tables yet.

Table privileges

policyengine_api/data/v2/privileges.py declares two new managed roles, and the Public API's runtime role's reads:

Tables	policyengine_v2_runtime (the Public API)	policyengine_v2_simulation_service (the simulation API on Cloud Run)	policyengine_v2_modal_worker (its Modal workers)
simulations, reports, report_run_simulation_runs	SELECT	SELECT, INSERT	SELECT
simulation_runs, report_runs, household_jobs	SELECT	SELECT, INSERT	SELECT, UPDATE
report_run_comparisons and the nine result tables	SELECT	—	SELECT, INSERT
policies, dynamics, households, tax_benefit_models, tax_benefit_model_versions, regions, datasets, parameters, parameter_values	unchanged	SELECT	SELECT
Every other table (including aggregates, change_aggregates)	unchanged	—	—

The catalog writer and the Stage 12 role keep exactly their privileges. The privileges are table-level, as for every managed role: the database doesn't limit the workers' UPDATE to the status and result columns, or to their own run; the workers' code does.

Declaring a role before it exists. reconcile_v2_privileges.py apply grants nothing, and verify fails, when a declared role is missing, so declaring these roles would otherwise fail every deploy until an operator creates them. Both roles are listed in a new ROLES_AWAITING_PROVISIONING: while a listed role is missing, apply and verify skip it and name it in their output and job summary ("Declared roles not created yet, skipped until an operator creates them"), and verify reports "Every other declared v2 table privilege is held". Once the role exists, the next deploy grants its privileges and verify checks them. A missing role that is not listed still blocks every grant and fails verification. A follow-up PR removes the two roles from the list once they exist in both environments. The rules registry's api.v2.deployed-privileges-match and api.v2.runtime-privileges-tested-as-runtime-role now describe this and the new role tests; docs/generated/rules.md is re-rendered.

tests/integration/test_v2_simulation_role_privileges.py runs the simulation API's writes as each role (the simulation service creates the rows but can't update a run or write a result; a worker updates its run and household job and inserts results but can't create a simulation or change a result; the Public API reads but can't write). CI's v2 integration job now runs it, and the v2 Alembic job runs tests/integration/test_alembic_v2_simulation_runs.py.

Revisions

All five chain onto c614b6bcccfc, the current master head, and were autogenerated with uv run alembic -c alembic-v2.ini revision --autogenerate against a disposable PostgreSQL 17 database at each parent. When master gained e86f19a6c4da, 3869eb814a90 and c614b6bcccfc, 2ffd50f47a2b was generated again on top of each, with --rev-id 2ffd50f47a2b; the operations were the same each time.

• 2ffd50f47a2b (parent c614b6bcccfc): simulation runs, report run members, results keyed by run, household jobs. Two post-generation corrections, each commented in the revision and covered by tests:
  1. Dialect: simulation_runs.status reuses v2_simulation_status, which the baseline created for simulations.status. The generated create_table would create the type again and fail (DuplicateObject), so the column uses postgresql.ENUM(..., create_type=False).
  2. Reversibility: the downgrade drops v2_simulation_role after its table, because Alembic does not drop a table's enum type with it.
• 7a83a87d029a: drops the 14 cross-service foreign keys; its downgrade restores them. No corrections.
• 7f43d2c7c3c6: adds the economy output's missing columns and relaxes program_statistics.entity and is_tax. No corrections.
• 2e3de7c572ed: adds decile_variable and filter_group and rebuilds eight unique keys with the run pair or group. No corrections.
• dce0caa329cc: creates report_run_comparisons and drops the four report_runs columns it takes over, which the earlier revisions added and nothing wrote. No corrections.
• V2_ALEMBIC_HEAD_REVISION is now dce0caa329cc. The Alembic guide documents the five revisions and the new roles. NULLS NOT DISTINCT needs PostgreSQL 15 or later.

Data

The upgrades add required columns without server defaults to household_jobs and the result tables, so they need those tables empty. They are while no managed role holds a write privilege on them (on master, the runtime, catalog writer and Stage 12 roles hold none on any table this PR changes). On a table with rows the upgrade fails inside its transaction and the release stops before deploying; nothing is migrated or deleted. The downgrades work only while simulations, household_jobs and the result tables are empty, while no simulation-side row names a Public API row that doesn't exist, and while no report run holds rows of more than one pair.

Compatibility with the previous Public API revision

The deploy upgrades the v2 schema before the new release starts, so the previous release runs against these revisions for a while, and after a rollback. It can't read or write any table they change: its managed roles hold no privilege on them, except the catalog writer, which counts the rows of simulations, reports and report_runs (a count reads no column). tests/integration/test_alembic_v2_simulation_runs.py checks that the revisions add the three tables and change only the simulation-side tables, and downgrades below each of them and upgrades again. The runtime role's new reads only add privileges.

Environment variables

This PR reads no new environment variable. The CI changes only add test files to existing steps of alembic-v2-check.yml (through .github/scripts/test_alembic_v2_lifecycle.sh) and v2-integration-check.yml, whose database URLs, V2_ALEMBIC_DISPOSABLE_TEST and DEPLOYMENT_ENVIRONMENT are literals in the job env:. The existing v2 seed workflow (.github/workflows/seed-v2-database.yml, called from push.yml with secrets: inherit and deployment_environment staging or production) applies the revisions and the privileges. Its job runs in that GitHub environment and reads:

• V2_MIGRATION_DATABASE_URL: GitHub environment secret, present in staging and production; passed as step env:.
• V2_SUPABASE_PROJECT_REF, V2_SUPABASE_ENVIRONMENT: GitHub environment variables, present in both; job env:.
• PRODUCTION_V2_SUPABASE_PROJECT_REF: repository variable, present; job env:.
• DEPLOYMENT_ENVIRONMENT: the workflow input. GITHUB_STEP_SUMMARY: set by GitHub Actions.

The two new roles' credentials are created and given to the simulation API outside this repository (below); until an operator confirms those steps, they are not verified.

What this waits for before merging

• Review, including a core developer's review of the rules registry change, and three design choices: dropping the foreign keys between the two services' tables, the report run member table (instead of two run columns on report_runs), and JSON columns in report_run_comparisons for labour supply, cliff impact and the SPM receipt (instead of row tables).
• The staging and production v2 databases run PostgreSQL 15 or later (SHOW server_version; in each project's SQL editor), for NULLS NOT DISTINCT.
• Other open PRs that add v2 revisions (for example Rename the v2 table user_policies to user_policy_associations #4010): whichever merges first, the other re-points its down_revision onto the new head.
• No operator step blocks the merge. The two new roles may be created before or after it. The required order:
  1. This PR merges. Each environment's deploy upgrades the schema, grants the runtime role's new reads, and names the two new roles as skipped.
  2. In staging, an operator creates the login roles policyengine_v2_simulation_service and policyengine_v2_modal_worker in the v2 database (no superuser, database, role-creation, replication or RLS-bypass attributes; no role membership; no table grants by hand), generates each password without printing it, and stores each URL in Secret Manager in the simulation API's project (simulation-service-staging-database-url, modal-worker-staging-database-url). Then the "Seed v2 database" workflow runs for staging (by hand, or with the next release), which grants and verifies the two roles' privileges.
  3. The same in production (…-production-database-url, workflow environment production).
  4. A follow-up PR removes the two roles from ROLES_AWAITING_PROVISIONING, so that a missing role fails the deploy again.
  5. Only after step 3 in an environment may the simulation API start connecting there with these roles: its Cloud Run service with SIMULATION_DATABASE_URL (from the first secret, Connect the Simulation Entrypoint to the Public API's database with its own role policyengine-sim-api#898) and its Modal workers with the Modal secret policyengine-modal-worker-database (key MODAL_WORKER_DATABASE_URL, from the second).
• How to check each step without printing a secret: pg_roles and pg_auth_members show the roles' attributes and no memberships; the seed workflow's summary shows the roles' grants and Every declared v2 table privilege is held; has_table_privilege spot checks (for example the simulation service can INSERT but not UPDATE simulation_runs); gcloud secrets versions list shows each secret's enabled version.

Testing

Each commit was written red-green: the tests first, seen failing for the expected reason, then the code. Only the tests each commit adds or modifies were run, each by node ID, against a disposable postgres:17 database named policyengine_v2_alembic_test (set up as alembic-v2-check.yml's service container is); the full suite runs in CI.

• 52b01a1ff (simulation runs, results by run, household jobs). Seen failing on master's code: the new model tests (ImportError: no SimulationRun), the revision and chain tests (no revision), and both tests then in tests/integration/test_alembic_v2_simulation_runs.py. Passing after the last rebase: 25, including the lifecycle test, the previous-revision compatibility tests and the parameter tree and search file revision tests, whose head assertions now name their own child.
• 019907f7e (cross-service foreign keys). Seen failing: the four foreign key and relationship tests, the revision test, the chain test and the live-schema test; and, after the catalog publication test, a downgrade that couldn't restore a key, because the publication tests' TRUNCATE tax_benefit_models CASCADE no longer reached simulations and reports (the fixture now names them). Passing: 41, including the 18 report-run unit tests whose shared helper no longer uses the removed relationships.
• 0ec365a4b (economy output fields). Seen failing: four model tests (KeyError per new column), the revision and chain tests, and the result-table checks. Passing: 14. The commit moves the revision-by-revision downgrade checks into tests/integration/test_alembic_v2_simulation_runs.py, because the lifecycle test file had reached its length limit.
• 13669e0e9 (skipping a role not created yet). Seen failing: four unit tests (TypeError: no roles_awaiting_creation, no skipped_roles) and two integration tests. Passing: 6.
• b874e2ff1 (the two roles' privileges). Seen failing: six manifest tests, the three role tests (ImportError: no MODAL_WORKER_ROLE) and two workflow tests. Passing: 11.
• Review round 1 found four bugs, fixed red-green in 205b2a6cc (wealth deciles colliding with income deciles; PolicyEngine.py's poverty breakdowns colliding on poverty's key; a budget window storing comparison rows for one year only; seen failing: 9 tests, passing: 17) and b5cd8d742 (the simulation API's roles couldn't read dynamics; verify named a skipped role only on stdout; seen failing: 3, passing: 12). Round 2 found one, fixed in 601561f2e (a budget window had one place for each year's labour supply, cliff impact, SPM receipts and data release; seen failing: the model tests on import, passing: 18). Round 3 found none; its quality findings are addressed in 1af1cf472 (seen failing: 2 verify tests; passing: 11), and the extra round on that commit found one inaccurate docstring, fixed with its quality findings in 23bfc02e5 (passing: 8).
• CI on the earlier pushed heads (3fdbd8b1c, dc617db3e, e7c362b16) failed one test, tests/unit/v2/test_import_side_effects.py::test_importing_v2_modules_opens_no_network_and_creates_no_files, which pins the number of v2 tables; d674032a2 counts the three new tables (45). Seen failing locally on the previous commit, passing after. No CI ran on c9c6eb6d3, which conflicted with master. CI on d674032a2 (pushed 2026-11-05) was still running when this was written, with no failure so far.
• On the final head: every test the branch adds or modifies, 60 migration and model tests and 7 role tests, by node ID, pass. alembic -c alembic-v2.ini check at head: No new upgrade operations detected. Each revision downgrades and upgrades again, and downgrade base leaves no v2_* enum type. ruff format --check ., ruff check ., shellcheck, uv run --frozen --extra dev mypy, python scripts/run_quality_guards.py, python -m scripts.rules validate and python scripts/export_migration_contracts.py (no generated changes): pass.

No inline ruff suppressions are added.

🤖 Generated with Claude Code

[codecov]

Codecov Report

❌ Patch coverage is 98.63014% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 88.41%. Comparing base (528cef1) to head (d674032).
⚠️ Report is 1 commits behind head on master.

Files with missing lines	Patch %	Lines
policyengine_api/data/v2/models/simulations.py	96.66%	1 Missing ⚠️
...licyengine_api/data/v2/privilege_reconciliation.py	96.55%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           master    #4162       +/-   ##
===========================================
+ Coverage   53.93%   88.41%   +34.48%     
===========================================
  Files         281      293       +12     
  Lines       16261    17498     +1237     
  Branches     2629     2869      +240     
===========================================
+ Hits         8770    15471     +6701     
+ Misses       6760     1318     -5442     
+ Partials      731      709       -22     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.