Fix v4 migrator: route annual techs' split rows to the annual split tables

[SutubraResearch]

Problem

master_migration.py routes TechInputSplit / TechOutputSplit rows to the seasonal v4 tables (limit_tech_input_split / limit_tech_output_split) unconditionally, ignoring Technology.annual. The seasonal split constraints only apply to non-annual techs, so every annual tech's split constraint silently disappears from migrated models — the rows are present in the DB, but no constraint is built from them, and nothing warns.

Impact (real-world case)

Migrating the Open Energy Outlook 9-region US database, 13,313 of 15,369 TechInputSplit rows belong to annual techs (transport fuel blends, industrial fuel-mix shares) and were all dropped from the model. Visible symptom: Fischer-Tropsch synfuel ran without its mandated hydrogen input, making captured CO₂ a free fuel source and distorting the whole electricity mix (a ~370 GW phantom coal-CCS build at $0 carbon). Re-routing the rows restored v3-equivalent behavior.

Fix

In _migrate_operator_tables, partition split rows by the tech's annual flag (reading Technology.annual, with a tech_annual table fallback for older v3 databases) and route annual techs' rows to limit_tech_input_split_annual / limit_tech_output_split_annual. The seasonal and annual targets are column-identical, so the existing operator-insertion transform applies unchanged.

v3.1-schema databases (explicit LimitTech*Split[Annual] tables) are unaffected — they already migrate by name.

Tests

Both migration mock datasets now carry one seasonal-tech and one annual-tech split row (input and output), with placement assertions after migration. The new v3-schema assertions fail without this fix and pass with it; the v3.1 path passes before and after (placement-preservation coverage).

Summary by CodeRabbit

• Bug Fixes

  • Improved v3-to-v4 migration for split operator tables containing both seasonal and annual technology rows.
  • Annual technologies are now migrated into the annual split constraints, while seasonal technologies go into the seasonal split constraints—preserving expected operator and proportion values.

• Tests

  • Added unit coverage to confirm annual-technology detection from legacy sources.
  • Extended migration test data and assertions to verify correct routing for both input and output split constraints.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: a4b90232-7a44-4df5-af3c-947a905dfc82

📥 Commits

Reviewing files that changed from the base of the PR and between 71a970c and 18e287c.

📒 Files selected for processing (1)

• tests/test_v4_migration.py

---

Walkthrough

The migration detects annual technologies from current or legacy metadata, routes legacy split-table rows to seasonal or annual v4 tables, and verifies both routing paths with expanded migration fixtures and tests.

Changes

Split-table migration

Layer / File(s)	Summary
Annual technology detection and fixtures temoa/utilities/master_migration.py, tests/testing_data/migration_v3_1_mock.sql, tests/testing_data/migration_v3_mock.sql, tests/test_v4_migration.py	Adds annual technology detection with a legacy-table fallback and seed data covering seasonal and annual split rows, including helper tests.
Split-row routing and validation temoa/utilities/master_migration.py, tests/test_v4_migration.py	Partitions legacy input/output split rows into seasonal and annual v4 tables and verifies the migrated values.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant LegacyDB
  participant Migration
  participant V4Tables
  LegacyDB->>Migration: Read annual technology flags
  Migration->>LegacyDB: Read split-table rows
  Migration->>V4Tables: Insert seasonal rows
  Migration->>V4Tables: Insert annual rows

Loading

Possibly related PRs

• TemoaProject/temoa#284: Modifies the same v3-to-v4 migration implementation and provides related migrator context.

Suggested labels: Maintenance, database-schema

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly summarizes the main change: routing annual tech split rows to the annual split tables.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[codecov]

Codecov Report

❌ Patch coverage is 75.00000% with 9 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
temoa/utilities/master_migration.py	47.05%	9 Missing ⚠️

📢 Thoughts on this report? Let us know!