Skip to content

feat: update promotion scripts to accommodate Airflow v1 and v2 (for Airflow 3 upgrade) - #454

Open
botanical wants to merge 4 commits into
mainfrom
feat/airflow3-flag
Open

botanical wants to merge 4 commits into
mainfrom
feat/airflow3-flag

Conversation

@botanical

@botanical botanical commented Sep 11, 2026

Copy link
Copy Markdown
Member

https://github.com/NASA-IMPACT/veda-architecture/issues/849

What Changed

  • I added scripts/airflow_api.py that triggers that DAGs according to the version it identifies based on env var settings
  • I updated the promotion scripts so that it would check the Airflow versions on Staging and Production (defaulting to use v2/ Airflow 3)
  • Notably, this refactor handles both Airflow 2 and Airflow 3 and allows staging and prod instances to be on different versions are we're staggering our rollout

@botanical
botanical requested a review from smohiudd as a code owner September 11, 2026 22:03

@anayeaye anayeaye left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These changes look good--was the main update just to the airflow base url?

Comment thread scripts/airflow_api.py
Comment on lines +29 to +37
def _build_request_body_v3(
conf: Dict[str, Any], dag_id: str, note: str = ""
) -> Dict[str, Any]:
"""Build the request body for Airflow 3 API v2"""
return {
"conf": conf,
"dag_run_id": f"{dag_id}-{uuid.uuid4()}",
"note": note or "Run from GitHub Actions veda-data workflow",
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this the same as the _v2 request body? I can't see the difference


api_token = b64encode(f"{username}:{password}".encode()).decode()

headers = {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like moving this logic to the common airflow_api.py!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants