Skip to content

Add IaC DB schema upgrade/downgrade scripts - #390

Merged
Chanel (chanel-y) merged 1 commit into
mainfrom
iac-downgrades-upgrades-scripts
Aug 12, 2026
Merged

Add IaC DB schema upgrade/downgrade scripts#390
Chanel (chanel-y) merged 1 commit into
mainfrom
iac-downgrades-upgrades-scripts

Conversation

@chanel-y

Copy link
Copy Markdown

The IaC extractor shipped no schema-migration scripts, so any mismatch between a database's dbscheme and a query pack's dbscheme was fatal (e.g. MRVA "database is not compatible with a QL library" errors). The dbscheme changed when Bicep support (82 bicep_* relations) was removed and yaml_comments/empty_location were added.

  • Add upgrade step iac/ql/lib/upgrades/e360fadd.../ (old bicep -> current): deletes the 82 bicep_* relations (compatibility: backwards).
  • Add downgrade step iac/downgrades/6b6bd68.../ (current -> old bicep): restores empty bicep tables, drops yaml_comments/empty_location (compatibility: partial).
  • Bootstrap iac/downgrades as a pack (qlpack.yml, initial/) and add iac/ql/lib/upgrades/initial/, matching go/rust/swift layout.
  • Teach misc/scripts/prepare-db-upgrade.sh about the iac language.
  • Fix iac/scripts/create-extractor-pack.ps1 (was a stale QL-for-QL copy) to mirror the .sh and ship the downgrades folder.

Verified with codeql 2.26.2: upgrading an old bicep DB to the current scheme and running an IaC query succeeds; downgrading a current DB back to the old scheme succeeds.

The IaC extractor shipped no schema-migration scripts, so any mismatch
between a database's dbscheme and a query pack's dbscheme was fatal (e.g.
MRVA "database is not compatible with a QL library" errors). The dbscheme
changed when Bicep support (82 bicep_* relations) was removed and
yaml_comments/empty_location were added.

- Add upgrade step iac/ql/lib/upgrades/e360fadd.../ (old bicep -> current):
  deletes the 82 bicep_* relations (compatibility: backwards).
- Add downgrade step iac/downgrades/6b6bd68.../ (current -> old bicep):
  restores empty bicep tables, drops yaml_comments/empty_location
  (compatibility: partial).
- Bootstrap iac/downgrades as a pack (qlpack.yml, initial/) and add
  iac/ql/lib/upgrades/initial/, matching go/rust/swift layout.
- Teach misc/scripts/prepare-db-upgrade.sh about the iac language.
- Fix iac/scripts/create-extractor-pack.ps1 (was a stale QL-for-QL copy)
  to mirror the .sh and ship the downgrades folder.

Verified with codeql 2.26.2: upgrading an old bicep DB to the current
scheme and running an IaC query succeeds; downgrading a current DB back
to the old scheme succeeds.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@MathiasVP

Copy link
Copy Markdown
Collaborator

I think this all makes sense. I'll just take a quick look to check if we're not being bitten by Windows vs. Unix line-endings like I fixed in #225 for PowerShell

@MathiasVP

Copy link
Copy Markdown
Collaborator

Yep, the hashes seems to all match up! If you're curious about what I did to check this we can have a chat about it if you want.

This upgrade/downgrade pair conflates two separate dbscheme changes: the removal of the Bicep extensionals from the extractor (which was done before we got ownership of the extractor), and the creation of the yaml_comments extensional. That is, this is the history:

|      A        |           B               |           C
| ------------- | ------------------------- | ------------------->
^ initial       ^ Bicep removed             ^ yaml_comments added

Since the upgrade/downgrade pair in this PR takes us all the way from A to Cany database created in periodB` won't be upgradeable/downgredeable. If that's not a problem then this PR is totally fine. But if that is a problem we need to split the upgrade/downgrade into a chain of length 2.

@chanel-y
Chanel (chanel-y) merged commit 40d658a into main Aug 12, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants