Add IaC DB schema upgrade/downgrade scripts - #390
Conversation
The IaC extractor shipped no schema-migration scripts, so any mismatch between a database's dbscheme and a query pack's dbscheme was fatal (e.g. MRVA "database is not compatible with a QL library" errors). The dbscheme changed when Bicep support (82 bicep_* relations) was removed and yaml_comments/empty_location were added. - Add upgrade step iac/ql/lib/upgrades/e360fadd.../ (old bicep -> current): deletes the 82 bicep_* relations (compatibility: backwards). - Add downgrade step iac/downgrades/6b6bd68.../ (current -> old bicep): restores empty bicep tables, drops yaml_comments/empty_location (compatibility: partial). - Bootstrap iac/downgrades as a pack (qlpack.yml, initial/) and add iac/ql/lib/upgrades/initial/, matching go/rust/swift layout. - Teach misc/scripts/prepare-db-upgrade.sh about the iac language. - Fix iac/scripts/create-extractor-pack.ps1 (was a stale QL-for-QL copy) to mirror the .sh and ship the downgrades folder. Verified with codeql 2.26.2: upgrading an old bicep DB to the current scheme and running an IaC query succeeds; downgrading a current DB back to the old scheme succeeds. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
I think this all makes sense. I'll just take a quick look to check if we're not being bitten by Windows vs. Unix line-endings like I fixed in #225 for PowerShell |
|
Yep, the hashes seems to all match up! If you're curious about what I did to check this we can have a chat about it if you want. This upgrade/downgrade pair conflates two separate dbscheme changes: the removal of the Bicep extensionals from the extractor (which was done before we got ownership of the extractor), and the creation of the Since the upgrade/downgrade pair in this PR takes us all the way from |
The IaC extractor shipped no schema-migration scripts, so any mismatch between a database's dbscheme and a query pack's dbscheme was fatal (e.g. MRVA "database is not compatible with a QL library" errors). The dbscheme changed when Bicep support (82 bicep_* relations) was removed and yaml_comments/empty_location were added.
Verified with codeql 2.26.2: upgrading an old bicep DB to the current scheme and running an IaC query succeeds; downgrading a current DB back to the old scheme succeeds.