Skip to content

perf: keep the build identity out of sqlbuild-rules so edits stop rebuilding it - #1510

Merged
kvlonge merged 2 commits into
mainfrom
perf/native-build-identity
Nov 16, 2026
Merged

kvlonge merged 2 commits into
mainfrom
perf/native-build-identity

Conversation

@kvlonge

@kvlonge kvlonge commented Nov 16, 2026

Copy link
Copy Markdown
Contributor

Why

The source hash that keys local caches and rule fingerprints was a compile-time constant in sqlbuild-rules. Its build script hashes every workspace crate, so any source edit rebuilt sqlbuild-rules, then sqlbuild-python-base, sqlbuild-cli and sqlbuild-python. The constant was also inlined into a few extension codegen units that changed on every build, and thin-local LTO then re-optimised about 100 of 256 objects for a one-line edit. Each later binding-crate split would add to it.

Changes

  • New leaf crate sqlbuild-build-identity: the unchanged build script plus one #[inline(never)] accessor. Only sqlbuild-cli depends on it; sqlbuild-python reaches it through sqlbuild-cli.
  • sqlbuild-rules holds the identity in a process-wide cell. sqb (run_launcher) and the _native module install it at startup. Reading it before then panics with the reason, never a default; empty or conflicting installs are refused.
  • Cache keys, rule fingerprints and _native.BUILD_IDENTITY keep the same value.
  • Tests that read the identity install a fixed test identity; a fresh-process test proves a missing identity fails.

No DuckShop case applies: only how the build identity reaches the rules engine changes, not compiler behaviour.

Verification

Incremental rebuild after a one-literal edit in sqlbuild-python (cargo build -p sqlbuild-python --profile fast), main and this branch alternating on one 8-core machine, 3 edit locations × 2 runs, load 3.7–5.5:

main this branch
median wall 46.5 s 31.3 s
median user CPU 194 s 61 s
_native objects rebuilt 100 50

cargo -v now rebuilds only sqlbuild-build-identity, sqlbuild-cli and sqlbuild-python.

  • Tests of sqlbuild-rules, sqlbuild-build-identity, sqlbuild-cli, sqlbuild-python lib; rule engine and macro bridge Python tests.
  • fensu check, layering, build-features and boundary checks, clippy; make test and make check-ci.

…uilding it

The source hash that keys local caches was a compile-time constant in
sqlbuild-rules, whose build script hashes every workspace crate. Any source
edit therefore rebuilt sqlbuild-rules and every crate above it, and the
constant changed the same codegen units of the extension on every build.

The hash now lives in a leaf crate, sqlbuild-build-identity, behind one
non-inlined accessor. sqb and the extension install it into sqlbuild-rules
at startup through sqlbuild-cli; reading it before then fails instead of
falling back. Cache keys and BUILD_IDENTITY keep the same value.
@kvlonge
kvlonge enabled auto-merge November 16, 2026 05:59
@kvlonge
kvlonge added this pull request to the merge queue Nov 16, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Nov 16, 2026
@kvlonge
kvlonge added this pull request to the merge queue Nov 16, 2026
Merged via the queue into main with commit 2172ec1 Nov 16, 2026
32 checks passed
@kvlonge
kvlonge deleted the perf/native-build-identity branch November 16, 2026 08:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant