Skip to content

refactor: move the shared runtime of the Python bindings into sqlbuild-python-base - #1514

Merged
kvlonge merged 2 commits into
mainfrom
refactor/python-base-infra
Nov 16, 2026
Merged

kvlonge merged 2 commits into
mainfrom
refactor/python-base-infra

Conversation

@kvlonge

@kvlonge kvlonge commented Nov 16, 2026

Copy link
Copy Markdown
Contributor

Why

sqlbuild-python is still far above the crate size limit. This second slice moves the shared runtime the command bindings stand on into sqlbuild-python-base, the bottom binding layer, and consolidates the duplicate clusters those files belonged to.

Changes

  • Moved into sqlbuild-python-base (about 8.5k code lines): boundary conversions, native classes, host calls and threads, compile context, diagnostics, event exporting, frontier, cache, CLI documents, progress, result output, warehouse access and presentation text.
  • sqlbuild-python imports from the base directly, with no shims; a script rewrote the imports and the paths in tests/native_field_fallbacks.txt.
  • Duplicates consolidated: the per-module Python error constructors now share new_plain_error and host_error, the UTF-16 conversions share one helper, and the context variables share context_variable; six [dupes] entries are removed. Three clusters the move newly exposes are listed with reasons.
  • Caps lowered: bindings crate 206,367 → 197,887 code lines, helpers container 101 → 98 buckets.

No DuckShop case applies: only where binding code lives changes, not compiler behaviour.

Verification

  • _native surface (names, kinds, signatures, class members) is identical to main.
  • Dense 3,000-model compile against main, alternating on one 8-core machine (load 3.2–3.6): compile JSON and all 11,175 compiled files identical. Cold, median of 7: wall 9.02 → 8.69 s, CPU 28.06 → 27.43 s. Warm and one-model edit, median of 3: wall 0.61 → 0.61 s and 1.84 → 1.78 s. Neutral.
  • Incremental rebuild after a one-literal edit in sqlbuild-python (--profile fast), 3 locations × 2 runs alternating, load 3.4–6.5: median wall 39.5 → 36.4 s, user CPU 90 → 100 s. Neutral.
  • Crate tests, script tests, fensu check, fensu dupes --fail-on-new, make check-ci; make test fails only environment-bound tests that fail on main here too.

…d-python-base

The native store, compile context, diagnostics, progress and terminal text,
warehouse access, orchestration reads and the other infra-layer helper modules
move into the base crate, one area per helper bucket. sqlbuild-python imports
their entry points directly; the extension module surface is unchanged.
@kvlonge
kvlonge enabled auto-merge November 16, 2026 08:04
@kvlonge
kvlonge added this pull request to the merge queue Nov 16, 2026
@kvlonge
kvlonge removed this pull request from the merge queue due to a manual request Nov 16, 2026
@kvlonge
kvlonge added this pull request to the merge queue Nov 16, 2026
Merged via the queue into main with commit b3392ad Nov 16, 2026
32 checks passed
@kvlonge
kvlonge deleted the refactor/python-base-infra branch November 16, 2026 09:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant