Skip to content

Commit de6301b

Browse files
committed
refactor: move the shared runtime of the Python bindings into sqlbuild-python-base
The native store, compile context, diagnostics, progress and terminal text, warehouse access, orchestration reads and the other infra-layer helper modules move into the base crate, one area per helper bucket. sqlbuild-python imports their entry points directly; the extension module surface is unchanged.
1 parent 2172ec1 commit de6301b

615 files changed

Lines changed: 7899 additions & 4973 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎Cargo.lock‎

Lines changed: 9 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎crates/README.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,7 @@ tested and benchmarked with `cargo test` alone.
2828
| `sqlbuild-refactor` | Native rename/move plans, contract-adoption edits, MODEL/SCHEMA/YAML text edits and atomic staging/commit with rollback. Shared scope placement delegates to the native scope implementation; user Python source inspection remains at the Python AST boundary. |
2929
| `sqlbuild-project` | The compiled project as plain Rust: `CompiledProject` with the producers' facts and each resource's records (model inputs and schemas, compiled models and their analysis, sources, functions, audits and seeds), and what those records answer without Python, such as lineage rows, resolved namespaces and dependency keys. Values the host owns, such as user config and discovered files, and the views it builds lazily live in one slot per record, whose types `sqlbuild-python` supplies through the `ProjectHost` trait; reads that need Python stay in the bindings as extension traits. |
3030
| `sqlbuild-cli` | The `sqb` and `sqlbuild` executables: `--version`, unchanged compiles replayed from the shared reuse store with Python's exact streams, exit code and compute log capture, and a handoff of every other invocation to the installed `sqlbuild-python` script. Built as its own wheel (`crates/sqlbuild-cli/pyproject.toml`). |
31-
| `sqlbuild-python-base` | The base of the PyO3 bindings: native record, enum and error classes with their specs, registration, construction and field reads; keyword calls, imports, conversions and panics at the Python boundary; host entry calls; and the process allocator. |
31+
| `sqlbuild-python-base` | The base of the PyO3 bindings: native record, enum and error classes with their specs, registration, construction and field reads; keyword calls, imports, conversions and panics at the Python boundary; host entry calls; the process allocator; and the shared runtime the binding modules build on: the native store, compile context, diagnostics, progress and terminal text, warehouse access and orchestration reads. |
3232
| `sqlbuild-python` | The `_native` module: its Python classes and functions and the conversions from Python objects, on top of `sqlbuild-python-base`. |
3333

3434
## Layering
@@ -67,7 +67,7 @@ entry carries a version other than its manifest's.
6767
`main/` entry points, `models.rs`, `types.rs` and `constants.rs`, not through `_helpers/`.
6868

6969
Inside `sqlbuild-python`, the custom rule XSB075 orders the `bindings/_helpers` modules in layers
70-
that follow the planned crate split: shared Python plumbing (`infra`, part of which is already
70+
that follow the planned crate split: shared Python plumbing (`infra`, which is almost all in
7171
`sqlbuild-python-base`, which Cargo keeps below every other layer), then the glue of the lower
7272
crates (`front`), the compiled project (`project`), planning and compilation as siblings (`plan`,
7373
`compile`), command output (`output`), command handlers (`commands`) and registration
@@ -89,7 +89,7 @@ empty: every import points down, and a new upward import fails the check.
8989
- module paths are never built at runtime: a `format!` or `concat!` template may not start with
9090
`sqlbuild.` or with a `{..PACKAGE..}.`/`{..MODULE..}.` constant. Write the complete module
9191
path as a constant instead;
92-
- every row of the host table (`bindings/_helpers/native_host/`) must resolve as an attribute of
92+
- every row of the host table (`entries` in `sqlbuild-host`) must resolve as an attribute of
9393
`sqlbuild._native_host`.
9494

9595
Staying-Python entry points (adapter, user code, executor and runtime, third-party and public

‎crates/sqlbuild-python-base/Cargo.toml‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,15 +8,24 @@ publish.workspace = true
88
rust-version.workspace = true
99

1010
[dependencies]
11+
chrono.workspace = true
1112
mimalloc.workspace = true
1213
pyo3.workspace = true
14+
rusqlite.workspace = true
15+
rustc-hash.workspace = true
16+
serde.workspace = true
1317
serde_json.workspace = true
18+
sha2.workspace = true
1419
sqlbuild-analysis.workspace = true
20+
sqlbuild-cache.workspace = true
1521
sqlbuild-config.workspace = true
1622
sqlbuild-core.workspace = true
1723
sqlbuild-discovery.workspace = true
1824
sqlbuild-host.workspace = true
25+
sqlbuild-model-config.workspace = true
26+
sqlbuild-orchestration.workspace = true
1927
sqlbuild-rules.workspace = true
28+
sqlbuild-state.workspace = true
2029

2130
[lints]
2231
workspace = true

‎crates/sqlbuild-python-base/src/boundary/_helpers/mod.rs‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,3 +9,4 @@ pub(crate) mod oracles;
99
pub(crate) mod panics;
1010
mod positional_records;
1111
pub(crate) mod python_calls;
12+
pub(crate) mod text_units;
Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
//! A Python `str` as UTF-16 code units, which keeps lone surrogates `to_str` rejects.
2+
3+
use pyo3::prelude::{Bound, PyAnyMethods, PyResult};
4+
use pyo3::types::{PyBytes, PyBytesMethods, PyString};
5+
6+
/// The UTF-16 code units of `text`, lone surrogates included.
7+
pub(crate) fn utf16_units(text: &Bound<'_, PyString>) -> PyResult<Vec<u16>> {
8+
let encoded: Bound<'_, PyBytes> = text
9+
.call_method1("encode", ("utf-16-le", "surrogatepass"))?
10+
.downcast_into::<PyBytes>()?;
11+
Ok(encoded
12+
.as_bytes()
13+
.as_chunks::<2>()
14+
.0
15+
.iter()
16+
.map(|pair| u16::from_le_bytes([pair[0], pair[1]]))
17+
.collect())
18+
}

‎crates/sqlbuild-python-base/src/boundary/main/mod.rs‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,5 +11,6 @@ pub mod native_compiler_error;
1111
pub mod register_expected_errors;
1212
pub mod register_native_compiler_error;
1313
pub mod register_oracles;
14+
pub mod utf16_units;
1415
pub mod value_error;
1516
pub mod within;
Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
//! The UTF-16 code units of a Python `str`, lone surrogates included.
2+
3+
use crate::boundary::_helpers::text_units;
4+
use pyo3::prelude::{Bound, PyResult};
5+
use pyo3::types::PyString;
6+
7+
/// The UTF-16 code units of a Python `str`, lone surrogates included.
8+
#[inline]
9+
pub fn utf16_units(text: &Bound<'_, PyString>) -> PyResult<Vec<u16>> {
10+
text_units::utf16_units(text)
11+
}
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
pub(crate) mod support;

crates/sqlbuild-python/src/bindings/_helpers/build_command/support.rs renamed to crates/sqlbuild-python-base/src/build_command/_helpers/support.rs

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ use pyo3::types::{
66
};
77
use pyo3::{IntoPyObject, pyclass, pymethods};
88

9-
use sqlbuild_python_base::boundary::main::imported::imported;
9+
use crate::boundary::main::imported::imported;
1010

1111
const DATETIME_MODULE: &str = "datetime";
1212

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
//! Boolean.
2+
3+
use crate::build_command::_helpers::support;
4+
use pyo3::prelude::Bound;
5+
use pyo3::prelude::PyAny;
6+
use pyo3::prelude::Python;
7+
8+
/// Boolean.
9+
#[inline]
10+
pub fn boolean(py: Python<'_>, value: bool) -> Bound<'_, PyAny> {
11+
support::boolean(py, value)
12+
}

0 commit comments

Comments
 (0)