Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
9 changes: 9 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 3 additions & 3 deletions crates/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ tested and benchmarked with `cargo test` alone.
| `sqlbuild-refactor` | Native rename/move plans, contract-adoption edits, MODEL/SCHEMA/YAML text edits and atomic staging/commit with rollback. Shared scope placement delegates to the native scope implementation; user Python source inspection remains at the Python AST boundary. |
| `sqlbuild-project` | The compiled project as plain Rust: `CompiledProject` with the producers' facts and each resource's records (model inputs and schemas, compiled models and their analysis, sources, functions, audits and seeds), and what those records answer without Python, such as lineage rows, resolved namespaces and dependency keys. Values the host owns, such as user config and discovered files, and the views it builds lazily live in one slot per record, whose types `sqlbuild-python` supplies through the `ProjectHost` trait; reads that need Python stay in the bindings as extension traits. |
| `sqlbuild-cli` | The `sqb` and `sqlbuild` executables: `--version`, unchanged compiles replayed from the shared reuse store with Python's exact streams, exit code and compute log capture, and a handoff of every other invocation to the installed `sqlbuild-python` script. Built as its own wheel (`crates/sqlbuild-cli/pyproject.toml`). |
| `sqlbuild-python-base` | The base of the PyO3 bindings: native record, enum and error classes with their specs, registration, construction and field reads; keyword calls, imports, conversions and panics at the Python boundary; host entry calls; and the process allocator. |
| `sqlbuild-python-base` | The base of the PyO3 bindings: native record, enum and error classes with their specs, registration, construction and field reads; keyword calls, imports, conversions and panics at the Python boundary; host entry calls; the process allocator; and the shared runtime the binding modules build on: the native store, compile context, diagnostics, progress and terminal text, warehouse access and orchestration reads. |
| `sqlbuild-python` | The `_native` module: its Python classes and functions and the conversions from Python objects, on top of `sqlbuild-python-base`. |

## Layering
Expand Down Expand Up @@ -67,7 +67,7 @@ entry carries a version other than its manifest's.
`main/` entry points, `models.rs`, `types.rs` and `constants.rs`, not through `_helpers/`.

Inside `sqlbuild-python`, the custom rule XSB075 orders the `bindings/_helpers` modules in layers
that follow the planned crate split: shared Python plumbing (`infra`, part of which is already
that follow the planned crate split: shared Python plumbing (`infra`, which is almost all in
`sqlbuild-python-base`, which Cargo keeps below every other layer), then the glue of the lower
crates (`front`), the compiled project (`project`), planning and compilation as siblings (`plan`,
`compile`), command output (`output`), command handlers (`commands`) and registration
Expand All @@ -89,7 +89,7 @@ empty: every import points down, and a new upward import fails the check.
- module paths are never built at runtime: a `format!` or `concat!` template may not start with
`sqlbuild.` or with a `{..PACKAGE..}.`/`{..MODULE..}.` constant. Write the complete module
path as a constant instead;
- every row of the host table (`bindings/_helpers/native_host/`) must resolve as an attribute of
- every row of the host table (`entries` in `sqlbuild-host`) must resolve as an attribute of
`sqlbuild._native_host`.

Staying-Python entry points (adapter, user code, executor and runtime, third-party and public
Expand Down
9 changes: 9 additions & 0 deletions crates/sqlbuild-python-base/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -8,15 +8,24 @@ publish.workspace = true
rust-version.workspace = true

[dependencies]
chrono.workspace = true
mimalloc.workspace = true
pyo3.workspace = true
rusqlite.workspace = true
rustc-hash.workspace = true
serde.workspace = true
serde_json.workspace = true
sha2.workspace = true
sqlbuild-analysis.workspace = true
sqlbuild-cache.workspace = true
sqlbuild-config.workspace = true
sqlbuild-core.workspace = true
sqlbuild-discovery.workspace = true
sqlbuild-host.workspace = true
sqlbuild-model-config.workspace = true
sqlbuild-orchestration.workspace = true
sqlbuild-rules.workspace = true
sqlbuild-state.workspace = true

[lints]
workspace = true
1 change: 1 addition & 0 deletions crates/sqlbuild-python-base/src/boundary/_helpers/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -9,3 +9,4 @@ pub(crate) mod oracles;
pub(crate) mod panics;
mod positional_records;
pub(crate) mod python_calls;
pub(crate) mod text_units;
18 changes: 18 additions & 0 deletions crates/sqlbuild-python-base/src/boundary/_helpers/text_units.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
//! A Python `str` as UTF-16 code units, which keeps lone surrogates `to_str` rejects.

use pyo3::prelude::{Bound, PyAnyMethods, PyResult};
use pyo3::types::{PyBytes, PyBytesMethods, PyString};

/// The UTF-16 code units of `text`, lone surrogates included.
pub(crate) fn utf16_units(text: &Bound<'_, PyString>) -> PyResult<Vec<u16>> {
let encoded: Bound<'_, PyBytes> = text
.call_method1("encode", ("utf-16-le", "surrogatepass"))?
.downcast_into::<PyBytes>()?;
Ok(encoded
.as_bytes()
.as_chunks::<2>()
.0
.iter()
.map(|pair| u16::from_le_bytes([pair[0], pair[1]]))
.collect())
}
1 change: 1 addition & 0 deletions crates/sqlbuild-python-base/src/boundary/main/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -11,5 +11,6 @@ pub mod native_compiler_error;
pub mod register_expected_errors;
pub mod register_native_compiler_error;
pub mod register_oracles;
pub mod utf16_units;
pub mod value_error;
pub mod within;
11 changes: 11 additions & 0 deletions crates/sqlbuild-python-base/src/boundary/main/utf16_units.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
//! The UTF-16 code units of a Python `str`, lone surrogates included.

use crate::boundary::_helpers::text_units;
use pyo3::prelude::{Bound, PyResult};
use pyo3::types::PyString;

/// The UTF-16 code units of a Python `str`, lone surrogates included.
#[inline]
pub fn utf16_units(text: &Bound<'_, PyString>) -> PyResult<Vec<u16>> {
text_units::utf16_units(text)
}
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
pub(crate) mod support;
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ use pyo3::types::{
};
use pyo3::{IntoPyObject, pyclass, pymethods};

use sqlbuild_python_base::boundary::main::imported::imported;
use crate::boundary::main::imported::imported;

const DATETIME_MODULE: &str = "datetime";

Expand Down
12 changes: 12 additions & 0 deletions crates/sqlbuild-python-base/src/build_command/main/boolean.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
//! Boolean.

use crate::build_command::_helpers::support;
use pyo3::prelude::Bound;
use pyo3::prelude::PyAny;
use pyo3::prelude::Python;

/// Boolean.
#[inline]
pub fn boolean(py: Python<'_>, value: bool) -> Bound<'_, PyAny> {
support::boolean(py, value)
}
12 changes: 12 additions & 0 deletions crates/sqlbuild-python-base/src/build_command/main/display_text.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
//! A string, integer or float shown as text; anything else is unset.

use crate::build_command::_helpers::support;
use pyo3::prelude::Bound;
use pyo3::prelude::PyAny;
use pyo3::prelude::PyResult;

/// A string, integer or float shown as text; anything else is unset.
#[inline]
pub fn display_text(value: &Bound<'_, PyAny>) -> PyResult<Option<String>> {
support::display_text(value)
}
12 changes: 12 additions & 0 deletions crates/sqlbuild-python-base/src/build_command/main/emit.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
//! Write `value` to the stream and flush it.

use crate::build_command::_helpers::support;
use pyo3::prelude::Bound;
use pyo3::prelude::PyAny;
use pyo3::prelude::PyResult;

/// Write `value` to the stream and flush it.
#[inline]
pub fn emit(stream: &Bound<'_, PyAny>, value: &str) -> PyResult<()> {
support::emit(stream, value)
}
16 changes: 16 additions & 0 deletions crates/sqlbuild-python-base/src/build_command/main/frozenset.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
//! Frozenset.

use crate::build_command::_helpers::support;
use pyo3::prelude::Bound;
use pyo3::prelude::PyAny;
use pyo3::prelude::PyResult;
use pyo3::prelude::Python;

/// Frozenset.
#[inline]
pub fn frozenset<'py>(
py: Python<'py>,
values: &[Bound<'py, PyAny>],
) -> PyResult<Bound<'py, PyAny>> {
support::frozenset(py, values)
}
12 changes: 12 additions & 0 deletions crates/sqlbuild-python-base/src/build_command/main/joined.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
//! `base / part`.

use crate::build_command::_helpers::support;
use pyo3::prelude::Bound;
use pyo3::prelude::PyAny;
use pyo3::prelude::PyResult;

/// `base / part`.
#[inline]
pub fn joined<'py>(base: &Bound<'py, PyAny>, parts: &[&str]) -> PyResult<Bound<'py, PyAny>> {
support::joined(base, parts)
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
//! A relay calling `target` with `names` as keywords.

use crate::build_command::_helpers::support;
use pyo3::prelude::Bound;
use pyo3::prelude::PyAny;
use pyo3::prelude::PyResult;

/// A relay calling `target` with `names` as keywords.
#[inline]
pub fn keyword_relay<'py>(
target: Bound<'py, PyAny>,
names: &[&'static str],
) -> PyResult<Bound<'py, PyAny>> {
support::keyword_relay(target, names)
}
12 changes: 12 additions & 0 deletions crates/sqlbuild-python-base/src/build_command/main/line_writer.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
//! A callable writing each message as a line on `stream`.

use crate::build_command::_helpers::support;
use pyo3::prelude::Bound;
use pyo3::prelude::PyAny;
use pyo3::prelude::PyResult;

/// A callable writing each message as a line on `stream`.
#[inline]
pub fn line_writer<'py>(stream: &Bound<'py, PyAny>) -> PyResult<Bound<'py, PyAny>> {
support::line_writer(stream)
}
12 changes: 12 additions & 0 deletions crates/sqlbuild-python-base/src/build_command/main/mod.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
pub mod boolean;
pub mod display_text;
pub mod emit;
pub mod frozenset;
pub mod joined;
pub mod keyword_relay;
pub mod line_writer;
pub mod none;
pub mod or_none;
pub mod text;
pub mod tuple;
pub mod utc_now;
12 changes: 12 additions & 0 deletions crates/sqlbuild-python-base/src/build_command/main/none.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
//! None.

use crate::build_command::_helpers::support;
use pyo3::prelude::Bound;
use pyo3::prelude::PyAny;
use pyo3::prelude::Python;

/// None.
#[inline]
pub fn none(py: Python<'_>) -> Bound<'_, PyAny> {
support::none(py)
}
15 changes: 15 additions & 0 deletions crates/sqlbuild-python-base/src/build_command/main/or_none.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
//! The value, or `default` when it is `None`.

use crate::build_command::_helpers::support;
use pyo3::prelude::Bound;
use pyo3::prelude::PyAny;
use pyo3::prelude::PyResult;

/// The value, or `default` when it is `None`.
#[inline]
pub fn or_none<'py>(
value: Bound<'py, PyAny>,
default: impl FnOnce() -> PyResult<Bound<'py, PyAny>>,
) -> PyResult<Bound<'py, PyAny>> {
support::or_none(value, default)
}
12 changes: 12 additions & 0 deletions crates/sqlbuild-python-base/src/build_command/main/text.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
//! Text.

use crate::build_command::_helpers::support;
use pyo3::prelude::Bound;
use pyo3::prelude::PyAny;
use pyo3::prelude::Python;

/// Text.
#[inline]
pub fn text<'py>(py: Python<'py>, value: &str) -> Bound<'py, PyAny> {
support::text(py, value)
}
13 changes: 13 additions & 0 deletions crates/sqlbuild-python-base/src/build_command/main/tuple.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
//! Tuple.

use crate::build_command::_helpers::support;
use pyo3::prelude::Bound;
use pyo3::prelude::PyAny;
use pyo3::prelude::PyResult;
use pyo3::prelude::Python;

/// Tuple.
#[inline]
pub fn tuple<'py>(py: Python<'py>, values: &[Bound<'py, PyAny>]) -> PyResult<Bound<'py, PyAny>> {
support::tuple(py, values)
}
13 changes: 13 additions & 0 deletions crates/sqlbuild-python-base/src/build_command/main/utc_now.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
//! `datetime.now(UTC)`.

use crate::build_command::_helpers::support;
use pyo3::prelude::Bound;
use pyo3::prelude::PyAny;
use pyo3::prelude::PyResult;
use pyo3::prelude::Python;

/// `datetime.now(UTC)`.
#[inline]
pub fn utc_now(py: Python<'_>) -> PyResult<Bound<'_, PyAny>> {
support::utc_now(py)
}
4 changes: 4 additions & 0 deletions crates/sqlbuild-python-base/src/build_command/mod.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
//! Python value helpers and callback relays the build command shares.

pub(crate) mod _helpers;
pub mod main;
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
pub(crate) mod progress;
pub(crate) mod terminal;
pub(crate) mod transient_lines;
pub(crate) mod transient_status;
Loading
Loading