[prrao87]

Why

lance-csr's engine could only be used through Python. The Rust crate had the parser and planner, but the part that makes lance-csr fast (the cached snapshot, CSR adjacencies, node indexes, join-key statistics and refresh()) lived inside the PyO3 bindings, tied to the GIL and Python exceptions. A Rust program, such as a desktop app backend, had to rebuild all of that by hand.

This PR makes CypherEngine a Rust library API and prepares the crates for crates.io and the 0.3.0 release. Upstream LanceDB owns lance-graph and lance-graph-catalog on crates.io, so the crates are also renamed to lance-csr*.

What changed

Rust engine. lance_csr::CypherEngine is now a public async API with the same behaviour as the Python class:

let engine = CypherEngine::new(config, sources, true).await?;
let batch = engine.execute(query, params).await?;
engine.refresh().await?;

• Inputs are EngineSource::open_lance(uri, version, storage_options) or EngineSource::Arrow(batch).
• execute, execute_stream, explain, execute_with_vector_rerank, refresh, csr_stats, node_index_stats and dataset_versions return Rust types and GraphError.
• The code moved from the bindings with block_on calls replaced by .await; the snapshot, CSR delta and statistics logic is unchanged.

Python bindings. CypherEngine in Python is now a thin wrapper: it converts Python inputs, calls the Rust engine and converts results. Its API, behaviour and exception types are unchanged. Errors map to the same ValueError, NotImplementedError and RuntimeError as before.

Concurrent refresh() fix. This fixes issue #12, "Prevent concurrent refreshes from publishing stale snapshots": overlapping refreshes could publish an older snapshot after a newer one. refresh() now holds an async lock for the whole operation, so concurrent calls run one at a time and each publishes the latest data. Queries never wait for it. A deterministic regression test covers it (see Validation).

Fixes #12

Crate rename. lance-graph → lance-csr, lance-graph-catalog → lance-csr-catalog, plus the unpublished benches and Python binding crates. Manifests point at this repository, and the published crates ship LICENSE and NOTICE. The Python package name and import path don't change.

Docs and examples.

• A new "Using from Rust" guide page, with links from getting started, the home page and the API reference.
• Every Rust snippet on that page is included from examples/embedded_query.rs (enabled through pymdownx.snippets), which CI now runs.
• The crate README, which becomes the crates.io page, leads with the engine.
• The benchmarks page says all numbers were measured through the Python package.

Version. 0.2.0 → 0.3.0 across the crates and the Python package.

CSR expansion memory (#11) is not addressed here; it needs a change to the expansion operator itself.

Validation

• cargo fmt --check, cargo clippy --workspace --all-targets -- -D warnings (and in the Python crate) pass.
• cargo test --workspace passes, including new engine tests:
  • Arrow input with CSR
  • parameters and streaming
  • Lance pinning and refresh()
  • configuration errors
  • a deterministic race test for Prevent concurrent refreshes from publishing stale snapshots #12 that holds the refresh lock and checks a second refresh waits and then publishes the newest version (confirmed failing with the lock removed)
• cargo run -p lance-csr --example embedded_query prints the documented output.
• In python/: make install, make test (332 passed) and make lint pass on the 0.3.0 build, and docs/check_snippets.py passes.
• zensical build --strict and typos pass. cargo publish --workspace --dry-run packages and verifies both crates at 0.3.0; nothing has been published.
• cargo doc reports three broken intra-doc links that predate this PR; nothing new.

Benchmarks

I compared the published 0.2.0 wheel from PyPI with a release wheel of this branch on both suites, using the published settings (Apple M5, macOS, 5+ rounds). Baseline and branch runs alternated. The table gives the geometric mean of the per-query median ratio (branch / 0.2.0, below 1 is faster):

Suite	In memory	Lazy
30-query (GDC SNB SF1)	0.987	1.022, rerun branch-first: 0.996
IC1–IC14 (GDC Interactive)	0.924	1.037, rerun branch-first: 1.010

All 88 result assertions passed on both sides. The lazy runs leaned slower when the baseline ran first and evened out when the order was reversed, so I read them as noise. engine_cache.py (20 tables) is flat: construction 78.0 → 76.8 ms, warm query 1.87 → 1.87 ms, unchanged refresh 1.14 → 1.15 ms. Published benchmark tables are unchanged.

Not run: the release wheel workflow on Linux.

🤖 Generated with Claude Code

[cloudflare-workers-and-pages]

Deploying lance-csr with    Cloudflare Pages

Latest commit:	9f28927
Status:	✅  Deploy successful!
Preview URL:	https://d268ab7f.lance-csr.pages.dev
Branch Preview URL:	https://chore-rename-crates-lance-cs.lance-csr.pages.dev

View logs