Skip to content

docs: state goals and scope, show the ldbc benchmark, move builds to uv - #4

Merged
prrao87 merged 5 commits into
mainfrom
docs/readme-goals
Oct 24, 2026
Merged

prrao87 merged 5 commits into
mainfrom
docs/readme-goals

Conversation

@prrao87

@prrao87 prrao87 commented Oct 24, 2026 •

Copy link
Copy Markdown
Member

Why

The README described what lance-csr adds but not what it is for, or what it will not do. Contributors need both before they open an issue or a pull request. It also still carried material from lance-graph: benchmark numbers from an old Criterion run nobody can reproduce, a link to lance-graph's DeepWiki page, and a project-structure proposal for crates that don't exist. This change states the project's goals and scope, shows the LDBC benchmark lance-csr is measured on, removes the stale upstream material, and credits the two projects lance-csr builds on.

What changed

  • A Goals section after the introduction. lance-csr targets single-machine execution, and aims to:
    • provide a fast, lightweight and portable openCypher-compatible engine on top of the Lance format;
    • enable multimodal knowledge graphs, where media and vectors live alongside the graph and path traversals run over a single copy of the data;
    • push DataFusion's execution engine for multi-hop graph traversal.
  • An out of scope note: graph queries across multiple machines are explicitly out of scope, and issues and pull requests that target distributed execution will be closed as out of scope. It points people with distributed graph workloads to HDC Labs' contact page.
  • Credits move to the end of the README and now also credit the contributors of Apache DataFusion, alongside lance-graph's maintainers and contributors.
  • Benchmarks now show the 30-query LDBC SNB SF1 suite from graph-benchmark-ldbc: lance-csr 0.1.0 in memory and lazy, next to Neo4j, Kuzu, Ladybug and lance-graph. The other engines' numbers are copied from that repository's 2026-10-16 run. lance-csr was measured on the same machine with the same settings on 2026-10-23, and the README notes that its runner is not yet in that repository. The Criterion table and its instructions are gone.
  • The DeepWiki link to lance-graph's documentation is removed.
  • The project structure document describes the actual workspace and the query pipeline, instead of lance-graph's proposal for lance-graph-core and lance-graph-planner crates.
  • The examples say lance-csr, install lance-csr from PyPI, list the snapshot and refresh example, and build with maturin develop from python/. The old maturin develop -m crates/lance-graph-python/Cargo.toml skipped the package metadata and produced a module without the lance_csr package.
  • Dependencies are managed with uv, never pip. Development tools (maturin, pytest, pandas, ruff, pyright) move from published extras into a uv dev dependency group, and uv.lock is regenerated for the 3.12 floor. Every documented setup installs with uv sync --locked --no-install-project, then makes a debug build with uv run --no-sync maturin develop. The READMEs explain both flags: uv would otherwise build the package in release mode, and a re-sync would remove the debug build. Users install with uv add lance-csr, and the examples run with uv run --with lance-csr. Building from source now lists protoc, and Rust commands run workspace-wide from the repository root.
  • The Makefile's targets work. make install, test, lint, format, wheel and clean wrap the uv commands. The old test target ran a single file through an invalid uv run --with tests, and the documented doctest target didn't exist. DEVELOPMENT.md now describes this project rather than Lance's tracing, S3 tests, vector benchmarks and manylinux2014 wheels. The contributor notes for agents follow the same commands.
  • The package metadata names the lance-csr authors instead of lance-graph's.
  • CI
    • Python Tests installs with uv sync --locked. Its doctest step is removed: it ran python -m doctest on the package's __init__.py, which can't import the package that way, and swallowed the error, so it never tested anything.
    • The publish workflow gets zig and twine through uv instead of pip, and keeps maturin-action and its sccache cache.
    • Each built wheel is now tested by its file path. Before, --find-links dist lance-csr could resolve to PyPI's copy of the same version. ci/check_wheel.py now checks that the extension loads from site-packages, because uv's environments can place packages outside sys.prefix.
    • Wheels now build on pull requests only when the wheel build changes (the publish workflow, the wheel check, python/.cargo/), not on every pyproject.toml or bindings Cargo change. Other changes rely on a manual dry_run before a release.
  • The repository layout gets its own heading. A leftover reference to an upstream lance-graph issue, and a stray fragment from an earlier edit, are removed.

Validation

  • No library code or tests change. The package metadata, the Makefile and the CI workflows do.
  • In python/, make install (uv sync --locked --no-install-project --inexact, then maturin develop), make test (280 passed) and make lint (clean) all run, and uv lock --check passes. The same commands also work in a fresh environment.
  • The publish workflow's new wheel-test step was run locally against a macOS release wheel: check_wheel.py passes, and the full suite (280 tests) passes without Rust on PATH. A manylinux_2_28 x86-64 wheel built on macOS with zig supplied only through uv (uv run --with 'maturin[zig]'). CI will run the edited publish workflow on this PR, since the workflow file changed.
  • Both workflows pass GitHub's workflow schema check.
  • The three examples run with uv run --with lance-csr against the published release.
  • The benchmark table was generated from the result files (graph-benchmark-ldbc's README table for the other engines, the two pytest-benchmark passes for lance-csr), not typed by hand.
  • Checked the section order and Markdown by reading the files; the links were not fetched.
  • This branch and feat(cypher): run the official ldbc ic queries as opencypher text #2 both edit the README, in different sections. feat(cypher): run the official ldbc ic queries as opencypher text #2 rewrites the "Much wider Cypher" bullet, which this change leaves alone, so whichever merges second needs at most a small rebase.

🤖 Generated with Claude Code

Add a Goals section (single-machine, openCypher on Lance, multimodal
knowledge graphs, DataFusion for graph traversal), mark distributed
execution as out of scope, and move the credits to the end with
Apache DataFusion's contributors alongside lance-graph's.
Replace the old Criterion numbers with lance-csr's results on
graph-benchmark-ldbc next to Neo4j, Kuzu, Ladybug and lance-graph,
remove the upstream DeepWiki link, describe the actual workspace layout,
and point the examples at lance-csr and a working build command.
@prrao87 prrao87 changed the title docs: state the project's goals and scope, credit datafusion docs: state goals and scope, show the ldbc benchmark, drop stale upstream references Oct 24, 2026
One build, maturin develop --extras tests,dev, instead of an editable
install followed by maturin develop; protoc listed as a requirement;
workspace-wide cargo commands from the repository root. The Makefile's
targets now work (test ran one file through a broken uv invocation), and
DEVELOPMENT.md describes this project instead of Lance's tracing, S3
tests, vector benchmarks and manylinux2014 wheels. The package authors
are lance-csr's, not lance-graph's.
@prrao87 prrao87 changed the title docs: state goals and scope, show the ldbc benchmark, drop stale upstream references docs: state goals and scope, show the ldbc benchmark, update build steps Oct 24, 2026
Development tools move from published extras into a uv dev dependency
group, and every documented setup, the Makefile and both Python
workflows install with uv sync --locked (relocked for the 3.12 floor).
The publish workflow gets zig and twine through uv, tests each built
wheel by path so PyPI's copy of the same version can't stand in, and
builds wheels on pull requests only when the wheel build itself
changes. The doctest step, which never imported the package, is gone.
@prrao87 prrao87 changed the title docs: state goals and scope, show the ldbc benchmark, update build steps docs: state goals and scope, show the ldbc benchmark, move builds to uv Oct 24, 2026
maturin-action runs python3 -m pip install, and the uv tool
environment's Python has no pip. The zig binary that ziglang ships is
found on PATH first, leaving python3 to the runner.
@prrao87
prrao87 marked this pull request as ready for review October 24, 2026 15:44
@prrao87
prrao87 merged commit 8c08a62 into main Oct 24, 2026
4 checks passed
@prrao87
prrao87 deleted the docs/readme-goals branch October 24, 2026 15:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant