Skip to content

feat(cypher): run the official ldbc ic queries as opencypher text - #2

Merged
prrao87 merged 5 commits into
mainfrom
feat/ic-full-cypher
Oct 24, 2026
Merged

prrao87 merged 5 commits into
mainfrom
feat/ic-full-cypher

Conversation

@prrao87

@prrao87 prrao87 commented Oct 24, 2026

Copy link
Copy Markdown
Member

Why

lance-csr 0.1.0 could answer all 14 LDBC Interactive complex (IC) queries, but only through hand-written ports that avoided syntax it lacked. This PR implements that syntax so the official IC Cypher runs as written. Features follow openCypher's standardisation scope rather than one vendor's dialect.

13 of the 14 queries now run from the unmodified official text (as adapted to the SF1 schema by ldbc-suite-test). IC14's text uses reduce(), which the scope excludes, so it runs as a port built only from in-scope constructs (UNWIND, range, nodes(path)[i], pattern comprehensions, sum).

What changed

Queries and parameters

  • CypherEngine.execute, execute_batches and explain take params for $name parameters, including datetime and date values.
  • Chained comparisons: a < b <= c means a < b AND b <= c.
  • Inline properties may read a variable: (t:Tag {id: tagId}).
  • An unlabeled node gets the label the configured endpoint labels pin down, so (friend)<-[:postHasCreator]-(post) knows post is a Post.

Nodes and lists as values

  • A node used as a value is its id and a path is its length. So a = b, a IN [x, y], count(DISTINCT a), uni IS NULL and path IS NULL behave as in Cypher.
  • collect(node) builds a list of nodes, and UNWIND of such a list gives nodes again.
  • x IN list works for any list expression, not only literals.

Expressions

  • Predicates as values (NOT (a)-[:knows]-(b) AS isNew), map literals (a node inside a map keeps its properties), and value.field on maps and dates (latestLike.msg.ID, birthday.month).
  • head, toInteger, toFloat, date, datetime and duration.inSeconds. List literals that mix types become lists of strings, since Arrow lists hold one type.
  • Aggregates inside expressions, such as head(collect(...)). collect of nothing gives [], and collect keeps the order of a preceding WITH ... ORDER BY.
  • List comprehensions [x IN list WHERE p | v], including pattern predicates on the element, and pattern comprehensions [(a)-[:R]->(b) WHERE p | v].

Scoping fixes

  • Semantic analysis now ends variable scope at WITH, as the planner already did.
  • A pattern after WITH may start at a node no earlier clause bound.
  • An OPTIONAL MATCH whose WHERE reads a list carried by WITH now plans.
  • Variable-length paths kept only their end nodes' columns and dropped everything else from earlier clauses. They now keep all of it.

Planning

  • EXISTS checks, pattern comprehensions and OPTIONAL MATCH seeds in one projection used to copy the rows built by the previous ones, so several of them grew the plan exponentially. They now all start from the original rows. IC14 goes from 641 to 225 physical operators.

Validation

  • Rust: cargo test --workspace passes, and cargo clippy --workspace --all-targets -- -D warnings and cargo fmt --check are clean.
  • Python: 329 tests pass (49 new cases for this syntax, each with and without the CSR); ruff is clean.
  • LDBC SF1, golden answers: all 14 IC queries match in all three modes (lazy CSR, in-memory CSR, in-memory joins), on debug and release builds.
  • Release build, two passes on macOS (Apple M5):
    • The 30-query suite is unchanged against 0.1.0, within 2.4% (geometric mean) in each mode.
    • The IC queries as openCypher text, compared with Neo4j 2026.09 and Kuzu 0.11.3 (geometric mean over all 14; Neo4j and Kuzu numbers are from an earlier run on the same machine):
lance-csr mode vs Neo4j wins vs Neo4j vs Kuzu wins vs Kuzu
in-memory, CSR 1.96× faster 9 of 14 4.36× faster 10 of 14
in-memory, joins 1.27× faster 8 of 14 2.82× faster 9 of 14
lazy, CSR 0.82× 4 of 14 1.83× faster 7 of 14
  • Known gaps, left for follow-up work:
    • IC12's unbounded *0.. alternation unrolls into 20 branches, and planning them dominates its run time.
    • IC1, IC7 and IC13 remain slower than Neo4j.
    • Integer list literals such as [1, 2, 3] still parse as float vectors (the inherited vector-search syntax).

🤖 Generated with Claude Code

…erred labels

Run more of the official LDBC IC Cypher unmodified: node comparison and
IN on ids, collect(node) and UNWIND back to nodes, IN over any list
expression, chained comparisons, inline properties that read variables,
labels inferred from relationship endpoints, mixed-type list literals,
aggregates nested in expressions, and $parameters on CypherEngine.
Predicates as values (path IS NULL, NOT (pattern) AS isNew), map literals
whose node values keep their properties, value.field on maps and
temporal values, head/toInteger/toFloat/date/datetime and
duration.inSeconds, and collect() in the order of a preceding WITH ...
ORDER BY.
[x IN list WHERE p | v] unnests each row's list with positions (keeping
rows whose list is empty), evaluates the predicate (pattern predicates
included) and value, and collects back in order. [(a)-[:R]->(b) WHERE p
| v] collects per binding of the row's variables, like an OPTIONAL
MATCH, and is [] without matches.
… rows

Each mark or comprehension planned its seed from the rows with the
previous ones already joined on, so k of them in one projection copied
the input 2^k times. Seeding all of them from the rows the projection
or WHERE started from keeps the copies linear: LDBC IC14 drops from 641
to 225 physical operators and from 188 to 62 ms.
@prrao87
prrao87 marked this pull request as ready for review October 24, 2026 14:56
@prrao87
prrao87 merged commit ddb6430 into main Oct 24, 2026
6 checks passed
@prrao87
prrao87 deleted the feat/ic-full-cypher branch October 24, 2026 14:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant