You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Shrink plans for comprehension-heavy queries such as IC10 #18
Queries built from pattern comprehensions, OPTIONAL MATCH, collect() and chained WITH produce very large physical plans. Two of them, IC10 and IC1, are where lance-csr trails other engines most; IC14 has the same plan growth, although it is already faster than the other engines.
Evidence
Physical operator counts on SF1, from explain():
Query
Operators
Notable
lance-csr in memory
Fastest other engine
IC10
217
38 AggregateExec, 32 CsrExpandExec
105.0 ms
13.7 ms (Ladybug)
IC14
225
49 RepartitionExec
64.4 ms
522.6 ms (Neo4j)
IC1
164
20 AggregateExec
26.8 ms
13.5 ms (Kuzu)
IC10 collects each friend's posts and counts those that share a tag with the person, with size([p IN posts WHERE (p)-[:postHasTag]->()<-[:hasInterest]-(person)]). An earlier change (seeding comprehensions from the original rows) cut IC14 from 641 to 225 operators and from 188 to 62 ms, so there is precedent for large wins here.
Suggested fix
Start with IC10: read its logical plan and find where the comprehension and the OPTIONAL MATCH ... collect() are planned as separate joins and aggregations over the same rows. Candidates are evaluating a pattern predicate inside a list comprehension as a semi-join over the unnested list, and merging aggregations that group by the same keys.
Acceptance criteria
IC10's operator count and latency before and after, in the pull request, with results unchanged.
No regression on the other IC queries or the 30-query workload.
Any new planning rule has focused tests.
Identified by profiling the LDBC IC benchmark; IC10's plan has not been analysed in detail.
Partial progress from #38, which counts size([x IN list WHERE ...]) without building the list. On current main (Apple M5, SF1, in memory), IC10 is down from 217 to 201 operators (AggregateExec 38 → 32, CsrExpandExec 32 → 24) and from 105.0 to 97.8 ms, still about 7x slower than Ladybug's 13.7 ms. IC1 is unchanged at 164 operators. The suggested fixes, a semi-join for the pattern predicate and merging aggregations with the same keys, are still open.
Activity
perf(cypher): count unmapped comprehensions without list aggregation (#…
feat: integrate cypher, csr and engine work for 0.3.0 (#55)
prrao87 commentedon Oct 29, 2026
Partial progress from #38, which counts
size([x IN list WHERE ...])without building the list. On currentmain(Apple M5, SF1, in memory), IC10 is down from 217 to 201 operators (AggregateExec38 → 32,CsrExpandExec32 → 24) and from 105.0 to 97.8 ms, still about 7x slower than Ladybug's 13.7 ms. IC1 is unchanged at 164 operators. The suggested fixes, a semi-join for the pattern predicate and merging aggregations with the same keys, are still open.