Repository navigation
fix(query): preserve empty-input distinct counts - #65
Merged
Merged
Conversation
Refs lance-format/lance-graph#123. Adapt the focused fix to the current 0.3.1 engine and crate layout. Submitted by David Hughes (@stugorf), with AI Collaboration.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem and change
MATCH (p:Person) WHERE p.id = 99 RETURN count(DISTINCT p.id) > 0 AS presentcan returntruewhen no row matches, because DataFusion's statistics shortcut turns the empty distinct count into 1.Construct a query-local frame that omits
aggregate_statisticsonly when the logical plan contains a distinct count. Preserve the caller session, the other optimizer rules, its query planner and the deadline context. Execution, streaming and explain share the guard. Current main already supports aggregates inside comparison/CASE expressions; this PR does not duplicate that work.Refs lance-format/lance-graph#123. This is a lance-csr fix; it does not close the upstream issue.
Validation
cargo fmt --all --check,cargo test --workspace --locked, workspace Clippy with warnings denied,make install,make test,make lint, executable documentation snippets, strict Zensical build andtypos@1.26.0.Empty and nonempty counts, booleans, CASE, WITH, duplicates, nulls, parameter reuse, output types, caller-session preservation, legacy/join/CSR execution and streaming over Arrow/Lance.
Above 1 is faster. These original suite means are retained unchanged; initial >10% per-query flags and their paired investigations are recorded in the final gate record. Both complete workloads preserve the original golden answers. These are local engineering measurements, not an audited SNB result or performance guarantee.
Measured serially after builds stopped, on Apple M4 Max (16 cores, 128 GiB), macOS arm64, Rust 1.96.0 and Python 3.13.17. The suites are pinned at
6260689(30 queries) andc185123(IC); settings, dependencies, SF1 manifests, timings and peak process RSS are recorded. Per-query before/after tables, reproduction instructions, and hashed raw evidence provide the commands and receipts.User impact
Correct empty-input distinct counts and derived predicates. The Cypher guide documents the count semantics and scoped optimizer guard. No Python signatures, stubs or dependencies change.
Independently based on
f5a986e822032a3aa8c52af55ad08e92504ee73c, head80d4438884872ce9d18c528caa1519441a9460ce. Latest main6effd0db0c002912cf969fd7447ea7a3ce2631dfadds documentation links/package URLs only; runtime sources and locks are unchanged, and this PR applies cleanly. Other platforms and hosted CI are not included in the local verification; check the PR checks for their current status.Submitted by David Hughes (@stugorf), with AI Collaboration.