[stugorf]

Problem and change

Addresses duplicate property reads in #16. Columns fetched for a pushed target predicate are filtered once and reused when projection also needs them. Remaining properties are fetched only for survivors. Reuse stays within one bounded output batch and its pinned snapshot; this adds no full-column cache, Python API or dependency. Existing projection pruning and Lance row sorting/deduplication retain their roles.

Real Arrow/Lance tests cover duplicate and out-of-order positions, null predicates, empty results, overlapping/disjoint/empty projections, reconstructed IDs and unused columns. A read-cell counter verifies six candidate ages plus four surviving names require ten cells, without repeated age or ID reads. Complete workspace tests/Clippy, 351 Python tests, lint/typing, strict docs, spelling and executable guides pass. The CSR guide and existing benchmark describe and exercise filter/projection reuse.

Validation

Head 447355dbaf90993d33926afc48ab6712e421505a; frozen parent 1bf6d42141b9cc6ee1824d000f49cdc0ebd9d8fe. Focused checks and regression details record commands, results, documentation and compatibility. The release receipt pins wheel/archive identity and 159 installed tests.

Both complete workloads pass unchanged goldens in memory and lazy modes. Retained first-pass geometric mean speedups are below (greater than one is faster). Per-query results and paired investigations remain in the audit archive.

Workload/mode	Main	Actual parent
Q30 memory	0.967	1.006
Q30 lazy	0.916	1.009
IC memory	1.197	1.003
IC lazy	1.191	1.056

Measurements use the same Apple M4 Max (16 cores, 128 GiB), macOS 26.6.2 arm64 and Python 3.13.17, with Q30 revision 6260689 and IC revision c185123. Reproduction instructions pin settings, data checksums, locks and commands. The frozen main baseline is code-identical to current main 90114f3; the equivalence check also verifies clean prospective main-target merges.

This is a coordinated draft. Preserve its frozen-head outcomes and apply the shared corrections in the qualified integration; changing the actual target requires requalification. The accepted combined engine source is 7d2ef3e; its current-main documentation merge is 5fe24cd.

User impact

Results and ordering are preserved while filter columns are reused within each output batch. There is no persistent property cache or Python API change. The CSR guide and existing benchmark cover the behavior.

Review ledger · Current contribution guide

Audit archive: 47,675,190 bytes; SHA-256 b3486fb30a34639c830b5d49353b3838f96369f95df959ac35f85511f7f8b2de. The small current-main supplement preserves the documentation update. Original absolute paths in receipts map to included records through the archive index.

Submitted by David Hughes (@stugorf), with AI Collaboration.

[prrao87]

This change merged into an intermediate branch, not main. It reached main through #55, which brought in the final integrated version of this work with every original commit kept on its branch. Follow-up fixes from review are listed there.