Skip to content

feat(graph): enforce cooperative query deadlines - #39

Merged
prrao87 merged 9 commits into
codex/issue-19-prerequisitesfrom
codex/issue-19-scan-budget
Oct 26, 2026
Merged

prrao87 merged 9 commits into
codex/issue-19-prerequisitesfrom
codex/issue-19-scan-budget

Conversation

@stugorf

@stugorf stugorf commented Oct 26, 2026

Copy link
Copy Markdown
Member

Problem and change

Relates to #19. Adds keyword-only timeout=None in seconds to CypherEngine.execute, execute_batches and CypherQuery.execute. One monotonic deadline includes planning and pauses between reads. Zero expires immediately; negative, nonfinite and unrepresentable values raise ValueError.

Native pyarrow.RecordBatchReader is preserved. Initial expiry raises TimeoutError; expired reads raise ArrowInvalid with Query timeout exceeded. A watchdog releases owned streams during consumer idle time. Close/drop and ordinary errors also release resources. Timed contexts install a private DataFusion join guard, including rejected nested-loop candidates; ordinary contexts remain reusable.

Validation

Head a641cf25cbe34db25245ce2543140af5b735d30e; frozen parent 81ef524be83db4abb626394dd4940121a45751e2. Focused checks and regression details record commands, results, documentation and compatibility. The release receipt pins wheel/archive identity and 487 installed tests.

Both complete workloads pass unchanged goldens in memory and lazy modes. Retained first-pass geometric mean speedups are below (greater than one is faster). Per-query results and paired investigations remain in the audit archive.

Workload/mode Main Actual parent
Q30 memory 0.970 0.985
Q30 lazy 0.969 1.010
IC memory 1.146 1.042
IC lazy 1.113 0.976

Measurements use the same Apple M4 Max (16 cores, 128 GiB), macOS 26.6.2 arm64 and Python 3.13.17, with Q30 revision 6260689 and IC revision c185123. Reproduction instructions pin settings, data checksums, locks and commands. The frozen main baseline is code-identical to current main 90114f3; the equivalence check also verifies clean prospective main-target merges.

This is a coordinated draft. Preserve its frozen-head outcomes and apply the shared corrections in the qualified integration; changing the actual target requires requalification. The accepted combined engine source is 7d2ef3e; its current-main documentation merge is 5fe24cd.

The standalone 8,192-work budget passes its own gates, including the fixed-size IC6 investigation. The combined target retains the tested 1,024-work bound and matching tests after rejecting the larger-budget combined variant. External runner adoption remains outside this submission; keep #19 open.

User impact

execute, execute_batches and CypherQuery.execute gain keyword-only timeout=None in seconds. Initial expiry raises TimeoutError; stream expiry raises ArrowInvalid with Query timeout exceeded. Native readers remain. API docs, stubs and examples cover cooperative limits; external runner adoption remains open.

Review ledger · Current contribution guide

Audit archive: 47,675,190 bytes; SHA-256 b3486fb30a34639c830b5d49353b3838f96369f95df959ac35f85511f7f8b2de. The small current-main supplement preserves the documentation update. Original absolute paths in receipts map to included records through the archive index.

Submitted by David Hughes (@stugorf), with AI Collaboration.

Add keyword-only timeout to the three Cypher execution entrypoints. Keep one deadline through streamed reads and release the owned stream while idle; preserve native Arrow reader errors and cooperative traversal cancellation.

Issue #19. Submitted by David Hughes (@stugorf), with AI Collaboration. External benchmark runner adoption and performance measurements remain pending.
Drop the owned DataFusion stream when its first error finishes the Arrow reader, preserving the original error. A real CAST failure and Weak reference to a later Arrow batch fail before the fix and pass after it. All 15 native binding tests pass; the Python reader-error regression and complete native rebuild are queued.

Submitted by David Hughes (@stugorf), with AI Collaboration.
Refs #19. Preserve bounded cancellation through rejected rows and tombstones while measuring the scheduling trade-off independently.
@prrao87
prrao87 marked this pull request as ready for review October 26, 2026 16:49
@prrao87
prrao87 merged commit e58721e into codex/issue-19-prerequisites Oct 26, 2026
8 checks passed
@prrao87
prrao87 deleted the codex/issue-19-scan-budget branch October 26, 2026 16:53
@prrao87

prrao87 commented Oct 26, 2026

Copy link
Copy Markdown
Member

This change merged into an intermediate branch, not main. It reached main through #55, which brought in the final integrated version of this work with every original commit kept on its branch. Follow-up fixes from review are listed there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants