Repository navigation
Reuse scalar query templates within each engine snapshot - #34
Merged
Merged
Conversation
Reuse unoptimized plans in the engine, binding fresh scalar values before optimization. Bound the cache to 128 templates and retain ordinary planning for containers and value-dependent output names. Include result, schema, refresh and concurrency regressions and scoped timing harnesses. Part of #14. Isolated core Rust suite: 344 passed, 1 optional timing diagnostic ignored. Formatting, focused Ruff and strict docs passed. Python runtime, bridge test linking, benchmark and Clippy evidence remain pending. Submitted by David Hughes (@stugorf), with AI Collaboration.
Keep the existing refresh test array downcast in scope after sharing its real snapshot fixture with cache tests. Submitted by David Hughes (@stugorf), with AI Collaboration.
Source and runtime checks confirm literals and arithmetic already use the stable expr column name. Remove the unnecessary fallback and correct its regression expectation and docs. Fresh-versus-cached Arrow schema/result tests pass for unaliased scalar parameters. Part of #14. Submitted by David Hughes (@stugorf), with AI Collaboration.
Member
|
This change merged into an intermediate branch, not |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem and change
Issue14. Proposed branch
codex/issue-14-curatedat74dd1e5d9ad680b79da40428e2f7e2cc18eb6671; actual comparison basecodex/issue-14-prerequisitesat248def899cea569f57e44fa447ad9ccf5f8f5eab.Repeated compatible scalar queries now reuse parsing, semantic analysis, graph logical planning and an unoptimized DataFusion logical template. Each execution binds its current values before optimization and physical planning. A bounded128-entry cache belongs to the engine snapshot; successful refresh creates a new cache and active readers retain the old snapshot. Concurrent executions keep parameter bindings separate.
Validation
Head
74dd1e5d9ad680b79da40428e2f7e2cc18eb6671; frozen parent248def899cea569f57e44fa447ad9ccf5f8f5eab. Focused checks and regression details record commands, results, documentation and compatibility. The release receipt pins wheel/archive identity and 57 installed tests.Both complete workloads pass unchanged goldens in memory and lazy modes. Retained first-pass geometric mean speedups are below (greater than one is faster). Per-query results and paired investigations remain in the audit archive.
Measurements use the same Apple M4 Max (16 cores, 128 GiB), macOS 26.6.2 arm64 and Python 3.13.17, with Q30 revision
6260689and IC revisionc185123. Reproduction instructions pin settings, data checksums, locks and commands. The frozen main baseline is code-identical to current main90114f3; the equivalence check also verifies clean prospective main-target merges.Individual review gates pass under the recorded repeat/controlled-investigation rules. Initial failed measurements remain intact in the audit record.
User impact
Each immutable snapshot keeps at most 128 unoptimized logical templates; parameters bind before optimization and refresh starts an empty cache. No public prepared-query API is added. Cache guides and examples are updated.
Review ledger · Current contribution guide
Audit archive: 47,675,190 bytes; SHA-256
b3486fb30a34639c830b5d49353b3838f96369f95df959ac35f85511f7f8b2de. The small current-main supplement preserves the documentation update. Original absolute paths in receipts map to included records through the archive index.Submitted by David Hughes (@stugorf), with AI Collaboration.