Repository navigation
feat: plan file for every selected plan; the Planner's executor check only reported (--plan-file) - #1402
Merged
Conversation
zzylol
force-pushed
the
ap1-plan-file
branch
from
November 17, 2026 16:49
0746edd to
50c4e90
Compare
… first) The selected plan written as an InstallablePlan plan file (--plan-file), with and without the executor's compile check (--executor-check), and InstallablePlan::of_dag_without_executor_check for a runtime other than the Planner's executor.
InstallablePlan::of_dag_without_executor_check builds the validated plan for a runtime other than the Planner's executor; of_dag is it plus the executor's compile check. asap_planner --plan-file writes the selected plan as InstallablePlan JSON, and --executor-check compile|skip says whether it must compile on the Planner's executor (default compile); with skip, unrestricted supported summaries stay unrestricted. A plan file left at the path is removed once the arguments are read.
The executor now runs last_over_time, mad_over_time and ts_of_max_over_time; absent_over_time still plans in Stages 1-3 and fails the executor's compile check.
… (failing first) The approved CLI: --plan-file alone writes the selected plan, also one the Planner's executor cannot run (reported in the document and a stderr note); no selected plan exits 1 and leaves the file at the path alone; summary families come only from --deployment-capabilities. InstallablePlan::of_dag validates without compiling.
…file Delete --executor-check. --plan-file writes the selected plan whether or not the Planner's executor can run it; the compile check's result is the document's planner_executor_can_run / planner_executor_error and one stderr note. No selected plan: exit 1, no file, an existing file untouched. Summary families come only from --deployment-capabilities (default: the executor's); every-summary-family.json is the ready-to-use unrestricted file. InstallablePlan::of_dag is the one public constructor (validated, not compiled); installable_plan_of adds the executor's compile check.
A plan file write error names --plan-file; materialized_views_error is documented (and shown in the viewer) as an invalid plan, not an executor gap; the misuse test drops a flag main never had.
zzylol
force-pushed
the
ap1-plan-file
branch
from
November 17, 2026 21:17
c186f5d to
67f17d5
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
asap_plannercould not write the selected plan for a runtime, and every installable plan passed through the Planner's executor's compile check, so a plan only that executor cannot run was shown as "not installable" to every runtime. For example,absent_over_time(data[1m])gets a selected plan from Stages 1-3, but the executor refuses it ("per-entity aggregate AbsentOverTime has no native implementation").asap_planner --plan-file <file>writes the plan Stage 3 selects asInstallablePlanJSON and exits 0, whether or not the Planner's executor can run it. The executor's compile check still runs, and its result is only reported: the document'splanner_executor_can_run/planner_executor_error, and one stderr linenote: the Planner's executor cannot run this plan: <reason>. Summary families come only from--deployment-capabilities; without them, the default is still the executor's families. A ready-to-useevery-summary-family.jsoncapabilities file plans for a runtime that builds every summary family.Review guide
main, not stacked. feat(devtools, executor, planner): the in-process ASAP lane of the engine comparison (asap_run bench-external --lane asap) #1410 is stacked on it.InstallablePlan::of_dagandinstallable_plan_ofincrates/planner/src/installable/mod.rs, then theinstallable/executor_can_run/--plan-fileblocks ofrunincrates/devtools/src/bin/asap_planner.rs.of_dagnow validates and does not compile.installable_plan_of, whichplan_installablereturns, adds the executor's compile check, soplan_installablebehaves as before.o11y_bench/q05,q06andq11used to havematerialized_views_error: "the executor cannot compile ...". Now they list their views and haveplanner_executor_errorwith the same reason.materialized_views_errornow means only that the views cannot be derived from the DAG.c186f5d7c): a plan-file write error names--plan-file;materialized_views_erroris documented and shown in the viewer as an invalid plan.docs/user_guide_docs/asap-planner.md: option table, the executor note, the capabilities file, a "Write a plan file" section), the viewer's views line and README, and the two workload-validation scripts. Those scripts still count a plan the executor cannot run as a failure, now throughplanner_executor_error.Behavior: before → after
absent.jsonholds the one-off exact PromQL queryabsent_over_time(data[1m])overdata(1,000 series, 15 s samples).main)--plan-file; nothing writes the selected plan--plan-file <path>writes it, exit 0absent_over_time)materialized_views_error: "the executor cannot compile ...", no viewsnote:line on stderr; document: views listed,planner_executor_can_run: false,planner_executor_errorplanner_executor_can_run: true(no plan file written to <path>)supported_summariesdeclared--deployment-capabilities tools/dag-viewer/deployment-capabilities/every-summary-family.jsonInterfaces
crates/planner/src/installable/mod.rs(file), compared withmainimpl InstallablePlan { - /// The plan of `dag`, with each view's and query-time scan's retention checked, - /// and each view's refresh and the query run compiled: a plan the executor - /// cannot run is refused here, not installed. - pub(crate) fn of_dag(dag: PhysicalASAPDAG, query_texts: Vec<String>) -> Result<Self, InstallablePlanError>; + /// The plan of `dag` answering `query_texts`, checked as `validate` checks a + /// plan file (DAG validity, one text per root, no standalone scalar query, + /// retention). Not compiled by the Planner's executor; `check_compiles` does that. + pub fn of_dag(dag: PhysicalASAPDAG, query_texts: Vec<String>) -> Result<Self, InstallablePlanError>; /// Unchanged: whether the Planner's executor compiles this plan. pub fn check_compiles(&self) -> Result<(), String>; } /// What `plan_installable` returns after planning, without its cost report: -/// `InstallablePlan::of_dag(plan, query_texts)`. +/// `InstallablePlan::of_dag`, then the executor's compile check (a plan the +/// Planner's executor cannot run is refused). pub fn installable_plan_of(plan: PhysicalASAPDAG, entries: &[QueryWorkloadEntry]) -> Result<InstallablePlan, InstallablePlanError>;crates/devtools/src/bin/asap_planner.rs(file): usage line, compared withmainThe JSON document's
stage3_plan_selection, with a selected plan:New file:
tools/dag-viewer/deployment-capabilities/every-summary-family.json, aDeploymentCapabilitiesfile whosesupported_summarieslists every sketch summary type with every readout its capability table row answers.Usage (end to end, as the test
a_plan_the_executor_cannot_run_is_written_with_a_noteruns it):Design scope
InstallablePlan::of_dag(plan/installer scope) becomes public and validates without compiling; it is the one constructor. The compile check moves intoinstallable_plan_of, whereplan_installablealready gets its plan.tools/dag-viewer/deployment-capabilities/every-summary-family.json(devtools inputs, next tocost-inputs/): an engine-neutral deployment input. It names what it supports, not an engine. The testthe_every_summary_family_file_rules_out_nothingkeeps it equal to the capability table.Algorithm changes
None.
Decisions and open questions
of_dagreturn(plan, executor result).check_compilesalready exists and is public. A pair would make every caller that only wants a plan (a runtime, a test) carry and ignore the executor's result. With this split,of_dag= validated plan,check_compiles= the Planner's executor's verdict, andinstallable_plan_of/plan_installable= both, refusing on a failure, as before. The CLI callsof_dag, thencheck_compiles, and reports the result.every-summary-family.jsonfile, not a CLI flag. A file that leaves outsupported_summariesstill means "the executor's families", the same asplan_installable.--plan-fileis never removed. A run that selects no plan leaves it, and the error says(no plan file written to <path>).validate(a planner bug, not an executor gap) still givesmaterialized_views_error, writes no plan file, and exits 1 with--plan-file <path>: <error>.Verification
Run on a build host at
c186f5d7c(the review fixes re-ran theasap_planner::tests, fmt and clippy). Only the tests covering the changed code were run, plus fmt and clippy; GitHub CI was not waited for.cargo nextest run -p asap-devtools --test devtools -E 'test(/^asap_planner::/)'cargo nextest run -p asap-planner --lib installableof_dag_validates_a_plan_the_executor_cannot_compile_without_refusing_itcargo nextest run -p asap-planner --test planner -E 'test(/runtime_install|metricsql_workloads|sql_table_scans|reduce_aggregates|exact_only/)'plan_installable/check_compilescallerscargo nextest run -p asap-integration-tests -E 'test(/stage3_objective/)'installable_plan_ofcallerUPDATE_SNAPSHOTS=1 cargo test --release -p asap-devtools --test corpus_snapshots -- --ignoredo11y_bench/q05,q06,q11(see Review guide)cargo fmt --check -p asap-planner -p asap-devtoolscargo clippy -p asap-planner -p asap-devtools --all-targets -- -D warningsThe tests were committed first (
5acdacd43), and at that commit all 8 new tests fail.test_viewer.pygets one new test for the views line. It was not run because the build host has nopy_mini_racer.New tests:
plan_file_round_trips_the_selected_plan:example1's plan file reads back, validates, holds the query texts in order, and is the selected DAG;planner_executor_can_run: true, no note.a_plan_the_executor_cannot_run_is_written_with_a_note:absent_over_time: exit 0, the file is written, exactly onenote:line with the document'splanner_executor_error, views listed, andcheck_compileson the file gives the same reason.no_selected_plan_exits_1_and_leaves_the_plan_file_path_alone: no selection (cost model 2): exit 1, the error says no plan file was written, and the earlier file is unchanged.a_failed_run_leaves_the_plan_file_path_alone: an unreadable workload: exit 1, and the earlier file is unchanged.summary_families_come_from_the_deployment_capabilities: with no file, and with a file withoutsupported_summaries, the families are the executor's;every-summary-family.jsonand a KLL-only file are used as given.the_every_summary_family_file_rules_out_nothing: the file lists every sketch algorithm with its capability-table readouts, and it causes no summary or readout exclusion.plan_file_flag_misuse_is_a_usage_error:--plan-filewithout a value and--plan-filebefore stage3 exit 2 and leave the file alone.Screenshots and measurements: not applicable (CLI and plan-file change, no algorithm).
Follow-ups
None.