Skip to content

feat: plan file for every selected plan; the Planner's executor check only reported (--plan-file) - #1402

Merged
zzylol merged 6 commits into
mainfrom
ap1-plan-file
Nov 17, 2026
Merged

zzylol merged 6 commits into
mainfrom
ap1-plan-file

Conversation

@zzylol

@zzylol zzylol commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Problem: asap_planner could not write the selected plan for a runtime, and every installable plan passed through the Planner's executor's compile check, so a plan only that executor cannot run was shown as "not installable" to every runtime. For example, absent_over_time(data[1m]) gets a selected plan from Stages 1-3, but the executor refuses it ("per-entity aggregate AbsentOverTime has no native implementation").
  • Change: asap_planner --plan-file <file> writes the plan Stage 3 selects as InstallablePlan JSON and exits 0, whether or not the Planner's executor can run it. The executor's compile check still runs, and its result is only reported: the document's planner_executor_can_run / planner_executor_error, and one stderr line note: the Planner's executor cannot run this plan: <reason>. Summary families come only from --deployment-capabilities; without them, the default is still the executor's families. A ready-to-use every-summary-family.json capabilities file plans for a runtime that builds every summary family.
  • Effect: every runtime gets the plan Stage 3 selects. The Planner's executor's gaps are visible, but they do not decide which plans exist. With no plan selected, the run exits 1, writes no file, and leaves a file already at the path as it is.

Review guide

  • Base: main, not stacked. feat(devtools, executor, planner): the in-process ASAP lane of the engine comparison (asap_run bench-external --lane asap) #1410 is stacked on it.
  • Read first: InstallablePlan::of_dag and installable_plan_of in crates/planner/src/installable/mod.rs, then the installable / executor_can_run / --plan-file blocks of run in crates/devtools/src/bin/asap_planner.rs.
  • Careful review: the library API (see Decisions). of_dag now validates and does not compile. installable_plan_of, which plan_installable returns, adds the executor's compile check, so plan_installable behaves as before.
  • Behavior change in snapshots: o11y_bench/q05, q06 and q11 used to have materialized_views_error: "the executor cannot compile ...". Now they list their views and have planner_executor_error with the same reason. materialized_views_error now means only that the views cannot be derived from the DAG.
  • Review fixes (c186f5d7c): a plan-file write error names --plan-file; materialized_views_error is documented and shown in the viewer as an invalid plan.
  • Mechanical: the user guide (docs/user_guide_docs/asap-planner.md: option table, the executor note, the capabilities file, a "Write a plan file" section), the viewer's views line and README, and the two workload-validation scripts. Those scripts still count a plan the executor cannot run as a failure, now through planner_executor_error.

Behavior: before → after

absent.json holds the one-off exact PromQL query absent_over_time(data[1m]) over data (1,000 series, 15 s samples).

$ asap_planner --workload absent.json --run-until stage3 --plan-file plan.json --format json > doc.json
note: the Planner's executor cannot run this plan: the executor cannot compile the plan's query run: invalid DAG: node 2: per-entity aggregate AbsentOverTime has no native implementation
$ echo $?
0
$ jq '.stage3_plan_selection | {planner_executor_can_run, planner_executor_error}' doc.json
{"planner_executor_can_run": false, "planner_executor_error": "the executor cannot compile the plan's query run: invalid DAG: node 2: per-entity aggregate AbsentOverTime has no native implementation"}
$ head -2 plan.json
{
  "wire_version": 15,
$ echo '{"excluded_candidates": [{"operator": "Scan"}]}' > no-scan.json
$ echo "an earlier plan" > plan.json
$ asap_planner --workload absent.json --run-until stage3 --deployment-capabilities no-scan.json --plan-file plan.json > out.txt
asap_planner: plan selection: no valid candidate: [Rejection { id: "workload", cause: ExcludedByDeployment, reason: "query 0 has no candidate left to answer it; excluded by the deployment: excluded_candidates[0] (2 candidates); the workload may be infeasible without these exclusions too" }] (no plan file written to plan.json)
$ echo $?; cat plan.json
1
an earlier plan
before (main) after
plan file no --plan-file; nothing writes the selected plan --plan-file <path> writes it, exit 0
plan the executor cannot run (absent_over_time) document: materialized_views_error: "the executor cannot compile ...", no views the file is written, exit 0, the note: line on stderr; document: views listed, planner_executor_can_run: false, planner_executor_error
plan the executor runs views listed views listed, planner_executor_can_run: true
no plan selected exit 1 exit 1; no file written, a file already at the path is left as it is; the error ends (no plan file written to <path>)
summary families with no supported_summaries declared the executor's the executor's; to plan for every family, pass --deployment-capabilities tools/dag-viewer/deployment-capabilities/every-summary-family.json

Interfaces

crates/planner/src/installable/mod.rs (file), compared with main

 impl InstallablePlan {
-    /// The plan of `dag`, with each view's and query-time scan's retention checked,
-    /// and each view's refresh and the query run compiled: a plan the executor
-    /// cannot run is refused here, not installed.
-    pub(crate) fn of_dag(dag: PhysicalASAPDAG, query_texts: Vec<String>) -> Result<Self, InstallablePlanError>;
+    /// The plan of `dag` answering `query_texts`, checked as `validate` checks a
+    /// plan file (DAG validity, one text per root, no standalone scalar query,
+    /// retention). Not compiled by the Planner's executor; `check_compiles` does that.
+    pub fn of_dag(dag: PhysicalASAPDAG, query_texts: Vec<String>) -> Result<Self, InstallablePlanError>;
     /// Unchanged: whether the Planner's executor compiles this plan.
     pub fn check_compiles(&self) -> Result<(), String>;
 }
 /// What `plan_installable` returns after planning, without its cost report:
-/// `InstallablePlan::of_dag(plan, query_texts)`.
+/// `InstallablePlan::of_dag`, then the executor's compile check (a plan the
+/// Planner's executor cannot run is refused).
 pub fn installable_plan_of(plan: PhysicalASAPDAG, entries: &[QueryWorkloadEntry]) -> Result<InstallablePlan, InstallablePlanError>;

crates/devtools/src/bin/asap_planner.rs (file): usage line, compared with main

 usage: asap_planner (--workload <file>)... --run-until stage0|stage1_pass1|stage1_pass2|stage2|stage3
   [--format text|json] [--verbose] [--out <file>] [--sql-catalog-table <json>]...
   [--cost-inputs <file>] [--deployment-capabilities <file>]
-  [--ignore-accuracy-guarantees]
+  [--ignore-accuracy-guarantees] [--plan-file <file>]

The JSON document's stage3_plan_selection, with a selected plan:

   "materialized_views": [...],          // now derived without the executor's compile check
+  "planner_executor_can_run": false,     // whether the Planner's executor compiles the plan
+  "planner_executor_error": "the executor cannot compile the plan's query run: ..."   // only when false

New file: tools/dag-viewer/deployment-capabilities/every-summary-family.json, a DeploymentCapabilities file whose supported_summaries lists every sketch summary type with every readout its capability table row answers.

Usage (end to end, as the test a_plan_the_executor_cannot_run_is_written_with_a_note runs it):

// The workload a user writes: one exact PromQL query the Planner's executor has no kernel for.
let workload = TempFile::json(&adhoc(&["absent_over_time(data[1m])"], exact(), &["data"]));
// asap_planner runs Stages 0-3, builds the plan with InstallablePlan::of_dag (validated),
// writes it, and reports the executor's compile check without acting on it.
let output = planner(&["--workload", workload.path(), "--run-until", "stage3",
    "--plan-file", file.path(), "--format", "json"]);
assert_eq!(output.status.code(), Some(0));
// stderr: "note: the Planner's executor cannot run this plan: <reason>"
// What any runtime reads: from_json + validate. The Planner's executor would refuse it.
let plan = InstallablePlan::from_json(&std::fs::read_to_string(file.path())?)?;
plan.validate()?;
assert_eq!(plan.check_compiles().unwrap_err(), reason);

Design scope

  • No new type.
  • InstallablePlan::of_dag (plan/installer scope) becomes public and validates without compiling; it is the one constructor. The compile check moves into installable_plan_of, where plan_installable already gets its plan.
  • tools/dag-viewer/deployment-capabilities/every-summary-family.json (devtools inputs, next to cost-inputs/): an engine-neutral deployment input. It names what it supports, not an engine. The test the_every_summary_family_file_rules_out_nothing keeps it equal to the capability table.

Algorithm changes

None.

Decisions and open questions

  • Decided (library API): one constructor plus the existing check. I did not make of_dag return (plan, executor result). check_compiles already exists and is public. A pair would make every caller that only wants a plan (a runtime, a test) carry and ignore the executor's result. With this split, of_dag = validated plan, check_compiles = the Planner's executor's verdict, and installable_plan_of / plan_installable = both, refusing on a failure, as before. The CLI calls of_dag, then check_compiles, and reports the result.
  • Decided: "unrestricted" is the explicit every-summary-family.json file, not a CLI flag. A file that leaves out supported_summaries still means "the executor's families", the same as plan_installable.
  • Decided: a file already at --plan-file is never removed. A run that selects no plan leaves it, and the error says (no plan file written to <path>).
  • Decided: a plan that fails validate (a planner bug, not an executor gap) still gives materialized_views_error, writes no plan file, and exits 1 with --plan-file <path>: <error>.

Verification

Run on a build host at c186f5d7c (the review fixes re-ran the asap_planner:: tests, fmt and clippy). Only the tests covering the changed code were run, plus fmt and clippy; GitHub CI was not waited for.

scope (command) passed / failed / ignored newly failing vs base notes
cargo nextest run -p asap-devtools --test devtools -E 'test(/^asap_planner::/)' 53 / 0 / 0 none the 7 CLI tests below
cargo nextest run -p asap-planner --lib installable 8 / 0 / 0 none adds of_dag_validates_a_plan_the_executor_cannot_compile_without_refusing_it
cargo nextest run -p asap-planner --test planner -E 'test(/runtime_install|metricsql_workloads|sql_table_scans|reduce_aggregates|exact_only/)' 50 / 0 / 0 none plan_installable / check_compiles callers
cargo nextest run -p asap-integration-tests -E 'test(/stage3_objective/)' 16 / 0 / 0 none the other installable_plan_of caller
UPDATE_SNAPSHOTS=1 cargo test --release -p asap-devtools --test corpus_snapshots -- --ignored 2 / 0 / 0 n/a rewrote o11y_bench/q05, q06, q11 (see Review guide)
cargo fmt --check -p asap-planner -p asap-devtools ok
cargo clippy -p asap-planner -p asap-devtools --all-targets -- -D warnings ok

The tests were committed first (5acdacd43), and at that commit all 8 new tests fail. test_viewer.py gets one new test for the views line. It was not run because the build host has no py_mini_racer.

New tests:

  • plan_file_round_trips_the_selected_plan: example1's plan file reads back, validates, holds the query texts in order, and is the selected DAG; planner_executor_can_run: true, no note.
  • a_plan_the_executor_cannot_run_is_written_with_a_note: absent_over_time: exit 0, the file is written, exactly one note: line with the document's planner_executor_error, views listed, and check_compiles on the file gives the same reason.
  • no_selected_plan_exits_1_and_leaves_the_plan_file_path_alone: no selection (cost model 2): exit 1, the error says no plan file was written, and the earlier file is unchanged.
  • a_failed_run_leaves_the_plan_file_path_alone: an unreadable workload: exit 1, and the earlier file is unchanged.
  • summary_families_come_from_the_deployment_capabilities: with no file, and with a file without supported_summaries, the families are the executor's; every-summary-family.json and a KLL-only file are used as given.
  • the_every_summary_family_file_rules_out_nothing: the file lists every sketch algorithm with its capability-table readouts, and it causes no summary or readout exclusion.
  • plan_file_flag_misuse_is_a_usage_error: --plan-file without a value and --plan-file before stage3 exit 2 and leave the file alone.

Screenshots and measurements: not applicable (CLI and plan-file change, no algorithm).

Follow-ups

None.

@zzylol zzylol added dev-tools Developer tools, diagnostics, and demos enhancement New feature or request labels Nov 17, 2026
@zzylol zzylol changed the title feat: plan file without the Planner executor's compile check (--plan-file, --executor-check) feat: plan file for every selected plan; the Planner's executor check only reported (--plan-file) Nov 17, 2026
… first)

The selected plan written as an InstallablePlan plan file (--plan-file),
with and without the executor's compile check (--executor-check), and
InstallablePlan::of_dag_without_executor_check for a runtime other than
the Planner's executor.
InstallablePlan::of_dag_without_executor_check builds the validated plan
for a runtime other than the Planner's executor; of_dag is it plus the
executor's compile check. asap_planner --plan-file writes the selected
plan as InstallablePlan JSON, and --executor-check compile|skip says
whether it must compile on the Planner's executor (default compile); with
skip, unrestricted supported summaries stay unrestricted. A plan file left
at the path is removed once the arguments are read.
The executor now runs last_over_time, mad_over_time and ts_of_max_over_time;
absent_over_time still plans in Stages 1-3 and fails the executor's compile check.
… (failing first)

The approved CLI: --plan-file alone writes the selected plan, also one the
Planner's executor cannot run (reported in the document and a stderr note);
no selected plan exits 1 and leaves the file at the path alone; summary
families come only from --deployment-capabilities. InstallablePlan::of_dag
validates without compiling.
…file

Delete --executor-check. --plan-file writes the selected plan whether or not
the Planner's executor can run it; the compile check's result is the
document's planner_executor_can_run / planner_executor_error and one
stderr note. No selected plan: exit 1, no file, an existing file untouched.
Summary families come only from --deployment-capabilities (default: the
executor's); every-summary-family.json is the ready-to-use unrestricted file.
InstallablePlan::of_dag is the one public constructor (validated, not
compiled); installable_plan_of adds the executor's compile check.
A plan file write error names --plan-file; materialized_views_error is
documented (and shown in the viewer) as an invalid plan, not an executor gap;
the misuse test drops a flag main never had.
@zzylol
zzylol merged commit c714d65 into main Nov 17, 2026
2 of 3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dev-tools Developer tools, diagnostics, and demos enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant