Repository navigation
feat(devtools, executor, planner): the in-process ASAP lane of the engine comparison (asap_run bench-external --lane asap) - #1410
Merged
Merged
Conversation
zzylol
force-pushed
the
ap2-asap-lane
branch
2 times, most recently
from
November 17, 2026 21:10
26cddec to
f865c26
Compare
zzylol
force-pushed
the
ap1-plan-file
branch
from
November 17, 2026 21:17
c186f5d to
67f17d5
Compare
AsapEngine installs a plan file written by asap_planner --plan-file and runs it with the Planner's executor under the sketch-bench harness: the harness's conformance kit, sum_over_time answered exactly from an ingestion-time chain and from raw samples at query time, a plan the executor cannot compile as a 'not executable' cell, install refusals, and asap_run bench-external --lane asap. The executor's in-memory store gains appended, time-ranged raw data dropped after its lookback.
AsapEngine (asap_devtools::asap_lane) installs a plan file written by asap_planner --plan-file and runs it with the Planner's executor as an engine under test of the sketch-bench harness; asap_run bench-external --lane asap runs it over a run directory. A plan the executor cannot compile is an install error starting 'not executable:'. The lane is a deployment runtime over the executor's public API: the executor's InMemoryDataStore gains raw tables bound for appending, read by the batches overlapping the requested range and dropped after their lookback, and byte accessors; InstallablePlan exposes the compiled query run and ingestion refreshes its compile check already built; the RawDataRetention accessors are public, since a runtime reads them.
One raw-table schema rule (PhysicalASAPDAG::raw_data_schemas) for the lane and the planner tests' stand-in runtime; samples routed once per series, with dictionary texts cached per table; views' windows before the data start refreshed empty at install; two answer rows of one series are an error; no empty raw appends; simpler raw binding in the in-memory store. Pins sketch-bench at the rebased head (harness unchanged).
last_over_time now compiles on the executor; absent_over_time is still planned but not compiled.
sketch-bench moved its harness into aqpbm-core (aqpbm_core::harness); pin aqpbm-core without default features (no sketch library, polars or CLI crate) at the bench-external head.
…no longer gates asap_planner --plan-file now writes every selected plan; a plan the Planner's executor cannot run comes with a stderr note. The lane tests write plan files without --executor-check, and the absent_over_time case checks the note, then the lane's not executable cell.
The check compares thread and process CPU; other tests running in the same process added process CPU the lane did not spend, so it failed whenever it ran beside them.
zzylol
force-pushed
the
ap2-asap-lane
branch
from
November 17, 2026 21:25
f865c26 to
172f63a
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
asap_devtools::asap_lane::AsapEngineinstalls a plan file written byasap_planner --plan-fileas given and runs it with the Planner's executor on the calling thread.asap_run bench-external --lane asap --run-dir Rruns it over a harness run directory. A plan the executor cannot compile becomes anot executable:cell, and the other lanes still run that plan.sum_over_time(m[5m])every 10 s, planned as a chain of 10 s tumbling windows built at ingestion time, runs over 2 series × 360 s. It gives 36 ingest steps and 7 evaluations, each answered exactly (e.g.{s="s_0"} 1525.73at 300 s). Footprint: 21,120 bytes of stored views and 4,232 bytes of raw samples.Review guide
ap1-plan-file(feat: plan file for every selected plan; the Planner's executor check only reported (--plan-file) #1402,--plan-file), stacked on it. Commits: the failing tests first, then the lane, then the independent review's fixes, then the executor-only example switched toabsent_over_time, then the harness taken fromaqpbm-core, then the lane tests adapted to feat: plan file for every selected plan; the Planner's executor check only reported (--plan-file) #1402's plan file (written for every selected plan; the Planner's executor check only reported).crates/devtools/src/asap_lane/installed_plan.rs(install, ingest, evaluate). Thencrates/executor/src/sources/memory.rs(raw tables bound for appending).window_ends, and the empty windows before the data start), and the raw lookback inInstalledPlan::install.InstallablePlan::compile_query_run/compile_ingestion_refreshreturn the DAGs the compile check already built.PhysicalASAPDAG::raw_data_schemasis the schema union that the planner tests' stand-in runtime had, moved out so the lane and the stand-in share it.aqpbm-corewithdefault-features = false(the harness isaqpbm_core::harness; its normal dependencies are aqpbm-datagen, anyhow, bitflags, chrono, libc, rand, rand_xoshiro, serde, serde_json, thiserror: no sketch library, polars, hdrhistogram or CLI crate) and, as a dev-dependency,aqpbm-datagen, both pinned to sketch-bench7c0d121192beaf3b68b1b1ec36aab8aafc90d0ae, the head of its bench-external PR. sketch-bench never depends on the planner.Behavior: before → after
The user writes a planning workload (
sum_over_time(m[5m])every 10 s, exact,mingesting at 1,000 rows/s from 1,000 series sampled every second) and a harness run directory withworkloads/w.json(the same query, 2 series, one sample per second for 360 s):Before: there is no
asap_run. The plan file can be translated for other engines, but the Planner's executor cannot run it as a lane.After:
asap_runprintsw ingesting Ranand writesresults/asap/w__ingesting.cell.jsonandresults/asap/w__ingesting.json. The arm result holds:stored_outputs ["n1 refreshed AtIngestion, kept 300000 ms"];{s="s_0"}: 1525.733751382602;For a plan the executor cannot compile (
absent_over_time(m[10s])),asap_planner --plan-filestill writes the plan file and printsnote: the Planner's executor cannot run this plan: .... On the ASAP lane, the cell isNotRun { error: "not executable: the executor cannot compile the plan's query run: ..." }, from the lane's own compile at install.This serves the engine comparison: the ASAP lane runs exactly the DAG that the other lanes translate.
Interfaces
crates/devtools/src/asap_lane/mod.rscrates/devtools/src/bin/asap_run.rscrates/executor/src/sources/memory.rsimpl InMemoryDataStore { /// Bind `batches`, all in `schema`, as the rows of `source`: every read returns every row. pub fn with_raw_data(self, source: Source, schema: SchemaRef, batches: Vec<Batch>) -> Result<Self, Error>; + /// Bind `source` for appending its rows as they arrive, each batch kept until the newest + /// end is `lookback_ms` past its end (`None`: forever). Bad input: already bound. + pub fn with_appended_raw_data(self, source: Source, schema: SchemaRef, lookback_ms: Option<u64>) -> Result<Self, Error>; + /// Append `batches` covering `(start, end]`, then drop the batches past the lookback; a read + /// returns the batches overlapping its range. Bad input: not bound for appending, another + /// schema, empty range or not after the last end. + pub fn append_raw_data(&mut self, source: &Source, time_range_ms: (i64, i64), batches: Vec<Batch>) -> Result<(), Error>; + /// Bytes of the raw rows held. + pub fn raw_data_bytes(&self) -> usize; + /// Bytes of the materialized views' stored rows. + pub fn materialized_view_bytes(&self) -> usize; /// Append `batches` as the rows of `view` covering `time_range_ms`. pub fn append_materialized_view(&mut self, view: &MaterializedView, time_range_ms: (i64, i64), batches: Vec<Batch>) -> Result<(), Error>; }crates/planner/src/installable/mod.rsimpl InstallablePlan { + /// The executor's query run, as a runtime runs it at each evaluation; `Err` is + /// `QueryRunNotExecutable` (unsatisfiable request) or why the views cannot be derived. + pub fn compile_query_run(&self) -> Result<CompiledPhysicalDAG, InstallablePlanError>; + /// The executor's refresh of one ingestion refresh group (its views as roots), as a runtime + /// runs it once per window; `Err` is `MaterializedViewNotExecutable` or why the views + /// cannot be derived. + pub fn compile_ingestion_refresh(&self, group: &IngestionRefreshGroup) -> Result<CompiledPhysicalDAG, InstallablePlanError>; }crates/types/src/plan/materialized_view/raw_data.rs,mod.rsimpl PhysicalASAPDAG { + /// Each raw-data source the DAG's `Scan`s read, with the union of their columns by name: + /// the raw table a runtime binds, e.g. `m` as `(ts, value, $promql_series_identity, job)`. + pub fn raw_data_schemas(&self) -> Vec<(Source, Schema)>; } impl RawDataRetention { - /// (test-support only) + /// The source; now public, since a runtime decides what raw data to keep from it. pub fn source(&self) -> &Source; - /// (test-support only) + /// The longest `retention_ms` of a raw-data `Scan` of it; `None` when one reads it unbounded. pub fn lookback_ms(&self) -> Option<u64>; }Usage (end to end; it runs as
asap_lane::the_asap_lane_runs_a_planned_sum_over_time_and_answers_exactlyandasap_run_runs_the_asap_lane_over_a_run_dir):Algorithm changes
The ASAP lane runtime (new;
installed_plan.rs):raw_data_schemas). It is kept back to the larger of its query-time lookback (raw_data_retention) and the longest refresh width plus one chunk.(a, b]: route samples once per series (a cached table and ordinal per label-map address) into one dictionary-encoded batch per series table, and append it.e ∈ (a, b]withe + phase ≡ 0 (mod width), run one ingestion run and append each view's output for(e − width, e].i→ queryi.ArmSchedule::for_workloadmakes them.unsupported by the ASAP lane:.Decisions and open questions
sum_over_time(m[5m])answers under{s="s_0"}, without__name__. The harness'sAnswersdoc example keeps__name__. A report that joins lanes needs one convention. Recommendation: PromQL's (the Prometheus ground truth drops it), and the harness doc follows.InMemoryDataStore, extended with raw tables bound for appending, rather than implementing its ownDataStore. The alternative would have duplicated the store's projection and view tiling.not executable:cell, not a missing plan.asap_plannerwrites the plan file either way, so the other lanes still get it.Verification
Run on a build host, at the PR head.
nextest -p asap-devtools --test devtools asap_lanenextest -p asap-executornextest -p asap-planner --lib installablenextest -p asap-planner --test plannersummary_sharing::identical_ungrouped_queries_share_their_producersfails identically at the base commitnextest -p asap-typesnextest -p asap-devtools --test devtools -E 'test(/^asap_lane::|^asap_planner::/)'(after the rebase onto #1402's new head)cargo clippy -p asap-devtools -p asap-executor -p asap-planner -p asap-types --all-targets -- -D warningscargo fmt --checkNew tests:
conformance_kit_passes_on_the_asap_engine: the harness's conformance kit on probe plans the planner library made.the_asap_lane_runs_a_planned_sum_over_time_and_answers_exactly: an ingestion-time chain and raw query-time scans answer exactly at every scheduled time, with 4 installs interleaved, and the footprint and plan summary are filled in.a_plan_the_executor_cannot_compile_is_a_not_executable_cell: forabsent_over_time,asap_planner --plan-fileexits 0, writes the file and prints the executor note; the lane givesNotRunstartingnot executable:.a_plan_for_other_queries_is_refused_naming_the_query: these are refused: no plan file, a missing file, not a plan, another text, a count mismatch. A plan text that appears as a query's second text matches.asap_run_runs_the_asap_lane_over_a_run_dir: the binary writes aRancell, and an unknown lane exits 2.appended_raw_data_is_read_by_range_and_dropped_after_its_lookback: the store reads by overlap, drops at the lookback boundary, is append-only, and keeps the whole-bound tables unchanged.End-to-end CLI evidence: see Behavior. That run used the build's debug binaries over the demo run directory.
Follow-ups
predicted()from Stage 3's report. The plan file does not carry it.node_profiles).