Repository navigation
Conversation
The engine traits, records and every module of the harness core, with todo!() bodies; ThreadCpuTime in aqpbm-core with its unit test. The implementation follows once the tests are approved.
82 tests from the approved S1 plan: machine key, protocol, fit with leave-one-setting-out, accuracy curves, data and data statistics, the fake engine and conformance kit, run directory, gate and sweep, and the dependency rule. 31 are moved or adapted from ASAPPlanner #1332. They compile against the todo!() skeleton and fail there, except the dependency rule, which already holds.
The engine traits, measurement protocol (idle check, pinning, warm-up, median of an odd run count), machine key, relative-weighted fit with leave-one-setting-out, accuracy curves, data chunks and statistics, run records and directory, gate and sweep arithmetic, the fake-clock engine and the conformance kit. aqpbm-core's ThreadCpuTime reads CLOCK_THREAD_CPUTIME_ID; aqpbm-datagen gains TimeSeriesCursor so a reader can own the generated series and draw samples lazily. All 82 tests pass except the two ignored timing tests (F7, P15).
- H1: a Processes engine is measured on its server pids. ProcessesCpuClock sums utime + stime under a ProcFs (thread CPU 0); run_arm and conformance::check take MeasurementClocks and build the engine's clock from cpu_accounting(). E10 now runs a fake server under a fake /proc. - M1: the calling-thread check repeats each step until 20 ms of CPU, then applies the 10% rule. - M2: answers are keyed by query id outside the labels; QUERY_LABEL deleted. - M3: measure_sizes runs on the protocol, takes a core and a LoadProbe, and returns a Result. - M4: one signed relative error per held-out point. - M5: samples share their series labels (Arc); run_arm documents the memory of its installs; consecutive jobs share one idle sample. - L1-L8 and nits: Option instead of NaN in records, NaN-cost gate failure and zero-duration refusal, private RunDir path and create_new result files, staleness message, FakeClock::ticks deleted, owning into_samples iterator, ConstantNotIdentified doc, relative units tolerance, renames.
Re-pinning the main thread, the process-CPU median, a zombie server pid, server CPU at ns resolution, a nonexistent core refused before the idle sample, and answers freed outside the timed window.
- Pinning reads the process's allowed cores once, before any pin, so the main thread can be re-pinned; a core outside the set is refused before the idle sample. - run_job takes the EngineClock and medians the CPU it measures the engine by (process CPU for Processes engines); MedianClock deleted; measure_sizes moves to protocol.rs and takes the EngineClock too. - Processes engines read their own cgroup v2 cpu.stat usage_usec, else their threads' schedstat (ns); a zombie pid is ProcessExited; the wall clock is read innermost around the call. - run_arm frees every run's answers after the job, outside the timed window. - NodeTerms and InstalledArm::node_terms, ProfilingMethod and MeasurementClocks::os deleted; ranking_agreement takes SelectedAndExactCosts rows; data_statistics groups by label Arc and keeps a running value range. - docs/aqpbm-harness.md, README link, into_samples in the datagen doc; test comments no longer carry PR numbers; generic machine-key example.
zzylol
force-pushed
the
engines-external/translator
branch
2 times, most recently
from
November 17, 2026 06:47
65f8061 to
6529c5a
Compare
This was referenced Nov 17, 2026
Draft
Draft
The aqpbm-harness crate becomes the harness module of aqpbm-core
(aqpbm_core::harness::{engine, protocol, machine, rundir, records, fit,
accuracy, data, data_stats, gate, sweep, fake, conformance, arm_run}).
Its tests move to aqpbm-core/tests/harness_*.rs, the dependency rule
now checks aqpbm-core with default features off, and the design doc
becomes docs/aqpbm-core-harness.md. aqpbm-core gains only thiserror and
serde's rc feature.
zzylol
force-pushed
the
engines-external/translator
branch
from
November 17, 2026 19:50
8a8dda4 to
4032055
Compare
harness::accuracy becomes harness::error_quantiles, so it no longer reads like core's accuracy comparators; the crate doc points to the harness; the last aqpbm_harness temp-file prefix is renamed.
…kHouse translator New crate `aqpbm-engines-external` (skeleton only) with its tests: - tests/plan_file.rs (R1-R6): reading an ASAPPlanner InstallablePlan file, wire version 15: node fields, label-equality predicates, other wire versions refused, unknown operators kept, standalone scalar queries refused, roots and readers. - tests/clickhouse_translate.rs (X1-X10): tables, materialized views and one query per root against hand-written goldens; the stored-rate substitute recorded; query-time sharing reported; every refused field named at the node that carries it; X10 (ignored) runs the goldens on ClickHouse 26.8.34.13 and checks hand-computed answers. - tests/capability_files.rs (K1): the four deployment-capabilities files equal the hand-computed values. Fixtures are plan files written by the planner for public synthetic workloads (tests/fixtures/plans/REV); edge cases change one field of a loaded fixture in test code.
- X6 also checks root 13, the query-time rate, against F3's query.
- X10's counter data now has three series, one per branch of Prometheus
extrapolatedRate: a reset, a start gap above 1.1 average gaps, and the
zero-point clamp; each rate is computed by hand in the test. X10 also
checks the ClickHouse version first.
- R3 nulls the DAG of the other-version file: the version is read first.
- Mutations use planner wire literals ({"Boolean": true}, {"Float64": 1.0}).
- X4 checks that a label value with a quote and a backslash is escaped.
- X2-X5 assert no substitutions / lost sharing where none are expected.
- REV records how the fixtures were made.
…lickHouse - plan_file.rs: decoder for InstallablePlan wire version 15 over JSON values (no ASAPPlanner dependency). The version is checked first; every field that can change an answer is decoded; unknown operators decode to Other; standalone scalar queries are refused. - clickhouse.rs: translate() refuses, in node id order, every operator and field it has no rule for, then translates each root: stored exact Sum/Count/Min/Max/Rate/Increase chains into AggregatingMergeTree tables and materialized views, query-time builds and Aggregates into SQL over the raw table, with right-closed windows and Prometheus extrapolatedRate. The stored counter chain is recorded as a substitution; query-time nodes shared by several readers are reported as lost sharing. - capabilities/: the four deployment-capabilities files and the README with each entry's class and evidence.
Correctness: - A stored chain's view keeps its Scan's label predicates (WHERE). - A stored read refuses a range that is not a whole number of windows, and every query reports the step its evaluation time must be a multiple of (query_time_steps_ms: the window width it reads, 1 if none). - The decoder checks every id: children, edges and roots name nodes, and a node reads only earlier nodes (no cycle); roots match query texts; versions beyond u32, overflowing or sub-millisecond ranges are refused. - Labels are the utf8 columns of the output schema, not names; a Reduce column must be a label of its input. - Query-time results kept across evaluations are refused (the ClickHouse files say they are not retained). - Measures over a column, unknown measure arguments and quantile levels outside [0, 1] are refused; a grouped SummaryAgg is refused at its node. - The ClickHouse files list no sketch until the translator builds sketch states; the README says so and states the time-order requirement of deltaSumTimestamp states. - Label columns are LowCardinality(String). Simplicity: one StoredChain per chain (BTreeMap), one ExactKind table, one free unsupported(), RawSamples instead of a Raw relation variant; unused fields deleted (wire_version, retention_ms, parameters, the estimate query, the table reference). ClickHouse version: the runs use 26.8.39.13, the 26.8 LTS build for CPUs without AVX2; the goldens pass on it (X10).
Tests first for the independent review of the translator: - Stored rate under parallel reads and partial merges, counter resets inside an insert, at an insert boundary and at a window boundary, single-sample segments, and overlapping segments (ClickHouse, ignored without a server). - Every committed public plan translates, and an equivalent-only one has no substitution; the plans are planner-written from public synthetic workloads at ASAPPlanner #1374 head 7834fed4d (the named fixtures are unchanged). - Grouped query-time summaries (the showcase q4c shape), regrouping merges, top_k, instant selectors, projections and arithmetic binary operators, with hand-computed answers on ClickHouse. - An unaligned evaluation time is an error; retention is recorded with TTL statements kept apart; a NULL value is refused on insert. - Predicate semantics, per-measure filters, the series identity type, the source sorting key, the binary's git hash, and one JSON format for the capability files.
- Stored rate/increase: each stored row is read as one time-ordered segment (finalizeAggregation) and a series' segments are joined in time order, so the answer no longer depends on how a parallel read merges deltaSumTimestamp states (it was 1.7x too high at 16 threads). The step into a segment follows deltaSumTimestamp's own rule; overlapping segments are an error. A new argMax state keeps each segment's last value. - New rules, so that every equivalent-only plan of the public workloads translates: query-time summaries grouped or over another relation (the showcase q4c shape), regrouping merges, grouped stored chains, top_k, instant selectors, Project and arithmetic BinaryOp. A checked division with a value that is not finite is an error. - A stored read fails at an evaluation time that is not a window end. - Retention is decoded and recorded per table; TTL statements are offered apart from the script, for wall-clock data only. - PromQL-semantics label equality only; per-measure filters; the series identity is a String; the source table is sorted by its filtered labels, the series identity and ts, with a minmax index on ts; inserts refuse NULL values; names follow the planner's; the query step travels with each relation; one JSON format for the capability files; the README cites the running binary (26.8.39.13 amd64compat, git hash d5bbb64d) with permalinks.
- A per-series binary operation between two metrics is refused: the series identity names the metric, so a join on it would match nothing where PromQL matches the series without their names. - A metric that only stored chains read is a Null table: its views still see every insert, and no raw rows are kept. - A projection must keep every label and take the value column. - A raw-data Scan read outside a time range or window chain keeps every row (no TTL), as in the planner's raw-data retention. - Names: Relation.unfinalized_kind, whole_number_argument, PlanNode::is_raw_data_scan; the retention_ms doc follows the planner. - README: floating-point summation and the cost of the stored counter read. - Test helpers folded into the common module.
…d capability files ASAPPlanner #1374 is merged into main as bc725bda8. Every committed plan fixture regenerated there is byte-identical, and the four capability files load there unchanged.
zzylol
force-pushed
the
engines-external/translator
branch
from
November 17, 2026 19:51
4032055 to
2ed3d47
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
sum by (job) (sum_over_time(latency[1h])), evaluated every 60 s. Nothing turns that plan into ClickHouse materialized views and a query, so the ClickHouse baseline cannot run the same plan as the ASAP executor (Translate an InstallablePlan into each engine's deployment form #253, Engine capability profiles with conformance tests #252).aqpbm-engines-external.InstallablePlan, wire version 15) with a small local decoder.-State/-Merge) and one SQL query per query root.asap_planner --deployment-capabilities capabilities/clickhouse.equivalent-only.jsonplans only what ClickHouse can do, and this crate turns every such plan of the public workloads into SQL with the same answers. All 206 public PromQL workloads were planned with each ClickHouse file: all 412 plans translate, and no equivalent-only plan has a substitution.Review guide
harness/core-tests). Draft.edb2270: the failing tests;1276b94: the fixes from the independent review of the tests;4b5df37: the implementation;6529c5a: the fixes from the independent review of the implementation;edacc69: the failing tests for the second independent review (below);7b6710a: its fixes;ebd5be9: the fixes from the independent review of those fixes (last table);8a8dda4: cites ASAPPlanner mainbc725bda8, where #1374 is merged; every committed fixture regenerated there is byte-identical.src/clickhouse.rs:translate,refuse_unsupported_fields,Translator::relation(the pattern dispatch),Translator::stored_read(the counter segment query),Translator::binary;src/plan_file.rs:PlanFile::from_value,operator_of.tests/fixtures/plans/**andtests/fixtures/workloads/**(planner output and its inputs, never hand-edited; seeREV).Second review: findings and fixes
deltaSumTimestampMergeof non-adjacent window states counts the increase between them twicefinalizeAggregation); a series' segments are joined in time order withlagInFrame(Σ in-segment increase + Σ max(0, first_k − last_(k−1)),deltaSumTimestamp's own rule); overlapping segments are an error; news_last_value(argMax) statestored_rate_is_right_under_parallel_reads(2,000 counters, 60 inserts, 1/16/40 threads, merged, partly merged),stored_rate_differs_from_promql_by_the_first_value_after_each_reset,overlapping_stored_segments_are_an_error, golden f2top_k(LIMIT k BY), instant selectors (argMax),Project, arithmeticBinaryOp; raw data kept at ingestion time is read like a query-time scanpublic_plans_translate(29 planner-written public plans), unit SQL tests, ClickHouse answers for each rule,public_translations_run(SELECT throwIf({t} % w != 0, …)) = 0(a scalar subquery, so it runs even when no row matches)unaligned_evaluation_time_is_an_error, goldensretention_msdecoded;ClickHouseDeployment.table_retentionper table (the planner's raw-data retention rule for sources);ttl_statements()kept out ofscript(): a TTL deletes by wall-clock time, so it applies only under wall-clock data (under replay the stored bytes include rows past retention)retention_is_recorded_and_ttl_is_separate,ttl_statements_apply_and_null_values_are_refused"semantics": "promql"label equality isLabelEqualsonly_promql_label_equality_is_label_equalsAggregate.filtersis one optional filter per measuremeasure_filters: Vec<Option<Value>>;[null]is no filteraggregate_filters_are_one_optional_predicate_per_measureString(one value per series), labelsLowCardinality(String)ORDER BY (<filtered labels>, $promql_series_identity, ts)+minmaxindex ontsa_label_filter_leads_the_source_sorting_key, golden f4INSERT_SETTINGS(input_format_null_as_default = 0, insert_null_as_default = 0)ttl_statements_apply_and_null_values_are_refusedd5bbb64d, its sha256; X10 checksversion()and the git hashSummaryTumblingWindowChain,FinalizeExactAccumulator,s_positive_delta_sumcapability_files_are_pretty_printedReview of the fixes (applied in
ebd5be9)BinaryOpbetween two metrics matched nothing (the series identity names the metric)per_series_binary_operation_across_metrics_is_refusedENGINE = Nullsource (its views still see every insert)a_metric_only_chains_read_keeps_no_rows, goldens f1, f2, f5Projectcould take a non-value column as the value, or drop a labelprojections_keep_every_label_and_one_valueunfinalized_kind,whole_number_argument,PlanNode::is_raw_data_scan); helpers intests/commonASAPPlanner #1374 moved to
7834fed4d(itsexcluded_candidatesdroppedcounter_semantics,merged_across_windowsandsketch_parameters, and addedsketch_parameter_range). The four files use none of those fields and stay valid; the six named fixtures regenerate byte-identical at that head.Checked on ClickHouse
All
#[ignore]tests (AQPBM_CLICKHOUSE_LOCAL) pass on ClickHouse 26.8.39.13: the goldens with hand-computed answers (X10), and every case inclickhouse_run.rs:v = s · (1 + k mod 7), 60 insertsManual check
At #1374 head
7834fed4d, the 206 public PromQL workloads were planned with each file (out-of-repo driver callinginstallable_plan_of): every file is accepted; ClickHouse equivalent-only and with-substitutes give a plan for all 206; VictoriaMetrics equivalent-only gives no plan for 80 (ExcludedByDeployment,counter_resets_not_handled), as its README says; forsum by (job) (rate(requests_total[5m]))the ClickHouse equivalent-only file selects the query-time rate.Design scope
TableRetention,ClickHouseDeployment::{table_retention, ttl_statements}INSERT_SETTINGSPlanNode::{retention_ms, timestamp_columns, is_value_column, is_raw_data_scan}PlanOperator::{Project, BinaryOp},measure_filtersRelation.{step_ms, unfinalized_kind, source_tables}(private)Interfaces
pub mod clickhouse { + pub const INSERT_SETTINGS: &str; pub struct ClickHouseDeployment { pub tables: Vec<String>, pub materialized_views: Vec<String>, pub queries: Vec<String>, pub query_time_steps_ms: Vec<u64>, + pub table_retention: Vec<TableRetention>, pub substitutions: Vec<Substitution>, pub sharing_lost: Vec<SharingLost>, } + pub struct TableRetention { pub table: String, pub row_time_sql: String, pub retention_ms: u64 } impl ClickHouseDeployment { pub fn script(&self) -> String; + pub fn ttl_statements(&self) -> Vec<String>; } pub fn translate(plan: &PlanFile) -> Result<ClickHouseDeployment, TranslationError>; } pub mod plan_file { pub struct PlanNode { pub id: NodeId, pub operator: PlanOperator, pub timing: Timing, pub lifetime: Lifetime, + pub retention_ms: Option<u64>, pub output_columns: Vec<String>, pub label_columns: Vec<String>, + pub timestamp_columns: Vec<String>, } + impl PlanNode { pub fn is_value_column(&self, index: usize) -> bool; pub fn is_raw_data_scan(&self) -> bool; } pub enum PlanOperator { - WindowChain { child, family, width_ms, reduction, input, filter }, + SummaryTumblingWindowChain { child, family, width_ms, reduction, input, filter }, - Finalize { child }, + FinalizeExactAccumulator { child }, - Aggregate { child, measures, reduction, filters: Vec<Value>, having }, + Aggregate { child, measures, reduction, measure_filters: Vec<Option<Value>>, having }, + Project { child: NodeId, expressions: Vec<Value> }, + BinaryOp { lhs: NodeId, rhs: NodeId, operator: Value, return_bool: bool }, ... } }Verification
cargo test -p aqpbm-engines-external: 40 pass (10 ClickHouse cases ignored without a server);cargo clippy -p aqpbm-engines-external --all-targets -- -D warnings: clean;cargo fmt --check: clean.#[ignore]tests withAQPBM_CLICKHOUSE_LOCAL="<clickhouse 26.8.39.13> local":clickhouse_run10 of 10, X10 pass.