Repository navigation
Description
Example
One series of latency_seconds (a gauge sampled at 10 Hz, values declared non-negative), every 60 s, with raw data not retained ("raw_data_retained": false):
| Query | Target | Plan today |
|---|---|---|
quantile_over_time(0.99, latency_seconds[1h]) |
rank 0.001, δ 0.01 | KLL, k = 2,863 |
max(quantile_over_time(0.99, latency_seconds[1h])) |
rank 0.05, δ 0.01 | no plan |
sum(...), min(...), abs(...), ... * 2, -..., ... > 0 over the same p99 |
rank 0.05 or 0.001 | no plan |
Each of the "no plan" rows fails Stage 3 with:
Selection(NoValidCandidate([Rejection { id: "workload", cause: Infeasible, reason: "no plan gives every query output a candidate whose every input has an admissible producer that delivers on its context (the selection MILP is infeasible)" }]))
The input has one series, so max, min and sum over it each return the one p99 value unchanged. The KLL that meets rank 0.001 for the bare p99 therefore meets the same target for max(p99). The planner should select that KLL plan instead of reporting the workload infeasible. With raw data retained, these queries plan only as exact.
Observed by planning each workload through the production stages (run_stages_until then select_plan) at main 2f3664d1f.
Current behavior
Rank error has no error contract through these operators, so the metric rule rejects every edge from a rank readout into them (asap_accuracy::error_contract::error_contract, crates/accuracy/src/error_contract/mod.rs):
AggregateMin/Maxtake a contract only in absolute or relative error (extremum_metric), andSum/Avgonly in relative error over values of one sign. Every other measure in rank error hits_ => return None.Projectitems:keeps_erroraccepts a copied column in any metric. It acceptsabs,clamp*and negation in absolute error, and negation,absand* c// cin relative error. It accepts no transform in rank error.- A comparison (
q > 0) is aFilteron a computed value, so it has no contract in any metric.
Pass 2 then lists the readout group as exact-only for these queries. With no raw data, no exact candidate exists, so the selection MILP is infeasible.
Expected behavior and invariant
A rank error ε means that the returned value lies between the true (q − ε) and (q + ε) quantiles of the series' samples. Whether an operator preserves it depends on the operator:
| Operator over a rank readout | Preserves rank error? | Condition |
|---|---|---|
copy, sort, label_replace, set operators |
yes, β = 1 | already supported |
max, min, sum, avg (and by (...)) |
yes, β = 1 | each output group reads exactly one readout value (one series, or group-by keys that identify one series). Over several series no common sample population exists, so there is no rank bound. |
* c, / c with c > 0, clamp_min(·, a), clamp_max(·, b), clamp |
yes, β = 1 | the transform is non-decreasing, so it maps the true (q ± ε) quantiles of the samples to those of the transformed samples. A clamp may make the bound loose but never wrong. |
unary minus, * c with c < 0 |
yes, β = 1, for the mirrored quantile 1 − q | the transform is non-increasing, so rank order reverses and the magnitude of the rank error stays the same |
abs |
yes, β = 1 | only over values of one sign (GroupProperties::state_column_sign, ValueSign). There it is the identity or a negation; over mixed signs it is not monotone. |
comparison filter (> 0) |
no | a value near the threshold may appear in or vanish from the result, so the series set itself may be wrong (this holds in every metric, as today) |
Minimal reproduction
The workload above, with --capabilities '{"raw_data_retained": false}', planned through asap_devtools::planning_stages::run_stages_until(..., RunUntil::Stage2) and asap_stage3_plan_selection::select_plan.
Impact
Dashboards that wrap a sketched quantile, such as max(...), sum by (job) (...) over one series per job, or a unit conversion * 1000, plan only as exact when raw data is kept, and do not plan at all when it is not. This is a missed-plan bug, not a wrong-answer bug.
Regression test
In crates/accuracy/src/error_contract/tests.rs: rank-metric contracts for max, min, sum and avg over one value per group, for * 2, / 1000, clamp_min and negation, and for abs over a non-negative state, each with β = 1. No contract for sum over two series or for > 0. End to end, in crates/integration-tests/tests/per_query_accuracy_targets.rs (the per-query accuracy checker), max(quantile_over_time(0.99, latency[1h])) at rank 0.001 over one series with no raw data selects a KLL with k ≥ 2,863. These fail today.
End-to-end verification
Plan the six "no plan" queries above with raw data off. Each should select the same KLL as the bare p99, and the per-query accuracy check on the selected plan should pass.
Proposal
- Add rank contracts to
error_contractin the conditions of the table above. One-readout-value-per-group needs the number of values each output group reads, whichreadout_capsalready counts per query (ReadoutsReadByQueries::readout_values) andMemo::queries()[q].number_of_values_per_query_evaluationholds. - Extend
keeps_errorwith a rank case for non-decreasing transforms (positive scale,clamp*), non-increasing ones (negation, negative scale), andabsover values of one sign. - Leave the comparison filter without a contract.
Related: #1238 (constraint (K) composes these contracts per query and per metric).
Activity