Repository navigation
test(integration): regression guard for each query meeting its own accuracy target over a shared readout group - #1384
Merged
Merged
Conversation
…ies share a readout group at different targets Regression tests through the production stages: one p99 text at rank 0.05 and 0.001 (KLL), one L2 text at relative 0.1 and 0.05 (UnivMon), two texts sharing the p99 readout group, and a fixed-seed property over query sets checking each query's readouts against its own target on the selected plan.
zzylol
force-pushed
the
test/per-query-accuracy-targets
branch
from
November 17, 2026 17:13
2b260fd to
f69c7cc
Compare
…ins the UnivMon and the shared readout, reuses the (K) test's planning helper
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
quantile_over_time(0.99, latency[1h])at rank 0.05 and again at rank 0.001. Before refactor(stage3): constraint (K) per execution and metric; sign-conditioned DDSketch guarantee, classic HLL certificate from the source's distinct bound (#1207 PR 8) #1238 both queries read one KLL with k = 52; with it they read k = 2,863.Review guide
main(cb78d07). Not stacked. Also passes unchanged on the head of fix(accuracy): constraint (K) checks the union bound over the estimates each query depends on #1405 (Stage 3 (K) union bound over dependent estimates), which stacks on fix(accuracy): split δ over the readout values the covered output values depend on #1360.crates/integration-tests/tests/per_query_accuracy_targets.rsfirst:query_meets_its_targetis the independent check, and the four tests follow it.crates/integration-tests/tests/accuracy_at_query_roots.rs: itsplanhelper (workload + production stages, no raw data) takes a series count and is reused here, so the two (K) test files share one workload builder.crates/integration-tests/tests/main.rsonly registers the module.Behavior: before → after
No planner behavior changes. What the tests pin, for one series of
latencysampled every second, queried every 10 s, no raw data retained:l2_over_timeat relative 0.1 + 0.05sort(p99)at rank 0.001This serves correctness: a plan must meet every query's own target.
Interfaces
None (tests only).
Algorithm changes
None.
Decisions and open questions
None.
Verification
cargo test -p asap-integration-tests --test integration -- per_query_accuracy_targets:: accuracy_at_query_roots::root_limitsreturnsNone) on maincargo clippy -p asap-integration-tests --tests,cargo fmt --checkNew tests:
one_query_text_at_two_rank_targets_reads_a_kll_meeting_the_stricter: the p99 at rank 0.05 and 0.001, in both orders. Each query meets its own target, and the plan holds a KLL with k ≥ 2,863.one_query_text_at_two_relative_targets_reads_a_univmon_meeting_the_stricter:l2_over_time(latency[24h])at relative 0.1 and 0.05, in both orders. The plan holds a UnivMon with at least 2,367 columns.two_query_texts_sharing_a_readout_group_each_meet_their_target:p99andsort(p99), which root at different groups over one readout group, with the targets swapped between runs. Asserts the roots differ and both read the same readout.every_query_meets_its_own_target_on_the_selected_plan: a property test over 16 fixed-seed sets of 2–3 queries drawn from 9 (query, target) pairs (p99 rank and relative, p50,sort(p99), L2). Every query's own check holds on the selected plan.The check is independent of the planner. From each query's root it walks to the one
SummaryEstimateon the path (no readout, more than one, or a state without θ is an error, so the check cannot pass vacuously), takes the θ of the state that readout reads, and requiresasap_accuracy::theoretical_guaranteeat the target's δ to be in an allowed metric and within ε.Probe evidence. The probe is a metamorphic relation: adding a query never lowers the optimal cost, so {q at ε, q at ε/5} must cost at least {q at ε/5} alone. For every query in the sketch-heavy workload set, it plans the pair in both orders, both as one text and with the looser query wrapped in
max(...), and compares the cost with the strict query alone. All plans go through the production path:Follow-ups
max/min/sum/abs/* c/negation over a sketched quantile has no plan without raw data. That is why the tests usesort(p99)and notmax(p99)as the second query text.