diff --git a/Cargo.lock b/Cargo.lock index 84b535f2d..daa30bee6 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -305,9 +305,9 @@ dependencies = [ name = "asap-devtools" version = "0.1.0" dependencies = [ - "asap-accuracy", "asap-cost", "asap-executor", + "asap-frontend-metricsql", "asap-frontend-promql", "asap-frontend-sql", "asap-planner", diff --git a/crates/devtools/Cargo.toml b/crates/devtools/Cargo.toml index 80079502b..71e496849 100644 --- a/crates/devtools/Cargo.toml +++ b/crates/devtools/Cargo.toml @@ -10,8 +10,8 @@ edition = "2021" # matching `asap-frontend-*` crate directly to avoid pulling the other parser. [dependencies] asap-frontend-promql = { path = "../frontend-promql" } +asap-frontend-metricsql = { path = "../frontend-metricsql" } asap-frontend-sql = { path = "../frontend-sql" } -asap-accuracy = { path = "../accuracy" } asap-stage0-memo = { path = "../stage0-memo" } asap-stage1-pass1-per-computation-candidates = { path = "../stage1-pass1-per-computation-candidates" } asap-stage1-pass2-reuse-candidates = { path = "../stage1-pass2-reuse-candidates" } @@ -30,7 +30,7 @@ asap-executor = { path = "../executor" } # examples. Regular deps, not dev-deps: `[[bin]]` targets can't see dev-deps. asap-types = { path = "../types" } serde = { version = "1", features = ["derive"] } -serde_json = "1" +serde_json = { version = "1", features = ["float_roundtrip"] } serde_yaml = "0.9" tokio = { version = "1", features = ["rt", "macros", "rt-multi-thread"] } diff --git a/crates/devtools/src/bin/asap_planner.rs b/crates/devtools/src/bin/asap_planner.rs index 26f6a3108..907a0270a 100644 --- a/crates/devtools/src/bin/asap_planner.rs +++ b/crates/devtools/src/bin/asap_planner.rs @@ -7,9 +7,13 @@ // Runs the planner stages over a workload up to and including // `--run-until` (`stage0`, `stage1_pass1`, `stage1_pass2`, `stage2` or // `stage3`) and writes each stage's output, as text (`--format text`, the default; -// `asap_devtools::planning_stages_text`) or as an `asap-stage-pipeline/v1` +// `asap_devtools::planning_stages_text`) or as an `asap-stage-pipeline/v2` // document for the Stage Viewer (`--format json`), to `--out` or standard -// output. +// output. The text folds each memo group's candidates that differ only in +// θ into one line (their indices, each θ parameter's values, how many Pass +// 2 pruned by cause), printing in full only those Pass 2 kept, and counts +// the pruned lists per cause; `--printing-full-candidates` prints every candidate, edge and +// pruned candidate. // // The stages run as `run_planning` runs them // (`asap_devtools::planning_stages::run_stages_until`), so the selected plan @@ -51,22 +55,30 @@ // - workload: the queries (`id`, `language`, `text`, `requirements`, // `recurrence`, `predictability`) and the `data_workload` planned with; // - stage0_query_language_frontends: the frontends' DAG, one root per query; -// - stage1_logical_asap_optimization (from `stage1_pass1`): `pruned_candidates`, -// the ways to compute a measure Pass 1 pruned (each on its aggregate's -// memo `group`, with `candidate` and `reason`); `error` when Pass 1 -// failed; -// - stage2_physical_asap_optimization (from `stage2`): `pruned_candidates`, -// the physical candidates Stage 2 pruned (each with its memo `group`, -// `operator_node`, `operator`, `implementation_root`, `timing`, -// `lifetime` and `reason`); +// - stage1_logical_asap_optimization (from `stage1_pass1`): from +// `stage1_pass2`, `pruned_candidates`, the candidates Pass 2 eliminated +// (`group`, `index_in_group`, and `causes`: each with +// `through_pruned_candidates` and its `root` reason, a serialized +// `RootPruneReason`, and `text`); `error` when Pass 1 or Pass 2 failed; +// - stage2_physical_asap_optimization (from `stage2`): +// `physical_candidates`, Stage 2's candidates (`id`, the logical +// candidate's `group` and `index_in_group`, `properties` (`timing`, +// `lifetime`), and per input its `group` and `edges`: each physical +// edge whose logical edge holds, with `producer`, +// `satisfies_physical_requirement`, `rejection` (why +// $\alpha^{\mathrm{phys}} = 0$, null when it holds) and `selected` (an +// edge of the plan)); `pruned_candidates`, the physical candidates +// Stage 2 pruned (`group`, `index_in_group`, `properties`, `reason`, a +// serialized `PruneReason`); `error` when Stage 2 failed; // - stage3_plan_selection (from `stage3`): the `plan` Stage 3 built (a -// `PhysicalASAPDAG`) and its `Selection`: `accepted_accuracy_per_query`, -// `pruned_candidates`, `milp_objective_cost_per_second` (the selected -// plan's cost per second), `milp_variables`, `milp_constraints`, -// `milp_construction_ms`, `milp_solve_ms` and `milp_solve_count` (2 when -// the solve at HiGHS's default tolerances missed an accuracy target and -// was repeated at 1e-10); `error` when no plan was -// selected. +// `PhysicalASAPDAG`), its `cost` (`asap_cost::plan_cost`: +// `cost_per_second`, `source`, `usage` and each node's charge), and its +// `Selection`: `accepted_accuracy_per_query`, `pruned_candidates` (as +// Stage 2's), `milp_variables`, `milp_constraints` and +// `milp_objective_cost_per_second` (the selected plan's cost per +// second); `error` when no plan was selected. `sharing_summary`: each +// plan node two or more queries read, by what it computes (`operator`, +// `parameters`, `timing`, `lifetime`) and the `queries` reading it. // `materialized_views` are the plan's materialized views as // `plan_installable` derives them; `materialized_views_error` when the // plan is not installable. `warnings`: a PromQL query applying `rate`, `increase` @@ -76,51 +88,76 @@ // saying whether answers may then be wrong beyond the accuracy target), and a peak // memory over 10^12 bytes when the cost model declares no memory capacity; // `assumed_data_facts`: those facts as records (`AssumedDataFact`), when there are any; -// - memo: `nodes`, one node table for the section (each distinct node -// once, children as indices before their parents), and `steps`, the -// memo after each step run (`stage0`: the frontends' DAG hash-consed, -// groups without candidates; `pass1`; `pass2`: Pass 2's output, the memo -// Stage 2 implements). Each -// step lists its groups (`id`, `label`, `children`, `parents`, -// `queries`, `roots_of`, `node`) and their candidates (`index`, -// `implements`, `children`, `origin`, `added` -// (not in the group at the step before), `label`, `node`, -// `group_of_node`). With `stage3`, `selected_choice` maps each group the -// plan uses to its candidate's index, or is null with -// `selected_choice_unavailable`; +// - memo: `steps`, the memo after each step run (`stage0`: the +// frontends' DAG hash-consed, groups without candidates; `pass1`; +// `pass2`: Pass 2's output, the memo Stage 2 expands). Each step lists +// its groups (`id`, `label`, `operator` (its expression, children as +// group ids), `children`, `parents`, the group properties +// `result_kind`, `column_lineage`, `column_signs`, +// `depends_on_evaluation_time`, `shareable_when_equal`, +// `aggregate_coverage` (as the text prints it, null for a group that +// is no aggregate), `scanned_sources` and, at `pass2`, +// `evaluation_grid`; `queries`, `roots_of`, `candidate_count`, +// `candidate_count_by_origin`) and their candidates +// (`index`, `origin`, `added` (Pass 2 added it), `label`, `operator`, +// `children` (its input groups), `fused_groups`, `parameters` (θ, null +// when exact) and `required_accuracy` per input); at `pass2` a +// candidate also has `pruned` and, per input, `inputs` (`group` and +// `edges`: the logical edge from each candidate of the input group, +// eliminated ones included, with its `producer` (index in the group), +// `satisfies_logical_requirement` and `rejection`, why +// $\alpha^{\log} = 0$ as Pass 2 recorded it). With a selected plan, `selected_candidates` +// maps each group the plan uses to the indices of its candidates there; // - `search`, `accuracy_guarantees` (`"enforced"` or `"ignored"`), // `deployment_capabilities` (as planned with) and `run_until`. // A section of a stage after `--run-until` is null. +// +// The document is stable, so it can be compared as text: memo group, +// candidate and physical candidate ids are canonical (renumbered depth +// first from the query roots, `asap_devtools::stable_output::CanonicalIds`), +// lists are in canonical id order, every non-integer number is rounded to +// 12 significant digits, and the MILP's times and solve count are left +// out. The text output keeps the planner's own ids, as in logs and error +// messages, and prints the times and solve count. use std::collections::btree_map::Entry; -use std::collections::BTreeSet; +use std::collections::{BTreeMap, BTreeSet}; -use asap_cost::CostInputs; -use asap_devtools::candidate_labels::node_label; +use asap_cost::{CostInputs, PlanCost}; +use asap_devtools::candidate_labels::operator_label; use asap_devtools::planning_stages::{ - lower_promql_roots, lower_sql_roots, run_stages_until, sql_catalog, PlanningStagesError, - PlanningStagesRun, RunUntil, + lower_metricsql_roots, lower_promql_roots, lower_sql_roots, run_stages_until, sql_catalog, + PlanningStagesError, PlanningStagesRun, RunUntil, +}; +use asap_devtools::planning_stages_text::{ + aggregate_coverage_text, column_lineage_text, edges_per_input, logical_edges_into, + physical_edge_rejection, planning_stages_text, root_prune_reason_text, selected_edges, }; -use asap_devtools::planning_stages_text::planning_stages_text; +use asap_devtools::stable_output::{rounded_to_12_significant_digits, CanonicalIds}; use asap_devtools::SqlCatalog; use asap_frontend_promql::{metric_type_warnings, promql_metric_names}; use asap_planner::{assumed_data_facts, installable_plan_of, PlanningModels, SelectionError}; use asap_stage3_plan_selection::{large_peak_memory_warning, AcceptedAccuracy, Selection}; use asap_types::accuracy::{default_metric_for_root, AccuracyTarget}; use asap_types::deployment_capabilities::DeploymentCapabilities; -use asap_types::ir::flat::{flatten, FlatDag, FlatNode}; +use asap_types::ir::flat::{flatten, FlatDag}; +use asap_types::ir::operator::Source; use asap_types::ir::QueryRoot; +use asap_types::ir::{NonASAPOp, Operator}; +use asap_types::plan::materialized_view::MaterializedView; use asap_types::plan::physical_asap_dag::PhysicalASAPDAG; -use asap_types::stages::PhysicalASAPMemoCandidates; -use asap_types::stages::{Memo, OriginOfCandidateForMemoGroup}; -use asap_types::stages::{MemoGroup, MemoGroupId}; +use asap_types::plan::physical_asap_dag::PhysicalASAPNodeId; +use asap_types::stages::{ + CandidateForMemoGroup, CandidateForMemoGroupId, LogicalASAPMemoCandidates, Memo, MemoGroupId, + PhysicalASAPMemoCandidates, PhysicalCandidate, PhysicalCandidateId, PrunedCandidate, +}; use asap_types::workload::{ DataArrival, DataWorkload, PlanningWorkload, QueryLanguage, QueryWorkloadEntry, }; use serde_json::{json, Value}; const USAGE: &str = "usage: asap_planner (--workload )... \ ---run-until stage0|stage1_pass1|stage1_pass2|stage2|stage3 [--format text|json] [--out ] \ +--run-until stage0|stage1_pass1|stage1_pass2|stage2|stage3 [--format text|json] [--printing-full-candidates] [--out ] \ [--sql-catalog-table ]... [--cost-inputs ] [--deployment-capabilities ] \ [--ignore-accuracy-guarantees]"; @@ -166,6 +203,7 @@ fn run(args: Vec) -> Result<(), Failure> { let mut cost_inputs = CostInputs::default(); let mut deployment_capabilities = DeploymentCapabilities::default(); let mut ignore_accuracy_guarantees = false; + let mut printing_full_candidates = false; let mut args = args.into_iter(); while let Some(flag) = args.next() { let mut value = || { @@ -189,6 +227,7 @@ fn run(args: Vec) -> Result<(), Failure> { "--cost-inputs" => cost_inputs = read_json(&flag, &value()?)?, "--deployment-capabilities" => deployment_capabilities = read_json(&flag, &value()?)?, "--ignore-accuracy-guarantees" => ignore_accuracy_guarantees = true, + "--printing-full-candidates" => printing_full_candidates = true, other => return Err(Failure::Usage(format!("unknown argument {other}"))), } } @@ -216,6 +255,19 @@ fn run(args: Vec) -> Result<(), Failure> { .with_deployment_capabilities(&deployment_capabilities) .with_ignore_accuracy_guarantees(ignore_accuracy_guarantees); let result = run_stages_until(roots.clone(), &entries, &data, models, run_until); + // The selected plan's cost, as `plan_installable` reports it. + let plan_cost = result + .as_ref() + .ok() + .and_then(|run| run.run.selection.as_ref().ok()) + .and_then(|(plan, _)| asap_cost::plan_cost(plan, &entries, &data, &cost_inputs).ok()) + .map(|(cost, _)| cost); + // The selected plan's materialized views, as `plan_installable` derives them. + let materialized_views = result + .as_ref() + .ok() + .and_then(|run| run.run.selection.as_ref().ok()) + .map(|(plan, _)| materialized_views_of(plan, &entries)); // A stage that failed: the output is still written, then the run exits 1. let failure = match &result { Err(PlanningStagesError::Pass1(error)) => Some(format!("Pass 1: {error}")), @@ -230,27 +282,26 @@ fn run(args: Vec) -> Result<(), Failure> { // The selected plan's data facts assumed because the data workload // does not declare them, and its very large peak memory when the // deployment declares no memory capacity to check it against. - let selected_plan_cost = result + let assumed_facts = result .as_ref() .ok() .and_then(|run| run.run.selection.as_ref().ok()) - .and_then(|(plan, _)| { - let (cost, _) = asap_cost::plan_cost(plan, &entries, &data, &cost_inputs).ok()?; - Some((plan, cost)) - }); - let assumed_facts = selected_plan_cost - .as_ref() - .map(|(plan, cost)| assumed_data_facts(plan, cost, &data)) + .zip(plan_cost.as_ref()) + .map(|((plan, _), cost)| assumed_data_facts(plan, cost, &data)) .unwrap_or_default(); warnings.extend(assumed_facts.iter().map(ToString::to_string)); - if let Some((_, cost)) = &selected_plan_cost { - warnings.extend(large_peak_memory_warning( - cost.usage.peak_memory_bytes, - &cost_inputs.cost_model, - )); - } + warnings.extend(plan_cost.as_ref().and_then(|cost| { + large_peak_memory_warning(cost.usage.peak_memory_bytes, &cost_inputs.cost_model) + })); let text = if json_format { - let mut document = document_of(described, &roots, &entries, result, run_until)?; + let mut document = document_of( + described, + &roots, + result, + run_until, + plan_cost.as_ref(), + materialized_views.as_ref(), + )?; // Stage 3 is null when Pass 1 rejected a query. if let Some(stage3) = document["stage3_plan_selection"].as_object_mut() { if !warnings.is_empty() { @@ -264,6 +315,8 @@ fn run(args: Vec) -> Result<(), Failure> { ); } } + // The planner's numbers; the inputs added below stay as given. + let mut document = rounded_to_12_significant_digits(document); document["workload"]["data_workload"] = serde_json::to_value(&data).map_err(|e| e.to_string())?; document["search"] = json!("candidate-connections"); @@ -278,7 +331,12 @@ fn run(args: Vec) -> Result<(), Failure> { serde_json::to_string_pretty(&document).map_err(|e| e.to_string())? + "\n" } else { let mut text = match result { - Ok(run) => planning_stages_text(&run), + Ok(run) => planning_stages_text( + &run, + plan_cost.as_ref(), + materialized_views.as_ref(), + printing_full_candidates, + ), Err(PlanningStagesError::Pass1(error)) => format!("error: Pass 1: {error}\n"), Err(error) => return Err(error.to_string().into()), }; @@ -303,22 +361,20 @@ fn run(args: Vec) -> Result<(), Failure> { fn document_of( queries: Vec, roots: &[QueryRoot], - entries: &[QueryWorkloadEntry], result: Result, run_until: RunUntil, + plan_cost: Option<&PlanCost>, + materialized_views: Option<&Result, String>>, ) -> Result { let queries = with_default_metrics(queries, roots); match result { - Ok(run) => Ok(document(queries, entries, &run)), + Ok(run) => Ok(document(queries, &run, plan_cost, materialized_views)), Err(PlanningStagesError::Pass1(error)) => Ok(json!({ - "format": "asap-stage-pipeline/v1", + "format": "asap-stage-pipeline/v2", "run_until": run_until.name(), "workload": { "queries": queries }, "stage0_query_language_frontends": { "dag": export(roots) }, - "stage1_logical_asap_optimization": { - "pruned_candidates": [], - "error": format!("Pass 1: {error}"), - }, + "stage1_logical_asap_optimization": { "error": format!("Pass 1: {error}") }, "stage2_physical_asap_optimization": null, "stage3_plan_selection": null, })), @@ -327,27 +383,38 @@ fn document_of( } /// The document for `run`, whose workload queries are `queries`: each -/// section of a stage `run` ran, the others null. -fn document(queries: Vec, entries: &[QueryWorkloadEntry], run: &PlanningStagesRun) -> Value { +/// section of a stage `run` ran, the others null. `plan_cost` is the +/// plan's cost and `materialized_views` its materialized views. +fn document( + queries: Vec, + run: &PlanningStagesRun, + plan_cost: Option<&PlanCost>, + materialized_views: Option<&Result, String>>, +) -> Value { let until = run.run_until; + let plan = run.run.selection.as_ref().ok().map(|(plan, _)| plan); + let ids = canonical_ids_of(run); let stage1 = (until >= RunUntil::Pass1).then(|| { let mut stage1 = json!({}); - // Pass 1 or Pass 2 failed, or the deployment's capabilities - // contradict the data workload. - if run.run.logical_asap_candidates.is_none() { - if let Err(error) = &run.run.selection { - if !matches!(error, SelectionError::NotRun) { - stage1["error"] = json!(error.to_string()); - } + if let Some(logical) = &run.run.logical_asap_candidates { + stage1["pruned_candidates"] = json!(pass2_pruned_json(logical, &ids)); + } else if let Err(error) = &run.run.selection { + // Pass 1 or Pass 2 failed, or the deployment's capabilities + // contradict the data workload. + if !matches!(error, SelectionError::NotRun) { + stage1["error"] = json!(error.to_string()); } } stage1 }); let stage2 = (until >= RunUntil::Stage2).then(|| { - let mut stage2 = json!({ - "pruned": [], - "pruned_candidates": stage2_pruned_candidates_json(run.run.physical_asap_candidates.as_ref()), - }); + let mut stage2 = match &run.run.physical_asap_candidates { + Some(physical) => json!({ + "physical_candidates": physical_candidates_json(physical, plan, &ids), + "pruned_candidates": pruned_candidates_json(&physical.pruned_candidates, &ids), + }), + None => json!({ "physical_candidates": [], "pruned_candidates": [] }), + }; // Stage 2 itself failed. if run.run.logical_asap_candidates.is_some() && run.run.physical_asap_candidates.is_none() { if let Err(error) = &run.run.selection { @@ -356,61 +423,104 @@ fn document(queries: Vec, entries: &[QueryWorkloadEntry], run: &PlanningS } stage2 }); - let stage3 = (until == RunUntil::Stage3).then(|| stage3_json(run, entries)); + let stage3 = + (until == RunUntil::Stage3).then(|| stage3_json(run, plan_cost, materialized_views, &ids)); json!({ - "format": "asap-stage-pipeline/v1", + "format": "asap-stage-pipeline/v2", "run_until": until.name(), "workload": { "queries": queries }, "stage0_query_language_frontends": { "dag": export(&run.stage0_query_language_frontends) }, "stage1_logical_asap_optimization": stage1, "stage2_physical_asap_optimization": stage2, "stage3_plan_selection": stage3, - "memo": memo_json(run), + "memo": memo_json(run, &ids), }) } -/// Stage 3's section: the plan, the selection and the plan's materialized -/// views. -fn stage3_json(run: &PlanningStagesRun, entries: &[QueryWorkloadEntry]) -> Value { - let mut stage3 = selection_json(&run.run.selection); - match &run.run.selection { - Ok((plan, _)) => match materialized_views_json(plan, entries) { - Ok(views) => stage3["materialized_views"] = Value::Array(views), - Err(error) => { - stage3["materialized_views"] = json!([]); - stage3["materialized_views_error"] = json!(error); - } - }, - Err(_) => stage3["materialized_views"] = json!([]), +/// The canonical ids of `run`'s last memo (Pass 2's, else Pass 1's, else +/// Stage 0's) and of its physical candidates. +fn canonical_ids_of(run: &PlanningStagesRun) -> CanonicalIds { + let logical = run.run.logical_asap_candidates.as_ref(); + let memo = logical + .map(|logical| &logical.memo) + .or(run.run.pass1_memo.as_ref()) + .or(run.stage0_memo_groups.as_ref()) + .expect("run_stages_until builds Stage 0's memo"); + CanonicalIds::of(memo, run.run.physical_asap_candidates.as_ref()) +} + +/// Stage 3's section: the plan, its cost, its sharing summary, the +/// selection and the plan's materialized views. +fn stage3_json( + run: &PlanningStagesRun, + plan_cost: Option<&PlanCost>, + materialized_views: Option<&Result, String>>, + ids: &CanonicalIds, +) -> Value { + let mut stage3 = selection_json(&run.run.selection, ids); + if let Ok((plan, _)) = &run.run.selection { + stage3["cost"] = plan_cost.map_or(Value::Null, cost_json); + stage3["sharing_summary"] = json!(sharing_summary_json(plan)); + } + stage3["materialized_views"] = json!([]); + match materialized_views { + Some(Ok(views)) => stage3["materialized_views"] = json!(views), + Some(Err(error)) => stage3["materialized_views_error"] = json!(error), + None => {} } stage3 } -/// The document's `memo` section: the memo after each step `run` ran. -fn memo_json(run: &PlanningStagesRun) -> Value { +/// The document's `memo` section: the memo after each step `run` ran, +/// and, with a selected plan, `selected_candidates`: each memo group the +/// plan uses, with the indices of its candidates there (a group may be +/// used through several candidates). Ids are `ids`' canonical ones. +fn memo_json(run: &PlanningStagesRun, ids: &CanonicalIds) -> Value { let stage0 = run .stage0_memo_groups .as_ref() .expect("run_stages_until builds Stage 0's memo"); - let mut named = vec![("stage0", "Stage 0, the frontends' DAG hash-consed", stage0)]; + let mut steps = vec![json!({ + "step": "stage0", + "title": "Stage 0, the frontends' DAG hash-consed", + "groups": memo_groups_json(stage0, None, ids), + })]; if let Some(pass1) = &run.run.pass1_memo { - named.push(("pass1", "Stage 1, Pass 1", pass1)); - } - if let Some(logical) = &run.run.logical_asap_candidates { - named.push(("pass2", "Stage 1, Pass 2", &logical.memo)); - } - let mut table = MemoNodes::default(); - let mut previous: Option<&Memo> = None; - let mut json_steps = Vec::new(); - for &(step, title, memo) in &named { - json_steps.push(json!({ - "step": step, - "title": title, - "groups": memo_groups_json(&mut table, memo, previous), + steps.push(json!({ + "step": "pass1", + "title": "Stage 1, Pass 1", + "groups": memo_groups_json(pass1, None, ids), })); - previous = Some(memo); + if let Some(logical) = &run.run.logical_asap_candidates { + steps.push(json!({ + "step": "pass2", + "title": "Stage 1, Pass 2", + "groups": memo_groups_json(&logical.memo, Some((pass1, logical)), ids), + })); + } + } + let mut memo = json!({ "steps": steps }); + if let (Ok((plan, _)), Some(physical)) = ( + &run.run.selection, + run.run.physical_asap_candidates.as_ref(), + ) { + let mut selected: BTreeMap> = BTreeMap::new(); + for candidate in plan + .nodes + .iter() + .filter_map(|node| node.selected_physical_candidate) + { + let logical = physical.physical_candidates[candidate.0].logical_candidate_id; + let (group, index_in_group) = ids.candidate(logical); + selected.entry(group).or_default().insert(index_in_group); + } + let selected: serde_json::Map = selected + .into_iter() + .map(|(group, indices)| (group.to_string(), json!(indices))) + .collect(); + memo["selected_candidates"] = json!(selected); } - json!({ "nodes": table.nodes, "steps": json_steps }) + memo } /// The [`metric_type_warnings`] of each PromQL query of `described` (the @@ -448,6 +558,7 @@ type Lowered = ( fn lowered(workload: &PlanningWorkload, catalog: &SqlCatalog) -> Result { let roots = match workload.query_workload.language { QueryLanguage::PromQL => lower_promql_roots(workload)?, + QueryLanguage::MetricsQL => lower_metricsql_roots(workload)?, _ => lower_sql_roots(workload, catalog)?, }; let entries = workload.query_workload.entries().collect(); @@ -465,7 +576,7 @@ fn lowered(workload: &PlanningWorkload, catalog: &SqlCatalog) -> Result = BTreeSet::new(); for (path, workload) in workloads { + let (queries, more_roots, more_entries, more_data) = + lowered(workload, catalog).map_err(|e| format!("--workload {path}: {e}"))?; match workload.query_workload.language { QueryLanguage::PromQL => { for entry in workload.query_workload.entries() { @@ -485,10 +598,9 @@ fn lowered_together( ); } } + QueryLanguage::MetricsQL => read_sources.extend(scanned_sources(&more_roots)), _ => read_sources.extend(sql_tables.iter().cloned()), } - let (queries, more_roots, more_entries, more_data) = - lowered(workload, catalog).map_err(|e| format!("--workload {path}: {e}"))?; described.extend(queries); roots.extend(more_roots); entries.extend(more_entries); @@ -541,7 +653,7 @@ fn lowered_together( } data.validate().map_err(|e| e.to_string())?; for (index, query) in described.iter_mut().enumerate() { - query["id"] = json!(format!("q{}", index + 1)); + query["id"] = json!(query_id(index)); } Ok((described, roots, entries, data)) } @@ -585,29 +697,188 @@ fn arrival_together(earlier: DataArrival, later: DataArrival) -> Option Result, String> { +) -> Result, String> { let plan = installable_plan_of(plan.clone(), entries).map_err(|e| e.to_string())?; - let views = plan.materialized_views().map_err(|e| e.to_string())?; - views + plan.materialized_views().map_err(|e| e.to_string()) +} + +/// Stage 2's physical candidates $\mathcal C$: each with its logical +/// candidate (`group`, `index_in_group`), its `properties` ($\tau$, +/// $\ell$) and, per input $(c, i)$, the input `group` and, as `edges`, +/// each physical edge whose logical edge holds: `producer`, +/// `satisfies_physical_requirement` ($\alpha^{\mathrm{phys}}$), `rejection` +/// (why it fails, null when it holds) and `selected` (an edge of `plan`). +/// In canonical id order (`ids`), each input's edges by producer. +fn physical_candidates_json( + physical: &PhysicalASAPMemoCandidates, + plan: Option<&PhysicalASAPDAG>, + ids: &CanonicalIds, +) -> Vec { + let memo = &physical.logical_candidates.memo; + let selected = selected_edges(plan); + let mut listed: Vec<(usize, Value)> = physical + .physical_candidates .iter() - .map(|view| serde_json::to_value(view).map_err(|e| e.to_string())) - .collect() + .zip(edges_per_input(physical)) + .enumerate() + .map(|(id, (candidate, edges_into))| { + let logical = candidate.logical_candidate_id; + let groups = memo.candidate(logical).expression.operator.children(); + let inputs: Vec = groups + .into_iter() + .zip(edges_into) + .enumerate() + .map(|(input, (group, edges))| { + let mut edges: Vec<(usize, Value)> = edges + .iter() + .filter(|edge| edge.satisfies_logical_requirement) + .map(|edge| { + let producer = edge.producer_candidate_id; + let canonical_producer = ids.physical(producer); + (canonical_producer, json!({ + "producer": canonical_producer, + "satisfies_physical_requirement": edge.satisfies_physical_requirement, + "rejection": physical_edge_rejection(physical, edge), + "selected": selected.contains(&(producer, PhysicalCandidateId(id), input)), + })) + }) + .collect(); + edges.sort_by_key(|(producer, _)| *producer); + let edges: Vec = edges.into_iter().map(|(_, edge)| edge).collect(); + json!({ "group": ids.group(*group), "edges": edges }) + }) + .collect(); + let canonical_id = ids.physical(PhysicalCandidateId(id)); + let (group, index_in_group) = ids.candidate(logical); + (canonical_id, json!({ + "id": canonical_id, + "group": group, + "index_in_group": index_in_group, + "properties": candidate.physical_properties, + "inputs": inputs, + })) + }) + .collect(); + listed.sort_by_key(|(id, _)| *id); + listed.into_iter().map(|(_, candidate)| candidate).collect() } -/// The physical candidates Stage 2 pruned -/// (`PhysicalASAPMemoCandidates::pruned_candidates`), in pruning order. -/// Empty without memo steps. -fn stage2_pruned_candidates_json(stage2: Option<&PhysicalASAPMemoCandidates>) -> Vec { - let Some(stage2) = stage2 else { - return Vec::new(); - }; - let _ = &stage2.pruned_candidates; - todo!("#1207 PR 9 (#1205): each pruned PhysicalCandidate with its reason") +/// Candidates Stage 2 or Stage 3 pruned: each logical candidate (`group`, +/// `index_in_group`), the `properties` it would have had and the `reason`; +/// by canonical candidate id (`ids`), then in pruning order. +fn pruned_candidates_json( + pruned: &[PrunedCandidate], + ids: &CanonicalIds, +) -> Vec { + let mut listed: Vec<((u32, usize), Value)> = pruned + .iter() + .map(|pruned| { + let (group, index_in_group) = ids.candidate(pruned.candidate.logical_candidate_id); + let reason = serde_json::to_value(pruned.reason).unwrap_or_default(); + ( + (group, index_in_group), + json!({ + "group": group, + "index_in_group": index_in_group, + "properties": pruned.candidate.physical_properties, + "reason": ids.in_serialized(reason), + }), + ) + }) + .collect(); + listed.sort_by_key(|(id, _)| *id); + listed.into_iter().map(|(_, pruned)| pruned).collect() +} + +/// The candidates Pass 2 eliminated: each candidate (`group`, +/// `index_in_group`) with its `causes`, each the eliminated consumers it +/// comes through (`through_pruned_candidates`), the `root` reason and its +/// `text`; by canonical candidate id (`ids`). +fn pass2_pruned_json(logical: &LogicalASAPMemoCandidates, ids: &CanonicalIds) -> Vec { + let mut listed: Vec<((u32, usize), Value)> = logical + .pruned + .iter() + .map(|pruned| { + let causes: Vec = pruned + .causes + .iter() + .map(|cause| { + let through: Vec = cause + .through_pruned_candidates + .iter() + .map(|id| ids.candidate_json(*id)) + .collect(); + let root = serde_json::to_value(&cause.root).unwrap_or_default(); + json!({ + "through_pruned_candidates": through, + "root": ids.in_serialized(root), + "text": root_prune_reason_text(&cause.root), + }) + }) + .collect(); + let (group, index_in_group) = ids.candidate(pruned.candidate); + ( + (group, index_in_group), + json!({ + "group": group, + "index_in_group": index_in_group, + "causes": causes, + }), + ) + }) + .collect(); + listed.sort_by_key(|(id, _)| *id); + listed.into_iter().map(|(_, pruned)| pruned).collect() +} + +/// One plan's cost: per second, its predicted CPU, its source, its usage +/// and each node's charge. +fn cost_json(cost: &PlanCost) -> Value { + let per_node: serde_json::Map<_, _> = cost + .per_node + .iter() + .map(|(node, charge)| { + ( + node.to_string(), + json!({ + "operator": charge.operator, + "timing": charge.timing, + "cpu_core_seconds_per_second": charge.cpu_core_seconds_per_second, + "cost_per_second": charge.cost_per_second, + "multiplier": charge.multiplier, + "detail": charge.detail, + "cost_source": charge.cost_source, + }), + ) + }) + .collect(); + let usage = &cost.usage; + json!({ + "cost_per_second": cost.cost_per_second, + // Report-only: the nodes' predicted CPU, summed as + // `plan_installable`'s predicted usage sums it. + "cpu_core_seconds_per_second": cost + .per_node + .values() + .map(|node| node.cpu_core_seconds_per_second) + .sum::(), + "source": cost.source, + "usage": { + "cpu_ops_per_second": usage.cpu_ops_per_second, + "scanned_bytes_per_second": usage.scanned_bytes_per_second, + "average_memory_bytes": usage.average_memory_bytes, + "peak_memory_bytes": usage.peak_memory_bytes, + "disk_bytes": usage.disk_bytes, + "peak_cpu_cores": usage.peak_cpu_cores, + }, + "per_node": per_node, + }) } /// Per query, the accuracy Stage 3 accepted: the accepted metric, the @@ -626,9 +897,14 @@ fn accepted_json(queries: &[AcceptedAccuracy]) -> Value { } /// Stage 3's plan and selection ([`Selection`]): the plan it built, the -/// accuracy accepted per query, the candidates it pruned and the selection -/// MILP's size and time; `error` when the facade selected no plan. -fn selection_json(selection: &Result<(PhysicalASAPDAG, Selection), SelectionError>) -> Value { +/// accuracy accepted per query, the candidates it pruned, the selection +/// MILP's size and its objective; `error` when the facade selected no +/// plan. Its times and solve count, which change from run to run or with +/// the solver, are in the text output only. +fn selection_json( + selection: &Result<(PhysicalASAPDAG, Selection), SelectionError>, + ids: &CanonicalIds, +) -> Value { let (plan, selection) = match selection { Ok(selected) => selected, Err(error) => return json!({ "plan": null, "error": error.to_string() }), @@ -636,17 +912,10 @@ fn selection_json(selection: &Result<(PhysicalASAPDAG, Selection), SelectionErro json!({ "plan": plan, "accepted_accuracy_per_query": accepted_json(&selection.accepted_accuracy_per_query), - "pruned_candidates": selection - .pruned_candidates - .iter() - .map(|p| json!({ "candidate": format!("{:?}", p.candidate), "reason": format!("{:?}", p.reason) })) - .collect::>(), - "milp_objective_cost_per_second": selection.selected_plan_cost_per_second, + "pruned_candidates": pruned_candidates_json(&selection.pruned_candidates, ids), "milp_variables": selection.milp_variable_count, "milp_constraints": selection.milp_constraint_count, - "milp_construction_ms": selection.milp_construction_ms, - "milp_solve_ms": selection.milp_solve_ms, - "milp_solve_count": selection.milp_solve_count, + "milp_objective_cost_per_second": selection.selected_plan_cost_per_second, }) } @@ -656,9 +925,26 @@ fn export(roots: &[QueryRoot]) -> FlatDag { flatten(roots).0 } +/// The sources the `Scan`s of `roots` read: metric names and table names. +fn scanned_sources(roots: &[QueryRoot]) -> BTreeSet { + flatten(roots) + .0 + .nodes + .into_iter() + .filter_map(|node| match node.operator { + Operator::NonASAP(NonASAPOp::Scan { source, .. }) => Some(match source { + Source::TimeSeries { metric } => metric, + Source::Table { table_ref } => table_ref, + }), + _ => None, + }) + .collect() +} + fn workload_queries(workload: &PlanningWorkload) -> Vec { let language = match workload.query_workload.language { QueryLanguage::PromQL => "promql", + QueryLanguage::MetricsQL => "metricsql", _ => "sql", }; workload @@ -678,7 +964,7 @@ fn workload_queries(workload: &PlanningWorkload) -> Vec { requirements["range_evaluation"] = json!(range); } json!({ - "id": format!("q{}", index + 1), + "id": query_id(index), "language": language, "text": entry.query.0, "requirements": requirements, @@ -708,69 +994,279 @@ fn with_default_metrics(mut queries: Vec, roots: &[QueryRoot]) -> Vec, -} - -/// `memo`'s groups, each with its candidates, their nodes added to -/// `table`; a candidate is `added` when `previous` (the memo at the step -/// before) has no such candidate. -fn memo_groups_json(table: &mut MemoNodes, memo: &Memo, previous: Option<&Memo>) -> Vec { - let ids = |groups: &[MemoGroupId]| groups.iter().map(|g| g.0).collect::>(); - memo.groups() +/// `memo`'s groups, each with its operator and its candidates, groups and +/// candidates in canonical id order (`ids`). At Pass 2 (`pass2`: the memo +/// at Pass 1 and Pass 2's output) a candidate is `added` when its index is +/// past the group's Pass 1 candidates (the memo only appends), has +/// `pruned`, and lists its logical edges per input, with the reasons Pass +/// 2 recorded. +fn memo_groups_json( + memo: &Memo, + pass2: Option<(&Memo, &LogicalASAPMemoCandidates)>, + ids: &CanonicalIds, +) -> Vec { + let group_ids = |groups: Vec<&MemoGroupId>| { + groups + .into_iter() + .map(|g| ids.group(*g)) + .collect::>() + }; + let pruned: BTreeSet = pass2 + .map(|(_, logical)| logical.pruned.iter().map(|p| p.candidate).collect()) + .unwrap_or_default(); + let mut listed_groups: Vec<(u32, Value)> = memo + .groups() .iter() .enumerate() .map(|(id, group)| { + let group_id = MemoGroupId(id as u32); let candidates = &group.candidates; - let node = group_node(table, group); let mut by_origin = serde_json::Map::new(); for candidate in candidates { let count = by_origin - .entry(origin_name(&candidate.origin)) + .entry(origin_name(candidate)) .or_insert(json!(0)); *count = json!(count.as_u64().unwrap_or(0) + 1); } - let listed: Vec = if candidates.is_empty() { - Vec::new() - } else { - let _ = previous; - todo!( - "#1207 PR 9 (#1205): each candidate's expression, properties and requirements" - ) - }; - json!({ - "id": id, - "label": node_label(&table.nodes[node]), - "children": group.expression.children().into_iter().map(|g| g.0).collect::>(), - "parents": ids(group.parent_groups()), + let mut listed: Vec<(usize, Value)> = candidates + .iter() + .enumerate() + .map(|(index, candidate)| { + let candidate_id = CandidateForMemoGroupId { + group: group_id, + index_in_group: index, + }; + let (_, canonical_index) = ids.candidate(candidate_id); + let mut value = candidate_json(canonical_index, candidate, ids); + if let Some((pass1, logical)) = pass2 { + value["added"] = json!(pass1 + .groups() + .get(id) + .is_none_or(|before| index >= before.candidates.len())); + value["pruned"] = json!(pruned.contains(&candidate_id)); + value["inputs"] = json!(logical_inputs_json(logical, candidate_id, ids)); + } + (canonical_index, value) + }) + .collect(); + listed.sort_by_key(|(index, _)| *index); + let listed: Vec = listed.into_iter().map(|(_, value)| value).collect(); + let canonical_group = ids.group(group_id); + (canonical_group, json!({ + "id": canonical_group, + "label": operator_label(&group.expression), + "operator": group.expression.map_children(|child| ids.group(*child)), + "children": group_ids(group.expression.children()), + "parents": group_ids(group.parent_groups().iter().collect()), + "result_kind": group.group_properties.result_kind, + "column_lineage": column_lineage_text(group, |group| ids.group(group)), + "column_signs": group.group_properties.column_signs, + "depends_on_evaluation_time": group.group_properties.depends_on_evaluation_time, + "shareable_when_equal": group.group_properties.shareable_when_equal, + "aggregate_coverage": group + .group_properties + .aggregate_coverage + .as_ref() + .map(|coverage| aggregate_coverage_text(coverage, |group| ids.group(group))), + "scanned_sources": group.group_properties.scanned_sources, + "evaluation_grid": pass2 + .and_then(|(_, logical)| logical.evaluation_grid_of_group[id].as_ref()) + .map(|grid| json!({"period_ms": grid.period_ms, "phase_ms": grid.phase_ms})), "queries": group.reached_by_queries(), - "roots_of": memo.queries_rooted_at(MemoGroupId(id as u32)), - "node": node, + "roots_of": memo.queries_rooted_at(group_id), "candidate_count": candidates.len(), "candidate_count_by_origin": by_origin, - "candidates_capped": false, "candidates": listed, - }) + })) + }) + .collect(); + listed_groups.sort_by_key(|(id, _)| *id); + listed_groups.into_iter().map(|(_, group)| group).collect() +} + +/// Per input of `consumer`, its `group` and, as `edges`, each candidate of +/// the group ([`logical_edges_into`], eliminated ones included): +/// `producer` (its index in the group), `satisfies_logical_requirement` +/// ($\alpha^{\log}$) and `rejection` (why it fails, null when it holds). +/// Ids are canonical (`ids`), edges by producer. +fn logical_inputs_json( + logical: &LogicalASAPMemoCandidates, + consumer: CandidateForMemoGroupId, + ids: &CanonicalIds, +) -> Vec { + let groups = logical + .memo + .candidate(consumer) + .expression + .operator + .children(); + groups + .into_iter() + .zip(logical_edges_into(logical, consumer)) + .map(|(group, edges)| { + let mut edges: Vec<(usize, Value)> = edges + .into_iter() + .enumerate() + .map(|(producer, status)| { + let (_, producer) = ids.candidate(CandidateForMemoGroupId { + group: *group, + index_in_group: producer, + }); + let rejection = status.rejection(); + ( + producer, + json!({ + "producer": producer, + "satisfies_logical_requirement": rejection.is_none(), + "rejection": rejection, + }), + ) + }) + .collect(); + edges.sort_by_key(|(producer, _)| *producer); + let edges: Vec = edges.into_iter().map(|(_, edge)| edge).collect(); + json!({ "group": ids.group(*group), "edges": edges }) }) .collect() } -/// The node of `group`'s own operator, added to `table`. -fn group_node(table: &mut MemoNodes, group: &MemoGroup) -> usize { - let _ = (table, group); - todo!("#1207 PR 9: a group's node from its expression") +/// One candidate $v = (e_v, p^{\log}_v)$ of a group: its `index`, +/// `origin`, `label`, `operator` $e_v$ (children are its input groups), +/// `children` (the input groups), `fused_groups`, `parameters` (θ) and +/// `required_accuracy` (one per input, null without one). Group ids are +/// canonical (`ids`). +fn candidate_json(index: usize, candidate: &CandidateForMemoGroup, ids: &CanonicalIds) -> Value { + let operator = &candidate.expression.operator; + let group_ids = |groups: Vec<&MemoGroupId>| { + groups + .into_iter() + .map(|g| ids.group(*g)) + .collect::>() + }; + json!({ + "index": index, + "origin": origin_name(candidate), + "added": false, + "label": operator_label(operator), + "operator": operator.map_children(|child| ids.group(*child)), + "children": group_ids(operator.children()), + "fused_groups": group_ids(candidate.fused_groups.iter().collect()), + "parameters": candidate.logical_properties.parameters(), + "required_accuracy": candidate + .input_requirements + .iter() + .map(|requirement| requirement.required_accuracy) + .collect::>(), + }) +} + +/// The plan's sharing summary: each node that two or more queries read, +/// directly or through other nodes, with its `operator`, `parameters` (θ), +/// `timing`, `lifetime` and the `queries` (`q1`, ...). Described by what +/// it computes, not by node id, so it stays the same when other parts of +/// the plan change; sorted by queries, then operator. Example: two +/// quantiles over one metric sharing a scan give `{"operator": "Scan", +/// "queries": ["q1", "q2"], ...}`. +fn sharing_summary_json(plan: &PhysicalASAPDAG) -> Vec { + let position: BTreeMap = plan + .nodes + .iter() + .enumerate() + .map(|(position, node)| (node.id, position)) + .collect(); + let mut queries_of_node: Vec> = vec![BTreeSet::new(); plan.nodes.len()]; + for (query, root) in plan.root_nodes_per_query().iter().enumerate() { + let mut pending: Vec = + root.operator_refs().into_iter().copied().collect(); + while let Some(id) = pending.pop() { + let node = position[&id]; + if queries_of_node[node].insert(query) { + pending.extend(plan.nodes[node].payload.children()); + } + } + } + // Sort key: queries, then operator, then the entry's JSON text (computed + // once per entry, not per comparison). + let mut shared: Vec<(Vec, String, String, Value)> = plan + .nodes + .iter() + .zip(queries_of_node) + .filter(|(_, queries)| queries.len() > 1) + .map(|(node, queries)| { + let operator = operator_label(&node.payload); + let names: Vec = queries.iter().map(|&q| query_id(q)).collect(); + let entry = json!({ + "operator": operator, + "parameters": node.parameters, + "timing": node.output_state.timing, + "lifetime": node.lifetime, + "queries": names, + }); + let text = entry.to_string(); + (queries.into_iter().collect(), operator, text, entry) + }) + .collect(); + shared.sort_by(|a, b| (&a.0, &a.1, &a.2).cmp(&(&b.0, &b.1, &b.2))); + shared.into_iter().map(|(_, _, _, entry)| entry).collect() +} + +/// The workload query id the documents use for query index `index` +/// (0-based): `0` gives `"q1"`, as in `workload.queries[].id`. +fn query_id(index: usize) -> String { + format!("q{}", index + 1) +} + +/// A candidate's origin, as `OriginOfCandidateForMemoGroup` serializes. +fn origin_name(candidate: &CandidateForMemoGroup) -> String { + json!(candidate.origin) + .as_str() + .unwrap_or_default() + .to_owned() } -/// `OriginOfCandidateForMemoGroup` in snake case. -fn origin_name(origin: &OriginOfCandidateForMemoGroup) -> String { - match origin { - OriginOfCandidateForMemoGroup::Pass1 => "pass1", - OriginOfCandidateForMemoGroup::TumblingWindows => "tumbling_windows", - OriginOfCandidateForMemoGroup::Rollup => "rollup", - OriginOfCandidateForMemoGroup::TopKReuse => "top_k_reuse", +#[cfg(test)] +mod tests { + use super::*; + + /// When Pass 1 fails, the document still carries the Stage 0 DAG, with + /// the error under Stage 1 and no Stage 2 or Stage 3 section, so the + /// viewer can show both. (Pass 1 fails only on a rule bug, so no + /// workload reaches this path end to end.) + #[test] + fn a_pass1_failure_still_writes_stage0_and_the_error() { + let workload: PlanningWorkload = serde_json::from_value(json!({ + "query_workload": {"language": "promql", "query_batch": [{ + "query": "sum(rate(m[5m]))", + "requirements": {"accuracy": {"max_error": 0.0}}, + "predictability": "unknown", + "invocations": 1, + }]}, + })) + .unwrap(); + let roots = lower_promql_roots(&workload).unwrap(); + let document = document_of( + workload_queries(&workload), + &roots, + Err(PlanningStagesError::Pass1( + "a rule built a bad group".into(), + )), + RunUntil::Stage3, + None, + None, + ) + .unwrap(); + assert_eq!( + document["stage0_query_language_frontends"]["dag"]["roots"] + .as_array() + .map(Vec::len), + Some(1), + "{document}" + ); + assert_eq!( + document["stage1_logical_asap_optimization"]["error"], + "Pass 1: a rule built a bad group" + ); + assert_eq!(document["stage2_physical_asap_optimization"], Value::Null); + assert_eq!(document["stage3_plan_selection"], Value::Null); } - .into() } diff --git a/crates/devtools/src/bin/planner_fuzz.rs b/crates/devtools/src/bin/planner_fuzz.rs index 7f0fc16ec..c08470612 100644 --- a/crates/devtools/src/bin/planner_fuzz.rs +++ b/crates/devtools/src/bin/planner_fuzz.rs @@ -1,7 +1,7 @@ // cargo run -p asap-devtools --bin planner_fuzz -- \ // --seed 1 --cases 200 --time-budget-s 600 --out /tmp/planner-fuzz // cargo run -p asap-devtools --bin planner_fuzz -- \ -// --seed 2 --cases 50 --relations a,b,tighter-accuracy --search candidate-connections \ +// --seed 2 --cases 50 --relations b,c,added-query --search candidate-connections \ // --seed-workload eval/rangeheavy.workload.json --out out // // Metamorphic fuzz testing for missed reuse and sharing @@ -14,11 +14,10 @@ // draft per distinct shrunk violation with its two workload files. Nothing // is filed. // -// `--relations` takes relation names or the letters a–o (default: all); -// `--search` is `candidate-connections` (the default search); -// `--ignore-accuracy-guarantees` accepts every output -// regardless of its accuracy guarantee (guarantees are enforced by -// default, as `plan_installable` does). A relation is violated when it misses by more +// `--relations` takes relation names or the letters b–o (default: all); +// `--search` is `candidate-connections` (the default search). Every +// workload is planned with accuracy not enforced (no query has an accuracy +// target), so a witness needs no accuracy check. A relation is violated when it misses by more // than max(`--relative-tolerance` (default 0.01) × cost, // `--absolute-tolerance` (default 1e-6)). `--max-shrink-attempts` (default // 60) bounds the checks one shrink runs. Each case and each shrink runs in @@ -34,7 +33,7 @@ use asap_devtools::planner_fuzz::{read_seed_file, FuzzSettings, SearchKind}; const USAGE: &str = "usage: planner_fuzz --seed --cases --time-budget-s \ [--relations ,...] [--search candidate-connections] \ -[--seed-workload ]... [--ignore-accuracy-guarantees] [--relative-tolerance ] \ +[--seed-workload ]... [--relative-tolerance ] \ [--absolute-tolerance ] [--max-shrink-attempts ] [--case-timeout-s ] --out "; fn main() { @@ -98,7 +97,6 @@ fn run(args: Vec) -> Result<(), String> { } } "--seed-workload" => seeds.push(read_seed_file(&PathBuf::from(value()?))?), - "--ignore-accuracy-guarantees" => settings.enforce_accuracy_guarantees = false, "--relative-tolerance" => { settings.relative_tolerance = value()?.parse().map_err(|e| format!("{e}"))? } diff --git a/crates/devtools/src/candidate_labels.rs b/crates/devtools/src/candidate_labels.rs index 2b5e06d3b..78cc51cda 100644 --- a/crates/devtools/src/candidate_labels.rs +++ b/crates/devtools/src/candidate_labels.rs @@ -1,16 +1,18 @@ -//! Display labels for the planning stages' candidates, for the -//! `asap_planner` viewer document and text -//! only. Planning never reads them: they -//! are pure functions of the candidates' planning data (the logical roots -//! and the physical properties assignment). +//! Display labels for the planning stages' operators and parameters, for +//! the `asap_planner` viewer document and text only. Planning never reads +//! them: they are pure functions of the candidates' planning data. use serde_json::Value; -/// One node's operator kind with its sketch family and parameters, its -/// summary query, its time range and its tumbling-window layout, when it has them: e.g. -/// `SummaryAgg Kll(k=200)`, `SummaryEstimate Quantile(q=0.5)`. -pub fn node_label(node: &asap_types::ir::flat::FlatNode) -> String { - let operator = serde_json::to_value(&node.operator).unwrap_or_default(); +use asap_types::ir::Operator; +use asap_types::summary_type::SketchParams; + +/// An operator's kind with its summary type, its summary query, its time +/// range and its tumbling-window width, when it has them, without its +/// inputs: e.g. `SummaryAgg Kll`, `SummaryEstimate Quantile(q=0.5)`, +/// `TimeRange 600s`. +pub fn operator_label(operator: &Operator) -> String { + let operator = serde_json::to_value(operator.map_children(|_| ())).unwrap_or_default(); // `{"ASAP": {"SummaryAgg": {...}}}` (or a unit variant's string). let (kind, fields) = match operator.as_object().and_then(|o| o.values().next()) { Some(Value::Object(inner)) => match inner.iter().next() { @@ -30,7 +32,22 @@ pub fn node_label(node: &asap_types::ir::flat::FlatNode) -> String { label } -/// Sketch algorithms with parameters, summary queries and time ranges found +/// A sketch's parameters θ: e.g. `Kll(k=269)`, `Cms(width=2719, depth=5)`. +pub fn parameters_label(parameters: &SketchParams) -> String { + let value = serde_json::to_value(parameters).unwrap_or_default(); + let Some((algorithm, fields)) = value.as_object().and_then(|o| o.iter().next()) else { + return value.to_string(); + }; + let fields: Vec = fields + .as_object() + .into_iter() + .flatten() + .map(|(key, value)| format!("{key}={value}")) + .collect(); + format!("{algorithm}({})", fields.join(", ")) +} + +/// Summary types, summary queries, measures, time ranges and windows found /// in an operator's fields (not inside its schemas). fn find_details(value: &Value, details: &mut Vec, depth: usize) { if depth > 6 || details.len() >= 4 { @@ -63,17 +80,6 @@ fn find_details(value: &Value, details: &mut Vec, depth: usize) { }; match value { Value::Object(fields) => { - if let (Some(Value::String(algorithm)), Some(params)) = - (fields.get("algorithm"), fields.get("params")) - { - let params = params - .as_object() - .and_then(|p| p.values().next()) - .map(compact) - .unwrap_or_default(); - details.push(format!("{algorithm}({params})")); - return; - } for (key, field) in fields { match key.as_str() { "schema" | "input_schema" | "output_schema" | "predicates" | "cols" => {} @@ -96,9 +102,11 @@ fn find_details(value: &Value, details: &mut Vec, depth: usize) { details.push(format!("{kind}({})", params.join(", "))); } } + // A summary type, e.g. `Kll` or `ExactAggregate(Rate)`. + "summary_type" => details.push(summary_type_text(field)), // A time range's length, e.g. `60s`. "range" => details.push(duration_text(field)), - // A window, and a tumbling-window chain's layout. + // A window, and a tumbling-window chain's width. "window" | "tumbling_window_width_ms" => details.push(format!("{key}={field}")), _ => find_details(field, details, depth + 1), } @@ -113,6 +121,33 @@ fn find_details(value: &Value, details: &mut Vec, depth: usize) { } } +/// A serialized summary type: `Kll` (`{"Sketch": "Kll"}`), +/// `ExactAggregate(Rate)` (`{"ExactAggregate": ["Rate", "Rate"]}`, the +/// distinct names), or the JSON of any other value. +fn summary_type_text(value: &Value) -> String { + match value { + Value::String(name) => name.clone(), + Value::Object(fields) if fields.len() == 1 => { + let (kind, inner) = fields.iter().next().expect("one field"); + let mut names: Vec = match inner { + Value::String(name) => vec![name.clone()], + Value::Array(items) => items + .iter() + .map(|item| item.as_str().map_or(item.to_string(), str::to_owned)) + .collect(), + other => vec![other.to_string()], + }; + names.dedup(); + if kind == "Sketch" { + names.join(", ") + } else { + format!("{kind}({})", names.join(", ")) + } + } + other => other.to_string(), + } +} + /// A serialized `Duration` (`{"secs": 60, "nanos": 0}`) as `60s`, or /// `1500ms` when it is not whole seconds; any other value as its JSON. fn duration_text(value: &Value) -> String { diff --git a/crates/devtools/src/lib.rs b/crates/devtools/src/lib.rs index 2b1e21f71..bf772cfdc 100644 --- a/crates/devtools/src/lib.rs +++ b/crates/devtools/src/lib.rs @@ -28,6 +28,7 @@ pub mod planner_layering_examples; pub mod planning_stages; pub mod planning_stages_text; pub mod rqe_atomic_costs; +pub mod stable_output; use asap_frontend_promql::lower_promql_query_workload; pub use asap_frontend_promql::PromqlError; diff --git a/crates/devtools/src/planner_fuzz/README.md b/crates/devtools/src/planner_fuzz/README.md index 56ebb9825..63feb5356 100644 --- a/crates/devtools/src/planner_fuzz/README.md +++ b/crates/devtools/src/planner_fuzz/README.md @@ -19,8 +19,8 @@ Audience: planner developers. ```sh cargo run --release -p asap-devtools --bin planner_fuzz -- \ --seed 1 --cases 1000 --time-budget-s 3600 --out out/run-1 \ - [--relations a,b,tighter-accuracy] [--search candidate-connections|exhaustive] \ - [--seed-workload eval/rangeheavy.workload.json]... [--ignore-accuracy-guarantees] \ + [--relations b,c,added-query] [--search candidate-connections] \ + [--seed-workload eval/rangeheavy.workload.json]... \ [--relative-tolerance 0.01] [--absolute-tolerance 1e-6] \ [--max-shrink-attempts 60] [--case-timeout-s 120] ``` @@ -79,7 +79,7 @@ planning that slow is a finding of its own. `tables.json` next to it. A file may instead be `{"workload": ..., "tables": [...]}`. Seed queries have no structured form. Only the relations that need no - structure apply to them: a, f, i, tighter-accuracy, added-query (when + structure apply to them: f, i, added-query (when structured), no-raw-retention, results-retained, relaxed-latency and reordered. @@ -88,13 +88,12 @@ planning that slow is a finding of its own. ### Marginal-cost relations For a query `q` of workload `W` and a query `q'` derivable from `q`, the tester -builds a **witness**. The witness is `W`'s selected logical plan (its Stage 1 -candidate) plus one more root that derives `q'` from `q`'s node in that plan, -by the operator the relation names. Three witnesses change that plan too: m -and n lengthen the tumbling-window chain `q` reads (so `q` merges its newest tumbling windows of -the longer chain); l's `q'` also reads a sibling query, so `W` is planned -with the sibling added; and h's second witness is `W ∪ {q'}` planned with -UnivMon as the only sketch family. Each is a plan for `W ∪ {q'}`. The tester +builds a **witness**. The witness is `W`'s selected plan (the +`PhysicalASAPDAG` Stage 3 built) plus one more root that derives `q'` from +`q`'s node in that plan, by the operator the relation names. Two witnesses +start from another plan: l's `q'` also reads a sibling query, so `W` is +planned with the sibling added; and h's second witness is `W ∪ {q'}` planned +with UnivMon as the only sketch algorithm. Each is a plan for `W ∪ {q'}`. The tester then checks: ```text @@ -102,47 +101,51 @@ cost(W ∪ {q'}) − cost(W) ≤ derive_cost + max(rel_tol · cost(W ∪ {q'}), derive_cost = cost(witness) − cost(W's plan) ``` -**How `derive_cost` is computed.** It comes from Stage 3's per-node prices, -never from a guess: - -1. Stage 2 and Stage 3 implement and price the witness the same way they price - any candidate. Each physical properties assignment goes through - `optimize_physical_asap` and `select_plan`, using the same cost model, - deployment capabilities and accuracy checks as the search. The cheapest assignment is kept. -2. `W`'s selected logical plan is priced the same way. This gives the witness +**How `derive_cost` is computed.** It comes from the cost model's per-node +costs, never from a guess: + +1. The witness is costed by `asap_cost::plan_cost`, as Stage 3 costs a + selected plan, with the same cost inputs as the search. Each node of `W`'s + plan keeps the timing, lifetime, θ, coverage and retention Stage 3 gave + it; each derived node runs at query time, not materialized, and a derived + `TimeRange` over a retained node (m's longer range over `q`'s chain) + widens that node's retention to its range. A node of `W`'s plan that `q'` + and a query in the same evaluation context (the same evaluation grid, as + Stage 3's contexts) both read at query time, not materialized, is + materialized for one evaluation, as Stage 3 may select. The witness holds + ASAP operators, which Pass 1 does not take as input, so it is not planned + again. +2. `W`'s selected plan is costed the same way. This gives the witness a reference point on the same footing. 3. `derive_cost` is the difference between the two. Under cost model 1, a - candidate's cost is the sum of its nodes' prices. So `derive_cost` is the - price of the derived operator alone, plus any change it causes in the - shared nodes' prices (for example, a shared node now materialized once - instead of computed per reader). + plan's cost is the sum of its nodes' costs. So `derive_cost` is the + cost of the derived operator alone, plus any change it causes in the + shared nodes' costs (for example, a shared node read once more). 4. Each draft also lists the witness's per-node cost of the nodes only `q'` reads, and their operators. Two rules decide whether a check counts: -- A witness that Stage 2, Stage 3 or validation rejects makes the check +- A witness that cannot be built, exported or costed makes the check *inconclusive*, never violated. -- A `Filter` on a group key keeps its input's guarantee in the witness. This - is because whole groups selected by an exact key keep each remaining row's - bound. Stage 1's guarantee composition gives no guarantee to any filter over - approximate rows. +- Every workload is planned with accuracy not enforced (no query has an + accuracy target), so no witness needs an accuracy check, and no relation + changes an accuracy target: the plannings would be of the same queries. | Relation | `q'` from `q` | Derived by (witness) | |---|---|---| -| `a-looser-accuracy` | `q` at the next looser target | reading `q`'s node | | `b-filter-on-group-key` | grouped `q` plus `label = 'value'` on a group key | `Filter` over `q`'s result | | `c-coarser-grouping` | `q` grouped by one key fewer | exact `Aggregate` (sum / max / min) over `q`'s result, or a sketch `SummaryMerge{group_by}` over `q`'s state | -| `d-nested-shorter-window` | `q` over a shorter window that divides it | `SummaryMerge{newest_tumbling_windows}` over `q`'s tumbling-window chain | +| `d-nested-shorter-window` | `q` over a shorter window that divides it | `q`'s `TimeRange` over its tumbling-window chain at the shorter window, over the same chain | | `e-other-quantile-rank` | `q` at another rank | `q`'s estimate (or exact quantile aggregate) at the other rank, same input | | `f-exact-duplicate` | `q` | reading `q`'s node | | `g-outer-over-result` | an operator over `q`, by `q`'s outer operator: `topk(3, q)` (sum), `bottomk(3, q)` (count), `max(q)` (max), `quantile(0.5, q)` (avg), `q > 0` (min, quantile) | the operator's lowering grafted onto `q`'s node | -| `h-other-statistic` | SQL, ungrouped: another frequency statistic of `q`'s column (distinct count → row count, L2 → distinct count, entropy → L2) | a `SummaryEstimate` of it from `q`'s state, when its family answers it (`SummaryTypeCapability::answers`); or `W ∪ {q'}` planned with UnivMon as the only sketch family, priced with every family | +| `h-other-statistic` | SQL, ungrouped: another frequency statistic of `q`'s column (distinct count → row count, L2 → distinct count, entropy → L2) | a `SummaryEstimate` of it from `q`'s state, when its summary type answers it (`SummaryTypeCapability::answers`); or `W ∪ {q'}` planned with UnivMon as the only sketch algorithm, costed with every algorithm | | `i-other-repeat-interval` | `q` at a longer repeat interval | reading `q`'s node | | `j-smaller-topk` | `q`'s top-k at k = 3 | `q`'s `Limit` / `TopK` at 3 over the same input | | `k-union-of-selections` | `q` over `label =~ "v1\|v2"`, with `q`'s `v2` twin added to `W` | `SummaryMerge` of the two states, then `q`'s estimate | | `l-binary-over-result` | PromQL: `q{label="v"} / q` on a label `q` does not group by, or `q / q_count` for a `sum_over_time` `q` with no outer operator or `sum` (avg as sum/count) | the division's lowering grafted onto `q`'s node | -| `m-longer-window` | `q` over the shortest longer window (up to 1 h) that is a multiple of `q`'s; not for `rate` or `increase` | `q`'s tumbling-window chain lengthened: every merge of it in `W`'s plan reads its old newest tumbling windows, and `q'` merges the whole chain | +| `m-longer-window` | `q` over the shortest longer window (up to 1 h) that is a multiple of `q`'s; not for `rate` or `increase` | `q`'s `TimeRange` over its tumbling-window chain at the longer window, over the same chain | | `n-longer-window-rollup` | `q` over a longer window, as in m, grouped by one key fewer | m's derivation, then c's rollup | | `o-equivalent-range-function` | `q` with `rate` and `increase` swapped (not under `count`) | `q`'s result times (or divided by) the window in seconds, grafted onto `q`'s node | @@ -152,7 +155,6 @@ Two rules decide whether a check counts: | Relation | `W'` | Holds when | |---|---|---| -| `tighter-accuracy` | one query's target one step tighter | `cost(W') ≥ cost(W) − tol` | | `added-query` | `W` plus a random query | `cost(W') ≥ cost(W) − tol` | | `no-raw-retention` | raw data not retained | `cost(W') ≥ cost(W) − tol` | | `results-retained` | query-time results retained across evaluations | `cost(W') ≤ cost(W) + tol` | @@ -161,8 +163,8 @@ Two rules decide whether a check counts: | `reordered` | queries in reverse order | `|cost(W') − cost(W)| ≤ tol` | | `disjoint-copy` | `W` plus a copy over renamed sources | `|cost(W') − 2·cost(W)| ≤ tol` | -When a monotonicity relation is violated, the draft also prices the other -side's logical plan for this side's workload. That is a concrete plan the +When a monotonicity relation is violated, the draft also costs the other +side's plan for this side's workload. That is a concrete plan the search could have selected. ## Shrinking @@ -196,7 +198,7 @@ status column, "conclusive" means the check either held or was violated. | Pattern | Doc reference | Relation | Status on main (one-hour run) | |---|---|---|---| -| Looser accuracy target reads the tighter build (an exact result meets any ε) | accuracy-models.md "Sharing across consumers" L308–368; planner-layering.md L530–533, L640–641 | `a-looser-accuracy` | VIOLATED (208 of 2,596 conclusive checks): #903 | +| Looser accuracy target reads the tighter build (an exact result meets any ε) | accuracy-models.md "Sharing across consumers" L308–368; planner-layering.md L530–533, L640–641 | — | not checked: the fuzzer plans with accuracy not enforced (an earlier run with accuracy enforced found #903) | | Slice: a filter on a group key over a grouped result or summary (planned) | asap-primitive-schema.md "Other uses of coverage" L846–848, L118 | `b-filter-on-group-key` | VIOLATED (995 of 1,003): #891, #902 (#689 for approximate inputs) | | Rollup to a coarser grouping: exact, or sketch `SummaryMerge{group_by}` | planner-layering.md rule table L588; asap-primitive-schema.md §5.6 L670, L754–794, L830–844; joint-planning-search.md L324 | `c-coarser-grouping` | VIOLATED (167 of 389): new drafts (exact rollup; sketch rollup over an inner range function) | | Scan widening: scans of one metric with different labels share one scan | planner-layering.md Pass 2 L597–607 | `c-coarser-grouping` (sketch rollup witness) | covered by c; no separate violation | @@ -208,7 +210,7 @@ status column, "conclusive" means the check either held or was violated. | gcd tumbling-window grid across repeat intervals | planner-layering.md L791–809 | `i-other-repeat-interval` | VIOLATED (49 of 2,717): new draft; compare #731 | | Exact top-k reuse (smaller k) | planner-layering.md rule table L589; joint-planning-search.md L325 | `j-smaller-topk` | holds (62 conclusive checks) | | Merge of disjoint selections | asap-primitive-schema.md §4.2 L194–232, §5.6 Example B L721–752 | `k-union-of-selections` | never conclusive: Stage 1 composes no guarantee for the merged estimate | -| Tumbling windows kept across evaluations when query-time results are retained | planner-layering.md Materialization L889–893 | `results-retained` | holds (1,442 conclusive checks) | +| Tumbling windows materialized across evaluations when query-time results are retained | planner-layering.md Materialization L889–893 | `results-retained` | holds (1,442 conclusive checks) | | Window = interval: the whole window as a one-tumbling-window chain | planner-layering.md Materialization L895–905 | — | not checked: a Stage 2 choice within one query | | Rollup over newest tumbling windows (windowed rollup) | planner-layering.md L809–823 | — | not checked: a composition of c and d | | avg as sum/count: shared exact accumulators | planner-layering.md L372–383, L502–503; ASAPWorkloads #8 | `l-binary-over-result` | holds (15 conclusive checks, with the ratio row below); checks that the sum is shared, not one accumulator for both | @@ -220,4 +222,3 @@ status column, "conclusive" means the check either held or was violated. Monotonicity and invariance relations (no doc pattern; they guard the search): - `added-query`, `no-raw-retention`, `relaxed-latency`, `renamed`, `reordered` and `disjoint-copy` held in every conclusive check. -- `tighter-accuracy` was violated 4 times in 2,152 checks. One case is a default-search miss: `--exhaustive` finds the plan. The others are #903. diff --git a/crates/devtools/src/planner_fuzz/generate.rs b/crates/devtools/src/planner_fuzz/generate.rs index 9efc183c3..078a82844 100644 --- a/crates/devtools/src/planner_fuzz/generate.rs +++ b/crates/devtools/src/planner_fuzz/generate.rs @@ -165,30 +165,6 @@ pub fn accuracy_ladder() -> [AccuracyTarget; 4] { ] } -/// The next looser target than `target` on [`accuracy_ladder`], or `None` -/// at the loosest. A target not on the ladder loosens to ε = 0.1. -pub fn looser(target: &AccuracyTarget) -> Option { - let ladder = accuracy_ladder(); - match ladder.iter().position(|t| t == target) { - Some(i) if i + 1 < ladder.len() => Some(ladder[i + 1].clone()), - Some(_) => None, - None if target.max_error < 0.1 => Some(ladder[3].clone()), - None => None, - } -} - -/// The next tighter target than `target` on [`accuracy_ladder`], or `None` -/// when it is exact. A target not on the ladder tightens to exact. -pub fn tighter(target: &AccuracyTarget) -> Option { - let ladder = accuracy_ladder(); - match ladder.iter().position(|t| t == target) { - Some(0) => None, - Some(i) => Some(ladder[i - 1].clone()), - None if target.is_exact() => None, - None => Some(AccuracyTarget::exact()), - } -} - impl PromqlSpec { pub fn render(&self) -> String { let selector = if self.matchers.is_empty() { @@ -743,14 +719,4 @@ mod tests { WHERE ts > now() - INTERVAL '5' MINUTE AND city = 'NYC' GROUP BY city" ); } - - /// The ladder loosens and tightens one step at a time. - #[test] - fn accuracy_ladder_steps() { - let ladder = accuracy_ladder(); - assert_eq!(looser(&ladder[0]), Some(ladder[1].clone())); - assert_eq!(looser(&ladder[3]), None); - assert_eq!(tighter(&ladder[1]), Some(ladder[0].clone())); - assert_eq!(tighter(&ladder[0]), None); - } } diff --git a/crates/devtools/src/planner_fuzz/mod.rs b/crates/devtools/src/planner_fuzz/mod.rs index 7ec726e99..91904f311 100644 --- a/crates/devtools/src/planner_fuzz/mod.rs +++ b/crates/devtools/src/planner_fuzz/mod.rs @@ -4,21 +4,27 @@ //! answers stay correct, and an exhaustive search over the generated //! candidates cannot see it. The tester finds one by relating two //! workloads. For a query `q` of a workload `W` and a query `q'` derivable -//! from `q`'s result or state (the same query with a looser accuracy -//! target, a filter on a group key, a coarser grouping, ...), it builds a +//! from `q`'s result or state (a filter on a group key, a coarser +//! grouping, ...), it builds a //! *witness*: `W`'s selected logical plan plus `q'` derived from `q`'s -//! node by one operator. Stage 2 and Stage 3 implement and cost the -//! witness exactly as they cost any candidate, and the relation +//! node by one operator. The witness is costed by `asap_cost::plan_cost`, +//! as Stage 3 costs its plan: each node of `W`'s plan keeps the timing, +//! lifetime, θ, coverage and retention Stage 3 gave it, and each derived +//! node runs at query time, not materialized (the witness holds ASAP +//! operators, which Pass 1 does not take as input, so it is not planned +//! again). The relation //! //! ```text //! cost(W ∪ {q'}) − cost(W) ≤ derive_cost + tolerance, //! derive_cost = cost(witness) − cost(W's plan costed the same way) //! ``` //! -//! must hold. `derive_cost` is what Stage 3 charges for the derived +//! must hold. `derive_cost` is what the cost model charges for the derived //! operator alone (its per-node cost, plus any change it causes in the -//! shared nodes' costs, such as a shared node now materialized); it is -//! never a guess. A violation means a cheaper plan for `W ∪ {q'}` exists +//! shared nodes' costs, such as a shared node now materialized for one +//! evaluation); it is never a guess. The tester plans with accuracy not +//! enforced (every query without an accuracy target), so a witness needs +//! no accuracy check. A violation means a cheaper plan for `W ∪ {q'}` exists //! that the planner did not select. Monotonicity and invariance relations //! (tightening accuracy, adding a query or dropping raw retention never //! lowers cost; relaxing a latency bound never raises it; renaming, @@ -34,21 +40,29 @@ pub mod relations; pub mod report; pub mod shrink; -use std::collections::{BTreeMap, BTreeSet, HashMap}; +use std::collections::{BTreeSet, HashMap}; use std::rc::Rc; use serde::{Deserialize, Serialize}; use serde_json::{json, Value}; -use asap_cost::CostInputs; +use asap_cost::{plan_cost, CostInputs}; use asap_planner::{PlanSearchSettings, PlanningModels}; use asap_types::accuracy::AccuracyTarget; use asap_types::deployment_capabilities::DeploymentCapabilities; -use asap_types::ir::QueryRoot; +use asap_types::ir::flat::flatten; +use asap_types::ir::properties::execution_data_state::{ + DataPrimitive, ExecutionDataState, ExecutionTiming, +}; +use asap_types::ir::{NonASAPOp, OperatorNode, QueryRoot}; +use asap_types::plan::physical_asap_dag::{ + PhysicalASAPDAG, PhysicalASAPDAGNode, PhysicalASAPNodeId, +}; +use asap_types::stages::{Lifetime, StorageMedium}; use asap_types::summary_type::SketchAlgorithm; use asap_types::workload::{ - DataArrival, DataWorkload, PlanningWorkload, QueryLanguage, QueryWorkload, RepeatingEntry, - SqlDialect, + DataArrival, DataWorkload, PlanningWorkload, QueryLanguage, QueryWorkload, QueryWorkloadEntry, + RepeatingEntry, SqlDialect, }; use crate::planning_stages::{lower_promql_roots, lower_sql_roots, run_search_over, sql_catalog}; @@ -217,9 +231,9 @@ impl FuzzWorkload { } /// The `asap_planner` invocation (a bash command) that plans this - /// workload, saved as `workload_file`, with `settings`' accuracy - /// guarantees. Its output document is `.out.json`. - pub fn asap_planner_command(&self, workload_file: &str, settings: &FuzzSettings) -> String { + /// workload, saved as `workload_file`, with accuracy not enforced, as + /// the tester plans. Its output document is `.out.json`. + pub fn asap_planner_command(&self, workload_file: &str) -> String { let mut command = String::new(); command.push_str(&format!( "cargo run -p asap-devtools --bin asap_planner -- --workload {workload_file} \ @@ -228,9 +242,7 @@ impl FuzzWorkload { for table in &self.tables { command.push_str(&format!(" \\\n --sql-catalog-table '{table}'")); } - if !settings.enforce_accuracy_guarantees { - command.push_str(" --ignore-accuracy-guarantees"); - } + command.push_str(" --ignore-accuracy-guarantees"); if !self.raw_data_retained || self.query_time_results_retained { command.push_str(&format!( " --deployment-capabilities <(echo '{{\"raw_data_retained\": {}, \"query_time_results_retained\": {}}}')", @@ -311,9 +323,6 @@ pub enum SearchKind { #[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] pub struct FuzzSettings { pub search: SearchKind, - /// Whether Stage 3 rejects an output whose accuracy guarantee misses its - /// target, as `plan_installable` does (default `true`). - pub enforce_accuracy_guarantees: bool, /// A relation is violated when it misses by more than /// `max(relative_tolerance × cost, absolute_tolerance)`, `cost` the /// larger side's cost per second. @@ -325,7 +334,6 @@ impl Default for FuzzSettings { fn default() -> Self { Self { search: SearchKind::CandidateConnections, - enforce_accuracy_guarantees: true, relative_tolerance: 0.01, absolute_tolerance: 1e-6, } @@ -341,21 +349,26 @@ impl FuzzSettings { /// One workload planned by the planner's search. #[derive(Debug)] pub struct Planned { - /// The selected physical plan's stable archive ID. + /// The selected plan, labelled for reports ([`plan_label`]). pub selected: String, /// Its cost per second under cost model 1. pub cost_per_second: f64, /// The frontends' roots, one per query. pub stage0_roots: Vec, - /// The selected Stage 1 (logical) candidate's roots, the plan a witness - /// extends; absent only when the selected logical plan is unavailable. - pub logical_roots: Option>, + /// The plan Stage 3 selected. + pub plan: PhysicalASAPDAG, + /// `plan`'s query roots as operator trees, one per query, a node the + /// plan shares shared: the roots a witness extends. + pub roots: Vec, + /// The plan node of each node of `roots`, by the node's address + /// (`roots` holds the nodes, so no address is reused). + node_of: HashMap<*const OperatorNode, PhysicalASAPNodeId>, } -/// A plan costed by Stage 2 and Stage 3 outside the search: the cheapest -/// physical properties assignment of given logical roots. +/// A plan costed outside the search ([`Planner::cost`]). #[derive(Debug, Clone, PartialEq, Serialize)] pub struct Costed { + /// The plan costed, labelled for reports ([`plan_label`]). pub selected: String, pub cost_per_second: f64, /// The nodes only the last root reads: in a witness, the operators @@ -372,8 +385,80 @@ pub struct PrivateNodes { pub operators: Vec, } +/// `plan`'s query roots as operator trees (each plan node one shared +/// node) and the plan node of each. +fn operator_trees( + plan: &PhysicalASAPDAG, +) -> ( + Vec, + HashMap<*const OperatorNode, PhysicalASAPNodeId>, +) { + // Stage 3 numbers producers first. + let mut tree_of: Vec> = Vec::with_capacity(plan.nodes.len()); + for node in &plan.nodes { + let operator = node + .payload + .map_children(|&child| Rc::clone(&tree_of[child])); + tree_of.push(Rc::new(OperatorNode::with_schema( + operator, + node.output_schema.clone(), + ))); + } + let node_of = tree_of + .iter() + .enumerate() + .map(|(id, tree)| (Rc::as_ptr(tree), id)) + .collect(); + let roots = plan + .root_nodes_per_query() + .into_iter() + .map(|root| match root { + QueryRoot::Operator(id) => QueryRoot::Operator(Rc::clone(&tree_of[id])), + QueryRoot::Scalar(expression) => { + QueryRoot::Scalar(expression.map_operator_refs(&mut |&id| Rc::clone(&tree_of[id]))) + } + }) + .collect(); + (roots, node_of) +} + +/// `plan`'s nodes in node order, each its operator, marked when it runs at +/// ingestion time or is materialized across evaluations: e.g. `Scan +/// (ingestion), SummaryTumblingWindowChain ExactAggregate(Sum) +/// tumbling_window_width_ms=60000 (ingestion, materialized), TimeRange +/// 1800s, ...`. +fn plan_label(plan: &PhysicalASAPDAG) -> String { + plan.nodes + .iter() + .map(|node| { + let mut marks = Vec::new(); + if node.output_state.timing == ExecutionTiming::IngestionTime { + marks.push("ingestion"); + } + if matches!(node.lifetime, Lifetime::AcrossEvaluations(_)) { + marks.push("materialized"); + } + let label = crate::candidate_labels::operator_label(&node.payload); + match marks.is_empty() { + true => label, + false => format!("{label} ({})", marks.join(", ")), + } + }) + .collect::>() + .join(", ") +} + +/// The grid of Stage 3's evaluation context of `entry`'s query: its +/// evaluation grid, or 1 ms when its evaluation times are not known. +fn evaluation_grid_ms(entry: &QueryWorkloadEntry) -> u64 { + entry + .recurrence + .evaluation_grid_ms(entry.range_evaluation) + .unwrap_or(1) +} + /// Plans and costs workloads, caching each workload's plan and the -/// reference cost of its selected logical plan. +/// reference cost of its selected plan. pub struct Planner { pub settings: FuzzSettings, cache: HashMap, String>>, @@ -405,7 +490,8 @@ impl Planner { } } - /// `workload` planned by the configured search. + /// `workload` planned by the configured search, with accuracy not + /// enforced. pub fn plan(&mut self, workload: &FuzzWorkload) -> Result, String> { let key = serde_json::to_string(workload).expect("a workload serializes"); if let Some(planned) = self.cache.get(&key) { @@ -426,7 +512,7 @@ impl Planner { let models = PlanningModels::builtin() .with_cost_inputs(&cost_inputs) .with_deployment_capabilities(&deployment) - .with_ignore_accuracy_guarantees(!self.settings.enforce_accuracy_guarantees); + .with_ignore_accuracy_guarantees(true); let run = match self.settings.search { SearchKind::CandidateConnections => run_search_over( roots.clone(), @@ -437,68 +523,154 @@ impl Planner { ), } .map_err(|e| e.to_string())?; - let (_plan, _selection) = run - .run - .selection - .as_ref() - .map_err(|e| format!("selection: {e}"))?; - todo!("#1207 PR 9: the selected plan's cost (plan_cost) and roots from its PhysicalASAPDAG") + let (plan, _) = run.run.selection.map_err(|e| format!("selection: {e}"))?; + let (cost, _) = plan_cost(&plan, &entries, &data, &cost_inputs) + .map_err(|(node, e)| format!("plan cost, node {node}: {e}"))?; + let cost_per_second = cost + .cost_per_second + .ok_or("no cost per second (cost model 3)")?; + let (plan_roots, node_of) = operator_trees(&plan); + Ok(Planned { + selected: plan_label(&plan), + cost_per_second, + stage0_roots: roots, + plan, + roots: plan_roots, + node_of, + }) } - /// The planning stages over the logical plan `roots` of `workload`'s - /// queries (one root per query, in order), with the same models as - /// [`plan`](Self::plan): the selected plan's cost (`plan_cost`) and the - /// cost of the nodes only its last root reads. - pub fn cost(&self, workload: &FuzzWorkload, roots: &[QueryRoot]) -> Result { + /// The cost of the plan `roots` of `workload`'s queries (one root per + /// query, in order) under the same cost inputs as [`plan`](Self::plan), + /// by `asap_cost::plan_cost`: each node of `planned`'s plan keeps the + /// timing, lifetime, θ, coverage and retention Stage 3 gave it, and + /// every other node (a witness's derived operators) runs at query time, + /// not materialized. A `TimeRange` derived over a node that is retained + /// widens its retention to its range. With `last_root_is_derived` (a + /// witness, whose last root is `q'`), a query-time node that is not + /// materialized and is read by the last root and by another root in the + /// same evaluation context (the same evaluation grid, as Stage 3's + /// contexts) is materialized for one evaluation instead + /// (`OneEvaluation(Memory)`): built once per evaluation and read by + /// both, as Stage 3 may select. `derived_root` is the nodes only the + /// last root reads. + pub fn cost( + &self, + workload: &FuzzWorkload, + planned: &Planned, + roots: &[QueryRoot], + last_root_is_derived: bool, + ) -> Result { let planning = workload.planning_workload(); let entries: Vec<_> = planning.query_workload.entries().collect(); - let cost_inputs = self.cost_inputs(); - let deployment = workload.deployment(); - let models = PlanningModels::builtin() - .with_cost_inputs(&cost_inputs) - .with_deployment_capabilities(&deployment) - .with_ignore_accuracy_guarantees(!self.settings.enforce_accuracy_guarantees); - let run = run_search_over( - roots.to_vec(), - &entries, - &workload.data, - models, - PlanSearchSettings::default(), - ) - .map_err(|e| format!("witness: {e}"))?; - let (plan, _) = run - .run - .selection - .as_ref() - .map_err(|e| format!("witness selection: {e}"))?; - let (cost, _) = asap_cost::plan_cost(plan, &entries, &workload.data, &cost_inputs) - .map_err(|(node, error)| format!("witness cost, node {node}: {error}"))?; - let cost_per_second = cost - .cost_per_second - .ok_or("no cost per second (cost model 3)")?; - let mut inputs: BTreeMap<_, Vec<_>> = BTreeMap::new(); - for edge in &plan.edges { - inputs.entry(edge.consumer).or_default().push(edge.producer); + if entries.len() != roots.len() { + return Err(format!( + "{} roots for {} queries", + roots.len(), + entries.len() + )); } - let reached = |root| { - let mut seen = BTreeSet::new(); - let mut stack = vec![root]; - while let Some(node) = stack.pop() { - if seen.insert(node) { - stack.extend(inputs.get(&node).into_iter().flatten().copied()); + let (flat, trees) = flatten(roots); + // The nodes each root reads, by id in `flat`. + let reached_by_root: Vec> = flat + .roots + .iter() + .map(|root| { + let mut seen = BTreeSet::new(); + let mut stack: Vec = root.operator_refs().into_iter().copied().collect(); + while let Some(node) = stack.pop() { + if seen.insert(node) { + stack.extend(flat.nodes[node].operator.children().into_iter().copied()); + } } + seen + }) + .collect(); + let (last, others) = reached_by_root.split_last().ok_or("no roots")?; + let shared: BTreeSet = match last_root_is_derived { + true => { + let grid = evaluation_grid_ms(&entries[entries.len() - 1]); + others + .iter() + .zip(&entries) + .filter(|(_, entry)| evaluation_grid_ms(entry) == grid) + .flat_map(|(reached, _)| reached.intersection(last).copied()) + .collect() } - seen + false => BTreeSet::new(), }; - let reached_by_root: Vec> = plan.roots.iter().map(|r| reached(*r)).collect(); - let (last, others) = reached_by_root.split_last().ok_or("no roots")?; + let mut nodes: Vec = flat + .nodes + .into_iter() + .zip(&trees) + .enumerate() + .map(|(id, (flat_node, tree))| { + let mut node = match planned.node_of.get(&Rc::as_ptr(tree)) { + Some(&planned_id) => PhysicalASAPDAGNode { + id, + payload: flat_node.operator, + ..planned.plan.nodes[planned_id].clone() + }, + None => PhysicalASAPDAGNode { + id, + output_state: ExecutionDataState { + timing: ExecutionTiming::QueryTime, + primitive: DataPrimitive::of(&flat_node.operator, tree.result_kind), + }, + payload: flat_node.operator, + lifetime: Lifetime::None, + parameters: None, + output_schema: tree.schema.clone(), + retention_ms: None, + view_coverage: None, + selected_physical_candidate: None, + }, + }; + if shared.contains(&id) + && node.output_state.timing == ExecutionTiming::QueryTime + && node.lifetime == Lifetime::None + { + node.lifetime = Lifetime::OneEvaluation(StorageMedium::Memory); + } + node + }) + .collect(); + for (id, tree) in trees.iter().enumerate() { + let Some(NonASAPOp::TimeRange { range, .. }) = tree.non_asap() else { + continue; + }; + if planned.node_of.contains_key(&Rc::as_ptr(tree)) { + continue; + } + let range_ms = u64::try_from(range.as_millis()).unwrap_or(u64::MAX); + for &child in nodes[id].payload.clone().children() { + if let Some(retention_ms) = &mut nodes[child].retention_ms { + *retention_ms = (*retention_ms).max(range_ms); + } + } + } + let mut operator_roots = Vec::new(); + let mut scalar_roots = Vec::new(); + for (query, root) in flat.roots.into_iter().enumerate() { + match root { + QueryRoot::Operator(id) => operator_roots.push(id), + QueryRoot::Scalar(expression) => scalar_roots.push((query, expression)), + } + } + let dag = PhysicalASAPDAG::new(nodes, operator_roots, scalar_roots) + .map_err(|e| format!("witness: {e}"))?; + let (cost, _) = plan_cost(&dag, &entries, &workload.data, &self.cost_inputs()) + .map_err(|(node, e)| format!("witness: node {node}: {e}"))?; + let cost_per_second = cost + .cost_per_second + .ok_or("no cost per second (cost model 3)")?; let private: Vec<_> = last .iter() - .filter(|node| others.iter().all(|set| !set.contains(node))) + .filter(|node| others.iter().all(|reached| !reached.contains(node))) .filter_map(|node| cost.per_node.get(node)) .collect(); Ok(Costed { - selected: "selected".into(), + selected: plan_label(&dag), cost_per_second, derived_root: PrivateNodes { cost_per_second: private.iter().filter_map(|n| n.cost_per_second).sum(), @@ -510,19 +682,19 @@ impl Planner { }) } - /// [`cost`](Self::cost) of `workload`'s selected logical plan - /// `roots`, cached per workload: every marginal check of a case costs - /// the same `W`. + /// [`cost`](Self::cost) of `planned`, `workload`'s selected plan, + /// cached per workload: every marginal check of a case costs the same + /// `W`. pub fn reference_cost( &mut self, workload: &FuzzWorkload, - roots: &[QueryRoot], + planned: &Planned, ) -> Result { let key = serde_json::to_string(workload).expect("a workload serializes"); if let Some(costed) = self.reference_costs.get(&key) { return costed.clone(); } - let costed = self.cost(workload, roots); + let costed = self.cost(workload, planned, &planned.roots, false); self.reference_costs.insert(key, costed.clone()); costed } diff --git a/crates/devtools/src/planner_fuzz/relations/mod.rs b/crates/devtools/src/planner_fuzz/relations/mod.rs index 127516b81..b5b6ab13d 100644 --- a/crates/devtools/src/planner_fuzz/relations/mod.rs +++ b/crates/devtools/src/planner_fuzz/relations/mod.rs @@ -26,8 +26,7 @@ use asap_types::summary_type::SummaryTypeCapability; use asap_types::summary_type::{SketchAlgorithm, SketchStatistic}; use super::generate::{ - label_value, looser, tighter, OuterAggregation, OuterOp, PromqlSpec, QuerySpec, RangeFunction, - SqlAggregate, + label_value, OuterAggregation, OuterOp, PromqlSpec, QuerySpec, RangeFunction, SqlAggregate, }; use super::{Costed, FuzzQuery, FuzzWorkload, Language, Planned, Planner}; @@ -35,8 +34,6 @@ use super::{Costed, FuzzQuery, FuzzWorkload, Language, Planned, Planner}; #[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, PartialOrd, Ord, Serialize, Deserialize)] #[serde(rename_all = "snake_case")] pub enum Relation { - /// (a) `q'` = `q` with a looser accuracy target: derived by reading `q`. - LooserAccuracy, /// (b) `q'` = grouped `q` plus an equality filter on a group key: /// derived by a selection (`Filter`) over `q`'s result. FilterOnGroupKey, @@ -84,8 +81,6 @@ pub enum Relation { /// (o) `q'` = `q` with `rate` and `increase` swapped: derived by scaling /// `q`'s result by the window length. EquivalentRangeFunction, - /// Tightening a query's accuracy target never lowers cost. - TighterAccuracy, /// Adding a query never lowers cost. AddedQuery, /// Dropping raw-data retention never lowers cost. @@ -104,8 +99,7 @@ pub enum Relation { } impl Relation { - pub const ALL: [Relation; 23] = [ - Relation::LooserAccuracy, + pub const ALL: [Relation; 21] = [ Relation::FilterOnGroupKey, Relation::CoarserGrouping, Relation::NestedShorterWindow, @@ -120,7 +114,6 @@ impl Relation { Relation::LongerWindow, Relation::LongerWindowRollup, Relation::EquivalentRangeFunction, - Relation::TighterAccuracy, Relation::AddedQuery, Relation::NoRawRetention, Relation::ResultsRetained, @@ -130,10 +123,9 @@ impl Relation { Relation::DisjointCopy, ]; - /// The CLI name, e.g. `a-looser-accuracy`. + /// The CLI name, e.g. `b-filter-on-group-key`. pub fn name(self) -> &'static str { match self { - Relation::LooserAccuracy => "a-looser-accuracy", Relation::FilterOnGroupKey => "b-filter-on-group-key", Relation::CoarserGrouping => "c-coarser-grouping", Relation::NestedShorterWindow => "d-nested-shorter-window", @@ -148,7 +140,6 @@ impl Relation { Relation::LongerWindow => "m-longer-window", Relation::LongerWindowRollup => "n-longer-window-rollup", Relation::EquivalentRangeFunction => "o-equivalent-range-function", - Relation::TighterAccuracy => "tighter-accuracy", Relation::AddedQuery => "added-query", Relation::NoRawRetention => "no-raw-retention", Relation::ResultsRetained => "results-retained", @@ -159,7 +150,7 @@ impl Relation { } } - /// Parses a CLI name or, for (a) to (o), its letter. + /// Parses a CLI name or, for (b) to (o), its letter. pub fn parse(text: &str) -> Option { Self::ALL.into_iter().find(|r| { r.name() == text @@ -173,8 +164,7 @@ impl Relation { pub fn is_marginal(self) -> bool { matches!( self, - Relation::LooserAccuracy - | Relation::FilterOnGroupKey + Relation::FilterOnGroupKey | Relation::CoarserGrouping | Relation::NestedShorterWindow | Relation::OtherQuantileRank @@ -194,16 +184,14 @@ impl Relation { /// Whether the relation is checked once per query of the workload /// (otherwise once per workload). pub fn per_query(self) -> bool { - self.is_marginal() || matches!(self, Relation::TighterAccuracy | Relation::RelaxedLatency) + self.is_marginal() || matches!(self, Relation::RelaxedLatency) } /// The statement checked, for reports. pub fn statement(self) -> &'static str { match self { r if r.is_marginal() => "cost(W ∪ {q'}) − cost(W) ≤ derive_cost + tolerance", - Relation::TighterAccuracy | Relation::AddedQuery | Relation::NoRawRetention => { - "cost(W') ≥ cost(W) − tolerance" - } + Relation::AddedQuery | Relation::NoRawRetention => "cost(W') ≥ cost(W) − tolerance", Relation::ResultsRetained | Relation::RelaxedLatency => { "cost(W') ≤ cost(W) + tolerance" } @@ -245,8 +233,8 @@ pub enum Witness { /// `q`'s root estimate reading a `SummaryMerge` with `group_by` the /// `labels` columns over its state, when the family merges groups. SketchRollup { labels: Vec }, - /// `q`'s `SummaryMerge` over a tumbling-window chain merging only the newest tumbling windows - /// covering `window_ms`. + /// `q`'s `TimeRange` over its tumbling-window chain at `window_ms` + /// instead, over the same chain: the windows a shorter range selects. NewestTumblingWindows { window_ms: u64 }, /// `q`'s topmost quantile estimate (or exact quantile aggregate) at rank /// `to` instead of `from`. @@ -257,9 +245,9 @@ pub enum Witness { /// `q`'s root estimate over a `SummaryMerge` of its state and the state /// of `W`'s last query (the other selection, added to `W`). UnionWithLast, - /// `q`'s tumbling-window chain lengthened to cover `window_ms`: every reader of - /// the chain in `W`'s plan merges its old newest tumbling windows of the longer - /// chain, and `q'` merges all of them, then applies `rollup` when set. + /// `q`'s `TimeRange` over its tumbling-window chain at the longer + /// `window_ms`, over the same chain (retained as long as the range + /// needs), then `rollup` when set. LongerWindow { window_ms: u64, rollup: Option>, @@ -304,7 +292,7 @@ pub struct Measurement { /// The derived or added query's text, for a relation that adds one. pub related_query: Option, /// The witness (marginal relations) or the confirming plan (a violated - /// monotonicity relation: the other side's logical plan costed for this + /// monotonicity relation: the other side's plan costed for this /// side). pub witness: Option, /// The bound `related_cost_per_second` was checked against, tolerance @@ -321,7 +309,7 @@ pub struct WitnessReport { pub derivation: Option, pub cost_per_second: f64, pub plan: String, - /// Marginal relations: `W`'s selected logical plan costed the same way. + /// Marginal relations: `W`'s selected plan costed the same way. pub base_reference_cost_per_second: Option, /// `cost_per_second − base_reference_cost_per_second`. pub derive_cost_per_second: Option, @@ -329,9 +317,7 @@ pub struct WitnessReport { /// operators alone) and their operators. pub derived_node_cost_per_second: Option, pub derived_operators: Vec, - /// Whether the witness's `Filter` on a group key kept its input's - /// guarantee, which Stage 1's guarantee composition does not give it - /// (so the witness passes Stage 3 only under that rule). + /// Whether the witness derives `q'` by a `Filter` on a group key. pub key_filter_keeps_input_guarantee: bool, } @@ -394,9 +380,6 @@ pub fn derive(relation: Relation, workload: &FuzzWorkload, target: usize) -> Opt }) }; match relation { - Relation::LooserAccuracy => { - derived(q.with_accuracy(looser(q.accuracy())?), vec![Witness::Same]) - } Relation::ExactDuplicate => derived(q.clone(), vec![Witness::Same]), Relation::OtherRepeatInterval => { let interval = repeat_interval_ms(q)?; @@ -1156,10 +1139,7 @@ pub fn witness_root( .map_err(|e| format!("rollup estimate: {e}"))?; with_root_estimate(q_root, estimate, &dropped) } - Witness::NewestTumblingWindows { window_ms } => { - let _ = window_ms; - todo!("#1207 PR 9: a witness merging the windows a TimeRange over q's chain selects") - } + Witness::NewestTumblingWindows { window_ms } => chain_window(q_root, *window_ms), Witness::QuantileRank { from, to } => { let mut done = false; rewrite_root(q_root, "quantile estimate", |node| { @@ -1264,8 +1244,19 @@ pub fn witness_root( .map_err(|e| format!("union estimate: {e}"))?; with_root_estimate(q_root, estimate, &[]) } - Witness::LongerWindow { .. } | Witness::UnivMonOnly | Witness::GraftWithOperand { .. } => { - Err("the derivation changes W's roots: see witness_roots".into()) + Witness::LongerWindow { window_ms, rollup } => { + let long_root = chain_window(q_root, *window_ms)?; + match rollup { + None => Ok(long_root), + Some(rollup) => { + let mut with_long = base_roots.to_vec(); + with_long[target] = QueryRoot::Operator(long_root); + witness_root(planner, rollup, base, planned, &with_long, target) + } + } + } + Witness::UnivMonOnly | Witness::GraftWithOperand { .. } => { + Err("the derivation plans W again: see witness_roots".into()) } } } @@ -1310,18 +1301,42 @@ fn graft( }) } -/// The witness's roots: `W`'s selected logical roots plus `q'`'s root, -/// with `W`'s roots rewritten when the derivation changes a node they -/// share ([`Witness::LongerWindow`]). +/// `q_root` with its `TimeRange` over a tumbling-window chain at +/// `window_ms`, over the same chain node, and the nodes above it rebuilt. +fn chain_window(q_root: &Rc, window_ms: u64) -> Result, String> { + rewrite_root(q_root, "time range over a tumbling-window chain", |node| { + let Some(NonASAPOp::TimeRange { range, kind, child }) = node.non_asap() else { + return None; + }; + let Some(ASAPOp::SummaryTumblingWindowChain { .. }) = child.asap() else { + return None; + }; + let range_ms = u64::try_from(range.as_millis()).ok()?; + (range_ms != window_ms).then(|| { + with_operator( + node, + Operator::NonASAP(NonASAPOp::TimeRange { + range: std::time::Duration::from_millis(window_ms), + kind: *kind, + child: child.clone(), + }), + ) + }) + }) +} + +/// The witness's roots, `W`'s selected plan's roots plus `q'`'s root, and +/// the planning whose plan they extend: `planned` (`W`'s), or `W` planned +/// again for [`Witness::GraftWithOperand`] (with `q`'s sibling operand) and +/// [`Witness::UnivMonOnly`]. pub fn witness_roots( planner: &mut Planner, witness: &Witness, base: &FuzzWorkload, related: &FuzzWorkload, - planned: &Planned, - base_roots: &[QueryRoot], + planned: &Rc, target: usize, -) -> Result, String> { +) -> Result<(Vec, Rc), String> { if let Witness::GraftWithOperand { text, operand } = witness { let mut sibling = base.queries[target].clone(); sibling.entry.query.0 = operand.clone(); @@ -1330,10 +1345,6 @@ pub fn witness_roots( let planned = planner .plan(&with_operand) .map_err(|e| format!("W with q's sibling operand does not plan: {e}"))?; - let roots = planned - .logical_roots - .clone() - .ok_or("the search keeps no logical plan for W with q's sibling operand")?; let n = base.queries.len(); let operator = |roots: &[QueryRoot], i: usize| -> Result, String> { roots[i] @@ -1345,7 +1356,10 @@ pub fn witness_roots( operator(&planned.stage0_roots, target)?, operator(&planned.stage0_roots, n)?, ); - let (q_root, operand_root) = (operator(&roots, target)?, operator(&roots, n)?); + let (q_root, operand_root) = ( + operator(&planned.roots, target)?, + operator(&planned.roots, n)?, + ); let root = graft( planner, base, @@ -1353,50 +1367,24 @@ pub fn witness_roots( text, &[(&lowered_q, &q_root), (&lowered_operand, &operand_root)], )?; - let mut roots = roots[..n].to_vec(); + let mut roots = planned.roots[..n].to_vec(); roots.push(QueryRoot::Operator(root)); - return Ok(roots); + return Ok((roots, planned)); } if let Witness::UnivMonOnly = witness { let univmon_only = FuzzWorkload { sketch_families: Some(vec![SketchAlgorithm::UnivMon]), ..related.clone() }; - return planner + let planned = planner .plan(&univmon_only) - .map_err(|e| format!("W ∪ {{q'}} with UnivMon only does not plan: {e}"))? - .logical_roots - .clone() - .ok_or_else(|| "the search keeps no logical plan with UnivMon only".into()); + .map_err(|e| format!("W ∪ {{q'}} with UnivMon only does not plan: {e}"))?; + return Ok((planned.roots.clone(), planned)); } - let Witness::LongerWindow { window_ms, rollup } = witness else { - let root = witness_root(planner, witness, base, planned, base_roots, target)?; - let mut roots = base_roots.to_vec(); - roots.push(QueryRoot::Operator(root)); - return Ok(roots); - }; - let (mut roots, long_root) = longer_chain_roots(base_roots, target, *window_ms)?; - let root = match rollup { - None => long_root, - Some(rollup) => { - let mut with_long = roots.clone(); - with_long[target] = QueryRoot::Operator(long_root); - witness_root(planner, rollup, base, planned, &with_long, target)? - } - }; + let root = witness_root(planner, witness, base, planned, &planned.roots, target)?; + let mut roots = planned.roots.clone(); roots.push(QueryRoot::Operator(root)); - Ok(roots) -} - -/// `W`'s roots with the reading of `q`'s tumbling-window chain lengthened -/// to cover `window_ms`, and `q`'s root reading the longer window. -fn longer_chain_roots( - base_roots: &[QueryRoot], - target: usize, - window_ms: u64, -) -> Result<(Vec, Rc), String> { - let _ = (base_roots, target, window_ms); - todo!("#1207 PR 9: lengthen the TimeRange over q's chain, which has no window count") + Ok((roots, planned.clone())) } fn planned( @@ -1430,12 +1418,6 @@ pub fn related_workloads( } let q = w.queries.get(check.target).ok_or("no query")?; let related = match check.relation { - Relation::TighterAccuracy => { - let Some(tight) = tighter(q.accuracy()) else { - return inconclusive("q is exact"); - }; - w.with_query_replaced(check.target, q.with_accuracy(tight)) - } Relation::RelaxedLatency => { if q.entry.requirements.max_latency_ms.is_none() { return inconclusive("q has no latency bound"); @@ -1481,15 +1463,17 @@ pub fn related_workloads( Ok((w.clone(), related, None)) } -/// The other side's logical plan costed for `workload` (its first -/// `keep` roots): a plan for `workload` the search could have selected. +/// The other side's plan costed for `workload` (its first `keep` roots): +/// a plan for `workload` the search could have selected. fn confirming( planner: &Planner, workload: &FuzzWorkload, - roots: Option<&Vec>, + other: &Planned, keep: usize, ) -> Option { - let costed = planner.cost(workload, &roots?[..keep]).ok()?; + let costed = planner + .cost(workload, other, &other.roots[..keep], false) + .ok()?; Some(WitnessReport { derivation: None, cost_per_second: costed.cost_per_second, @@ -1517,12 +1501,12 @@ fn evaluate_inner(planner: &mut Planner, check: &Check) -> Result { + Relation::AddedQuery | Relation::NoRawRetention => { let tolerance = planner.settings.tolerance(b); let violated = r < b - tolerance; // The related plan, minus an added query, is a plan for W. let witness = violated - .then(|| confirming(planner, &w, other.logical_roots.as_ref(), n)) + .then(|| confirming(planner, &w, &other, n)) .flatten(); (b - tolerance, tolerance, violated, witness) } @@ -1530,7 +1514,7 @@ fn evaluate_inner(planner: &mut Planner, check: &Check) -> Result b + tolerance; let witness = violated - .then(|| confirming(planner, &related, base.logical_roots.as_ref(), n)) + .then(|| confirming(planner, &related, &base, n)) .flatten(); (b + tolerance, tolerance, violated, witness) } @@ -1573,36 +1557,25 @@ fn evaluate_inner(planner: &mut Planner, check: &Check) -> Result derive_cost + tolerance`) and its plain reading (the /// witness is cheaper than the planner's `W ∪ {q'}` by more than the -/// tolerance) fail, so a search that costs `W` below Stage 3's cost of -/// the same plan cannot make a false violation. +/// tolerance) fail, so a search that costs `W` below `plan_cost` of the +/// same plan cannot make a false violation. fn evaluate_marginal( planner: &mut Planner, check: &Check, base_workload: &FuzzWorkload, related: &FuzzWorkload, derived: &Derived, - base: &Planned, + base: &Rc, with: &Planned, ) -> Result { - let Some(base_roots) = base.logical_roots.clone() else { - return inconclusive("the search keeps no logical plan for W"); - }; let base_reference = planner - .reference_cost(base_workload, &base_roots) + .reference_cost(base_workload, base) .or_else(|e| inconclusive(format!("W's plan does not cost: {e}")))?; let mut best: Option<(Witness, Costed)> = None; let mut failures = Vec::new(); for witness in &derived.witnesses { - let costed = witness_roots( - planner, - witness, - base_workload, - related, - base, - &base_roots, - check.target, - ) - .and_then(|roots| planner.cost(related, &roots)); + let costed = witness_roots(planner, witness, base_workload, related, base, check.target) + .and_then(|(roots, source)| planner.cost(related, &source, &roots, true)); match costed { Ok(costed) if best diff --git a/crates/devtools/src/planner_fuzz/relations/tests.rs b/crates/devtools/src/planner_fuzz/relations/tests.rs index 7432d3dd6..121c04af0 100644 --- a/crates/devtools/src/planner_fuzz/relations/tests.rs +++ b/crates/devtools/src/planner_fuzz/relations/tests.rs @@ -59,8 +59,6 @@ fn marginal_relations_derive_the_documented_query() { text(Relation::ExactDuplicate).as_deref(), Some(w.queries[0].text()) ); - let looser = derive(Relation::LooserAccuracy, &w, 0).unwrap(); - assert_eq!(looser.query.accuracy().max_error, 0.01); let interval = derive(Relation::OtherRepeatInterval, &w, 0).unwrap(); assert_eq!(repeat_interval_ms(&interval.query), Some(30_000)); } @@ -256,7 +254,8 @@ fn relation_names_parse() { for relation in Relation::ALL { assert_eq!(Relation::parse(relation.name()), Some(relation)); } - assert_eq!(Relation::parse("a"), Some(Relation::LooserAccuracy)); + assert_eq!(Relation::parse("a"), None); + assert_eq!(Relation::parse("b"), Some(Relation::FilterOnGroupKey)); assert_eq!(Relation::parse("k"), Some(Relation::UnionOfSelections)); assert_eq!(Relation::parse("z"), None); } diff --git a/crates/devtools/src/planner_fuzz/report.rs b/crates/devtools/src/planner_fuzz/report.rs index 703543743..8da208569 100644 --- a/crates/devtools/src/planner_fuzz/report.rs +++ b/crates/devtools/src/planner_fuzz/report.rs @@ -35,8 +35,8 @@ pub fn documented_patterns() -> Vec { DocumentedPattern { pattern: "Looser accuracy target reads the tighter build (max-sized shared summary; exact meets any ε)", doc_reference: "accuracy-models.md \"Sharing across consumers\" L308–368; planner-layering.md Pass 2 summary-capability rule L530–533, L640–641", - relation: Some(Relation::LooserAccuracy), - note: "", + relation: None, + note: "the fuzzer plans with accuracy not enforced, so no relation can check an accuracy target", }, DocumentedPattern { pattern: "Slice: a filter on a group key over a grouped result or summary", @@ -105,7 +105,7 @@ pub fn documented_patterns() -> Vec { note: "the merge operator is implemented; the docs name no Pass 2 rule that creates it", }, DocumentedPattern { - pattern: "Tumbling windows kept across evaluations when the deployment retains query-time results", + pattern: "Tumbling windows materialized across evaluations when the deployment retains query-time results", doc_reference: "planner-layering.md Materialization L889–893", relation: Some(Relation::ResultsRetained), note: "a capability never raises cost", @@ -192,7 +192,7 @@ pub struct Finding { pub shrink_attempts: Option, pub check: Check, pub measurement: Measurement, - /// The open issue a reader should compare it with, when the relation + /// The open issue to compare it with, when the relation /// matches one (a guess from the relation, not a proof). pub possibly_known_issue: Option<&'static str>, pub draft: PathBuf, @@ -356,7 +356,6 @@ pub fn draft_key(check: &Check, measurement: &Measurement) -> String { fn possibly_known_issue(relation: Relation, check: &Check) -> Option<&'static str> { let sql = check.workload.language == Language::Sql; match relation { - Relation::LooserAccuracy => Some("#903"), Relation::FilterOnGroupKey if sql => Some("#902"), Relation::FilterOnGroupKey => Some("#891"), _ => None, @@ -743,7 +742,7 @@ fn derivation_text(witness: &Witness) -> String { plan, planned with `W`" ), Witness::UnivMonOnly => { - "`W ∪ {q'}` planned with UnivMon as the only sketch family, priced with every family" + "`W ∪ {q'}` planned with UnivMon as the only sketch family, costed with every family" .into() } Witness::LongerWindow { window_ms, rollup } => { @@ -796,9 +795,6 @@ fn write_draft( .map(|q| q.text().to_string()) .unwrap_or_default(); let title = match check.relation { - Relation::LooserAccuracy => { - format!("`{q}` at a looser accuracy target does not read the tighter query's result") - } Relation::FilterOnGroupKey => { format!("A query filtered on a group key does not read `{q}`'s grouped result") } @@ -841,7 +837,6 @@ fn write_draft( Relation::EquivalentRangeFunction => { format!("`{q}` with `rate` and `increase` swapped does not scale its result") } - Relation::TighterAccuracy => "Tightening an accuracy target lowers the planned cost".into(), Relation::AddedQuery => "Adding a query lowers the planned cost".into(), Relation::NoRawRetention => "Dropping raw-data retention lowers the planned cost".into(), Relation::ResultsRetained => "Retaining query-time results raises the planned cost".into(), @@ -863,15 +858,10 @@ fn write_draft( }; text.push_str(&format!( "Found by `planner_fuzz` (seed {seed}, case {}, relation `{}`), {shrunk}. \ - Search: {:?}; accuracy guarantees {}.\n\n", + Search: {:?}; accuracy not enforced.\n\n", finding.case, check.relation.name(), settings.search, - if settings.enforce_accuracy_guarantees { - "enforced" - } else { - "ignored" - }, )); text.push_str(&format!( "`{base_name}` (save as `base.json`):\n\n```json\n{}\n```\n\n", @@ -889,8 +879,8 @@ fn write_draft( text.push_str(&format!("```json\n{}\n```\n\n", workload_json(&related))); text.push_str(&format!( "```sh\n{}\n{}\n```\n\n", - base.asap_planner_command("base.json", settings), - related.asap_planner_command("related.json", settings), + base.asap_planner_command("base.json"), + related.asap_planner_command("related.json"), )); text.push_str("| Workload | Selected plan | Cost per second |\n|---|---|---|\n"); text.push_str(&format!( @@ -907,7 +897,7 @@ fn write_draft( "Witness: `W`'s plan with `q'` derived by {}", derivation_text(d) ), - None => "The other side's plan, priced for this workload".into(), + None => "The other side's plan, costed for this workload".into(), }; text.push_str(&format!( "| {label} | `{}` | {:.6} |\n", @@ -923,7 +913,7 @@ fn write_draft( match witness.and_then(|w| w.derive_cost_per_second.map(|d| (w, d))) { Some((w, derive_cost)) => text.push_str(&format!( "**Expected:** a candidate for `{related_name}` costing at most about {:.6} per second \ - (the witness): `W`'s plan ({:.6} as Stage 3 prices it) plus `q'` derived for \ + (the witness): `W`'s plan ({:.6} as `asap_cost::plan_cost` costs it) plus `q'` derived for \ {derive_cost:.6}. Of that, the nodes only `q'` reads cost {:.6} ({}).\n\n", w.cost_per_second, w.base_reference_cost_per_second.unwrap_or_default(), @@ -939,7 +929,7 @@ fn write_draft( check.relation.statement(), witness .map(|w| format!( - " The other side's plan prices at {:.6} per second for this workload.", + " The other side's plan costs at {:.6} per second for this workload.", w.cost_per_second )) .unwrap_or_default() @@ -948,9 +938,10 @@ fn write_draft( text.push_str("## Why it happens\n\nNot diagnosed yet. "); if marginal { text.push_str( - "The witness is priced by Stage 2 and Stage 3 (`optimize_physical_asap`, \ - `select_plan`) with the same models as the search, so the cheaper plan passes \ - Stage 3's checks, but the search did not select it.", + "The witness is costed by `asap_cost::plan_cost` with the same cost inputs as \ + the search, its nodes from `W`'s plan with the timing, lifetime and parameters \ + that plan gave them, but the search did not select it. Its accuracy is not \ + checked: the fuzzer plans with accuracy not enforced.", ); if witness.is_some_and(|w| w.key_filter_keeps_input_guarantee) { text.push_str( @@ -967,8 +958,8 @@ fn write_draft( ); if witness.is_some() { text.push_str( - " The other side's logical plan, priced for this workload by Stage 2 and \ - Stage 3, is that plan.", + " The other side's plan, costed for this workload by \ + `asap_cost::plan_cost`, is that plan.", ); } } @@ -996,17 +987,12 @@ fn write_draft( /// design docs' reuse patterns with each relation's status in this run. pub fn summary_markdown(summary: &Summary) -> String { let mut text = format!( - "# planner_fuzz summary\n\nSeed {}, {} cases, {} checks, {:.0} s. Search {:?}, accuracy guarantees {}.\n\n", + "# planner_fuzz summary\n\nSeed {}, {} cases, {} checks, {:.0} s. Search {:?}, accuracy not enforced.\n\n", summary.seed, summary.cases_run, summary.checks_run, summary.elapsed_s, summary.settings.search, - if summary.settings.enforce_accuracy_guarantees { - "enforced" - } else { - "ignored" - }, ); text.push_str("| Relation | Holds | Violated | Distinct violations | Inconclusive | Crashed |\n|---|---|---|---|---|---|\n"); for (relation, tally) in &summary.tallies { diff --git a/crates/devtools/src/planning_stages.rs b/crates/devtools/src/planning_stages.rs index b4ab56335..dba4d1fe6 100644 --- a/crates/devtools/src/planning_stages.rs +++ b/crates/devtools/src/planning_stages.rs @@ -76,8 +76,25 @@ pub struct PlanningStagesRun { /// Lower a PromQL workload, each series' identity as a column (the row /// representation per-series state needs at runtime). pub fn lower_promql_roots(workload: &PlanningWorkload) -> Result, String> { - asap_frontend_promql::lower_promql_query_workload(workload) - .map_err(|e| format!("lowering: {e}"))? + with_series_identity( + asap_frontend_promql::lower_promql_query_workload(workload) + .map_err(|e| format!("lowering: {e}"))?, + ) +} + +/// Lower a MetricsQL workload, each series' identity as a column, as +/// [`lower_promql_roots`] does for PromQL. +pub fn lower_metricsql_roots(workload: &PlanningWorkload) -> Result, String> { + with_series_identity( + asap_frontend_metricsql::lower_metricsql_query_workload(workload) + .map_err(|e| format!("lowering: {e}"))?, + ) +} + +/// `roots` (a PromQL-family frontend's) with each series' identity as a +/// column. +fn with_series_identity(roots: Vec) -> Result, String> { + roots .into_iter() .map(|root| match root { QueryRoot::Operator(node) => with_promql_series_identity(&node) diff --git a/crates/devtools/src/planning_stages_text.rs b/crates/devtools/src/planning_stages_text.rs index e6faccf28..f74cceee2 100644 --- a/crates/devtools/src/planning_stages_text.rs +++ b/crates/devtools/src/planning_stages_text.rs @@ -1,36 +1,74 @@ //! The planner stages' output as plain text: `asap_planner --format text`. //! It prints, up to the run's last stage, the frontends' roots and their //! memo groups (Stage 0), Pass 1's `Memo`, Pass 2's -//! `LogicalASAPMemoCandidates` (Stage 2's input), Stage 2's -//! `PhysicalASAPMemoCandidates` (Stage 3's input), and Stage 3's -//! `Selection` with the selected memo choice, each node's cost and each -//! query's accepted accuracy. Planning never reads it. +//! `LogicalASAPMemoCandidates` (Stage 2's input: each candidate with its +//! logical edges and $\alpha^{\log}$, then the candidates Pass 2 pruned +//! with their causes), Stage 2's `PhysicalASAPMemoCandidates` (Stage 3's +//! input: each physical candidate with its physical edges and +//! $\alpha^{\mathrm{phys}}$, then the candidates Stage 2 pruned), and +//! Stage 3's plan (each node with its candidate, the plan's cost, each +//! query's accepted accuracy and the candidates Stage 3 pruned). Planning +//! never reads it. //! //! Notation: `G` is memo group `n` and `G[]` its candidate `i`; -//! `O` is Stage 2's operator node `n` and `C` physical candidate `n`; -//! `<- ...` lists what a line reads; `new` marks a candidate the step -//! added. A candidate's chain lists its own nodes root first, down to its -//! input groups: +//! `C` is physical candidate `n` and `n` node `k` of the plan; `<- +//! ...` lists what a line reads; `new` marks a candidate the step added +//! and `pruned` one Pass 2 eliminated; `θ` is a candidate's parameters and +//! `requires` its required accuracy per input. An edge is `✓` when its +//! layer's requirement holds, else `✗` with why; a logical edge's producer +//! is any candidate of the input group (eliminated ones included), a +//! physical edge's a physical candidate whose logical edge holds: //! //! ```text -//! G1: TimeRange 60s <- G0 -//! G2: Aggregate rate() <- G1 -//! [0] Pass1 FinalizeExactAccumulator ← SummaryAgg inputs=[G1] -//! [1] TumblingWindows new FinalizeExactAccumulator ← SummaryMerge ← SummaryTumblingWindowChain tumbling_window_width_ms=10000 inputs=[G0] +//! G4: SummaryEstimate Quantile(q=0.99) <- G3 +//! [0] Pass1 SummaryEstimate Quantile(q=0.99) <- G3 exact requires=[Rank ≤ 0.01 (δ 0.01)] +//! input 0 <- G3: +//! G3[0] ✗ α^log = 0: the producer's guarantee exceeds Rank ≤ 0.01 (δ 0.01) +//! G3[4] ✓ +//! +//! C6 = G2[0] (QueryTime, None) Aggregate quantile(q=0.99) +//! input 0 <- G1: +//! C3 = G1[0] (QueryTime, None) ✓ selected +//! C5 = G1[0] (IngestionTime, None) ✗ α^phys = 0: consumer (QueryTime, None) cannot read producer (IngestionTime, None) //! ``` +use std::collections::{BTreeMap, BTreeSet}; use std::fmt::Write; +use asap_cost::PlanCost; use asap_types::ir::flat::flatten; +use asap_types::ir::operator::Source; use asap_types::ir::QueryRoot; -use asap_types::stages::{Memo, MemoGroup, PhysicalASAPMemoCandidates}; +use asap_types::plan::materialized_view::MaterializedView; +use asap_types::plan::physical_asap_dag::PhysicalASAPDAG; +use asap_types::stages::{ + AggregateCoverage, CandidateForMemoGroup, CandidateForMemoGroupId, ColumnLineage, + EvaluationGrid, LogicalASAPMemoCandidates, Memo, MemoGroup, MemoGroupId, + PhysicalASAPMemoCandidates, PhysicalCandidateId, PhysicalEdge, PhysicalProperties, + RequiredAccuracy, RootPruneReason, +}; +use asap_types::summary_type::SketchParams; -use crate::candidate_labels::node_label; +use crate::candidate_labels::{operator_label, parameters_label}; use crate::planning_stages::{PlanningStagesRun, RunUntil}; /// The output of every stage `stages` ran, as text, one section per stage -/// boundary. A failure ends the text with its error. -pub fn planning_stages_text(stages: &PlanningStagesRun) -> String { +/// boundary. `plan_cost` is the selected plan's `asap_cost::plan_cost` and +/// `materialized_views` its materialized views as `plan_installable` +/// derives them (or why it is not installable), each printed with the plan +/// when given. A failure ends the text with its error. +/// +/// Unless `printing_full_candidates`, the candidates of a memo group that differ only in θ +/// are folded ([`write_memo`]): one line gives their indices, each θ +/// parameter's values and how many Pass 2 pruned, by cause; those not +/// pruned are still printed in full with their edges. The pruned lists of +/// Pass 2 and Stage 2 are then counts per cause. +pub fn planning_stages_text( + stages: &PlanningStagesRun, + plan_cost: Option<&PlanCost>, + materialized_views: Option<&Result, String>>, + printing_full_candidates: bool, +) -> String { let mut out = String::new(); let run = &stages.run; let until = stages.run_until; @@ -51,7 +89,7 @@ pub fn planning_stages_text(stages: &PlanningStagesRun) -> String { "Stage 0 → Stage 1: Memo (the roots hash-consed, without candidates)", ); if let Some(memo) = &stages.stage0_memo_groups { - write_memo(&mut out, memo, None, false); + write_memo(&mut out, memo, None, printing_full_candidates); } if until == RunUntil::Stage0 { return out; @@ -62,7 +100,12 @@ pub fn planning_stages_text(stages: &PlanningStagesRun) -> String { return out; }; section(&mut out, "Stage 1 Pass 1 output: Memo"); - write_memo(&mut out, pass1, None, true); + write_memo( + &mut out, + pass1, + Some(MemoStep::Pass1), + printing_full_candidates, + ); if until == RunUntil::Pass1 { return out; } @@ -75,7 +118,13 @@ pub fn planning_stages_text(stages: &PlanningStagesRun) -> String { &mut out, "Stage 1 Pass 2 output = Stage 2 input: LogicalASAPMemoCandidates", ); - write_memo(&mut out, &logical.memo, Some(pass1), true); + write_memo( + &mut out, + &logical.memo, + Some(MemoStep::Pass2 { pass1, logical }), + printing_full_candidates, + ); + write_pass2_pruned(&mut out, logical, printing_full_candidates); if until == RunUntil::Pass2 { return out; } @@ -88,13 +137,14 @@ pub fn planning_stages_text(stages: &PlanningStagesRun) -> String { &mut out, "Stage 2 output = Stage 3 input: PhysicalASAPMemoCandidates", ); - write_physical(&mut out, physical); + let plan = run.selection.as_ref().ok().map(|(plan, _)| plan); + write_physical(&mut out, physical, plan, printing_full_candidates); if until == RunUntil::Stage2 { return out; } - section(&mut out, "Stage 3 output: Selection"); - write_selection(&mut out, stages); + section(&mut out, "Stage 3 output: PhysicalASAPDAG and Selection"); + write_selection(&mut out, stages, plan_cost, materialized_views); out } @@ -117,7 +167,11 @@ fn write_dag(out: &mut String, roots: &[QueryRoot]) { .into_iter() .map(|child| format!("n{child}")) .collect(); - vec![format!("n{id}:"), node_label(node), reads(&children)] + vec![ + format!("n{id}:"), + operator_label(&node.operator), + reads(&children), + ] }) .collect(); write_rows(out, "", rows); @@ -160,6 +214,121 @@ fn write_rows(out: &mut String, indent: &str, rows: Vec>) { } } +/// Each output column's lineage, one per schema field: `source.column` +/// for a column copied from a scan (`source#1.column` for a self-join's +/// second scan), `G.` for one computed by column `column` of +/// group `n`, where `n` is `group_id` of the group (the planner's id in +/// the text, the canonical id in the JSON document). +/// +/// Example: `Scan latency` → `["latency.ts", "latency.value", +/// "latency.job"]`; G2 = `Aggregate sum()` over it → `["G2.0"]`. +pub fn column_lineage_text( + group: &MemoGroup, + group_id: impl Fn(MemoGroupId) -> u32, +) -> Vec { + group + .group_properties + .column_lineage + .iter() + .map(|lineage| match lineage { + ColumnLineage::SourceColumn(column) => { + let source = match &column.source { + Source::TimeSeries { metric } => metric, + Source::Table { table_ref } => table_ref, + }; + match column.scan_ordinal { + 0 => format!("{source}.{}", column.column), + ordinal => format!("{source}#{ordinal}.{}", column.column), + } + } + ColumnLineage::Computed { group, column } => format!("G{}.{column}", group_id(*group)), + }) + .collect() +} + +/// A group's properties on one line: its result kind, each column's name, +/// lineage and sign, whether its result depends on the evaluation time, +/// whether two equal expressions of it may not be one group, the sources +/// it scans, (from Pass 2 on, `grid`) the grid a result of it is evaluated +/// on, and an aggregate's coverage ([`aggregate_coverage_text`]): +/// `State ts=latency.ts:Unknown state=G4.1:Unknown · depends on the +/// evaluation time · scans latency · evaluated every 60000 ms at 0 · +/// covers G1 through [SummaryAgg Kll] selecting [...]`. +fn group_properties_text(group: &MemoGroup, grid: Option>) -> String { + let properties = &group.group_properties; + let columns: Vec = properties + .schema + .fields + .iter() + .zip(column_lineage_text(group, |group| group.0)) + .zip(&properties.column_signs) + .map(|((field, lineage), sign)| format!("{}={lineage}:{sign:?}", field.name)) + .collect(); + let mut text = format!("{:?} {}", properties.result_kind, columns.join(" ")); + if properties.depends_on_evaluation_time { + text += " · depends on the evaluation time"; + } + if !properties.shareable_when_equal { + text += " · not shareable when equal"; + } + let sources: Vec<&str> = properties + .scanned_sources + .iter() + .map(|source| match source { + Source::TimeSeries { metric } => metric.as_str(), + Source::Table { table_ref } => table_ref.as_str(), + }) + .collect(); + if !sources.is_empty() { + let _ = write!(text, " · scans {}", sources.join(", ")); + } + match grid { + Some(Some(grid)) => { + let _ = write!( + text, + " · evaluated every {} ms at {}", + grid.period_ms, grid.phase_ms + ); + } + Some(None) => text += " · no known evaluation grid", + None => {} + } + if let Some(coverage) = &properties.aggregate_coverage { + let _ = write!( + text, + " · {}", + aggregate_coverage_text(coverage, |group| group.0) + ); + } + text +} + +/// What an aggregate group covers (`GroupProperties::aggregate_coverage`), +/// which decides whether a rollup or a view match may read it: its input +/// group (`a scan` when the scan's matchers are lifted too), its +/// operators not lifted into the selection, top down, and its selection as +/// JSON, the input group named by `group_id`. Example: `covers G1 through +/// [SummaryAgg Kll] selecting [{"columns":[...]}]`. +pub fn aggregate_coverage_text( + coverage: &AggregateCoverage, + group_id: impl Fn(MemoGroupId) -> u32, +) -> String { + let definition = coverage.definition(); + let input = definition + .input + .map_or("a scan".to_owned(), |group| format!("G{}", group_id(group))); + let operators: Vec = definition + .unlifted_operators + .iter() + .map(operator_label) + .collect(); + format!( + "covers {input} through [{}] selecting {}", + operators.join(", "), + serde_json::to_string(coverage.selection()).unwrap_or_default() + ) +} + /// `<- a, b`, or nothing when there is nothing read. fn reads(inputs: &[String]) -> String { if inputs.is_empty() { @@ -169,10 +338,49 @@ fn reads(inputs: &[String]) -> String { } } -/// `memo`'s groups, one line each with the child groups it reads, and with -/// `with_candidates` each group's candidates below it, marked `new` when -/// `previous` (the memo at the step before) has no such candidate. -fn write_memo(out: &mut String, memo: &Memo, previous: Option<&Memo>, with_candidates: bool) { +/// The step a memo with candidates is printed at: Pass 1, or Pass 2 with +/// the memo at Pass 1 (to mark the candidates Pass 2 added) and its output +/// (its logical edges and pruned candidates). +#[derive(Clone, Copy)] +enum MemoStep<'a> { + Pass1, + Pass2 { + pass1: &'a Memo, + logical: &'a LogicalASAPMemoCandidates, + }, +} + +/// `memo`'s groups, one line each with the child groups it reads, and at a +/// `step` each group's candidates below it: index, origin, `new` (Pass 2 +/// added it: the memo only appends, so a candidate is new when its index is +/// past the group's Pass 1 candidates), `pruned` (Pass 2 eliminated it), +/// the operator $e_v$ with its input groups, θ, the fused groups and the +/// required accuracy per input; at Pass 2, under each candidate, the +/// logical edges into each input with $\alpha^{\log}$ and, when it is 0, +/// the reason Pass 2 recorded. +/// +/// Unless `printing_full_candidates`, candidates that differ only in θ (same origin, `new`, +/// operator, fused groups and requirements) are folded when there are +/// several: at Pass 1 into one line, at Pass 2 when Pass 2 pruned some of +/// them, into one line followed by those it kept, in full; their edges +/// from pruned producers and producers of another θ are counted, not +/// listed: +/// +/// ```text +/// G9: SummaryAgg CmsWithHeap <- G1 +/// [0..279] Pass1 SummaryAgg CmsWithHeap <- G1 fuses=[G4] +/// θ: depth ∈ {5, 7, 10, 12}, heap_size ∈ {10..10000} (7 values), width ∈ {28..2719} (10 values) +/// 280 candidates: 278 pruned (BelowRequiredAccuracy 270, MetricRule 8), 2 kept +/// [57] Pass1 SummaryAgg CmsWithHeap <- G1 θ=CmsWithHeap(depth=7, heap_size=10, width=544) fuses=[G4] +/// input 0 <- G1: +/// G1[0] ✓ +/// ``` +fn write_memo( + out: &mut String, + memo: &Memo, + step: Option>, + printing_full_candidates: bool, +) { // The group lines share one set of column widths, with each group's // candidates printed under its line. let group_rows: Vec> = memo @@ -186,18 +394,152 @@ fn write_memo(out: &mut String, memo: &Memo, previous: Option<&Memo>, with_candi .into_iter() .map(|child| format!("G{}", child.0)) .collect(); - vec![format!("G{id}:"), group_label(group), reads(&children)] + vec![ + format!("G{id}:"), + operator_label(&group.expression), + reads(&children), + ] }) .collect(); let mut group_lines = String::new(); write_rows(&mut group_lines, "", group_rows); + let (earlier, logical) = match step { + Some(MemoStep::Pass2 { pass1, logical }) => (Some(pass1), Some(logical)), + _ => (None, None), + }; + let pruned: BTreeMap = logical + .map(|logical| { + logical + .pruned + .iter() + .map(|p| { + let cause = p + .causes + .first() + .map_or("no cause", |cause| root_prune_reason_name(&cause.root)); + (p.candidate, cause) + }) + .collect() + }) + .unwrap_or_default(); for (id, line) in group_lines.lines().enumerate() { let _ = writeln!(out, "{line}"); - if !with_candidates { + let grid = logical.map(|logical| logical.evaluation_grid_of_group[id].as_ref()); + let _ = writeln!( + out, + " {}", + group_properties_text(&memo.groups()[id], grid) + ); + if step.is_none() { continue; } - let _ = (previous, &memo.groups()[id].candidates); - todo!("#1207 PR 9: print each candidate's expression, logical properties and input requirements") + let group = MemoGroupId(id as u32); + let first_added = earlier.map_or(usize::MAX, |memo| { + memo.groups().get(id).map_or(0, |g| g.candidates.len()) + }); + let candidates = &memo.groups()[id].candidates; + let id_of = |index_in_group| CandidateForMemoGroupId { + group, + index_in_group, + }; + let flag = |set: bool, name: &str| if set { name.to_owned() } else { String::new() }; + let row = |index: String, candidate: &CandidateForMemoGroup, new: bool, text: String| { + vec![ + index, + format!("{:?}", candidate.origin), + flag(new, "new"), + text, + ] + }; + // The candidates printed in full, in index order; unless `printing_full_candidates`, + // each class of candidates differing only in θ that is folded + // prints its line where its first candidate is. + let mut classes: Vec<(String, Vec)> = Vec::new(); + for (index, candidate) in candidates.iter().enumerate() { + let key = format!( + "{:?} {} {}", + candidate.origin, + index >= first_added, + candidate_text(candidate, false) + ); + match classes.iter_mut().find(|(known, _)| *known == key) { + Some((_, members)) => members.push(index), + None => classes.push((key, vec![index])), + } + } + let mut printed = Vec::new(); + for (_, members) in &classes { + let pruned_members = members + .iter() + .filter(|&&index| pruned.contains_key(&id_of(index))) + .count(); + let folded = !printing_full_candidates + && members.len() > 1 + && (logical.is_none() || pruned_members > 0); + if !folded { + printed.extend(members.iter().map(|&index| (index, None))); + continue; + } + printed.push((members[0], Some(members))); + printed.extend( + members + .iter() + .filter(|&&index| logical.is_some() && !pruned.contains_key(&id_of(index))) + .map(|&index| (index, None)), + ); + } + printed.sort_by_key(|&(index, fold)| (index, fold.is_none())); + for (index, fold) in printed { + let candidate = &candidates[index]; + let candidate_id = id_of(index); + if let Some(members) = fold { + write_rows( + out, + " ", + vec![row( + format!("[{}]", index_ranges(members)), + candidate, + index >= first_added, + candidate_text(candidate, false), + )], + ); + let parameters: Vec<&SketchParams> = members + .iter() + .filter_map(|&index| candidates[index].logical_properties.parameters()) + .collect(); + if !parameters.is_empty() { + let _ = writeln!(out, " θ: {}", parameter_values_text(¶meters)); + } + let mut counts = format!("{} candidates", members.len()); + if logical.is_some() { + let causes: Vec<&str> = members + .iter() + .filter_map(|&index| pruned.get(&id_of(index)).copied()) + .collect(); + let _ = write!( + counts, + ": {} pruned ({}), {} kept", + causes.len(), + counts_text(causes.iter().copied()), + members.len() - causes.len() + ); + } + let _ = writeln!(out, " {counts}"); + continue; + } + let mut cells = row( + format!("[{index}]"), + candidate, + index >= first_added, + candidate_text(candidate, true), + ); + cells.insert(3, flag(pruned.contains_key(&candidate_id), "pruned")); + write_rows(out, " ", vec![cells]); + if let Some(logical) = logical { + let folded_producers = (!printing_full_candidates).then_some(&pruned); + write_logical_edges(out, logical, candidate_id, folded_producers); + } + } } let roots: Vec = memo .queries() @@ -210,22 +552,751 @@ fn write_memo(out: &mut String, memo: &Memo, previous: Option<&Memo>, with_candi let _ = writeln!(out, "query_roots: [{}]", roots.join(", ")); } -fn write_physical(out: &mut String, physical: &PhysicalASAPMemoCandidates) { - let _ = (out, physical); - todo!("#1207 PR 9: print each physical candidate's properties and its edges with α") +/// The logical edges into each input of `consumer`: each producer of the +/// input group with `✓`, or `✗` and why ([`logical_edges_into`]). With +/// `folded` (Pass 2's pruned candidates), the edges from those and from +/// producers of another θ are counted on one line, not listed. +fn write_logical_edges( + out: &mut String, + logical: &LogicalASAPMemoCandidates, + consumer: CandidateForMemoGroupId, + folded: Option<&BTreeMap>, +) { + let groups = logical + .memo + .candidate(consumer) + .expression + .operator + .children(); + for ((input, group), edges) in groups + .into_iter() + .enumerate() + .zip(logical_edges_into(logical, consumer)) + { + let (mut pruned_producers, mut other_parameters) = (0, 0); + let rows: Vec> = edges + .into_iter() + .enumerate() + .filter_map(|(index_in_group, status)| { + let producer = CandidateForMemoGroupId { + group: *group, + index_in_group, + }; + if let Some(folded) = folded { + if folded.contains_key(&producer) { + pruned_producers += 1; + return None; + } + if matches!(status, LogicalEdgeStatus::NoEdgeOtherTheta) { + other_parameters += 1; + return None; + } + } + let producer = candidate_id_text(producer); + Some(match status.rejection() { + None => vec![producer, "✓".into()], + Some(reason) => vec![producer, format!("✗ {reason}")], + }) + }) + .collect(); + let _ = writeln!(out, " input {input} <- G{}:", group.0); + let listed = !rows.is_empty(); + write_rows(out, " ", rows); + let counted: Vec = [ + (pruned_producers, "pruned producers"), + (other_parameters, "producers of another θ (no edge)"), + ] + .into_iter() + .filter(|&(count, _)| count > 0) + .map(|(count, what)| format!("{count} {what}")) + .collect(); + if !counted.is_empty() { + let and = if listed { "and " } else { "" }; + let _ = writeln!(out, " {and}{}", counted.join(", ")); + } + } +} + +/// Whether a possible logical edge $(u, c, i)$ holds, as Pass 2 decided. +#[derive(Debug, Clone, Copy)] +pub enum LogicalEdgeStatus<'a> { + /// $\alpha^{\log} = 1$: Pass 2 recorded no failure for it. + Holds, + /// No edge: the consumer reads only producers of its own θ + /// (`Memo::producers`), e.g. a `SummaryMerge` with θ = `Kll(k=200)` + /// and a `Kll(k=400)` build. + NoEdgeOtherTheta, + /// $\alpha^{\log} = 0$, with the reason Pass 2 recorded + /// (`LogicalASAPMemoCandidates::failing_logical_edges`). + Fails(&'a RootPruneReason), +} + +impl LogicalEdgeStatus<'_> { + /// Why the edge fails, `None` when it holds: `no edge: the producer's + /// θ differs`, or `α^log = 0: ` and [`root_prune_reason_text`]. + pub fn rejection(&self) -> Option { + match self { + Self::Holds => None, + Self::NoEdgeOtherTheta => Some("no edge: the producer's θ differs".to_owned()), + Self::Fails(reason) => Some(format!("α^log = 0: {}", root_prune_reason_text(reason))), + } + } +} + +/// Per input $i$ of `consumer` $c$, per candidate $u$ of the input group +/// $G_{c,i}$ (eliminated ones included), in group order: whether the edge +/// holds, read from Pass 2's output. A producer the consumer cannot read +/// has no edge; a readable one fails with the reason Pass 2 recorded in +/// `failing_logical_edges`, and holds when it is not listed there. +pub fn logical_edges_into( + logical: &LogicalASAPMemoCandidates, + consumer: CandidateForMemoGroupId, +) -> Vec>> { + let memo = &logical.memo; + // `failing_logical_edges` is sorted by consumer. + let first = logical + .failing_logical_edges + .partition_point(|(edge, _)| edge.consumer_candidate_id < consumer); + let failing: BTreeMap<(usize, CandidateForMemoGroupId), &RootPruneReason> = logical + .failing_logical_edges[first..] + .iter() + .take_while(|(edge, _)| edge.consumer_candidate_id == consumer) + .map(|(edge, reason)| { + ( + (edge.input_index_of_consumer, edge.producer_candidate_id), + reason, + ) + }) + .collect(); + let groups = memo.candidate(consumer).expression.operator.children(); + groups + .into_iter() + .enumerate() + .map(|(input, group)| { + let readable: BTreeSet = + memo.producers(consumer, input).into_iter().collect(); + (0..memo.groups()[group.0 as usize].candidates.len()) + .map(|index_in_group| { + let producer = CandidateForMemoGroupId { + group: *group, + index_in_group, + }; + if !readable.contains(&producer) { + return LogicalEdgeStatus::NoEdgeOtherTheta; + } + failing + .get(&(input, producer)) + .map_or(LogicalEdgeStatus::Holds, |reason| { + LogicalEdgeStatus::Fails(reason) + }) + }) + .collect() + }) + .collect() } -/// Stage 3's selection: the search's outcome, the selected DAG (each node's -/// candidate, operator and producer nodes), the plan's cost with each plan -/// node's charge, and each query's accepted accuracy; or why none was -/// selected. -fn write_selection(out: &mut String, stages: &PlanningStagesRun) { - let _ = (out, stages); - todo!("#1207 PR 9: print the selected DAG over the new physical candidates") +/// `0..279`, or `0..3, 7, 9..12`: sorted indices as ranges. +fn index_ranges(indices: &[usize]) -> String { + let mut ranges: Vec<(usize, usize)> = Vec::new(); + for &index in indices { + match ranges.last_mut() { + Some((_, end)) if *end + 1 == index => *end = index, + _ => ranges.push((index, index)), + } + } + ranges + .iter() + .map(|&(start, end)| match start == end { + true => start.to_string(), + false => format!("{start}..{end}"), + }) + .collect::>() + .join(", ") } -/// The label of `group`'s own operator, without its input groups. -fn group_label(group: &MemoGroup) -> String { - let _ = group; - todo!("#1207 PR 9: label a group by its expression") +/// Each θ parameter's values among `parameters`, sorted: +/// `depth ∈ {5, 7}, width ∈ {28..2719} (10 values)` (a range past four +/// values). +fn parameter_values_text(parameters: &[&SketchParams]) -> String { + let mut values: BTreeMap> = BTreeMap::new(); + for parameters in parameters { + let value = serde_json::to_value(parameters).unwrap_or_default(); + let fields = value + .as_object() + .and_then(|algorithm| algorithm.values().next()) + .and_then(|fields| fields.as_object()); + for (name, value) in fields.into_iter().flatten() { + let known = values.entry(name.clone()).or_default(); + if !known.contains(value) { + known.push(value.clone()); + } + } + } + values + .into_iter() + .map(|(name, mut known)| { + known.sort_by(|a, b| match (a.as_f64(), b.as_f64()) { + (Some(a), Some(b)) => a.total_cmp(&b), + _ => a.to_string().cmp(&b.to_string()), + }); + let set = match known.len() { + 0..=4 => format!( + "{{{}}}", + known + .iter() + .map(|value| value.to_string()) + .collect::>() + .join(", ") + ), + n => format!("{{{}..{}}} ({n} values)", known[0], known[n - 1]), + }; + format!("{name} ∈ {set}") + }) + .collect::>() + .join(", ") +} + +/// `BelowRequiredAccuracy 270, MetricRule 8`: each name with how often it +/// occurs, most frequent first. +fn counts_text<'a>(names: impl IntoIterator) -> String { + let mut counts: BTreeMap<&str, usize> = BTreeMap::new(); + for name in names { + *counts.entry(name).or_default() += 1; + } + let mut counts: Vec<(&str, usize)> = counts.into_iter().collect(); + counts.sort_by(|a, b| b.1.cmp(&a.1).then(a.0.cmp(b.0))); + counts + .iter() + .map(|(name, count)| format!("{name} {count}")) + .collect::>() + .join(", ") +} + +/// A root cause's variant name: `BelowRequiredAccuracy`. +fn root_prune_reason_name(reason: &RootPruneReason) -> &'static str { + match reason { + RootPruneReason::BelowRequiredAccuracy { .. } => "BelowRequiredAccuracy", + RootPruneReason::MetricRule { .. } => "MetricRule", + RootPruneReason::NoGuarantee { .. } => "NoGuarantee", + RootPruneReason::ValuesPerEvaluationUnknown { .. } => "ValuesPerEvaluationUnknown", + RootPruneReason::HeapSmallerThanTopK { .. } => "HeapSmallerThanTopK", + RootPruneReason::ChainWidthDoesNotDivideEvaluationGrid { .. } => { + "ChainWidthDoesNotDivideEvaluationGrid" + } + RootPruneReason::NoConsumer => "NoConsumer", + } +} + +/// `G3[4]`. +fn candidate_id_text(id: CandidateForMemoGroupId) -> String { + format!("G{}[{}]", id.group.0, id.index_in_group) +} + +/// A candidate on one line: `SummaryAgg Kll <- G1 θ=Kll(k=269)` (θ only +/// with `parameters`), then `fuses=[..]` and `requires=[..]` when it has +/// them. +fn candidate_text(candidate: &CandidateForMemoGroup, parameters: bool) -> String { + let operator = &candidate.expression.operator; + let inputs: Vec = operator + .children() + .into_iter() + .map(|group| format!("G{}", group.0)) + .collect(); + let mut text = operator_label(operator); + if !inputs.is_empty() { + let _ = write!(text, " {}", reads(&inputs)); + } + match candidate.logical_properties.parameters() { + None => text += " exact", + Some(theta) if parameters => { + let _ = write!(text, " θ={}", parameters_label(theta)); + } + Some(_) => {} + } + if !candidate.fused_groups.is_empty() { + let fused: Vec = candidate + .fused_groups + .iter() + .map(|group| format!("G{}", group.0)) + .collect(); + let _ = write!(text, " fuses=[{}]", fused.join(", ")); + } + if candidate + .input_requirements + .iter() + .any(|requirement| requirement.required_accuracy.is_some()) + { + let required: Vec = candidate + .input_requirements + .iter() + .map(|requirement| { + requirement + .required_accuracy + .as_ref() + .map_or("-".to_owned(), required_accuracy_text) + }) + .collect(); + let _ = write!(text, " requires=[{}]", required.join(", ")); + } + text +} + +/// `Rank ≤ 0.01 (δ 0.01)`. +fn required_accuracy_text(required: &RequiredAccuracy) -> String { + format!( + "{:?} ≤ {} (δ {})", + required.metric, required.max_error, required.failure_probability + ) +} + +/// Why an edge fails, or a candidate has none, without where: +/// `the producer's guarantee exceeds Rank ≤ 0.01 (δ 0.01)`. +pub fn root_prune_reason_text(reason: &RootPruneReason) -> String { + match reason { + RootPruneReason::BelowRequiredAccuracy { required, .. } => format!( + "the producer's guarantee exceeds {}", + required_accuracy_text(required) + ), + RootPruneReason::MetricRule { + producer_metric, + accepted_metrics, + .. + } => format!( + "metric rule: the consumer accepts {accepted_metrics:?} on this input, not the \ + producer's {producer_metric:?}" + ), + RootPruneReason::NoGuarantee { .. } => { + "the readout's error has no bound at the failure probability its queries require: \ + no metric, or no guarantee known to hold there (a randomized sketch under an exact \ + target's δ 0; HLL, whose failure probability is unknown; DDSketch over values not \ + of one sign)" + .to_owned() + } + RootPruneReason::ValuesPerEvaluationUnknown { query, .. } => { + format!( + "query {query}: the query's failure probability covers its whole result, and the \ + number of values one evaluation returns is unknown" + ) + } + RootPruneReason::ChainWidthDoesNotDivideEvaluationGrid { + tumbling_window_width_ms, + evaluation_period_ms, + .. + } => match evaluation_period_ms { + Some(period_ms) => format!( + "the chain's {tumbling_window_width_ms} ms windows do not end on the consumer's \ + evaluation grid of {period_ms} ms" + ), + None => format!( + "the chain's {tumbling_window_width_ms} ms windows cannot end on the consumer's \ + evaluation times: they are on no known grid" + ), + }, + RootPruneReason::HeapSmallerThanTopK { k, .. } => { + format!("the producer's heap is smaller than the top-{k}") + } + RootPruneReason::NoConsumer => "no consumer, and not a query output".to_owned(), + } +} + +/// The candidates Pass 2 eliminated, each with its θ and one line per +/// cause: the failing edge (`at G4[0] input 0`) or none, the eliminated +/// consumers the cause comes through, and why. Unless `printing_full_candidates`, per group +/// the count of its pruned candidates by their first cause. +fn write_pass2_pruned( + out: &mut String, + logical: &LogicalASAPMemoCandidates, + printing_full_candidates: bool, +) { + let _ = writeln!(out, "pruned by Pass 2 ({}):", logical.pruned.len()); + if !printing_full_candidates { + let mut causes: BTreeMap> = BTreeMap::new(); + for pruned in &logical.pruned { + causes.entry(pruned.candidate.group).or_default().push( + pruned + .causes + .first() + .map_or("no cause", |cause| root_prune_reason_name(&cause.root)), + ); + } + for (group, causes) in causes { + let _ = writeln!( + out, + " G{}: {} ({})", + group.0, + causes.len(), + counts_text(causes) + ); + } + return; + } + for pruned in &logical.pruned { + let candidate = logical.memo.candidate(pruned.candidate); + let _ = writeln!( + out, + " {} {}", + candidate_id_text(pruned.candidate), + candidate_text(candidate, true) + ); + for cause in &pruned.causes { + let mut line = String::new(); + if !cause.through_pruned_candidates.is_empty() { + let through: Vec = cause + .through_pruned_candidates + .iter() + .map(|id| candidate_id_text(*id)) + .collect(); + let _ = write!(line, "through {}, ", through.join(" → ")); + } + match &cause.root { + RootPruneReason::BelowRequiredAccuracy { + consumer, input, .. + } + | RootPruneReason::MetricRule { + consumer, input, .. + } + | RootPruneReason::HeapSmallerThanTopK { + consumer, input, .. + } + | RootPruneReason::ChainWidthDoesNotDivideEvaluationGrid { + consumer, input, .. + } + | RootPruneReason::NoGuarantee { consumer, input } => { + let _ = write!(line, "at {} input {input}: ", candidate_id_text(*consumer)); + } + RootPruneReason::ValuesPerEvaluationUnknown { consumer, .. } => { + let _ = write!(line, "at {}: ", candidate_id_text(*consumer)); + } + RootPruneReason::NoConsumer => {} + } + line.push_str(&root_prune_reason_text(&cause.root)); + let _ = writeln!(out, " {line}"); + } + } +} + +/// `(QueryTime, None)`. +fn physical_properties_text(properties: &PhysicalProperties) -> String { + format!("({:?}, {:?})", properties.timing(), properties.lifetime()) +} + +/// Why physical edge `edge` fails $\alpha^{\mathrm{phys}}$ (the producer's +/// timing and lifetime cannot serve the consumer's), or `None` when it +/// holds. +pub fn physical_edge_rejection( + physical: &PhysicalASAPMemoCandidates, + edge: &PhysicalEdge, +) -> Option { + (!edge.satisfies_physical_requirement).then(|| { + let properties = |id: PhysicalCandidateId| { + physical_properties_text(&physical.physical_candidates[id.0].physical_properties) + }; + format!( + "α^phys = 0: consumer {} cannot read producer {}", + properties(edge.consumer_candidate_id), + properties(edge.producer_candidate_id) + ) + }) +} + +/// The physical edges $(u, c, i)$ of `plan`: for each node, its candidate +/// and each input's producer node's candidate. Empty without a plan. +pub fn selected_edges( + plan: Option<&PhysicalASAPDAG>, +) -> BTreeSet<(PhysicalCandidateId, PhysicalCandidateId, usize)> { + let Some(plan) = plan else { + return BTreeSet::new(); + }; + let candidate_of = |node: usize| plan.nodes[node].selected_physical_candidate; + plan.nodes + .iter() + .filter_map(|node| Some((node, node.selected_physical_candidate?))) + .flat_map(|(node, consumer)| { + node.payload + .children() + .into_iter() + .enumerate() + .filter_map(move |(input, &producer)| { + Some((candidate_of(producer)?, consumer, input)) + }) + }) + .collect() +} + +/// Stage 2's edges by consumer and input: `edges[c][i]` lists the edges +/// $(u, c, i)$ in Stage 2's order. +pub fn edges_per_input(physical: &PhysicalASAPMemoCandidates) -> Vec>> { + let memo = &physical.logical_candidates.memo; + let mut edges: Vec>> = physical + .physical_candidates + .iter() + .map(|candidate| { + let inputs = memo + .candidate(candidate.logical_candidate_id) + .expression + .operator + .children() + .len(); + vec![Vec::new(); inputs] + }) + .collect(); + for (_, edge) in physical.edges() { + edges[edge.consumer_candidate_id.0][edge.input_index_of_consumer].push(edge); + } + edges +} + +/// `C3 = G2[0] (QueryTime, None)`. +fn physical_candidate_text( + physical: &PhysicalASAPMemoCandidates, + id: PhysicalCandidateId, +) -> String { + let candidate = &physical.physical_candidates[id.0]; + format!( + "C{} = {} {}", + id.0, + candidate_id_text(candidate.logical_candidate_id), + physical_properties_text(&candidate.physical_properties) + ) +} + +/// Stage 2's output: each physical candidate $v$ with its logical +/// candidate and $p^{\mathrm{phys}}_v$; under it, per input $(c, i)$, the +/// physical edges whose logical edge holds, with $\alpha^{\mathrm{phys}}$ +/// (`✓`, or `✗` and why not) and `selected` for an edge of `plan`; then the +/// candidates Stage 2 pruned (unless `printing_full_candidates`, their count per reason). +fn write_physical( + out: &mut String, + physical: &PhysicalASAPMemoCandidates, + plan: Option<&PhysicalASAPDAG>, + printing_full_candidates: bool, +) { + let memo = &physical.logical_candidates.memo; + let selected = selected_edges(plan); + let _ = writeln!( + out, + "physical candidates ({}):", + physical.physical_candidates.len() + ); + let edges_into = edges_per_input(physical); + for (id, candidate) in physical.physical_candidates.iter().enumerate() { + let operator = &memo + .candidate(candidate.logical_candidate_id) + .expression + .operator; + let _ = writeln!( + out, + " {} {}", + physical_candidate_text(physical, PhysicalCandidateId(id)), + operator_label(operator) + ); + for ((input, group), edges) in operator + .children() + .into_iter() + .enumerate() + .zip(&edges_into[id]) + { + let rows = edges + .iter() + .filter(|edge| edge.satisfies_logical_requirement) + .map(|edge| { + let producer = physical_candidate_text(physical, edge.producer_candidate_id); + let is_selected = selected.contains(&( + edge.producer_candidate_id, + PhysicalCandidateId(id), + input, + )); + match physical_edge_rejection(physical, edge) { + None => vec![ + producer, + "✓".to_owned(), + if is_selected { + "selected".into() + } else { + String::new() + }, + ], + Some(reason) => vec![producer, format!("✗ {reason}")], + } + }) + .collect(); + let _ = writeln!(out, " input {input} <- G{}:", group.0); + write_rows(out, " ", rows); + } + } + let _ = writeln!( + out, + "pruned by Stage 2 ({}):", + physical.pruned_candidates.len() + ); + if !printing_full_candidates { + let reasons: Vec = physical + .pruned_candidates + .iter() + .map(|pruned| { + let reason = format!("{:?}", pruned.reason); + reason + .split(['(', ' ']) + .next() + .unwrap_or_default() + .to_owned() + }) + .collect(); + if !reasons.is_empty() { + let _ = writeln!( + out, + " {}", + counts_text(reasons.iter().map(String::as_str)) + ); + } + return; + } + let rows = physical + .pruned_candidates + .iter() + .map(|pruned| { + vec![ + candidate_id_text(pruned.candidate.logical_candidate_id), + physical_properties_text(&pruned.candidate.physical_properties), + format!("{:?}", pruned.reason), + ] + }) + .collect(); + write_rows(out, " ", rows); +} + +/// Stage 3's plan, each node with its physical candidate, operator and +/// input nodes; the plan's cost (`plan_cost`) with each node's charge; its +/// materialized views (`materialized_views`), one line each with its node, +/// refresh and retention; each query's accepted accuracy; the selection +/// MILP's size; and the candidates Stage 3 pruned. Or why no plan was +/// selected. Costs print as `{:.6e}`. +fn write_selection( + out: &mut String, + stages: &PlanningStagesRun, + plan_cost: Option<&PlanCost>, + materialized_views: Option<&Result, String>>, +) { + let run = &stages.run; + let (plan, selection) = match &run.selection { + Ok(selected) => selected, + Err(error) => { + let _ = writeln!(out, "error: {error}"); + return; + } + }; + let _ = writeln!( + out, + "selection MILP: {} variables, {} constraints, {} ms building, {} ms solving \ + ({} {}), objective {:.6e}/s", + selection.milp_variable_count, + selection.milp_constraint_count, + selection.milp_construction_ms, + selection.milp_solve_ms, + selection.milp_solve_count, + if selection.milp_solve_count == 1 { + "solve" + } else { + "solves" + }, + selection.selected_plan_cost_per_second + ); + let physical = run.physical_asap_candidates.as_ref(); + let rows = plan + .nodes + .iter() + .map(|node| { + let inputs: Vec = node + .payload + .children() + .into_iter() + .map(|input| format!("n{input}")) + .collect(); + let candidate = match (physical, node.selected_physical_candidate) { + (Some(physical), Some(id)) => physical_candidate_text(physical, id), + _ => format!("({:?}, {:?})", node.output_state.timing, node.lifetime), + }; + let parameters = node + .parameters + .as_ref() + .map_or(String::new(), |parameters| { + format!("θ={}", parameters_label(parameters)) + }); + vec![ + format!("n{}:", node.id), + candidate, + operator_label(&node.payload), + parameters, + reads(&inputs), + ] + }) + .collect(); + write_rows(out, "", rows); + let roots: Vec = plan.roots.iter().map(|node| format!("n{node}")).collect(); + let _ = writeln!(out, "roots: [{}]", roots.join(", ")); + if let Some(cost) = plan_cost { + let _ = writeln!( + out, + "cost_per_second: {}", + cost.cost_per_second + .map_or("none".to_owned(), |cost| format!("{cost:.6e}")) + ); + let rows = cost + .per_node + .iter() + .map(|(node, charge)| { + vec![ + format!("n{node}"), + format!("{:?}", charge.operator), + format!("{:?}", charge.timing), + charge + .cost_per_second + .map_or("none".to_owned(), |cost| format!("{cost:.6e}")), + charge.detail.clone(), + ] + }) + .collect(); + write_rows(out, " ", rows); + } + match materialized_views { + Some(Ok(views)) => { + let _ = writeln!(out, "materialized views ({}):", views.len()); + let rows = views + .iter() + .map(|view| { + vec![ + format!("n{}", view.node_id), + format!("{:?}", view.refresh()), + format!("retention {} ms", view.retention_ms()), + ] + }) + .collect(); + write_rows(out, " ", rows); + } + Some(Err(error)) => { + let _ = writeln!(out, "materialized views: not installable: {error}"); + } + None => {} + } + let _ = writeln!(out, "accepted accuracy:"); + for (query, accepted) in selection.accepted_accuracy_per_query.iter().enumerate() { + let _ = writeln!(out, " q{}: {accepted}", query + 1); + } + let _ = writeln!( + out, + "pruned by Stage 3 ({}):", + selection.pruned_candidates.len() + ); + let rows = selection + .pruned_candidates + .iter() + .map(|pruned| { + vec![ + candidate_id_text(pruned.candidate.logical_candidate_id), + physical_properties_text(&pruned.candidate.physical_properties), + format!("{:?}", pruned.reason), + ] + }) + .collect(); + write_rows(out, " ", rows); } diff --git a/crates/devtools/src/rqe_atomic_costs.rs b/crates/devtools/src/rqe_atomic_costs.rs index ac137e19d..faccfed55 100644 --- a/crates/devtools/src/rqe_atomic_costs.rs +++ b/crates/devtools/src/rqe_atomic_costs.rs @@ -18,7 +18,7 @@ //! ``` //! //! Only this converter reads the sketch-bench format; the planner reads the -//! table. Rows plan selection cannot price from the table are skipped with +//! table. Rows plan selection cannot cost from the table are skipped with //! the reason: exact accumulators, Hydra structures (shared across groups, //! measured at one label width) and sketch names with no planner sketch. //! @@ -33,7 +33,7 @@ use serde_json::Value; pub struct SkippedRow { /// The row's `sketch` name, e.g. `exact-sum`. pub sketch_name: String, - /// Why plan selection cannot price from it. + /// Why plan selection cannot cost from it. pub reason: String, } @@ -110,9 +110,9 @@ pub fn convert_rqe_atomic_costs(rows: &Value) -> Result /// Why a row whose name has no planner sketch is skipped. fn skip_reason(sketch_name: &str) -> &'static str { if sketch_name.starts_with("exact-") { - "an exact accumulator: the table prices sketches only" + "an exact accumulator: the table costs sketches only" } else if sketch_name.starts_with("hydra-") { - "a Hydra structure shared across groups: the table prices one sketch per group" + "a Hydra structure shared across groups: the table costs one sketch per group" } else { "no planner sketch has this name" } diff --git a/crates/devtools/src/stable_output.rs b/crates/devtools/src/stable_output.rs new file mode 100644 index 000000000..909fec0c2 --- /dev/null +++ b/crates/devtools/src/stable_output.rs @@ -0,0 +1,254 @@ +//! What keeps `asap_planner`'s JSON document the same across changes that +//! do not change the plan's meaning, so it can be compared as text: +//! canonical memo ids ([`CanonicalIds`]) and numbers rounded to 12 +//! significant digits ([`rounded_to_12_significant_digits`]). +//! +//! The planner numbers memo groups and their candidates in insertion +//! order, so a new rule or a reordered loop renumbers every later group, +//! and a query added to a workload renumbers the groups Pass 1 adds for +//! the queries before it. +//! The document renumbers them instead: depth first from the query roots, +//! in query order. The text output keeps the planner's own ids, which +//! logs and error messages use. + +use asap_types::stages::{ + CandidateForMemoGroup, CandidateForMemoGroupId, Memo, MemoGroupId, PhysicalASAPMemoCandidates, + PhysicalCandidateId, +}; +use serde_json::{json, Value}; + +/// The canonical ids of one run's memo groups, logical candidates and +/// physical candidates, from the run's last memo (memo ids only append, so +/// a group or candidate keeps its id in every step). +/// +/// - Groups: depth first from each query's root groups, in query order; +/// below a group, its expression's inputs, then each candidate's inputs +/// and fused groups, candidates in canonical order. A group no query +/// reaches comes after, in planner order. +/// - Candidates of a group: ordered by origin, then operator (without its +/// inputs), neither of which depends on ids; candidates equal in both +/// keep planner order, e.g. a sketch's θ list in the order its rule +/// enumerates it (`Kll(k=26)`, `Kll(k=52)`, ...). +/// - Physical candidates: by their logical candidate's canonical id, then +/// in Stage 2's order (one per timing and lifetime it enumerates). +/// +/// Example: `quantile_over_time(0.9, m[5m])` gives its `Aggregate +/// quantile` root group id 0, the `TimeRange 300s` below it 1 and the +/// `Scan` 2 (the planner numbers them the other way round, `Scan` first), +/// then the groups its candidates read: the `SummaryEstimate` readout 3 +/// and its `SummaryAgg Kll` 4. +pub struct CanonicalIds { + /// Indexed by the planner's group id. + canonical_group_of_planner_group: Vec, + /// Per planner group, indexed by the planner's candidate index. + canonical_index_of_planner_index: Vec>, + /// Indexed by the planner's physical candidate id; empty before Stage 2. + canonical_physical_of_planner_physical: Vec, +} + +impl CanonicalIds { + /// The canonical ids of `memo` (the run's last memo) and, after Stage + /// 2, of its `physical` candidates. + pub fn of(memo: &Memo, physical: Option<&PhysicalASAPMemoCandidates>) -> Self { + let groups = memo.groups(); + let candidate_order: Vec> = groups + .iter() + .map(|group| { + let keys: Vec = group.candidates.iter().map(candidate_order_key).collect(); + let mut order: Vec = (0..keys.len()).collect(); + // Stable: equal keys keep planner order. + order.sort_by(|a, b| keys[*a].cmp(&keys[*b])); + order + }) + .collect(); + let mut group_order: Vec = Vec::with_capacity(groups.len()); + let mut seen = vec![false; groups.len()]; + let query_roots = memo + .queries() + .iter() + .flat_map(|query| query.root.operator_refs().into_iter().copied()); + let all_groups = (0..groups.len() as u32).map(MemoGroupId); + for start in query_roots.chain(all_groups) { + let mut pending = vec![start]; + while let Some(group) = pending.pop() { + if std::mem::replace(&mut seen[group.0 as usize], true) { + continue; + } + group_order.push(group); + let memo_group = &groups[group.0 as usize]; + let mut below: Vec = memo_group + .expression + .children() + .into_iter() + .copied() + .collect(); + for &index in &candidate_order[group.0 as usize] { + let candidate = &memo_group.candidates[index]; + below.extend( + candidate + .expression + .operator + .children() + .into_iter() + .copied(), + ); + below.extend(candidate.fused_groups.iter().copied()); + } + // Reversed, so the first input is visited first. + pending.extend(below.into_iter().rev()); + } + } + let mut canonical_group_of_planner_group = vec![0; groups.len()]; + for (canonical, group) in group_order.iter().enumerate() { + canonical_group_of_planner_group[group.0 as usize] = canonical as u32; + } + let canonical_index_of_planner_index = candidate_order + .iter() + .map(|order| { + let mut canonical_of = vec![0; order.len()]; + for (canonical, &index) in order.iter().enumerate() { + canonical_of[index] = canonical; + } + canonical_of + }) + .collect(); + let mut ids = Self { + canonical_group_of_planner_group, + canonical_index_of_planner_index, + canonical_physical_of_planner_physical: Vec::new(), + }; + if let Some(physical) = physical { + let mut order: Vec = (0..physical.physical_candidates.len()).collect(); + order.sort_by_key(|&id| { + ids.candidate(physical.physical_candidates[id].logical_candidate_id) + }); + let mut canonical_of = vec![0; order.len()]; + for (canonical, &id) in order.iter().enumerate() { + canonical_of[id] = canonical; + } + ids.canonical_physical_of_planner_physical = canonical_of; + } + ids + } + + /// `group`'s canonical id. + pub fn group(&self, group: MemoGroupId) -> u32 { + self.canonical_group_of_planner_group[group.0 as usize] + } + + /// The canonical (group, index in group) of a logical candidate. + pub fn candidate(&self, id: CandidateForMemoGroupId) -> (u32, usize) { + ( + self.group(id.group), + self.canonical_index_of_planner_index[id.group.0 as usize][id.index_in_group], + ) + } + + /// A physical candidate's canonical id. + pub fn physical(&self, id: PhysicalCandidateId) -> usize { + self.canonical_physical_of_planner_physical[id.0] + } + + /// A logical candidate as the document prints it: + /// `{"group": .., "index_in_group": ..}`. + pub fn candidate_json(&self, id: CandidateForMemoGroupId) -> Value { + let (group, index_in_group) = self.candidate(id); + json!({ "group": group, "index_in_group": index_in_group }) + } + + /// `value`, a serialized planner value (a `RootPruneReason` or a + /// `PruneReason`), with each logical candidate id (an object of exactly + /// `group` and `index_in_group`) and each `dominated_by` physical + /// candidate id made canonical. + pub fn in_serialized(&self, value: Value) -> Value { + match value { + Value::Object(mut map) => { + let is_candidate_id = map.len() == 2 + && map.get("group").is_some_and(Value::is_u64) + && map.get("index_in_group").is_some_and(Value::is_u64); + if is_candidate_id { + let id = CandidateForMemoGroupId { + group: MemoGroupId(map["group"].as_u64().unwrap_or_default() as u32), + index_in_group: map["index_in_group"].as_u64().unwrap_or_default() as usize, + }; + return self.candidate_json(id); + } + if let Some(id) = map.get("dominated_by").and_then(Value::as_u64) { + map.insert( + "dominated_by".into(), + json!(self.physical(PhysicalCandidateId(id as usize))), + ); + return Value::Object(map); + } + Value::Object( + map.into_iter() + .map(|(key, value)| (key, self.in_serialized(value))) + .collect(), + ) + } + Value::Array(items) => { + Value::Array(items.into_iter().map(|v| self.in_serialized(v)).collect()) + } + other => other, + } + } +} + +/// What orders a group's candidates, none of it an id: the origin and the +/// operator without its inputs. +fn candidate_order_key(candidate: &CandidateForMemoGroup) -> String { + json!([ + candidate.origin, + candidate.expression.operator.map_children(|_| ()) + ]) + .to_string() +} + +/// `value` with every non-integer number rounded to 12 significant digits, +/// e.g. `0.30000000000000004` to `0.3`: the last digits of a cost move +/// when the planner sums the same terms in another order. +pub fn rounded_to_12_significant_digits(value: Value) -> Value { + match value { + Value::Number(n) if n.is_f64() => { + let x = n.as_f64().unwrap_or_default(); + json!(format!("{x:.11e}").parse::().unwrap_or(x)) + } + Value::Object(map) => Value::Object( + map.into_iter() + .map(|(key, value)| (key, rounded_to_12_significant_digits(value))) + .collect(), + ), + Value::Array(items) => Value::Array( + items + .into_iter() + .map(rounded_to_12_significant_digits) + .collect(), + ), + other => other, + } +} + +#[cfg(test)] +mod tests { + use super::*; + + /// A float keeps 12 significant digits; integers and other values are + /// untouched. + #[test] + fn floats_keep_12_significant_digits() { + assert_eq!( + rounded_to_12_significant_digits(json!({ + "cost": 0.1 + 0.2, + "small": 1.234567890123456e-9, + "count": 3, + "list": [2.0000000000001, "x"], + })), + json!({ + "cost": 0.3, + "small": 1.23456789012e-9, + "count": 3, + "list": [2.0, "x"], + }) + ); + } +} diff --git a/crates/devtools/tests/asap_planner.rs b/crates/devtools/tests/asap_planner.rs index c2606f96b..8e4728bb9 100644 --- a/crates/devtools/tests/asap_planner.rs +++ b/crates/devtools/tests/asap_planner.rs @@ -12,10 +12,6 @@ use asap_types::workload::PlanningWorkload; use serde_json::{json, Value}; -/// The id of the selected plan among the physical candidates: the -/// selection is always shown as `P1`. -const SELECTED: &str = "P1"; - /// A file under the system temp directory, removed when dropped. struct TempFile(PathBuf); @@ -168,7 +164,7 @@ fn cost_inputs(inputs: &asap_cost::CostInputs) -> TempFile { TempFile::new(&serde_json::to_string(inputs).unwrap()) } -/// Cost model 1 at AWS Fargate's us-east-1 prices. +/// Cost model 1 at AWS Fargate's us-east-1 costs. fn aws_fargate() -> asap_cost::CostInputs { let defaults = asap_cost::CostInputs::default(); asap_cost::CostInputs { @@ -179,11 +175,10 @@ fn aws_fargate() -> asap_cost::CostInputs { } } -/// Stage 3 selected a plan: `stage3_plan_selection` is its `Selection`, -/// whose plan is the physical candidate [`SELECTED`] with its `cost` and -/// `accepted_accuracy_per_query`. +/// Stage 3 selected a plan: `stage3_plan_selection` has its `plan` and the +/// plan's `cost`. fn selects_a_plan(selection: &Value) -> bool { - selection["selected_dag"].is_object() && selection["cost"].is_object() + selection["plan"].is_object() && selection["cost"].is_object() } fn validated_dag(value: &Value, queries: usize) { @@ -204,36 +199,16 @@ fn validated_dag(value: &Value, queries: usize) { } } -/// The selected plan `P1`, validated, with one root per query. +/// Stage 3's plan, validated, with one root per operator query of +/// `queries`. fn selected_physical(document: &Value, queries: usize) -> PhysicalASAPDAG { - let candidate = document["stage2_physical_asap_optimization"]["candidates"] - .as_array() - .unwrap() - .iter() - .find(|c| c["id"] == SELECTED) - .expect("the selected plan is shown"); - let dag: PhysicalASAPDAG = serde_json::from_value(candidate["dag"].clone()).unwrap(); + let dag: PhysicalASAPDAG = + serde_json::from_value(document["stage3_plan_selection"]["plan"].clone()).unwrap(); dag.validate().unwrap(); - assert_eq!(dag.roots.len(), queries); + assert_eq!(dag.roots.len() + dag.scalar_roots.len(), queries); dag } -/// Stage 1's `pruned_candidates`: each candidate Pass 1 pruned, with its -/// reason, in pruning order. -fn pass1_pruned(document: &Value) -> Vec<(String, String)> { - document["stage1_logical_asap_optimization"]["pruned_candidates"] - .as_array() - .unwrap() - .iter() - .map(|pruned| { - ( - pruned["candidate"].as_str().unwrap().to_string(), - pruned["reason"].as_str().unwrap().to_string(), - ) - }) - .collect() -} - /// Every candidate label of the memo step `step`. fn memo_labels(document: &Value, step: &str) -> Vec { document["memo"]["steps"] @@ -250,57 +225,49 @@ fn memo_labels(document: &Value, step: &str) -> Vec { .collect() } -/// Example 1's document: every logical DAG is a well-formed flat DAG and -/// every physical DAG validates; each has one root per query; ids are -/// unique; Stage 2's candidates come from the shown logical ones and carry -/// no cost; Stage 3 selects `P1`. +/// Example 1's document: the frontends' DAG is a well-formed flat DAG with +/// one root per query; each physical candidate names a candidate of the +/// final memo, with unique ids and no cost; Stage 3's plan validates. #[test] fn example1_document_validates_its_stage_dags() { let document = example_document("example1", &["--run-until", "stage3"]); - assert_eq!(document["format"], "asap-stage-pipeline/v1"); + assert_eq!(document["format"], "asap-stage-pipeline/v2"); let queries = document["workload"]["queries"].as_array().unwrap().len(); assert_eq!(queries, 2); validated_dag(&document["stage0_query_language_frontends"]["dag"], queries); - let logical = &document["stage1_logical_asap_optimization"]; - let candidates = logical["candidates"].as_array().unwrap(); - assert_eq!(logical["capped"], false); - assert_eq!(logical["combinations"], candidates.len()); - let mut ids = HashSet::new(); - let mut composed = HashSet::new(); - for candidate in candidates { - let id = candidate["id"].as_str().unwrap(); - assert!(ids.insert(id)); - if candidate.get("dag").is_some() { - composed.insert(id); - validated_dag(&candidate["dag"], queries); - } - } - - let physical = document["stage2_physical_asap_optimization"]["candidates"] + let pass2 = memo_step(&document, "pass2"); + let physical = document["stage2_physical_asap_optimization"]["physical_candidates"] .as_array() .unwrap(); - let sources: HashSet<_> = physical - .iter() - .map(|p| p["from_logical"].as_str().unwrap()) - .collect(); - assert_eq!(sources, composed); - for candidate in physical { + assert!(!physical.is_empty()); + for (id, candidate) in physical.iter().enumerate() { + assert_eq!(candidate["id"], id); assert!(candidate.get("cost").is_none(), "Stage 2 has no cost"); - let dag: PhysicalASAPDAG = serde_json::from_value(candidate["dag"].clone()).unwrap(); - assert_eq!(dag.roots.len(), queries); - dag.validate().unwrap(); + let group = &pass2["groups"][candidate["group"].as_u64().unwrap() as usize]; + assert!( + group["candidates"][candidate["index_in_group"].as_u64().unwrap() as usize].is_object() + ); } assert!(selects_a_plan(&document["stage3_plan_selection"])); selected_physical(&document, queries); } +/// The memo step `step` of `document`. +fn memo_step<'a>(document: &'a Value, step: &str) -> &'a Value { + document["memo"]["steps"] + .as_array() + .unwrap() + .iter() + .find(|s| s["step"] == step) + .unwrap_or_else(|| panic!("no memo step {step}")) +} + /// A top-k partitioned by every label of its input has no item, so a heap -/// over it could not be composed; such a heap reads one row per item and -/// is pruned in Pass 1 (`TopKHeapOverOneRowPerItem`), listed under -/// `pruned_candidates`, and the selected plan's logical DAG composes. +/// over it could not be composed; Pass 1 offers no heap for it (a heap +/// would read one row per item), and a plan is selected. #[test] -fn a_topk_heap_over_one_row_per_item_is_pruned_with_its_reason() { +fn a_topk_heap_over_one_row_per_item_is_not_offered() { let document = document( &adhoc( &["topk by (job) (3, sum by (job) (rate(m[5m])))"], @@ -309,23 +276,12 @@ fn a_topk_heap_over_one_row_per_item_is_pruned_with_its_reason() { ), &["--run-until", "stage3", "--ignore-accuracy-guarantees"], ); - assert_eq!( - pass1_pruned(&document), - [ - ("CMS+heap".into(), "TopKHeapOverOneRowPerItem".into()), - ( - "CountSketch+heap".into(), - "TopKHeapOverOneRowPerItem".into() - ), - ] + let labels = memo_labels(&document, "pass1"); + assert!( + labels.iter().all(|label| !label.contains("WithHeap")), + "{labels:?}" ); - let candidates = document["stage1_logical_asap_optimization"]["candidates"] - .as_array() - .unwrap(); - assert!(!candidates.is_empty()); - for candidate in candidates { - validated_dag(&candidate["dag"], 1); - } + selected_physical(&document, 1); } /// Example 3a (243 Pass 1 candidates) selects a plan whose physical DAG @@ -459,7 +415,8 @@ fn nine_quantile_queries() -> Value { /// Nine quantile queries plan with the candidate-connection search, which /// searches Stage 2's catalog without enumerating the combinations. The -/// document holds the selected plan and the search's report. +/// document holds Stage 2's candidates, the plan and the selection MILP's +/// size. #[test] fn nine_queries_plan_without_cartesian_enumeration() { let document = document( @@ -469,12 +426,14 @@ fn nine_queries_plan_without_cartesian_enumeration() { assert_eq!(document["search"], "candidate-connections"); let selection = &document["stage3_plan_selection"]; assert!(selects_a_plan(selection), "{selection}"); - let physical = document["stage2_physical_asap_optimization"]["candidates"] + let physical = document["stage2_physical_asap_optimization"]["physical_candidates"] .as_array() .unwrap(); - assert_eq!(physical.len(), 1); - assert_eq!(physical[0]["id"], SELECTED); - assert!(selection["search_report"].is_object(), "{selection}"); + assert!(!physical.is_empty()); + assert!( + selection["milp_variables"].as_u64().unwrap() > 0, + "{selection}" + ); } /// A deployment that keeps neither raw data nor query-time results. @@ -485,7 +444,7 @@ const NINE_QUERY_WORKLOAD: &str = "../../tools/dag-viewer/workloads/random-mixed-seed-862/workload-01.json"; /// The nine-query workload's document up to `run_until`, planned as a -/// deployment that keeps no raw data at AWS Fargate prices. +/// deployment that keeps no raw data at AWS Fargate costs. fn nine_query_document(run_until: &str) -> Value { let path = PathBuf::from(env!("CARGO_MANIFEST_DIR")).join(NINE_QUERY_WORKLOAD); let capabilities = TempFile::new(NO_RAW); @@ -503,7 +462,7 @@ fn nine_query_document(run_until: &str) -> Value { } /// The nine-query workload, planned as a deployment that keeps no raw data -/// at AWS Fargate prices with guarantees enforced: `asap_planner` lowers it +/// at AWS Fargate costs with guarantees enforced: `asap_planner` lowers it /// to one root per query, and installation returns a validated plan that /// accepts no query without its guarantee. #[test] @@ -539,25 +498,16 @@ fn the_nine_query_workload_lowers_and_installs() { } /// `asap_planner` selects a validated plan for the nine-query workload -/// (no raw data, AWS Fargate prices, guarantees enforced), with an -/// accepted accuracy per query and a MILP search report. +/// (no raw data, AWS Fargate costs, guarantees enforced), with an +/// accepted accuracy per query and the selection MILP's size. #[test] fn the_nine_query_workload_selects_a_valid_plan() { let document = nine_query_document("stage3"); let queries = 9; let selection = &document["stage3_plan_selection"]; assert!(selects_a_plan(selection), "{selection}"); - let physical = document["stage2_physical_asap_optimization"]["candidates"] - .as_array() - .unwrap(); - assert_eq!(physical.len(), 1); selected_physical(&document, queries); - assert!( - selection["search_report"]["milp_variables"] - .as_u64() - .unwrap() - > 0 - ); + assert!(selection["milp_variables"].as_u64().unwrap() > 0); assert_eq!( selection["accepted_accuracy_per_query"] .as_array() @@ -607,63 +557,6 @@ fn table_declared_twice_is_rejected() { ); } -/// When Pass 1 fails, the document still carries the Stage 0 DAG, with the -/// error under Stage 1 and no Stage 2 or Stage 3 section, so the viewer can -/// show both, and the run exits 1. -#[test] -fn pass1_failure_still_writes_stage0_and_the_error() { - let document = document( - &adhoc(&["topk(5, rate(m[5m]))"], within(1.5, 0.1), &["m"]), - &["--run-until", "stage3", "--ignore-accuracy-guarantees"], - ); - validated_dag(&document["stage0_query_language_frontends"]["dag"], 1); - let logical = &document["stage1_logical_asap_optimization"]; - assert!( - logical["error"] - .as_str() - .unwrap() - .starts_with("Pass 1: approximate accuracy"), - "{logical}" - ); - assert_eq!(logical["combinations"], 0); - assert_eq!(logical["candidates"], json!([])); - assert_eq!(document["stage2_physical_asap_optimization"], Value::Null); - assert!(!selects_a_plan(&document["stage3_plan_selection"])); - // Text mode also prints the error and exits 1. - let file = TempFile::json(&adhoc(&["topk(5, rate(m[5m]))"], within(1.5, 0.1), &["m"])); - let output = planner(&["--workload", file.path(), "--run-until", "stage3"]); - assert_eq!(output.status.code(), Some(1)); - assert!(String::from_utf8(output.stdout) - .unwrap() - .contains("error: Pass 1: ")); -} - -/// A query Pass 1 rejects is reported before deployment capabilities that -/// contradict the data workload (as `run_planning` orders them), and a -/// metric-type warning does not turn the null Stage 3 section into one. -#[test] -fn pass1_failure_comes_before_capabilities_and_keeps_stage3_null() { - let mut workload = adhoc(&["topk(5, rate(m[5m]))"], within(1.5, 0.1), &["m"]); - workload["data_workload"]["arrival"] = json!("at_rest"); - workload["data_workload"]["sources"]["m"]["metric_type"] = json!("gauge"); - let no_raw = TempFile::new(NO_RAW); - let document = document( - &workload, - &[ - "--run-until", - "stage3", - "--ignore-accuracy-guarantees", - "--deployment-capabilities", - no_raw.path(), - ], - ); - let error = document["stage1_logical_asap_optimization"]["error"] - .as_str() - .unwrap(); - assert!(error.starts_with("Pass 1: approximate accuracy"), "{error}"); - assert_eq!(document["stage3_plan_selection"], Value::Null); -} - /// Standalone scalar queries first, in the middle and last keep their /// query positions in the selected physical DAG's `scalar_roots` (the /// viewer's test_viewer.py draws this workload). @@ -684,13 +577,7 @@ fn scalar_queries_keep_their_positions_in_scalar_roots() { &["--run-until", "stage3", "--ignore-accuracy-guarantees"], ); assert!(selects_a_plan(&document["stage3_plan_selection"])); - let physical = document["stage2_physical_asap_optimization"]["candidates"] - .as_array() - .unwrap() - .iter() - .find(|c| c["id"] == SELECTED) - .unwrap(); - let positions: Vec<_> = physical["dag"]["scalar_roots"] + let positions: Vec<_> = document["stage3_plan_selection"]["plan"]["scalar_roots"] .as_array() .unwrap() .iter() @@ -741,9 +628,9 @@ fn a_target_without_a_metric_shows_the_query_default() { /// A quantile query's target may leave its metric unset: then Pass 1 keeps /// both KLL (rank) and DDSketch (relative) candidates, which compete on -/// cost, and the selection reports the metric it accepted and that the -/// target's metric was not set. With `accuracy_metric: rank`, Pass 1 prunes -/// DDSketch and the selection accepts rank (or exact). The workload's +/// cost, and the selection reports what it accepted. With +/// `accuracy_metric: rank`, the metric rule leaves DDSketch no edge and the +/// selection accepts rank (or exact). The workload's /// accuracy is the target's own JSON form. Guarantees are enforced (the /// default), since the test is about which guarantees meet the target. #[test] @@ -784,11 +671,6 @@ fn a_quantile_target_without_a_metric_accepts_rank_or_relative() { assert!(unset["workload"]["queries"][0]["requirements"] .get("default_metric") .is_some_and(Value::is_null)); - assert!( - pass1_pruned(&unset).is_empty(), - "{:?}", - pass1_pruned(&unset) - ); let labels = memo_labels(&unset, "pass1"); for family in ["Kll", "DDSketch"] { assert!( @@ -797,31 +679,22 @@ fn a_quantile_target_without_a_metric_accepts_rank_or_relative() { ); } let accepted = summary(&unset); - assert!( - accepted.starts_with("metric not set; accepted "), - "{accepted}" - ); - assert!( - unset["stage3_plan_selection"]["accepted_accuracy_per_query"][0]["accepted_metric"] - .is_string() - ); + assert!(accepted.starts_with("accepted "), "{accepted}"); + // A DDSketch states a relative error, which a rank target cannot + // read (the metric rule): only KLL or an exact plan is accepted. let rank = query(Some("rank")); - assert_eq!( - pass1_pruned(&rank), - [("DDSketch".into(), "NoErrorBoundForQueryTarget".into())] - ); let accepted = summary(&rank); assert!( - accepted == "metric rank; accepted rank (Kll)" || accepted == "metric rank; accepted exact", + accepted == "accepted rank (Kll)" || accepted == "accepted exact", "{accepted}" ); } /// The `--cost-inputs` cost model decides how Example 4's candidates are -/// compared: cost model 1 (the default) selects a priced plan, and Stage 3's -/// selection MILP refuses cost models 2 and 3 with an error naming the -/// model (#888). +/// compared: cost model 1 (the default) selects a plan with a cost per +/// second, and Stage 3's selection MILP refuses cost models 2 and 3 with +/// an error saying it implements cost model 1 only (#888). #[test] fn the_cost_model_chooses_how_candidates_are_compared() { use asap_cost::{ClusterLimits, ClusterObjective, CostInputs, CostModel, ResourceUnitCost}; @@ -836,34 +709,33 @@ fn the_cost_model_chooses_how_candidates_are_compared() { let usage = selected(CostModel::UsageOverTime(ResourceUnitCost::illustrative())); assert!(selects_a_plan(&usage), "{usage}"); assert!(usage["cost"]["cost_per_second"].is_number()); - for (cost_model, name) in [ - ( - CostModel::ProvisionedCapacity(ResourceUnitCost::illustrative()), - "ProvisionedCapacity (cost model 2)", - ), - ( - CostModel::ClusterCapacity(ClusterLimits { - memory_bytes: 256_000_000_000, - disk_bytes: 1_000_000_000_000, - cpu_cores: None, - objective: ClusterObjective::LowestLatency, - }), - "ClusterCapacity (cost model 3)", - ), + for cost_model in [ + CostModel::ProvisionedCapacity(ResourceUnitCost::illustrative()), + CostModel::ClusterCapacity(ClusterLimits { + memory_bytes: 256_000_000_000, + disk_bytes: 1_000_000_000_000, + cpu_cores: None, + objective: ClusterObjective::LowestLatency, + }), ] { let refused = selected(cost_model); assert!(!selects_a_plan(&refused), "{refused}"); assert!( - refused["error"].as_str().unwrap().contains(name), + refused["error"] + .as_str() + .unwrap() + .contains("implements cost model 1 only"), "{refused}" ); } } -/// AWS Fargate's us-east-1 prices Example 4 under cost model 1 in US -/// dollars per second: the selected plan costs no more than the -/// whole-window KLL, $2.60278e-8/s (its 2.1 CPU-ms/s at $1.1244e-8 per -/// CPU-ms, and its states held while the query runs). +/// AWS Fargate's us-east-1 costs Example 4 under cost model 1 in US +/// dollars per second: the selected plan's cost is its usage at the page's +/// prices, $1.1244e-5 per vCPU-second (1.1244e-14 per CPU operation of +/// 1 ns, 1.1244e-15 per scanned byte) and $1.235e-6 per GiB-second of +/// memory. E.g. 1.17e8 CPU operations, 1.07e7 scanned bytes and 2.17e7 +/// average bytes held per second cost $1.3525e-6/s. #[test] fn aws_fargate_prices_example_4_in_dollars() { let fargate = cost_inputs(&aws_fargate()); @@ -877,20 +749,27 @@ fn aws_fargate_prices_example_4_in_dollars() { fargate.path(), ], ); - let cost = document["stage3_plan_selection"]["cost"]["cost_per_second"] - .as_f64() - .unwrap(); - assert!(cost.is_finite() && cost > 0.0); - assert!(cost <= 2.602778532914e-8 * (1.0 + 1e-6), "{cost}"); + let cost = &document["stage3_plan_selection"]["cost"]; + let cost_per_second = cost["cost_per_second"].as_f64().unwrap(); + let usage = |field: &str| cost["usage"][field].as_f64().unwrap(); + let at_page_prices = usage("cpu_ops_per_second") * 1.1244e-14 + + usage("scanned_bytes_per_second") * 1.1244e-15 + + usage("average_memory_bytes") * 1.235e-6 / (1u64 << 30) as f64; + assert!(cost_per_second > 0.0, "{cost}"); + // 1e-9: the document's numbers are rounded to 12 significant digits. + assert!( + (cost_per_second - at_page_prices).abs() <= 1e-9 * at_page_prices, + "{cost_per_second} != {at_page_prices}: {cost}" + ); } /// `--deployment-capabilities` declares the executor's capabilities, which /// the document's `deployment_capabilities` shows (defaults filled in; /// without the flag, every summary supported, raw data retained, no /// query-time result retained). For Pattern B with KLL only and query-time -/// results retained: no DDSketch candidate is offered. Without raw data, a -/// plan is still selected with guarantees enforced, from materialized -/// views. A deployment with an unknown field is refused. +/// results retained: Pass 1 offers no DDSketch candidate. Without raw data, a plan is still selected +/// with guarantees enforced, from materialized views. A deployment with an +/// unknown field is refused. #[test] fn deployment_capabilities_flag_sets_the_capabilities_and_shows_them() { let with_capabilities = |capabilities: &str, extra: &[&str]| { @@ -929,14 +808,17 @@ fn deployment_capabilities_flag_sets_the_capabilities_and_shows_them() { "query_time_results_retained": true, }) ); + let has_ddsketch = |document: &Value| { + memo_labels(document, "pass1") + .iter() + .any(|l| l.contains("DDSketch")) + }; + assert!(!has_ddsketch(&kll_kept)); + assert!(has_ddsketch(&default)); assert!(selects_a_plan(&kll_kept["stage3_plan_selection"])); - let physical = kll_kept["stage2_physical_asap_optimization"]["candidates"] - .as_array() - .unwrap(); - assert!(!physical.is_empty()); - assert!(physical - .iter() - .all(|c| !c["dag"].to_string().contains("DDSketch"))); + assert!(!kll_kept["stage3_plan_selection"]["plan"] + .to_string() + .contains("DDSketch")); let no_raw = with_capabilities(r#"{"raw_data_retained": false}"#, &[]); let selection = &no_raw["stage3_plan_selection"]; @@ -961,99 +843,12 @@ fn deployment_capabilities_flag_sets_the_capabilities_and_shows_them() { ); } -/// Two sums and two maxima over two metrics share each metric's scan and -/// range between its own pair of queries: the selected logical candidate's -/// label names which queries share each node. -#[test] -fn logical_candidate_labels_name_each_query() { - let document = document( - &adhoc( - &["sum(a)", "max(a)", "sum(b)", "max(b)"], - exact(), - &["a", "b"], - ), - &["--run-until", "stage3", "--ignore-accuracy-guarantees"], - ); - let labels: Vec<&str> = document["stage1_logical_asap_optimization"]["candidates"] - .as_array() - .unwrap() - .iter() - .map(|c| c["display_name"].as_str().unwrap()) - .collect(); - assert!(!labels.is_empty()); - assert!(labels - .iter() - .all(|label| (1..=4).all(|q| label.contains(&format!("Q{q} "))))); -} - -/// The MILP search's reported lower bound never exceeds the selected -/// plan's cost, and an optimal search reports a relative gap of at most -/// 1e-4, on `topk(32, sum by (label_0) (sum_over_time(data[1h])))`, whose -/// largest charge (the query-time top-k tumbling windows) is about 308 times the -/// selected plan's. -#[test] -fn milp_lower_bound_is_consistent_with_the_selected_cost() { - let workload = json!({ - "query_workload": { - "language": "promql", - "query_batch": null, - "repeating_queries": [{ - "query": "topk(32, sum by (label_0) (sum_over_time(data[1h])))", - "demand": {"fixed_interval_at": {"interval": 60000, "evaluation_phase": 0}}, - "requirements": { - "accuracy": { - "max_error": 0.05, - "failure_probability": 0.01, - "accuracy_scope": "whole_result", - }, - "max_latency_ms": null, - }, - "predictability": {"predictable": {"known_at": null}}, - }], - }, - "data_workload": { - "arrival": {"ingesting": {"rows_per_second": 2000000.0}}, - "sources": {"data": { - "series_count": 10000, - "metric_type": "gauge", - "column_signs": {"value": "non_negative"}, - "sample_interval": 5, - }}, - }, - }); - let capabilities = - TempFile::new(r#"{"raw_data_retained": true, "query_time_results_retained": false}"#); - let fargate = cost_inputs(&aws_fargate()); - let document = document( - &workload, - &[ - "--run-until", - "stage3", - "--deployment-capabilities", - capabilities.path(), - "--cost-inputs", - fargate.path(), - ], - ); - let selection = &document["stage3_plan_selection"]; - assert!(selects_a_plan(selection), "{selection}"); - let report = &selection["search_report"]; - let incumbent = selection["cost"]["cost_per_second"].as_f64().unwrap(); - if let Some(lower_bound) = report["cost_lower_bound_per_second"].as_f64() { - assert!(lower_bound <= incumbent * (1.0 + 1e-9), "{report}"); - if report["status"] == "optimal" { - let relative_gap = (incumbent - lower_bound).max(0.0) / incumbent; - assert!(relative_gap <= 1e-4, "{report}"); - } - } -} - -/// The default is to enforce accuracy guarantees: Pass 1 prunes every -/// sketch candidate of an average of p99s, which has no guarantee -/// (`NoErrorBoundForQueryTarget`). With `--ignore-accuracy-guarantees`, the -/// selected sketch is accepted regardless of its guarantee and records why -/// the guarantee check would have rejected it. The document says which -/// mode it planned under, and a `--cost-inputs` file cannot carry the mode. +/// The default is to enforce accuracy guarantees: an average of p99s has +/// no guarantee from any sketch, so constraint (K) leaves only an exact +/// plan (or none). With `--ignore-accuracy-guarantees`, no query has a +/// target and a plan is selected on cost alone. The document says which +/// mode it planned under, and a `--cost-inputs` file cannot carry the +/// mode. #[test] fn accuracy_guarantees_are_enforced_by_default_and_ignored_on_request() { let workload = adhoc( @@ -1061,32 +856,20 @@ fn accuracy_guarantees_are_enforced_by_default_and_ignored_on_request() { json!({"max_error": 0.01, "failure_probability": 0.01, "accuracy_metric": "rank"}), &["x"], ); - let enforced = document(&workload, &["--run-until", "stage1_pass1"]); + let enforced = document(&workload, &["--run-until", "stage3"]); assert_eq!(enforced["accuracy_guarantees"], "enforced"); - let no_bound: Vec<&str> = enforced["stage1_logical_asap_optimization"]["pruned_candidates"] - .as_array() - .unwrap() - .iter() - .filter(|pruned| pruned["reason"] == "NoErrorBoundForQueryTarget") - .map(|pruned| pruned["candidate"].as_str().unwrap()) - .collect(); - assert!(!no_bound.is_empty(), "{enforced}"); + let selection = &enforced["stage3_plan_selection"]; + if selects_a_plan(selection) { + let accepted = &selection["accepted_accuracy_per_query"][0]; + assert_eq!(accepted["sketches"], json!([]), "{accepted}"); + } let ignored = document( &workload, &["--run-until", "stage3", "--ignore-accuracy-guarantees"], ); assert_eq!(ignored["accuracy_guarantees"], "ignored"); - assert!(pass1_pruned(&ignored).is_empty()); - let accepted = &ignored["stage3_plan_selection"]["accepted_accuracy_per_query"][0]; - assert_eq!( - accepted["accepted_regardless_of_guarantee"], true, - "{accepted}" - ); - assert_eq!( - accepted["guarantee_check_rejection"], - "no_accuracy_guarantee" - ); + assert!(selects_a_plan(&ignored["stage3_plan_selection"])); let settings = TempFile::new(r#"{"ignore_accuracy_guarantees": true}"#); let file = TempFile::json(&workload); @@ -1234,12 +1017,6 @@ fn check_workload_example(root: &Path, example: &Value) { let dag = selected_physical(&document, queries.len()); let accepted = selection["accepted_accuracy_per_query"].as_array().unwrap(); assert_eq!(accepted.len(), queries.len()); - if document["accuracy_guarantees"] == "enforced" { - for query in accepted { - assert_eq!(query["accepted_regardless_of_guarantee"], false); - assert!(query["guarantee_check_rejection"].is_null()); - } - } assert!( selection.get("materialized_views_error").is_none(), "{selection}" @@ -1248,16 +1025,11 @@ fn check_workload_example(root: &Path, example: &Value) { for view in selection["materialized_views"].as_array().unwrap() { assert!(nodes.contains(&view["node_id"].as_u64().unwrap())); } - assert!( - selection["search_report"].is_object(), - "{}: {selection}", - example["id"] - ); } /// With no supported summaries declared, `asap_planner` uses the executor's /// families, as `plan_installable` does. CountSketch cannot answer a total, so -/// Pass 1 neither generates it for count nor reports it deployment-pruned. +/// Pass 1 does not offer it for count. #[test] fn undeclared_supported_summaries_default_to_the_executors_families() { let document = document( @@ -1276,7 +1048,6 @@ fn undeclared_supported_summaries_default_to_the_executors_families() { ); } assert!(labels.iter().all(|label| !label.contains("CountSketch"))); - assert!(pass1_pruned(&document).is_empty()); assert_eq!( document["deployment_capabilities"]["supported_summaries"], serde_json::to_value(asap_executor::supported_summary_families()).unwrap() @@ -1285,8 +1056,10 @@ fn undeclared_supported_summaries_default_to_the_executors_families() { /// Two windowed quantiles over one metric (Q1 over 1 m, Q2 over 5 m, both /// every 10 s): with a shared scan, Stage 2 also offers the scan built once -/// per evaluation, which reads the 5 m window once (≈0.1364/s) instead of -/// once per query (0.2203/s); the selected plan costs at most 0.143/s. +/// per evaluation, which reads the 5 m window once instead of once per +/// query. The selected plan has that one scan, and it is charged for the +/// 5 m window's 10,000 series × 300 s / 15 s = 200,000 samples per +/// evaluation. #[test] fn a_scan_two_queries_share_is_offered_built_once_per_evaluation() { let workload = json!({ @@ -1307,18 +1080,38 @@ fn a_scan_two_queries_share_is_offered_built_once_per_evaluation() { &workload, &["--run-until", "stage3", "--ignore-accuracy-guarantees"], ); - let physical = document["stage2_physical_asap_optimization"]["candidates"] + let selection = &document["stage3_plan_selection"]; + // The plan's one scan, built once per evaluation. + let selected_scans: Vec<&Value> = selection["plan"]["nodes"] .as_array() - .unwrap(); - assert!(physical.iter().any(|p| p["id"] == SELECTED)); - let cost = document["stage3_plan_selection"]["cost"]["cost_per_second"] - .as_f64() - .unwrap(); - assert!(cost <= 0.143, "{cost}"); + .unwrap() + .iter() + .filter(|node| node["payload"]["NonASAP"]["Scan"].is_object()) + .collect(); + assert_eq!(selected_scans.len(), 1, "{selected_scans:?}"); + assert_eq!( + selected_scans[0]["lifetime"], + json!({"OneEvaluation": "Memory"}) + ); + let scan_charges: Vec<&Value> = selection["cost"]["per_node"] + .as_object() + .unwrap() + .values() + .filter(|charge| charge["operator"] == "Scan") + .collect(); + assert_eq!(scan_charges.len(), 1, "{scan_charges:?}"); + assert!( + scan_charges[0]["detail"] + .as_str() + .unwrap() + .starts_with("scan 200000 samples;"), + "{scan_charges:?}" + ); } -/// The search is the candidate-connection search; its report has no -/// legacy route. +/// The search is the candidate-connection search, whose selection MILP's +/// size Stage 3's section reports. Its times and solve count, which +/// change from run to run, are left out of the document. #[test] fn the_search_reports_candidate_connections() { let document = document( @@ -1326,44 +1119,109 @@ fn the_search_reports_candidate_connections() { &["--run-until", "stage3"], ); assert_eq!(document["search"], "candidate-connections"); - let report = &document["stage3_plan_selection"]["search_report"]; - assert!(report.is_object(), "{report}"); - assert!(report.get("routed_to_legacy").is_none()); - assert!(report.get("algorithm").is_none(), "{report}"); + let selection = &document["stage3_plan_selection"]; + for field in ["milp_variables", "milp_constraints"] { + assert!(selection[field].is_u64(), "{field}: {selection}"); + } + for field in ["milp_construction_ms", "milp_solve_ms", "milp_solve_count"] { + assert!(selection.get(field).is_none(), "{field}: {selection}"); + } } -/// The document names its selected plan's logical candidate under the id -/// its physical candidates read (`from_logical`), as the viewer's -/// validator requires. +/// Memo ids are canonical: depth first from the query roots, so adding a +/// query over another metric after `quantile_over_time(0.9, a[5m])` keeps +/// every id of the first query's groups and candidates, at every step +/// (the planner's own ids would interleave the two queries' groups). #[test] -fn the_document_names_the_selected_logical_candidate() { - let document = document( +fn memo_ids_of_a_query_do_not_move_when_another_query_is_added() { + let first = "quantile_over_time(0.9, a[5m])"; + let target = json!({"max_error": 0.01, "failure_probability": 0.001}); + let args = ["--run-until", "stage3"]; + let alone = document(&adhoc(&[first], target.clone(), &["a"]), &args); + let with_another = document( &adhoc( - &[ - "quantile_over_time(0.5, data[5m])", - "quantile_over_time(0.9, data[5m])", - ], - exact(), - &["data"], + &[first, "quantile_over_time(0.5, b[1m])"], + target, + &["a", "b"], ), - &["--run-until", "stage3", "--ignore-accuracy-guarantees"], + &args, ); - let logical: HashSet<&str> = document["stage1_logical_asap_optimization"]["candidates"] - .as_array() - .unwrap() - .iter() - .map(|c| c["id"].as_str().unwrap()) - .collect(); - let physical = document["stage2_physical_asap_optimization"]["candidates"] - .as_array() - .unwrap(); - assert!(!physical.is_empty()); - for candidate in physical { - let from = candidate["from_logical"].as_str().unwrap(); - assert!(logical.contains(from), "{from} is not in {logical:?}"); + // A group and its candidates by what they are, without the queries + // that reach them. + let shape = |group: &Value| { + let mut group = group.clone(); + let object = group.as_object_mut().unwrap(); + object.remove("queries"); + object.remove("roots_of"); + group + }; + for step in ["stage0", "pass1", "pass2"] { + let groups_alone = memo_step(&alone, step)["groups"].as_array().unwrap(); + let groups_with_another = memo_step(&with_another, step)["groups"].as_array().unwrap(); + assert!(groups_with_another.len() > groups_alone.len(), "{step}"); + for group in groups_alone { + let same = groups_with_another + .iter() + .find(|other| other["id"] == group["id"]) + .unwrap_or_else(|| panic!("{step}: no group {}", group["id"])); + assert_eq!(shape(same), shape(group), "{step}"); + } } - assert!(selects_a_plan(&document["stage3_plan_selection"])); - assert!(physical.iter().any(|c| c["id"] == SELECTED)); + // The first query's root is group 0. + assert_eq!( + memo_step(&with_another, "stage0")["groups"][0]["roots_of"], + json!([0]) + ); +} + +/// Stage 3 reports which plan nodes several queries read, by what they +/// compute: the two windowed quantiles over one metric share their scan. +/// One query shares nothing. Every number of the planner's output keeps +/// at most 12 significant digits. +#[test] +fn the_sharing_summary_names_the_nodes_several_queries_read() { + let workload = json!({ + "query_workload": { + "language": "promql", + "query_batch": null, + "repeating_queries": [ + every_10s_within_1_percent("quantile_over_time(0.5, data[1m])"), + every_10s_within_1_percent("quantile_over_time(0.9, data[5m])"), + ], + }, + "data_workload": { + "arrival": {"ingesting": {"rows_per_second": 666.67}}, + "sources": {"data": {"series_count": 10000, "metric_type": "gauge", "sample_interval": 15000}}, + }, + }); + let args = ["--run-until", "stage3", "--ignore-accuracy-guarantees"]; + let selection = &document(&workload, &args)["stage3_plan_selection"]; + let shared = selection["sharing_summary"].as_array().unwrap(); + assert!( + shared + .iter() + .any(|node| node["operator"] == "Scan" && node["queries"] == json!(["q1", "q2"])), + "{shared:?}" + ); + assert!(shared.iter().all(|node| node.get("id").is_none())); + let alone = document( + &adhoc(&["quantile_over_time(0.9, m[5m])"], exact(), &["m"]), + &["--run-until", "stage3"], + ); + assert_eq!(alone["stage3_plan_selection"]["sharing_summary"], json!([])); + + fn assert_rounded(value: &Value) { + match value { + Value::Number(n) if n.is_f64() => { + let x = n.as_f64().unwrap(); + assert_eq!(format!("{x:.11e}").parse::().unwrap(), x); + } + Value::Array(items) => items.iter().for_each(assert_rounded), + Value::Object(map) => map.values().for_each(assert_rounded), + _ => {} + } + } + assert_rounded(selection); } /// The selected plan's `per_node` entries carry the node's operator, @@ -1476,17 +1334,20 @@ fn a_counter_function_over_a_declared_gauge_plans_with_a_warning() { assert!(counter["stage3_plan_selection"].get("warnings").is_none()); } -/// A relative-error quantile over `m` (1,000 series, non-negative values) -/// whose `value` declares `column_magnitudes` when `magnitudes` is set. +/// A relative-error quantile over `m` (1,000 series, non-negative values, +/// ingesting) every 10 s, whose `value` declares `column_magnitudes` when +/// `magnitudes` is set. Repeating, so a deployment without raw data can +/// still answer it from a state built at ingestion. fn relative_quantile_over(magnitudes: Option) -> Value { let mut workload = json!({ - "query_workload": {"language": "promql", "query_batch": [{ + "query_workload": {"language": "promql", "query_batch": null, "repeating_queries": [{ "query": "quantile_over_time(0.99, m[5m])", + "demand": {"fixed_interval_at": {"interval": 10000, "evaluation_phase": 0}}, "requirements": {"accuracy": {"max_error": 0.01, "accuracy_metric": "relative"}}, - "predictability": "unknown", "invocations": 1 + "predictability": {"predictable": {"known_at": null}} }]}, - "data_workload": {"arrival": "unknown", "sources": {"m": { - "sample_interval": 1000, "series_count": 1000, "rows_held": 600000, + "data_workload": {"arrival": {"ingesting": {"rows_per_second": 1000.0}}, "sources": {"m": { + "sample_interval": 1000, "series_count": 1000, "column_signs": {"value": "non_negative"} }}} }); @@ -1739,7 +1600,7 @@ fn several_workload_files_in_different_languages_are_planned_together() { /// source they both declare: a run is refused rather than picking one. /// While data arrives, each source must also declare its own rate: plan /// selection shares a workload's `rows_per_second` out over all its -/// sources, so `flows` (10 groups, no sample interval) would be priced at +/// sources, so `flows` (10 groups, no sample interval) would be costed at /// 1500 × 10 / 15010 ≈ 1 row/s instead of its file's 500. #[test] fn workload_files_that_disagree_on_the_data_are_refused() { @@ -1820,13 +1681,13 @@ fn a_classic_histogram_quantile_gets_a_selected_plan() { const MEMO_RATIO_OF_QUANTILES: &str = "../../tools/dag-viewer/memo-examples/promql/ratio-of-quantiles-5m.json"; -/// The `memo` section: the memo after each step and its node table, on the -/// issue's example with guarantees ignored (so Pass 1 keeps every sketch), -/// and with guarantees enforced the selected choice. +/// The `memo` section: the memo after each step, each candidate with its +/// operator over its input groups, θ and one required accuracy per input, +/// on the issue's example with guarantees ignored, and the selected +/// candidates. #[test] -fn memo_section_lists_each_step_and_the_selected_choice() { +fn memo_section_lists_each_step_and_the_selected_candidates() { let path = PathBuf::from(env!("CARGO_MANIFEST_DIR")).join(MEMO_RATIO_OF_QUANTILES); - // Up to Stage 2: Stage 3 over every sketch candidate takes minutes. let document = document_of(&[ "--workload", path.to_str().unwrap(), @@ -1848,53 +1709,248 @@ fn memo_section_lists_each_step_and_the_selected_choice() { }; // The PromQL frontend gives each series its identity as a column, so the // two `quantile_over_time(0.5, data[5m])` have a unique key and share one - // group: 7 groups (scan, range, two quantile groups, ratio, sum, avg), - // 5 + 5 + 5 Pass 1 candidates (each quantile group: exact, DDSketch and - // a KLL sized for the whole result for the whole budget, DDSketch and a - // KLL for half of it). + // group: 7 groups at Stage 0 (scan, range, two quantile groups, ratio, + // sum, avg). Pass 1 adds the summary-state and readout groups (19) and + // one candidate per θ of the lists. assert_eq!(steps[0]["groups"].as_array().unwrap().len(), 7); assert_eq!(candidates(&steps[0]), 0); - assert_eq!(steps[1]["groups"].as_array().unwrap().len(), 7); - assert_eq!(candidates(&steps[1]), 15); - // The node table is a valid FlatDag (children before parents) holding - // each node once; every group's and candidate's `node` names a node of - // it, and each boundary names a node and one of the candidate's child - // groups. - let nodes = memo["nodes"].as_array().unwrap(); - validated_dag(&json!({ "nodes": nodes, "roots": [] }), 0); - let distinct: HashSet = nodes.iter().map(Value::to_string).collect(); - assert_eq!(distinct.len(), nodes.len()); - let in_table = |id: &Value| (id.as_u64().unwrap() as usize) < nodes.len(); + assert_eq!(steps[1]["groups"].as_array().unwrap().len(), 19); + assert_eq!(candidates(&steps[1]), 36); + // Every candidate is one operator over its input groups, with one + // requirement per input, and every group it reads is in the step. for step in steps { - for group in step["groups"].as_array().unwrap() { - assert!(in_table(&group["node"]) && group.get("dag").is_none()); + let groups = step["groups"].as_array().unwrap(); + for group in groups { + assert!(group.get("node").is_none() && group["operator"].is_object()); for candidate in group["candidates"].as_array().unwrap() { - assert!(in_table(&candidate["node"]) && candidate.get("dag").is_none()); let children = candidate["children"].as_array().unwrap(); - for (node, child) in candidate["group_of_node"].as_object().unwrap() { - assert!(node.parse::().unwrap() < nodes.len()); - assert!(children.contains(child)); - } + assert_eq!( + candidate["required_accuracy"].as_array().unwrap().len(), + children.len() + ); + assert!(children + .iter() + .all(|child| groups.iter().any(|group| group["id"] == *child))); + assert!(candidate.get("parameters").is_some()); } } } - assert!(memo.get("selected_choice").is_none()); - // Every candidate of the selected choice exists in the last step. - let enforced = document_of(&[ + assert!(memo.get("selected_candidates").is_none()); + // Every selected candidate exists in the last step. + let selected = document_of(&[ "--workload", path.to_str().unwrap(), "--run-until", "stage3", + "--ignore-accuracy-guarantees", ]); - let memo = &enforced["memo"]; + let memo = &selected["memo"]; let last = memo["steps"].as_array().unwrap().last().unwrap()["groups"] .as_array() .unwrap(); - let choice = memo["selected_choice"].as_object().unwrap(); - assert!(!choice.is_empty()); - for (group, index) in choice { + let selected = memo["selected_candidates"].as_object().unwrap(); + assert!(!selected.is_empty()); + for (group, indices) in selected { let group = &last[group.parse::().unwrap()]; - assert!(index.as_u64().unwrap() < group["candidate_count"].as_u64().unwrap()); + for index in indices.as_array().unwrap() { + assert!(index.as_u64().unwrap() < group["candidate_count"].as_u64().unwrap()); + } + } +} + +/// Edges are listed by layer, on a p90 over 5 m within rank 0.01 every +/// minute at phase 0 over arriving data: Pass 2's memo step lists, per +/// input of each candidate, every producer of the input group with +/// $\alpha^{\log}$ and the reason when it is 0 (KLL sizes too small for +/// the readout's required accuracy), and Stage 1's `pruned_candidates` the +/// candidates Pass 2 eliminated with their causes; Stage 2's +/// `physical_candidates` list, per input, the physical edges whose logical +/// edge holds, with $\alpha^{\mathrm{phys}}$ and the reason when it is 0 +/// (a query-time consumer of an ingestion-time result that is not +/// materialized), the plan's edges marked; and Stage 2's pruned candidates +/// carry their properties and reason. +#[test] +fn edges_are_listed_by_layer_with_their_alpha_and_rejection() { + let workload = json!({ + "query_workload": {"language": "promql", "query_batch": null, "repeating_queries": [{ + "query": "quantile_over_time(0.9, data[5m])", + "demand": {"fixed_interval_at": {"interval": 60000, "evaluation_phase": 0}}, + "requirements": {"accuracy": {"max_error": 0.01, "failure_probability": 0.01, "accuracy_metric": "rank"}}, + "predictability": {"predictable": {"known_at": null}}, + }]}, + "data_workload": { + "arrival": {"ingesting": {"rows_per_second": 1000.0}}, + "sources": {"data": {"series_count": 1000, "sample_interval": 1000, "metric_type": "gauge"}}, + }, + }); + let document = document(&workload, &["--run-until", "stage3"]); + let pruned = document["stage1_logical_asap_optimization"]["pruned_candidates"] + .as_array() + .unwrap(); + assert!(!pruned.is_empty()); + for candidate in pruned { + for cause in candidate["causes"].as_array().unwrap() { + assert!( + cause["root"].is_object() || cause["root"].is_string(), + "{cause}" + ); + assert!(cause["text"].is_string(), "{cause}"); + } + } + let pass2 = memo_step(&document, "pass2")["groups"].as_array().unwrap(); + let (mut logical_holds, mut logical_fails) = (0, 0); + for group in pass2 { + for candidate in group["candidates"].as_array().unwrap() { + for input in candidate["inputs"].as_array().unwrap() { + let producers = pass2[input["group"].as_u64().unwrap() as usize]["candidate_count"] + .as_u64() + .unwrap(); + for edge in input["edges"].as_array().unwrap() { + assert!(edge["producer"].as_u64().unwrap() < producers, "{edge}"); + let holds = edge["satisfies_logical_requirement"] == true; + assert_eq!(edge["rejection"].is_null(), holds, "{edge}"); + if holds { + logical_holds += 1; + } else { + logical_fails += 1; + // α^log fails, or the consumer reads only producers + // of its own θ. + let rejection = edge["rejection"].as_str().unwrap(); + assert!( + rejection.starts_with("α^log = 0: ") + || rejection == "no edge: the producer's θ differs", + "{rejection}" + ); + } + } + } + } + } + assert!(logical_holds > 0 && logical_fails > 0); + let stage2 = &document["stage2_physical_asap_optimization"]; + let candidates = stage2["physical_candidates"].as_array().unwrap(); + assert!(!candidates.is_empty()); + let (mut physical_holds, mut physical_fails, mut selected) = (0, 0, 0); + for candidate in candidates { + assert!(candidate["properties"]["timing"].is_string()); + for input in candidate["inputs"].as_array().unwrap() { + for edge in input["edges"].as_array().unwrap() { + assert!((edge["producer"].as_u64().unwrap() as usize) < candidates.len()); + let holds = edge["satisfies_physical_requirement"] == true; + assert_eq!(edge["rejection"].is_null(), holds, "{edge}"); + if holds { + physical_holds += 1; + } else { + physical_fails += 1; + assert!(edge["rejection"] + .as_str() + .unwrap() + .starts_with("α^phys = 0: ")); + } + if edge["selected"] == true { + assert!(holds, "a selected edge is admissible: {edge}"); + selected += 1; + } + } + } + } + assert!(physical_holds > 0 && physical_fails > 0); + let plan_edges = document["stage3_plan_selection"]["plan"]["edges"] + .as_array() + .unwrap() + .len(); + assert!(selected > 0 && selected <= plan_edges); + for pruned in stage2["pruned_candidates"].as_array().unwrap() { + assert!( + pruned["properties"]["lifetime"].is_string() + || pruned["properties"]["lifetime"].is_object() + ); + assert!(pruned["reason"].is_string() || pruned["reason"].is_object()); + } +} + +/// Each failing logical edge shows the reason Pass 2 recorded for it, +/// eliminated endpoints included. A q0.9 with δ over its whole result and +/// no declared `series_count` (so $n_q$ is unknown) fails every edge into +/// its readouts, the KLL one (G5[0] <- G4) and the DDSketch one (G7[0] <- +/// G6), with `ValuesPerEvaluationUnknown`: Pass 2 checks that rule before +/// the metric and accuracy rules, so the DDSketch edges do not show "no +/// guarantee" and the KLL ones do not show `✓`. +#[test] +fn failing_logical_edges_show_the_reason_pass_2_recorded() { + let workload = json!({ + "query_workload": {"language": "promql", "repeating_queries": [ + {"query": "quantile_over_time(0.9, latency[5m])", + "demand": {"fixed_interval_at": {"interval": 60000, "evaluation_phase": 0}}, + "requirements": {"accuracy": {"max_error": 0.01, "failure_probability": 0.01, "accuracy_scope": "whole_result"}}, + "predictability": {"predictable": {"known_at": null}}}, + {"query": "quantile_over_time(0.5, latency[5m])", + "demand": {"fixed_interval_at": {"interval": 60000, "evaluation_phase": 0}}, + "requirements": {"accuracy": {"max_error": 0.01}}, + "predictability": {"predictable": {"known_at": null}}}, + ]}, + "data_workload": { + "arrival": {"ingesting": {"rows_per_second": 2000.0}}, + "sources": {"latency": {"metric_type": "gauge", "sample_interval": 1000}}, + }, + }); + const VALUES_PER_EVALUATION_UNKNOWN: &str = "α^log = 0: query 0: the query's failure \ + probability covers its whole result, and the number of values one evaluation returns is \ + unknown"; + let args = ["--run-until", "stage1_pass2", "--printing-full-candidates"]; + let document = document(&workload, &args); + let pass2 = memo_step(&document, "pass2")["groups"].as_array().unwrap(); + let readouts: Vec<&Value> = pass2 + .iter() + .filter(|group| group["label"] == "SummaryEstimate Quantile(q=0.9)") + .collect(); + assert_eq!(readouts.len(), 2, "the KLL and the DDSketch readout"); + for group in &readouts { + let candidate = &group["candidates"][0]; + assert_eq!(candidate["pruned"], true, "{candidate}"); + let edges = candidate["inputs"][0]["edges"].as_array().unwrap(); + assert!(!edges.is_empty()); + for edge in edges { + assert_eq!(edge["satisfies_logical_requirement"], false, "{edge}"); + assert_eq!(edge["rejection"], VALUES_PER_EVALUATION_UNKNOWN, "{edge}"); + } + } + + let file = TempFile::json(&workload); + let mut text_args = vec!["--workload", file.path(), "--format", "text"]; + text_args.extend(args); + let output = planner(&text_args); + assert!(output.status.success()); + let text = String::from_utf8(output.stdout).unwrap(); + let pass2 = text + .split("==================== Stage 1 Pass 2 output") + .nth(1) + .expect("a Pass 2 section"); + // The text keeps the planner's group ids, so its readouts are found by + // label. + let blocks: Vec<&str> = pass2 + .split("\nG") + .filter(|block| { + block + .lines() + .next() + .is_some_and(|header| header.contains("SummaryEstimate Quantile(q=0.9)")) + }) + .collect(); + assert_eq!(blocks.len(), 2, "{pass2}"); + for block in blocks { + let edges: Vec<&str> = block + .lines() + .map(str::trim_start) + .filter(|line| line.starts_with('G') && line.contains('[')) + .collect(); + assert!(!edges.is_empty(), "{block}"); + for edge in edges { + assert!( + edge.ends_with(&format!("✗ {VALUES_PER_EVALUATION_UNKNOWN}")), + "{edge}" + ); + } } } @@ -1905,7 +1961,10 @@ fn text_output_shows_the_selected_plans_cost() { let output = planner(&["--workload", &example("example1"), "--run-until", "stage3"]); assert!(output.status.success()); let text = String::from_utf8(output.stdout).unwrap(); - assert!(text.contains("Stage 3 output: Selection"), "{text}"); + assert!( + text.contains("Stage 3 output: PhysicalASAPDAG and Selection"), + "{text}" + ); assert!(text.contains("\ncost_per_second: "), "{text}"); } @@ -1920,7 +1979,7 @@ fn memo_example(path: &str) -> String { } /// The arguments for raw data retained, query-time results not retained, -/// AWS Fargate prices and enforced guarantees, with `capabilities` the +/// AWS Fargate costs and enforced guarantees, with `capabilities` the /// temporary deployment capabilities file they name. fn milp_args(capabilities: &TempFile) -> Vec { let aws = PathBuf::from(env!("CARGO_MANIFEST_DIR")) @@ -1940,11 +1999,9 @@ fn raw_data_retained() -> TempFile { } /// #1095 C5: example 1 and the RangeHeavy T5 q0.5 + q0.9 workload select a -/// plan whose `plan_cost` equals the MILP's objective (no -/// `inexact_objective`), solved to optimality, and which passes every -/// query's accuracy check. +/// plan through the selection MILP, with an accepted accuracy per query. #[test] -fn milp_search_objective_equals_plan_cost_on_example1_and_rangeheavy() { +fn milp_search_selects_a_plan_on_example1_and_rangeheavy() { let capabilities = raw_data_retained(); let rangeheavy = memo_example("promql/rangeheavy-t5-q05-q09-1h.json"); let example1 = example("example1"); @@ -1960,19 +2017,10 @@ fn milp_search_objective_equals_plan_cost_on_example1_and_rangeheavy() { let document = document_of(&args); let selection = &document["stage3_plan_selection"]; assert!(selects_a_plan(selection), "{args:?}: {selection}"); - let report = &selection["search_report"]; - assert_eq!(report["status"], "optimal", "{args:?}: {report}"); - assert!( - report.get("inexact_objective").is_none(), - "{args:?}: {report}" - ); - assert!(report["milp_variables"].as_u64().unwrap() > 0); + assert!(selection["milp_variables"].as_u64().unwrap() > 0); let queries = document["workload"]["queries"].as_array().unwrap().len(); let accepted = selection["accepted_accuracy_per_query"].as_array().unwrap(); assert_eq!(accepted.len(), queries); - for query in accepted { - assert_eq!(query["accepted_regardless_of_guarantee"], false, "{query}"); - } } } @@ -2005,7 +2053,6 @@ fn milp_search_selects_checked_plans_on_ratio_and_rollupheavy() { let accepted = selection["accepted_accuracy_per_query"].as_array().unwrap(); assert_eq!(accepted.len(), queries); for query in accepted { - assert_eq!(query["accepted_regardless_of_guarantee"], false, "{query}"); assert!( !query["sketches"] .as_array() @@ -2015,12 +2062,6 @@ fn milp_search_selects_checked_plans_on_ratio_and_rollupheavy() { "{workload}: {query}" ); } - let pruned = pass1_pruned(&document); - assert!( - pruned.iter().any(|(family, reason)| family == pruned_family - && reason == "NoErrorBoundForQueryTarget"), - "{workload}: {pruned:?}" - ); } } @@ -2039,12 +2080,11 @@ fn milp_search_selects_a_checked_ddsketch_plan_for_a_ratio_and_its_operands() { let selection = &document["stage3_plan_selection"]; assert!(selects_a_plan(selection), "{selection}"); for query in selection["accepted_accuracy_per_query"].as_array().unwrap() { - assert_eq!(query["accepted_regardless_of_guarantee"], false, "{query}"); assert_eq!(query["sketches"], json!(["DDSketch"]), "{query}"); } } -/// The viewer's cost-input files carry the prices of the presets they are +/// The viewer's cost-input files carry the costs of the presets they are /// named after, so a change to `crates/cost`'s constants fails here instead /// of leaving the viewer and the snapshots on the old numbers. #[test] @@ -2072,3 +2112,183 @@ fn cost_input_files_match_their_presets() { assert_eq!(loaded, expected, "{file}"); } } + +/// The text output's memo after Pass 2 is complete: every group has its +/// operator and a properties line (each column's lineage and sign, whether +/// the sources it scans, its evaluation grid); a sketch build shows +/// its θ and a readout its required accuracy; Pass 2's and Stage 2's +/// pruned candidates are listed with their causes; every `G` named +/// resolves to a printed group. Workload: two windowed quantiles every +/// 10 s within 1 % (the shared-scan workload), printed with `--printing-full-candidates`. +#[test] +fn the_text_memo_after_pass_2_prints_every_group_with_its_properties() { + let workload = two_windowed_quantiles_within_1_percent(); + let file = TempFile::json(&workload); + let output = planner(&[ + "--workload", + file.path(), + "--run-until", + "stage3", + "--format", + "text", + "--printing-full-candidates", + ]); + assert!( + output.status.success(), + "{}", + String::from_utf8_lossy(&output.stderr) + ); + let text = String::from_utf8(output.stdout).unwrap(); + let pass2 = text + .split("==================== Stage 1 Pass 2 output") + .nth(1) + .and_then(|rest| rest.split("==================== Stage 2 output").next()) + .expect("a Pass 2 section"); + let lines: Vec<&str> = pass2.lines().collect(); + let mut groups = Vec::new(); + for (index, line) in lines.iter().enumerate() { + let Some(id) = line + .strip_prefix('G') + .and_then(|rest| rest.split(':').next()) + else { + continue; + }; + let Ok(id) = id.parse::() else { + continue; + }; + groups.push(id); + let properties = lines[index + 1]; + assert!( + properties.contains('=') && properties.contains(':'), + "G{id}'s columns with lineage and sign: {properties}" + ); + assert!(properties.contains(" · scans "), "{properties}"); + assert!( + properties.contains("evaluated every") + || properties.contains("no known evaluation grid"), + "{properties}" + ); + } + assert_eq!( + groups, + (0..groups.len()).collect::>(), + "groups in order" + ); + for line in &lines { + let candidate = line.trim_start(); + if candidate.contains("SummaryAgg") && candidate.starts_with('[') { + assert!(candidate.contains("θ="), "a build shows its θ: {candidate}"); + } + if candidate.contains("SummaryEstimate") + && candidate.starts_with('[') + && !candidate.contains("pruned") + { + assert!( + candidate.contains("requires="), + "a readout shows its requirement: {candidate}" + ); + } + for word in line.split(|c: char| !c.is_ascii_alphanumeric()) { + if let Some(id) = word + .strip_prefix('G') + .and_then(|id| id.parse::().ok()) + { + assert!(id < groups.len(), "G{id} resolves: {line}"); + } + } + } + assert!( + pass2.contains("pruned by Pass 2"), + "Pass 2's pruned candidates" + ); + assert!( + text.contains("pruned by Stage 2"), + "Stage 2's pruned candidates" + ); + assert!(text.contains("==================== Stage 3 output")); +} + +/// Two windowed quantiles every 10 s within 1 % over 10,000 series every +/// 15 s (the shared-scan workload). +fn two_windowed_quantiles_within_1_percent() -> serde_json::Value { + json!({ + "query_workload": { + "language": "promql", + "query_batch": null, + "repeating_queries": [ + every_10s_within_1_percent("quantile_over_time(0.5, data[1m])"), + every_10s_within_1_percent("quantile_over_time(0.9, data[5m])"), + ], + }, + "data_workload": { + "arrival": {"ingesting": {"rows_per_second": 666.67}}, + "sources": {"data": {"series_count": 10000, "metric_type": "gauge", "sample_interval": 15000}}, + }, + }) +} + +/// Without `--printing-full-candidates` the text folds the candidates of a group that differ +/// only in θ: a line with their indices (`[0..39]`), one with each θ +/// parameter's values (`θ: k ∈ {...}`) and one with how many Pass 2 pruned +/// by cause and kept, whose counts add up; the candidates Pass 2 kept are +/// still printed with their θ. The folded text is shorter than `--printing-full-candidates`. +#[test] +fn the_text_folds_candidates_that_differ_only_in_theta() { + let file = TempFile::json(&two_windowed_quantiles_within_1_percent()); + let text = |extra: &[&str]| { + let mut args = vec![ + "--workload", + file.path(), + "--run-until", + "stage3", + "--format", + "text", + ]; + args.extend(extra); + let output = planner(&args); + assert!( + output.status.success(), + "{}", + String::from_utf8_lossy(&output.stderr) + ); + String::from_utf8(output.stdout).unwrap() + }; + let (folded, printed_in_full) = (text(&[]), text(&["--printing-full-candidates"])); + assert!(folded.lines().count() < printed_in_full.lines().count()); + let pass2 = folded + .split("==================== Stage 1 Pass 2 output") + .nth(1) + .and_then(|rest| rest.split("==================== Stage 2 output").next()) + .expect("a Pass 2 section"); + let lines: Vec<&str> = pass2.lines().map(str::trim_start).collect(); + let mut folds = 0; + for (index, line) in lines.iter().enumerate() { + let Some(counts) = line.split_once(" candidates: ") else { + continue; + }; + folds += 1; + assert!(lines[index - 1].starts_with("θ: "), "{}", lines[index - 1]); + assert!(lines[index - 2].starts_with('['), "{}", lines[index - 2]); + let total: usize = counts.0.parse().unwrap(); + let pruned: usize = counts.1.split(' ').next().unwrap().parse().unwrap(); + let kept: usize = counts + .1 + .rsplit(", ") + .next() + .and_then(|kept| kept.strip_suffix(" kept")) + .unwrap() + .parse() + .unwrap(); + assert!(pruned > 0, "{line}"); + assert_eq!(pruned + kept, total, "{line}"); + } + assert!(folds > 0, "a folded group:\n{pass2}"); + for (index, line) in lines.iter().enumerate() { + let fold_line = lines + .get(index + 1) + .is_some_and(|next| next.starts_with("θ: ")); + if line.starts_with('[') && line.contains("SummaryAgg") && !fold_line { + assert!(line.contains("θ="), "a kept build shows its θ: {line}"); + } + } +} diff --git a/crates/devtools/tests/asap_planner_acceptance.rs b/crates/devtools/tests/asap_planner_acceptance.rs index 503753464..e3aecf2af 100644 --- a/crates/devtools/tests/asap_planner_acceptance.rs +++ b/crates/devtools/tests/asap_planner_acceptance.rs @@ -85,7 +85,7 @@ const SECTIONS: [(&str, &[&str]); 5] = [ "stage2", &["Stage 2 output = Stage 3 input: PhysicalASAPMemoCandidates"], ), - ("stage3", &["Stage 3 output: Selection"]), + ("stage3", &["Stage 3 output: PhysicalASAPDAG and Selection"]), ]; /// `--run-until ` prints exactly the sections of every stage up to @@ -113,13 +113,14 @@ fn run_until_prints_exactly_the_stages_up_to_it() { } } -/// Pass 2 composes tumbling-window candidates over every legal width, not only -/// the seed grid: a 5 m quantile within 1% every 60 s gets a mergeable -/// summary's tumbling-window chain for every width that divides 60 s (in ms), from -/// 1 ms to 60 s. +/// Pass 2 adds chain groups at the greatest common divisor of the ranges +/// and evaluation steps: a 5 m quantile within 1% every 60 s at phase 0 +/// gets its mergeable summary's chain of 60 s tumbling windows only. #[test] -fn pass2_holds_a_tumbling_window_candidate_for_every_width_dividing_the_grid() { - let quantile = workload("quantile_over_time(0.5, data[5m])", 60_000, 0.01, "data"); +fn pass2_holds_tumbling_windows_at_the_gcd_of_ranges_and_steps() { + let mut quantile = workload("quantile_over_time(0.5, data[5m])", 60_000, 0.01, "data"); + quantile["query_workload"]["repeating_queries"][0]["demand"] = + json!({"fixed_interval_at": {"interval": 60_000, "evaluation_phase": 0}}); let text = stdout(planner(&quantile, &["--run-until", "stage1_pass2"])); let widths: BTreeSet = text .split("tumbling_window_width_ms=") @@ -129,25 +130,26 @@ fn pass2_holds_a_tumbling_window_candidate_for_every_width_dividing_the_grid() { digits.parse().unwrap() }) .collect(); - let divisors: BTreeSet = (1..=60_000).filter(|w| 60_000 % w == 0).collect(); - assert_eq!(widths, divisors); + assert_eq!(widths, BTreeSet::from([60_000])); } /// Stage 3 selects among the candidates Stage 2 lists: every candidate a -/// selected node runs (`nodeN: C`) is a line of Stage 2's output. +/// plan node runs (`n: C = ...`) is a line of Stage 2's output. #[test] fn stage3_selects_candidates_stage2_lists() { let text = stdout(planner(&rate_workload(), &["--run-until", "stage3"])); - let (stage2, stage3) = text.split_once("Stage 3 output: Selection").unwrap(); + let (stage2, stage3) = text + .split_once("Stage 3 output: PhysicalASAPDAG and Selection") + .unwrap(); let selected: Vec<&str> = stage3 .lines() - .filter(|line| line.starts_with("node") && line.contains(": C")) + .filter(|line| line.starts_with('n') && line.contains(": C")) .filter_map(|line| line.split_whitespace().nth(1)) .collect(); assert!(!selected.is_empty(), "{stage3}"); for candidate in selected { assert!( - stage2.contains(&format!("\n {candidate}:")), + stage2.contains(&format!("\n {candidate} = ")), "{candidate} is not in Stage 2's output" ); } @@ -181,15 +183,7 @@ fn json_sections_of_stages_not_run_are_null() { let stage3 = document("stage3"); assert!(stage3["stage2_physical_asap_optimization"].is_object()); - assert!(stage3["stage3_plan_selection"]["selected_dag"].is_object()); - assert_eq!( - stage3["stage1_logical_asap_optimization"]["candidates"][0]["id"], - "L1" - ); - assert_eq!( - stage3["stage2_physical_asap_optimization"]["candidates"][0]["id"], - "P1" - ); + assert!(stage3["stage3_plan_selection"]["plan"].is_object()); } /// A missing `--run-until`, the old `pass1`/`pass2` stage names, and each diff --git a/crates/devtools/tests/corpus_snapshots.rs b/crates/devtools/tests/corpus_snapshots.rs index 40bf88751..8f163b991 100644 --- a/crates/devtools/tests/corpus_snapshots.rs +++ b/crates/devtools/tests/corpus_snapshots.rs @@ -13,13 +13,12 @@ //! Each query is planned by `asap_planner --run-until stage3`: one exact //! search over every legal tumbling-window width, with no time limit. //! -//! A snapshot holds only the plan, never the search that found it: the -//! selected plan's DAG (Stage 2 candidate `P1`) and display name, its cost -//! and accepted accuracy per query, the materialized views, tumbling-window retention, -//! and whether the search proved the plan optimal; for a query that does not -//! plan, the error. Candidate ids, timings, the Stage 1 and Stage 2 -//! candidate lists and the memo are left out. Floats are rounded to 12 -//! significant digits. A query with a `skip_reason` is skipped. +//! A snapshot holds only the plan, never the search that found it: Stage +//! 3's plan (its `PhysicalASAPDAG`, retention on its nodes), its cost and +//! accepted accuracy per query, and the materialized views; for a query +//! that does not plan, the error. The selection MILP's size and timings, +//! the Stage 1 and Stage 2 candidate lists and the memo are left out. Floats are as +//! `asap_planner` prints them, rounded to 12 significant digits. A query with a `skip_reason` is skipped. //! //! The same binary holds the sketch plan snapshots (`sketch_plan_snapshots.rs`), //! which reuse these helpers on multi-query workloads with accuracy targets. @@ -64,8 +63,9 @@ fn plan(workload: &Value, args: &[String], scratch: &Path) -> Value { } /// Runs `asap_planner --run-until stage3 --format json` with `args`, which -/// name its input (paths relative to the repository root); panics (a -/// planner crash) on failure. +/// name its input (paths relative to the repository root). A run whose plan +/// selection failed still writes its document, with Stage 3's `error`, +/// which is returned; any other failure (a planner crash) panics. fn asap_planner(args: &[String], scratch: &Path) -> Value { let output = scratch.join("document.json"); let mut command = Command::new(env!("CARGO_BIN_EXE_asap_planner")); @@ -76,36 +76,20 @@ fn asap_planner(args: &[String], scratch: &Path) -> Value { .arg("--out") .arg(&output); let result = command.output().unwrap(); - assert!( - result.status.success(), - "{command:?} failed: {}", - String::from_utf8_lossy(&result.stderr) - ); - serde_json::from_str(&std::fs::read_to_string(output).unwrap()).unwrap() -} - -/// The selected plan's search status (`optimal`), or null without a plan. -fn search_status(document: &Value) -> &Value { - &document["stage3_plan_selection"]["search_report"]["status"] -} - -/// Whether the plan is proven optimal: its search says so. -fn proven_optimal(document: &Value) -> bool { - search_status(document) == "optimal" -} - -/// `value` with every non-integer number rounded to 12 significant digits. -fn rounded(value: &Value) -> Value { - match value { - Value::Number(n) if n.is_f64() => { - let x = n.as_f64().unwrap(); - json!(format!("{x:.11e}").parse::().unwrap()) + let document: Option = std::fs::read_to_string(&output) + .ok() + .and_then(|text| serde_json::from_str(&text).ok()); + match document { + Some(document) + if result.status.success() + || document["stage3_plan_selection"]["error"].is_string() => + { + document } - Value::Object(map) => { - Value::Object(map.iter().map(|(k, v)| (k.clone(), rounded(v))).collect()) - } - Value::Array(items) => Value::Array(items.iter().map(rounded).collect()), - other => other.clone(), + _ => panic!( + "{command:?} failed: {}", + String::from_utf8_lossy(&result.stderr) + ), } } @@ -117,10 +101,10 @@ fn snapshot_of(document: &Value) -> Value { "query".into(), document["workload"]["queries"][0]["text"].clone(), ); - rounded(&Value::Object(snapshot)) + Value::Object(snapshot) } -/// The selected plan and its cost, or why there is none, unrounded: the +/// The selected plan and its cost, or why there is none: the /// part of a snapshot the sketch plan snapshots share. fn plan_snapshot(document: &Value) -> Map { let stage3 = &document["stage3_plan_selection"]; @@ -129,28 +113,16 @@ fn plan_snapshot(document: &Value) -> Map { "accuracy_guarantees".into(), document["accuracy_guarantees"].clone(), ); - if stage3["cost"].is_object() { - // The document shows the selected plan as Stage 2 candidate `P1`. - let candidate = document["stage2_physical_asap_optimization"]["candidates"] - .as_array() - .and_then(|candidates| candidates.iter().find(|c| c["id"] == "P1")) - .unwrap_or_else(|| panic!("the selected plan P1 is not shown")); + if stage3["plan"].is_object() { snapshot.insert( "selected".into(), json!({ - "display_name": candidate["display_name"], - "dag": candidate["dag"], + "dag": stage3["plan"], "cost": stage3["cost"], "accepted": stage3["accepted_accuracy_per_query"], }), ); - snapshot.insert("proven_optimal".into(), json!(proven_optimal(document))); - for key in [ - "materialized_views", - "materialized_views_error", - "tumbling_window_retention_ms", - "warnings", - ] { + for key in ["materialized_views", "materialized_views_error", "warnings"] { if let Some(value) = stage3.get(key) { snapshot.insert(key.into(), value.clone()); } @@ -158,7 +130,6 @@ fn plan_snapshot(document: &Value) -> Map { } else { snapshot.insert("selected".into(), Value::Null); if let Some(error) = stage3["error"].as_str() { - let error = error.split(" Search report: ").next().unwrap(); snapshot.insert("error".into(), json!(error)); } } @@ -305,23 +276,18 @@ fn corpus_plans_match_snapshots() { ); } -/// The snapshot keeps the selected plan without the search report, and -/// rounds floats. +/// The snapshot keeps the plan without the selection MILP's size. #[test] fn snapshot_keeps_only_the_plan() { let document = json!({ "accuracy_guarantees": "enforced", "workload": {"queries": [{"text": "sum(up)"}]}, - "stage2_physical_asap_optimization": {"candidates": [ - {"id": "P1", "display_name": "plan A", "dag": {"nodes": [1]}}, - ]}, "stage3_plan_selection": { - "selected_dag": {"candidate_of_node": [0]}, - "cost": {"cost_per_second": 0.1 + 0.2}, + "plan": {"nodes": [1]}, + "cost": {"cost_per_second": 0.3}, "accepted_accuracy_per_query": [], "materialized_views": [], - "tumbling_window_retention_ms": {"1": 60000}, - "search_report": {"solver_ms": 1}, + "milp_variables": 1, }, }); assert_eq!( @@ -330,28 +296,22 @@ fn snapshot_keeps_only_the_plan() { "query": "sum(up)", "accuracy_guarantees": "enforced", "selected": { - "display_name": "plan A", "dag": {"nodes": [1]}, "cost": {"cost_per_second": 0.3}, "accepted": [], }, - "proven_optimal": false, "materialized_views": [], - "tumbling_window_retention_ms": {"1": 60000}, }) ); } -/// A query without a plan keeps the error up to the embedded search report. +/// A query without a plan keeps the error. #[test] fn snapshot_of_a_failed_query() { let document = json!({ "accuracy_guarantees": "enforced", "workload": {"queries": [{"text": "sum(up)"}]}, - "stage3_plan_selection": { - "selected_dag": null, - "error": "infeasible. Search report: {\"solver_ms\":1.5}", - }, + "stage3_plan_selection": {"plan": null, "error": "infeasible"}, }); assert_eq!( snapshot_of(&document), @@ -359,20 +319,11 @@ fn snapshot_of_a_failed_query() { "query": "sum(up)", "accuracy_guarantees": "enforced", "selected": null, - "error": "infeasible.", + "error": "infeasible", }) ); } -/// Only an `optimal` search status proves the plan optimal. -#[test] -fn proven_optimal_needs_an_optimal_status() { - let document = - |status: Value| json!({"stage3_plan_selection": {"search_report": {"status": status}}}); - assert!(proven_optimal(&document(json!("optimal")))); - assert!(!proven_optimal(&document(Value::Null))); -} - /// The failure message names the first differing JSON path. #[test] fn first_differing_path_names_the_path() { diff --git a/crates/devtools/tests/corpus_snapshots/o11y_bench/q02.json b/crates/devtools/tests/corpus_snapshots/o11y_bench/q02.json index fae02b115..4fa01d6ac 100644 --- a/crates/devtools/tests/corpus_snapshots/o11y_bench/q02.json +++ b/crates/devtools/tests/corpus_snapshots/o11y_bench/q02.json @@ -3,222 +3,19 @@ "materialized_views": [ { "coverage": { - "definition_id": "4f999e82e92588ae4def6db7bcbd4fd7", + "definition_id": "be9c29c7ba723e7271224b1879a6c7b4", "selection": [ { "columns": [ { "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "order-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -60000 - }, - { - "included": 0 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 0, - "source": { - "TimeSeries": { - "metric": "http_request_duration_seconds_bucket" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 5, - "installed_position": 0, - "node_id": 2, - "refresh": "at_ingestion", - "retention_ms": 3900000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "4f999e82e92588ae4def6db7bcbd4fd7", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "order-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -120000 - }, - { - "included": -60000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 60000, - "source": { - "TimeSeries": { - "metric": "http_request_duration_seconds_bucket" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 5, - "installed_position": 1, - "node_id": 5, - "refresh": "at_ingestion", - "retention_ms": 3900000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "4f999e82e92588ae4def6db7bcbd4fd7", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "order-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -180000 - }, - { - "included": -120000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 120000, - "source": { - "TimeSeries": { - "metric": "http_request_duration_seconds_bucket" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 5, - "installed_position": 2, - "node_id": 8, - "refresh": "at_ingestion", - "retention_ms": 3900000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "4f999e82e92588ae4def6db7bcbd4fd7", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "order-service" + "column": "job", + "scan_ordinal": 0, + "source": { + "TimeSeries": { + "metric": "http_request_duration_seconds_bucket" } - ] - } - } - ], - "relative_time": [ - { - "excluded": -240000 - }, - { - "included": -180000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 180000, - "source": { - "TimeSeries": { - "metric": "http_request_duration_seconds_bucket" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 5, - "installed_position": 3, - "node_id": 11, - "refresh": "at_ingestion", - "retention_ms": 3900000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "4f999e82e92588ae4def6db7bcbd4fd7", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null + } }, "constraint": { "in": [ @@ -228,135 +25,102 @@ ] } } - ], - "relative_time": [ - { - "excluded": -300000 - }, - { - "included": -240000 - } ] } ] }, "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 240000, - "source": { - "TimeSeries": { - "metric": "http_request_duration_seconds_bucket" - } - }, - "width_ms": 60000 - } - ] + "materialized_views": [] }, - "installed_count": 5, - "installed_position": 4, - "node_id": 14, + "node_id": 1, "refresh": "at_ingestion", - "retention_ms": 3900000, - "stage2_node_id": 1 + "retention_ms": 3900000 } ], - "proven_optimal": true, "query": "histogram_quantile(0.95, sum(rate(http_request_duration_seconds_bucket{job=\"order-service\"}[5m])) by (le))", "selected": { "accepted": [ { "accepted_metric": null, - "accepted_regardless_of_guarantee": false, - "guarantee_check_rejection": null, "sketches": [], - "summary": "metric not set; accepted exact", - "target_metric": null + "summary": "accepted exact" } ], "cost": { - "cost_per_second": 0.481566859833, - "cost_unit": "cost_per_second", - "cpu_core_seconds_per_second": 0.0000195666666667, + "cost_per_second": 12.4552352995, + "cpu_core_seconds_per_second": 0.01113515, "per_node": { "0": { "cost_per_second": 0.0074, + "cost_source": "analytical", "cpu_core_seconds_per_second": 7.4e-6, "detail": "scan 1000 samples; x 1.0000/s", "multiplier": 1, "operator": "Scan", - "price_source": "analytical", "timing": "ingestion_time" }, "1": { - "cost_per_second": 0.464, - "cpu_core_seconds_per_second": 2e-6, - "detail": "build exact Rate accumulator in 5 tumbling windows of 60000 ms for 1000 groups: 1000 rows x (1 + depth 1); x 1.0000/s; keeps 3696000 bytes", + "cost_per_second": 1.483, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.000163, + "detail": "build exact Rate accumulator in tumbling windows of 60000 ms for 1000 groups: 1000 rows x 107 ops, 100 group visits, 16.7 states emitted; x 1.0000/s; keeps 10560000 bytes", "multiplier": 1, "operator": "SummaryTumblingWindowChain", - "price_source": "analytical", "timing": "ingestion_time" }, "2": { - "cost_per_second": 0.0050834045, - "cpu_core_seconds_per_second": 5.08333333333e-6, - "detail": "merge 5000 states into 1000; x 1.0167/s; holds 56000 bytes for 0.0 ms", + "cost_per_second": 0.055916667785, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.0000559166666667, + "detail": "time range 300s: 5 tumbling windows of states; x 1.0167/s; holds 160 bytes for 0.055 ms", "multiplier": 1, - "operator": "SummaryMerge", - "price_source": "analytical", + "operator": "TimeRange", "timing": "query_time" }, "3": { - "cost_per_second": 0.00101666666667, - "cpu_core_seconds_per_second": 1.01666666667e-6, - "detail": "finalize 1000 accumulators; x 1.0167/s", + "cost_per_second": 2.89755795, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.0028975, + "detail": "merge 5000 states into 1000; x 1.0167/s; holds 160000 bytes for 2.850 ms", "multiplier": 1, - "operator": "FinalizeExactAccumulator", - "price_source": "analytical", + "operator": "SummaryMerge", "timing": "query_time" }, "4": { - "cost_per_second": 0.00101673783333, - "cpu_core_seconds_per_second": 1.01666666667e-6, - "detail": "build exact Sum accumulator into 1000 states: 1000 rows x depth 1; x 1.0167/s; holds 56000 bytes for 0.0 ms", + "cost_per_second": 2.54166666667, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.00254166666667, + "detail": "finalize 1000 accumulators; x 1.0167/s", "multiplier": 1, - "operator": "SummaryAgg", - "price_source": "analytical", + "operator": "FinalizeExactAccumulator", "timing": "query_time" }, "5": { - "cost_per_second": 0.00101666666667, - "cpu_core_seconds_per_second": 1.01666666667e-6, - "detail": "finalize 1000 accumulators; x 1.0167/s", + "cost_per_second": 2.7348470075, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.00273483333333, + "detail": "hash aggregate 1000 rows into 1000 groups: 20 ops per row, 1000 group visits; x 1.0167/s; holds 40000 bytes for 2.690 ms", "multiplier": 1, - "operator": "FinalizeExactAccumulator", - "price_source": "analytical", + "operator": "Aggregate", "timing": "query_time" }, "6": { - "cost_per_second": 0.00203338416667, - "cpu_core_seconds_per_second": 2.03333333333e-6, - "detail": "hash aggregate 1000 rows into 1000 groups; x 1.0167/s; holds 40000 bytes for 0.0 ms", + "cost_per_second": 2.7348470075, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.00273483333333, + "detail": "hash aggregate 1000 rows into 1000 groups: 20 ops per row, 1000 group visits; x 1.0167/s; holds 40000 bytes for 2.690 ms", "multiplier": 1, "operator": "Aggregate", - "price_source": "analytical", "timing": "query_time" } }, "source": "analytical-cost-v2 (illustrative statistics)", "usage": { - "average_memory_bytes": 3696001.54533, - "cpu_ops_per_second": 13166.6666667, + "average_memory_bytes": 10560682.3956, + "cpu_ops_per_second": 11128750.0, "disk_bytes": 0, - "latency_per_query": [ - { - "compaction_ms": 0.0, - "query_ms": 0.01 - } - ], "peak_cpu_cores": 2, - "peak_memory_bytes": 3848064, + "peak_memory_bytes": 10800224, "scanned_bytes_per_second": 64000.0 } }, @@ -377,40 +141,35 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "le", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -423,7 +182,7 @@ }, "producer": 0, "role": "Input", - "window": "RequiresAlignedPanePhaseOrExactBoundaryResidual" + "window": "NotApplicable" }, { "consumer": 2, @@ -439,9 +198,8 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { @@ -451,39 +209,37 @@ ] }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "le", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 4, - 0 + 0, + 2, + 3, + 4 ] ] }, @@ -505,9 +261,8 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { @@ -517,39 +272,37 @@ ] }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "le", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 4, - 0 + 0, + 2, + 3, + 4 ] ] }, @@ -560,7 +313,7 @@ { "consumer": 4, "data_state": { - "primitive": "Raw", + "primitive": "SummaryState", "timing": "query_time" }, "grouping": "NotApplicable", @@ -572,40 +325,38 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { - "Plain": "float64" + "ExactAggregate": [ + "Rate", + "Rate" + ] }, - "name": "value", - "nullable": false, - "table": null + "name": "state", + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "le", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -623,36 +374,53 @@ { "consumer": 5, "data_state": { - "primitive": "SummaryState", + "primitive": "Raw", "timing": "query_time" }, "grouping": "NotApplicable", "intermediate_schema": { "closed": true, "fields": [ + { + "dtype": { + "Plain": "timestamp" + }, + "name": "ts", + "nullable": false + }, + { + "dtype": { + "Plain": "float64" + }, + "name": "value", + "nullable": false + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "job", + "nullable": true + }, { "dtype": { "Plain": "utf8" }, "name": "le", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { - "ExactAggregate": [ - "Sum", - "Sum" - ] + "Plain": "utf8" }, - "name": "state", - "nullable": false, - "table": null + "name": "$promql_series_identity", + "nullable": false } ], - "time_index": null, + "time_index": 0, "unique_keys": [ [ + 4, 0 ] ] @@ -676,16 +444,14 @@ "Plain": "utf8" }, "name": "le", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "float64" }, "name": "sum", - "nullable": false, - "table": null + "nullable": false } ], "time_index": null, @@ -702,24 +468,6 @@ ], "nodes": [ { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact over exact inputs" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 0, "lifetime": "None", "output_schema": { @@ -730,40 +478,35 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "le", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -805,40 +548,35 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "le", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -859,7 +597,6 @@ } }, { - "guarantee": null, "id": 1, "lifetime": { "AcrossEvaluations": "Memory" @@ -871,9 +608,8 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { @@ -883,39 +619,37 @@ ] }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "le", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 4, - 0 + 0, + 2, + 3, + 4 ] ] }, @@ -927,14 +661,7 @@ "ASAP": { "SummaryTumblingWindowChain": { "child": 0, - "family": { - "ExactAggregate": [ - "Rate", - "Rate" - ] - }, "filter": null, - "grouping": "PerSubpopulationInstance", "input": { "item": null, "weight": { @@ -942,14 +669,46 @@ } }, "reduction": "PerEntity", - "tumbling_window_count": 5, + "summary_type": { + "ExactAggregate": [ + "Rate", + "Rate" + ] + }, "tumbling_window_width_ms": 60000 } } + }, + "retention_ms": 3900000, + "view_coverage": { + "definition_id": "be9c29c7ba723e7271224b1879a6c7b4", + "selection": [ + { + "columns": [ + { + "column": { + "column": "job", + "scan_ordinal": 0, + "source": { + "TimeSeries": { + "metric": "http_request_duration_seconds_bucket" + } + } + }, + "constraint": { + "in": [ + { + "Utf8": "order-service" + } + ] + } + } + ] + } + ] } }, { - "guarantee": null, "id": 2, "lifetime": "None", "output_schema": { @@ -959,9 +718,8 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { @@ -971,39 +729,37 @@ ] }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "le", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 4, - 0 + 0, + 2, + 3, + 4 ] ] }, @@ -1012,36 +768,23 @@ "timing": "query_time" }, "payload": { - "ASAP": { - "SummaryMerge": { - "children": [ - 1 - ] + "NonASAP": { + "TimeRange": { + "child": 1, + "kind": "range", + "range": { + "nanos": 0, + "secs": 300 + } } } } }, { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact accumulator over exact input" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 3, - "lifetime": "None", + "lifetime": { + "OneEvaluation": "Memory" + }, "output_schema": { "closed": true, "fields": [ @@ -1050,40 +793,38 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { - "Plain": "float64" + "ExactAggregate": [ + "Rate", + "Rate" + ] }, - "name": "value", - "nullable": false, - "table": null + "name": "state", + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "le", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -1095,103 +836,86 @@ ] }, "output_state": { - "primitive": "Raw", + "primitive": "SummaryState", "timing": "query_time" }, "payload": { "ASAP": { - "FinalizeExactAccumulator": { - "child": 2 + "SummaryMerge": { + "children": [ + 2 + ] } } } }, { - "guarantee": null, "id": 4, "lifetime": "None", "output_schema": { "closed": true, "fields": [ + { + "dtype": { + "Plain": "timestamp" + }, + "name": "ts", + "nullable": false + }, + { + "dtype": { + "Plain": "float64" + }, + "name": "value", + "nullable": false + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "job", + "nullable": true + }, { "dtype": { "Plain": "utf8" }, "name": "le", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { - "ExactAggregate": [ - "Sum", - "Sum" - ] + "Plain": "utf8" }, - "name": "state", - "nullable": false, - "table": null + "name": "$promql_series_identity", + "nullable": false } ], - "time_index": null, + "time_index": 0, "unique_keys": [ [ + 4, 0 ] ] }, "output_state": { - "primitive": "SummaryState", + "primitive": "Raw", "timing": "query_time" }, "payload": { "ASAP": { - "SummaryAgg": { - "child": 3, - "family": { - "ExactAggregate": [ - "Sum", - "Sum" - ] - }, - "filter": null, - "grouping": "PerSubpopulationInstance", - "input": { - "item": null, - "weight": { - "Column": "SampleValue" - } - }, - "reduction": { - "Reduce": [ - 3 - ] - } + "FinalizeExactAccumulator": { + "child": 3 } } } }, { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact accumulator over exact input" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 5, - "lifetime": "None", + "lifetime": { + "OneEvaluation": "Memory" + }, "output_schema": { "closed": true, "fields": [ @@ -1200,16 +924,14 @@ "Plain": "utf8" }, "name": "le", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "float64" }, "name": "sum", - "nullable": false, - "table": null + "nullable": false } ], "time_index": null, @@ -1224,34 +946,34 @@ "timing": "query_time" }, "payload": { - "ASAP": { - "FinalizeExactAccumulator": { - "child": 4 + "NonASAP": { + "Aggregate": { + "child": 4, + "filters": [], + "having": null, + "measures": [ + { + "col": null, + "kind": "sum" + } + ], + "output_names": [ + "" + ], + "reduction": { + "Reduce": [ + 3 + ] + } } } } }, { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact over exact inputs" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 6, - "lifetime": "None", + "lifetime": { + "OneEvaluation": "Memory" + }, "output_schema": { "closed": false, "fields": [ @@ -1260,8 +982,7 @@ "Plain": "float64" }, "name": "histogram_quantile", - "nullable": false, - "table": null + "nullable": false } ], "time_index": null, @@ -1303,10 +1024,9 @@ 6 ], "scalar_roots": [] - }, - "display_name": "Q1 Sum acc + Rate acc ×5 tumbling windows · ingestion time: Rate acc for Q1, 5 × 1m tumbling windows" + } }, - "tumbling_window_retention_ms": { - "1": 3900000 - } + "warnings": [ + "assumed data fact: the data arrival declares no max_out_of_order_ms, so the samples of source http_request_duration_seconds_bucket that an ingestion-time rate, increase or irate state reads are taken to arrive in timestamp order; if this does not hold, answers may be wrong by an amount no accuracy target bounds (a late sample can be read as a counter reset), or ingestion may fail" + ] } diff --git a/crates/devtools/tests/corpus_snapshots/o11y_bench/q05.json b/crates/devtools/tests/corpus_snapshots/o11y_bench/q05.json index 35e9d5196..621527658 100644 --- a/crates/devtools/tests/corpus_snapshots/o11y_bench/q05.json +++ b/crates/devtools/tests/corpus_snapshots/o11y_bench/q05.json @@ -1,183 +1,163 @@ { "accuracy_guarantees": "enforced", "materialized_views": [], - "materialized_views_error": "raw-data window of node 1 is unsupported: a subquery or a TimeShift above the range", - "proven_optimal": true, + "materialized_views_error": "the executor cannot compile the plan's query run: invalid DAG: node 13: invalid DAG: value operation has no native implementation", "query": "max_over_time((sum(rate(http_requests_total{job=\"order-service\",status=~\"5..\"}[5m])) / sum(rate(http_requests_total{job=\"order-service\"}[5m])))[6h:1m])", "selected": { "accepted": [ { "accepted_metric": null, - "accepted_regardless_of_guarantee": false, - "guarantee_check_rejection": null, "sketches": [], - "summary": "metric not set; accepted exact", - "target_metric": null + "summary": "accepted exact" } ], "cost": { - "cost_per_second": 0.955439633013, - "cost_unit": "cost_per_second", - "cpu_core_seconds_per_second": 0.0000314394333333, + "cost_per_second": 13.8167759738, + "cpu_core_seconds_per_second": 0.0113086658667, "per_node": { "0": { "cost_per_second": 0.0074, + "cost_source": "analytical", "cpu_core_seconds_per_second": 7.4e-6, "detail": "scan 1000 samples; x 1.0000/s", "multiplier": 1, "operator": "Scan", - "price_source": "analytical", "timing": "ingestion_time" }, "1": { - "cost_per_second": 0.464, - "cpu_core_seconds_per_second": 2e-6, - "detail": "build exact Rate accumulator in 5 tumbling windows of 60000 ms for 1000 groups: 1000 rows x (1 + depth 1); x 1.0000/s; keeps 3696000 bytes", + "cost_per_second": 1.483, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.000163, + "detail": "build exact Rate accumulator in tumbling windows of 60000 ms for 1000 groups: 1000 rows x 107 ops, 100 group visits, 16.7 states emitted; x 1.0000/s; keeps 10560000 bytes", "multiplier": 1, "operator": "SummaryTumblingWindowChain", - "price_source": "analytical", "timing": "ingestion_time" }, "10": { - "cost_per_second": 0.00101666676634, - "cpu_core_seconds_per_second": 1.01666666667e-6, - "detail": "build exact Sum accumulator into 1 states: 1000 rows x depth 1; x 1.0167/s; holds 56 bytes for 0.0 ms", + "cost_per_second": 2.54166666667, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.00254166666667, + "detail": "finalize 1000 accumulators; x 1.0167/s", "multiplier": 1, - "operator": "SummaryAgg", - "price_source": "analytical", + "operator": "FinalizeExactAccumulator", "timing": "query_time" }, "11": { - "cost_per_second": 1.01666666667e-6, - "cpu_core_seconds_per_second": 1.01666666667e-9, - "detail": "finalize 1 accumulators; x 1.0167/s", + "cost_per_second": 0.00678116668701, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 6.78116666667e-6, + "detail": "hash aggregate 1000 rows into 1 groups: 4 ops per row, 1 group visits; x 1.0167/s; holds 24 bytes for 0.007 ms", "multiplier": 1, - "operator": "FinalizeExactAccumulator", - "price_source": "analytical", + "operator": "Aggregate", "timing": "query_time" }, "12": { - "cost_per_second": 1.0166809061e-6, + "cost_per_second": 1.01666666768e-6, + "cost_source": "analytical", "cpu_core_seconds_per_second": 1.01666666667e-9, - "detail": "operator: pass 1 rows; x 1.0167/s; holds 8 bytes for 0.0 ms", + "detail": "operator: pass 1 rows; x 1.0167/s; holds 8 bytes for 0.000 ms", "multiplier": 1, "operator": "BinaryOp", - "price_source": "analytical", "timing": "query_time" }, "13": { - "cost_per_second": 1.0166809061e-6, + "cost_per_second": 1.01666666768e-6, + "cost_source": "analytical", "cpu_core_seconds_per_second": 1.01666666667e-9, - "detail": "operator: pass 1 rows; x 1.0167/s; holds 8 bytes for 0.0 ms", + "detail": "operator: pass 1 rows; x 1.0167/s; holds 8 bytes for 0.000 ms", "multiplier": 1, "operator": "PromqlSubquery", - "price_source": "analytical", "timing": "query_time" }, "14": { - "cost_per_second": 1.0167663427e-6, - "cpu_core_seconds_per_second": 1.01666666667e-9, - "detail": "build exact Max accumulator into 1 states: 1 rows x depth 1; x 1.0167/s; holds 56 bytes for 0.0 ms", - "multiplier": 1, - "operator": "SummaryAgg", - "price_source": "analytical", - "timing": "query_time" - }, - "15": { - "cost_per_second": 1.01666666667e-6, - "cpu_core_seconds_per_second": 1.01666666667e-9, - "detail": "finalize 1 accumulators; x 1.0167/s", + "cost_per_second": 0.00273483334701, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 2.73483333333e-6, + "detail": "hash aggregate 1 rows into 1 groups: 20 ops per row, 1 group visits; x 1.0167/s; holds 40 bytes for 0.003 ms", "multiplier": 1, - "operator": "FinalizeExactAccumulator", - "price_source": "analytical", + "operator": "Aggregate", "timing": "query_time" }, "2": { - "cost_per_second": 0.00508343300937, - "cpu_core_seconds_per_second": 5.08333333333e-6, - "detail": "merge 5000 states into 1000; x 1.0167/s; holds 56000 bytes for 0.0 ms", + "cost_per_second": 0.055916667785, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.0000559166666667, + "detail": "time range 300s: 5 tumbling windows of states; x 1.0167/s; holds 160 bytes for 0.055 ms", "multiplier": 1, - "operator": "SummaryMerge", - "price_source": "analytical", + "operator": "TimeRange", "timing": "query_time" }, "3": { - "cost_per_second": 0.00101666666667, - "cpu_core_seconds_per_second": 1.01666666667e-6, - "detail": "finalize 1000 accumulators; x 1.0167/s", + "cost_per_second": 2.89755795, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.0028975, + "detail": "merge 5000 states into 1000; x 1.0167/s; holds 160000 bytes for 2.850 ms", "multiplier": 1, - "operator": "FinalizeExactAccumulator", - "price_source": "analytical", + "operator": "SummaryMerge", "timing": "query_time" }, "4": { - "cost_per_second": 0.00101666676634, - "cpu_core_seconds_per_second": 1.01666666667e-6, - "detail": "build exact Sum accumulator into 1 states: 1000 rows x depth 1; x 1.0167/s; holds 56 bytes for 0.0 ms", + "cost_per_second": 2.54166666667, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.00254166666667, + "detail": "finalize 1000 accumulators; x 1.0167/s", "multiplier": 1, - "operator": "SummaryAgg", - "price_source": "analytical", + "operator": "FinalizeExactAccumulator", "timing": "query_time" }, "5": { - "cost_per_second": 1.01666666667e-6, - "cpu_core_seconds_per_second": 1.01666666667e-9, - "detail": "finalize 1 accumulators; x 1.0167/s", + "cost_per_second": 0.00678116668701, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 6.78116666667e-6, + "detail": "hash aggregate 1000 rows into 1 groups: 4 ops per row, 1 group visits; x 1.0167/s; holds 24 bytes for 0.007 ms", "multiplier": 1, - "operator": "FinalizeExactAccumulator", - "price_source": "analytical", + "operator": "Aggregate", "timing": "query_time" }, "6": { "cost_per_second": 0.0058, + "cost_source": "analytical", "cpu_core_seconds_per_second": 5.8e-6, "detail": "scan 1000 samples; x 1.0000/s", "multiplier": 1, "operator": "Scan", - "price_source": "analytical", "timing": "ingestion_time" }, "7": { - "cost_per_second": 0.464, - "cpu_core_seconds_per_second": 2e-6, - "detail": "build exact Rate accumulator in 5 tumbling windows of 60000 ms for 1000 groups: 1000 rows x (1 + depth 1); x 1.0000/s; keeps 3696000 bytes", + "cost_per_second": 1.314, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.000126, + "detail": "build exact Rate accumulator in tumbling windows of 60000 ms for 1000 groups: 1000 rows x 70 ops, 100 group visits, 16.7 states emitted; x 1.0000/s; keeps 9504000 bytes", "multiplier": 1, "operator": "SummaryTumblingWindowChain", - "price_source": "analytical", "timing": "ingestion_time" }, "8": { - "cost_per_second": 0.00508343300937, - "cpu_core_seconds_per_second": 5.08333333333e-6, - "detail": "merge 5000 states into 1000; x 1.0167/s; holds 56000 bytes for 0.0 ms", + "cost_per_second": 0.0559166676732, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.0000559166666667, + "detail": "time range 300s: 5 tumbling windows of states; x 1.0167/s; holds 144 bytes for 0.055 ms", "multiplier": 1, - "operator": "SummaryMerge", - "price_source": "analytical", + "operator": "TimeRange", "timing": "query_time" }, "9": { - "cost_per_second": 0.00101666666667, - "cpu_core_seconds_per_second": 1.01666666667e-6, - "detail": "finalize 1000 accumulators; x 1.0167/s", + "cost_per_second": 2.897552155, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.0028975, + "detail": "merge 5000 states into 1000; x 1.0167/s; holds 144000 bytes for 2.850 ms", "multiplier": 1, - "operator": "FinalizeExactAccumulator", - "price_source": "analytical", + "operator": "SummaryMerge", "timing": "query_time" } }, "source": "analytical-cost-v2 (illustrative statistics)", "usage": { - "average_memory_bytes": 7392001.59744, - "cpu_ops_per_second": 20239.4333333, + "average_memory_bytes": 20064880.8574, + "cpu_ops_per_second": 11297465.8667, "disk_bytes": 0, - "latency_per_query": [ - { - "compaction_ms": 0.0, - "query_ms": 0.014006 - } - ], "peak_cpu_cores": 2, - "peak_memory_bytes": 7504296, + "peak_memory_bytes": 20368520, "scanned_bytes_per_second": 112000.0 } }, @@ -198,40 +178,35 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "status", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -244,7 +219,7 @@ }, "producer": 0, "role": "Input", - "window": "RequiresAlignedPanePhaseOrExactBoundaryResidual" + "window": "NotApplicable" }, { "consumer": 2, @@ -260,9 +235,8 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { @@ -272,39 +246,37 @@ ] }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "status", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 4, - 0 + 0, + 2, + 3, + 4 ] ] }, @@ -326,9 +298,8 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { @@ -338,39 +309,37 @@ ] }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "status", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 4, - 0 + 0, + 2, + 3, + 4 ] ] }, @@ -381,7 +350,7 @@ { "consumer": 4, "data_state": { - "primitive": "Raw", + "primitive": "SummaryState", "timing": "query_time" }, "grouping": "NotApplicable", @@ -393,40 +362,38 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { - "Plain": "float64" + "ExactAggregate": [ + "Rate", + "Rate" + ] }, - "name": "value", - "nullable": false, - "table": null + "name": "state", + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "status", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -444,7 +411,7 @@ { "consumer": 5, "data_state": { - "primitive": "SummaryState", + "primitive": "Raw", "timing": "query_time" }, "grouping": "NotApplicable", @@ -453,18 +420,47 @@ "fields": [ { "dtype": { - "ExactAggregate": [ - "Sum", - "Sum" - ] + "Plain": "timestamp" }, - "name": "state", - "nullable": false, - "table": null + "name": "ts", + "nullable": false + }, + { + "dtype": { + "Plain": "float64" + }, + "name": "value", + "nullable": false + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "job", + "nullable": true + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "status", + "nullable": true + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false } ], - "time_index": null, - "unique_keys": [] + "time_index": 0, + "unique_keys": [ + [ + 4, + 0 + ] + ] }, "producer": 4, "role": "Input", @@ -485,32 +481,28 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -523,7 +515,7 @@ }, "producer": 6, "role": "Input", - "window": "RequiresAlignedPanePhaseOrExactBoundaryResidual" + "window": "NotApplicable" }, { "consumer": 8, @@ -539,9 +531,8 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { @@ -551,31 +542,29 @@ ] }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 3, - 0 + 0, + 2, + 3 ] ] }, @@ -597,9 +586,8 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { @@ -609,31 +597,29 @@ ] }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 3, - 0 + 0, + 2, + 3 ] ] }, @@ -644,7 +630,7 @@ { "consumer": 10, "data_state": { - "primitive": "Raw", + "primitive": "SummaryState", "timing": "query_time" }, "grouping": "NotApplicable", @@ -656,32 +642,31 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { - "Plain": "float64" + "ExactAggregate": [ + "Rate", + "Rate" + ] }, - "name": "value", - "nullable": false, - "table": null + "name": "state", + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -699,7 +684,7 @@ { "consumer": 11, "data_state": { - "primitive": "SummaryState", + "primitive": "Raw", "timing": "query_time" }, "grouping": "NotApplicable", @@ -708,18 +693,40 @@ "fields": [ { "dtype": { - "ExactAggregate": [ - "Sum", - "Sum" - ] + "Plain": "timestamp" }, - "name": "state", - "nullable": false, - "table": null + "name": "ts", + "nullable": false + }, + { + "dtype": { + "Plain": "float64" + }, + "name": "value", + "nullable": false + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "job", + "nullable": true + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false } ], - "time_index": null, - "unique_keys": [] + "time_index": 0, + "unique_keys": [ + [ + 3, + 0 + ] + ] }, "producer": 10, "role": "Input", @@ -740,8 +747,7 @@ "Plain": "float64" }, "name": "sum", - "nullable": false, - "table": null + "nullable": false } ], "time_index": null, @@ -766,8 +772,7 @@ "Plain": "float64" }, "name": "sum", - "nullable": false, - "table": null + "nullable": false } ], "time_index": null, @@ -792,8 +797,7 @@ "Plain": "float64" }, "name": "sum", - "nullable": false, - "table": null + "nullable": false } ], "time_index": null, @@ -818,8 +822,7 @@ "Plain": "float64" }, "name": "sum", - "nullable": false, - "table": null + "nullable": false } ], "time_index": null, @@ -828,57 +831,10 @@ "producer": 13, "role": "Input", "window": "NotApplicable" - }, - { - "consumer": 15, - "data_state": { - "primitive": "SummaryState", - "timing": "query_time" - }, - "grouping": "NotApplicable", - "intermediate_schema": { - "closed": true, - "fields": [ - { - "dtype": { - "ExactAggregate": [ - "Max", - "Max" - ] - }, - "name": "state", - "nullable": false, - "table": null - } - ], - "time_index": null, - "unique_keys": [] - }, - "producer": 14, - "role": "Input", - "window": "NotApplicable" } ], "nodes": [ { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact over exact inputs" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 0, "lifetime": "None", "output_schema": { @@ -889,40 +845,35 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "status", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -978,40 +929,35 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "status", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -1032,7 +978,6 @@ } }, { - "guarantee": null, "id": 1, "lifetime": { "AcrossEvaluations": "Memory" @@ -1044,9 +989,8 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { @@ -1056,39 +1000,37 @@ ] }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "status", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 4, - 0 + 0, + 2, + 3, + 4 ] ] }, @@ -1100,14 +1042,7 @@ "ASAP": { "SummaryTumblingWindowChain": { "child": 0, - "family": { - "ExactAggregate": [ - "Rate", - "Rate" - ] - }, "filter": null, - "grouping": "PerSubpopulationInstance", "input": { "item": null, "weight": { @@ -1115,14 +1050,46 @@ } }, "reduction": "PerEntity", - "tumbling_window_count": 5, + "summary_type": { + "ExactAggregate": [ + "Rate", + "Rate" + ] + }, "tumbling_window_width_ms": 60000 } } + }, + "retention_ms": 3900000, + "view_coverage": { + "definition_id": "7e48d8927e84a32d499eaaf08711a04d", + "selection": [ + { + "columns": [ + { + "column": { + "column": "job", + "scan_ordinal": 0, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + } + }, + "constraint": { + "in": [ + { + "Utf8": "order-service" + } + ] + } + } + ] + } + ] } }, { - "guarantee": null, "id": 2, "lifetime": "None", "output_schema": { @@ -1132,9 +1099,8 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { @@ -1144,39 +1110,37 @@ ] }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "status", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 4, - 0 + 0, + 2, + 3, + 4 ] ] }, @@ -1185,36 +1149,23 @@ "timing": "query_time" }, "payload": { - "ASAP": { - "SummaryMerge": { - "children": [ - 1 - ] + "NonASAP": { + "TimeRange": { + "child": 1, + "kind": "range", + "range": { + "nanos": 0, + "secs": 300 + } } } } }, { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact accumulator over exact input" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 3, - "lifetime": "None", + "lifetime": { + "OneEvaluation": "Memory" + }, "output_schema": { "closed": true, "fields": [ @@ -1223,40 +1174,38 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { - "Plain": "float64" + "ExactAggregate": [ + "Rate", + "Rate" + ] }, - "name": "value", - "nullable": false, - "table": null + "name": "state", + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "status", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -1268,19 +1217,20 @@ ] }, "output_state": { - "primitive": "Raw", + "primitive": "SummaryState", "timing": "query_time" }, "payload": { "ASAP": { - "FinalizeExactAccumulator": { - "child": 2 + "SummaryMerge": { + "children": [ + 2 + ] } } } }, { - "guarantee": null, "id": 4, "lifetime": "None", "output_schema": { @@ -1288,69 +1238,65 @@ "fields": [ { "dtype": { - "ExactAggregate": [ - "Sum", - "Sum" - ] + "Plain": "timestamp" }, - "name": "state", - "nullable": false, - "table": null + "name": "ts", + "nullable": false + }, + { + "dtype": { + "Plain": "float64" + }, + "name": "value", + "nullable": false + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "job", + "nullable": true + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "status", + "nullable": true + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false } ], - "time_index": null, - "unique_keys": [] + "time_index": 0, + "unique_keys": [ + [ + 4, + 0 + ] + ] }, "output_state": { - "primitive": "SummaryState", + "primitive": "Raw", "timing": "query_time" }, "payload": { "ASAP": { - "SummaryAgg": { - "child": 3, - "family": { - "ExactAggregate": [ - "Sum", - "Sum" - ] - }, - "filter": null, - "grouping": "PerSubpopulationInstance", - "input": { - "item": null, - "weight": { - "Column": "SampleValue" - } - }, - "reduction": { - "Reduce": [] - } + "FinalizeExactAccumulator": { + "child": 3 } } } }, { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact accumulator over exact input" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 5, - "lifetime": "None", + "lifetime": { + "OneEvaluation": "Memory" + }, "output_schema": { "closed": true, "fields": [ @@ -1359,8 +1305,7 @@ "Plain": "float64" }, "name": "sum", - "nullable": false, - "table": null + "nullable": false } ], "time_index": null, @@ -1371,32 +1316,28 @@ "timing": "query_time" }, "payload": { - "ASAP": { - "FinalizeExactAccumulator": { - "child": 4 + "NonASAP": { + "Aggregate": { + "child": 4, + "filters": [], + "having": null, + "measures": [ + { + "col": null, + "kind": "sum" + } + ], + "output_names": [ + "" + ], + "reduction": { + "Reduce": [] + } } } } }, { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact over exact inputs" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 6, "lifetime": "None", "output_schema": { @@ -1407,32 +1348,28 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -1474,32 +1411,28 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -1520,7 +1453,6 @@ } }, { - "guarantee": null, "id": 7, "lifetime": { "AcrossEvaluations": "Memory" @@ -1532,9 +1464,8 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { @@ -1544,31 +1475,29 @@ ] }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 3, - 0 + 0, + 2, + 3 ] ] }, @@ -1580,14 +1509,7 @@ "ASAP": { "SummaryTumblingWindowChain": { "child": 6, - "family": { - "ExactAggregate": [ - "Rate", - "Rate" - ] - }, "filter": null, - "grouping": "PerSubpopulationInstance", "input": { "item": null, "weight": { @@ -1595,14 +1517,46 @@ } }, "reduction": "PerEntity", - "tumbling_window_count": 5, + "summary_type": { + "ExactAggregate": [ + "Rate", + "Rate" + ] + }, "tumbling_window_width_ms": 60000 } } + }, + "retention_ms": 3900000, + "view_coverage": { + "definition_id": "58395555aae268f4f68a8a3e82a6c45b", + "selection": [ + { + "columns": [ + { + "column": { + "column": "job", + "scan_ordinal": 0, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + } + }, + "constraint": { + "in": [ + { + "Utf8": "order-service" + } + ] + } + } + ] + } + ] } }, { - "guarantee": null, "id": 8, "lifetime": "None", "output_schema": { @@ -1612,9 +1566,8 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { @@ -1624,31 +1577,29 @@ ] }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 3, - 0 + 0, + 2, + 3 ] ] }, @@ -1657,36 +1608,23 @@ "timing": "query_time" }, "payload": { - "ASAP": { - "SummaryMerge": { - "children": [ - 7 - ] + "NonASAP": { + "TimeRange": { + "child": 7, + "kind": "range", + "range": { + "nanos": 0, + "secs": 300 + } } } } }, { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact accumulator over exact input" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 9, - "lifetime": "None", + "lifetime": { + "OneEvaluation": "Memory" + }, "output_schema": { "closed": true, "fields": [ @@ -1695,32 +1633,31 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { - "Plain": "float64" + "ExactAggregate": [ + "Rate", + "Rate" + ] }, - "name": "value", - "nullable": false, - "table": null + "name": "state", + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -1732,19 +1669,20 @@ ] }, "output_state": { - "primitive": "Raw", + "primitive": "SummaryState", "timing": "query_time" }, "payload": { "ASAP": { - "FinalizeExactAccumulator": { - "child": 8 + "SummaryMerge": { + "children": [ + 8 + ] } } } }, { - "guarantee": null, "id": 10, "lifetime": "None", "output_schema": { @@ -1752,69 +1690,58 @@ "fields": [ { "dtype": { - "ExactAggregate": [ - "Sum", - "Sum" - ] + "Plain": "timestamp" }, - "name": "state", - "nullable": false, - "table": null + "name": "ts", + "nullable": false + }, + { + "dtype": { + "Plain": "float64" + }, + "name": "value", + "nullable": false + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "job", + "nullable": true + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false } ], - "time_index": null, - "unique_keys": [] + "time_index": 0, + "unique_keys": [ + [ + 3, + 0 + ] + ] }, "output_state": { - "primitive": "SummaryState", + "primitive": "Raw", "timing": "query_time" }, "payload": { "ASAP": { - "SummaryAgg": { - "child": 9, - "family": { - "ExactAggregate": [ - "Sum", - "Sum" - ] - }, - "filter": null, - "grouping": "PerSubpopulationInstance", - "input": { - "item": null, - "weight": { - "Column": "SampleValue" - } - }, - "reduction": { - "Reduce": [] - } + "FinalizeExactAccumulator": { + "child": 9 } } } }, { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact accumulator over exact input" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 11, - "lifetime": "None", + "lifetime": { + "OneEvaluation": "Memory" + }, "output_schema": { "closed": true, "fields": [ @@ -1823,8 +1750,7 @@ "Plain": "float64" }, "name": "sum", - "nullable": false, - "table": null + "nullable": false } ], "time_index": null, @@ -1835,34 +1761,32 @@ "timing": "query_time" }, "payload": { - "ASAP": { - "FinalizeExactAccumulator": { - "child": 10 + "NonASAP": { + "Aggregate": { + "child": 10, + "filters": [], + "having": null, + "measures": [ + { + "col": null, + "kind": "sum" + } + ], + "output_names": [ + "" + ], + "reduction": { + "Reduce": [] + } } } } }, { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact over exact inputs" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 12, - "lifetime": "None", + "lifetime": { + "OneEvaluation": "Memory" + }, "output_schema": { "closed": true, "fields": [ @@ -1871,8 +1795,7 @@ "Plain": "float64" }, "name": "sum", - "nullable": false, - "table": null + "nullable": false } ], "time_index": null, @@ -1901,26 +1824,10 @@ } }, { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact over exact inputs" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 13, - "lifetime": "None", + "lifetime": { + "OneEvaluation": "Memory" + }, "output_schema": { "closed": true, "fields": [ @@ -1929,8 +1836,7 @@ "Plain": "float64" }, "name": "sum", - "nullable": false, - "table": null + "nullable": false } ], "time_index": null, @@ -1957,75 +1863,10 @@ } }, { - "guarantee": null, "id": 14, - "lifetime": "None", - "output_schema": { - "closed": true, - "fields": [ - { - "dtype": { - "ExactAggregate": [ - "Max", - "Max" - ] - }, - "name": "state", - "nullable": false, - "table": null - } - ], - "time_index": null, - "unique_keys": [] - }, - "output_state": { - "primitive": "SummaryState", - "timing": "query_time" - }, - "payload": { - "ASAP": { - "SummaryAgg": { - "child": 13, - "family": { - "ExactAggregate": [ - "Max", - "Max" - ] - }, - "filter": null, - "grouping": "PerSubpopulationInstance", - "input": { - "item": null, - "weight": { - "Column": "SampleValue" - } - }, - "reduction": "PerEntity" - } - } - } - }, - { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact accumulator over exact input" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } + "lifetime": { + "OneEvaluation": "Memory" }, - "id": 15, - "lifetime": "None", "output_schema": { "closed": true, "fields": [ @@ -2034,8 +1875,7 @@ "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false } ], "time_index": null, @@ -2046,23 +1886,33 @@ "timing": "query_time" }, "payload": { - "ASAP": { - "FinalizeExactAccumulator": { - "child": 14 + "NonASAP": { + "Aggregate": { + "child": 13, + "filters": [], + "having": null, + "measures": [ + { + "col": null, + "kind": "max" + } + ], + "output_names": [ + "" + ], + "reduction": "PerEntity" } } } } ], "roots": [ - 15 + 14 ], "scalar_roots": [] - }, - "display_name": "Q1 Max acc + Sum acc + Rate acc ×5 tumbling windows · ingestion time: Rate acc for Q1, 5 × 1m tumbling windows, Rate acc for Q1, 5 × 1m tumbling windows" + } }, - "tumbling_window_retention_ms": { - "1": 3900000, - "7": 3900000 - } + "warnings": [ + "assumed data fact: the data arrival declares no max_out_of_order_ms, so the samples of source http_requests_total that an ingestion-time rate, increase or irate state reads are taken to arrive in timestamp order; if this does not hold, answers may be wrong by an amount no accuracy target bounds (a late sample can be read as a counter reset), or ingestion may fail" + ] } diff --git a/crates/devtools/tests/corpus_snapshots/o11y_bench/q06.json b/crates/devtools/tests/corpus_snapshots/o11y_bench/q06.json index 934e7a5d6..1f26986f6 100644 --- a/crates/devtools/tests/corpus_snapshots/o11y_bench/q06.json +++ b/crates/devtools/tests/corpus_snapshots/o11y_bench/q06.json @@ -1,129 +1,109 @@ { "accuracy_guarantees": "enforced", "materialized_views": [], - "materialized_views_error": "raw-data window of node 1 is unsupported: a subquery or a TimeShift above the range", - "proven_optimal": true, + "materialized_views_error": "the executor cannot compile the plan's query run: invalid DAG: node 7: invalid DAG: value operation has no native implementation", "query": "max_over_time(histogram_quantile(0.95, sum(rate(http_request_duration_seconds_bucket{job=\"user-service\"}[5m])) by (le))[12h:1m])", "selected": { "accepted": [ { "accepted_metric": null, - "accepted_regardless_of_guarantee": false, - "guarantee_check_rejection": null, "sketches": [], - "summary": "metric not set; accepted exact", - "target_metric": null + "summary": "accepted exact" } ], "cost": { - "cost_per_second": 0.484617010313, - "cost_unit": "cost_per_second", - "cpu_core_seconds_per_second": 0.0000226166666667, + "cost_per_second": 15.1910989736, + "cpu_core_seconds_per_second": 0.013871, "per_node": { "0": { "cost_per_second": 0.0074, + "cost_source": "analytical", "cpu_core_seconds_per_second": 7.4e-6, "detail": "scan 1000 samples; x 1.0000/s", "multiplier": 1, "operator": "Scan", - "price_source": "analytical", "timing": "ingestion_time" }, "1": { - "cost_per_second": 0.464, - "cpu_core_seconds_per_second": 2e-6, - "detail": "build exact Rate accumulator in 5 tumbling windows of 60000 ms for 1000 groups: 1000 rows x (1 + depth 1); x 1.0000/s; keeps 3696000 bytes", + "cost_per_second": 1.483, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.000163, + "detail": "build exact Rate accumulator in tumbling windows of 60000 ms for 1000 groups: 1000 rows x 107 ops, 100 group visits, 16.7 states emitted; x 1.0000/s; keeps 10560000 bytes", "multiplier": 1, "operator": "SummaryTumblingWindowChain", - "price_source": "analytical", "timing": "ingestion_time" }, "2": { - "cost_per_second": 0.00508342585, - "cpu_core_seconds_per_second": 5.08333333333e-6, - "detail": "merge 5000 states into 1000; x 1.0167/s; holds 56000 bytes for 0.0 ms", + "cost_per_second": 0.055916667785, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.0000559166666667, + "detail": "time range 300s: 5 tumbling windows of states; x 1.0167/s; holds 160 bytes for 0.055 ms", "multiplier": 1, - "operator": "SummaryMerge", - "price_source": "analytical", + "operator": "TimeRange", "timing": "query_time" }, "3": { - "cost_per_second": 0.00101666666667, - "cpu_core_seconds_per_second": 1.01666666667e-6, - "detail": "finalize 1000 accumulators; x 1.0167/s", + "cost_per_second": 2.89755795, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.0028975, + "detail": "merge 5000 states into 1000; x 1.0167/s; holds 160000 bytes for 2.850 ms", "multiplier": 1, - "operator": "FinalizeExactAccumulator", - "price_source": "analytical", + "operator": "SummaryMerge", "timing": "query_time" }, "4": { - "cost_per_second": 0.00101675918333, - "cpu_core_seconds_per_second": 1.01666666667e-6, - "detail": "build exact Sum accumulator into 1000 states: 1000 rows x depth 1; x 1.0167/s; holds 56000 bytes for 0.0 ms", + "cost_per_second": 2.54166666667, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.00254166666667, + "detail": "finalize 1000 accumulators; x 1.0167/s", "multiplier": 1, - "operator": "SummaryAgg", - "price_source": "analytical", + "operator": "FinalizeExactAccumulator", "timing": "query_time" }, "5": { - "cost_per_second": 0.00101666666667, - "cpu_core_seconds_per_second": 1.01666666667e-6, - "detail": "finalize 1000 accumulators; x 1.0167/s", + "cost_per_second": 2.7348470075, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.00273483333333, + "detail": "hash aggregate 1000 rows into 1000 groups: 20 ops per row, 1000 group visits; x 1.0167/s; holds 40000 bytes for 2.690 ms", "multiplier": 1, - "operator": "FinalizeExactAccumulator", - "price_source": "analytical", + "operator": "Aggregate", "timing": "query_time" }, "6": { - "cost_per_second": 0.00203339941667, - "cpu_core_seconds_per_second": 2.03333333333e-6, - "detail": "hash aggregate 1000 rows into 1000 groups; x 1.0167/s; holds 40000 bytes for 0.0 ms", + "cost_per_second": 2.7348470075, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.00273483333333, + "detail": "hash aggregate 1000 rows into 1000 groups: 20 ops per row, 1000 group visits; x 1.0167/s; holds 40000 bytes for 2.690 ms", "multiplier": 1, "operator": "Aggregate", - "price_source": "analytical", "timing": "query_time" }, "7": { - "cost_per_second": 0.00101666667988, + "cost_per_second": 0.00101666666768, + "cost_source": "analytical", "cpu_core_seconds_per_second": 1.01666666667e-6, - "detail": "operator: pass 1000 rows; x 1.0167/s; holds 8 bytes for 0.0 ms", + "detail": "operator: pass 1000 rows; x 1.0167/s; holds 8 bytes for 0.001 ms", "multiplier": 1, "operator": "PromqlSubquery", - "price_source": "analytical", "timing": "query_time" }, "8": { - "cost_per_second": 0.00101675918333, - "cpu_core_seconds_per_second": 1.01666666667e-6, - "detail": "build exact Max accumulator into 1000 states: 1000 rows x depth 1; x 1.0167/s; holds 56000 bytes for 0.0 ms", - "multiplier": 1, - "operator": "SummaryAgg", - "price_source": "analytical", - "timing": "query_time" - }, - "9": { - "cost_per_second": 0.00101666666667, - "cpu_core_seconds_per_second": 1.01666666667e-6, - "detail": "finalize 1000 accumulators; x 1.0167/s", + "cost_per_second": 2.7348470075, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.00273483333333, + "detail": "hash aggregate 1000 rows into 1000 groups: 20 ops per row, 1000 group visits; x 1.0167/s; holds 40000 bytes for 2.690 ms", "multiplier": 1, - "operator": "FinalizeExactAccumulator", - "price_source": "analytical", + "operator": "Aggregate", "timing": "query_time" } }, "source": "analytical-cost-v2 (illustrative statistics)", "usage": { - "average_memory_bytes": 3696002.74917, - "cpu_ops_per_second": 16216.6666667, + "average_memory_bytes": 10560791.789, + "cpu_ops_per_second": 13864600.0, "disk_bytes": 0, - "latency_per_query": [ - { - "compaction_ms": 0.0, - "query_ms": 0.013 - } - ], "peak_cpu_cores": 2, - "peak_memory_bytes": 3904072, + "peak_memory_bytes": 10840232, "scanned_bytes_per_second": 64000.0 } }, @@ -144,40 +124,35 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "le", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -190,7 +165,7 @@ }, "producer": 0, "role": "Input", - "window": "RequiresAlignedPanePhaseOrExactBoundaryResidual" + "window": "NotApplicable" }, { "consumer": 2, @@ -206,9 +181,8 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { @@ -218,39 +192,37 @@ ] }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "le", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 4, - 0 + 0, + 2, + 3, + 4 ] ] }, @@ -272,9 +244,8 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { @@ -284,39 +255,37 @@ ] }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "le", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 4, - 0 + 0, + 2, + 3, + 4 ] ] }, @@ -327,7 +296,7 @@ { "consumer": 4, "data_state": { - "primitive": "Raw", + "primitive": "SummaryState", "timing": "query_time" }, "grouping": "NotApplicable", @@ -339,40 +308,38 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { - "Plain": "float64" + "ExactAggregate": [ + "Rate", + "Rate" + ] }, - "name": "value", - "nullable": false, - "table": null + "name": "state", + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "le", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -390,36 +357,53 @@ { "consumer": 5, "data_state": { - "primitive": "SummaryState", + "primitive": "Raw", "timing": "query_time" }, "grouping": "NotApplicable", "intermediate_schema": { "closed": true, "fields": [ + { + "dtype": { + "Plain": "timestamp" + }, + "name": "ts", + "nullable": false + }, + { + "dtype": { + "Plain": "float64" + }, + "name": "value", + "nullable": false + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "job", + "nullable": true + }, { "dtype": { "Plain": "utf8" }, "name": "le", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { - "ExactAggregate": [ - "Sum", - "Sum" - ] + "Plain": "utf8" }, - "name": "state", - "nullable": false, - "table": null + "name": "$promql_series_identity", + "nullable": false } ], - "time_index": null, + "time_index": 0, "unique_keys": [ [ + 4, 0 ] ] @@ -443,16 +427,14 @@ "Plain": "utf8" }, "name": "le", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "float64" }, "name": "sum", - "nullable": false, - "table": null + "nullable": false } ], "time_index": null, @@ -481,8 +463,7 @@ "Plain": "float64" }, "name": "histogram_quantile", - "nullable": false, - "table": null + "nullable": false } ], "time_index": null, @@ -507,8 +488,7 @@ "Plain": "float64" }, "name": "histogram_quantile", - "nullable": false, - "table": null + "nullable": false } ], "time_index": null, @@ -517,57 +497,10 @@ "producer": 7, "role": "Input", "window": "NotApplicable" - }, - { - "consumer": 9, - "data_state": { - "primitive": "SummaryState", - "timing": "query_time" - }, - "grouping": "NotApplicable", - "intermediate_schema": { - "closed": false, - "fields": [ - { - "dtype": { - "ExactAggregate": [ - "Max", - "Max" - ] - }, - "name": "state", - "nullable": false, - "table": null - } - ], - "time_index": null, - "unique_keys": [] - }, - "producer": 8, - "role": "Input", - "window": "NotApplicable" } ], "nodes": [ { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact over exact inputs" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 0, "lifetime": "None", "output_schema": { @@ -578,40 +511,35 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "le", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -653,40 +581,35 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "le", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -707,7 +630,6 @@ } }, { - "guarantee": null, "id": 1, "lifetime": { "AcrossEvaluations": "Memory" @@ -719,9 +641,8 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { @@ -731,39 +652,37 @@ ] }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "le", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 4, - 0 + 0, + 2, + 3, + 4 ] ] }, @@ -775,14 +694,7 @@ "ASAP": { "SummaryTumblingWindowChain": { "child": 0, - "family": { - "ExactAggregate": [ - "Rate", - "Rate" - ] - }, "filter": null, - "grouping": "PerSubpopulationInstance", "input": { "item": null, "weight": { @@ -790,14 +702,46 @@ } }, "reduction": "PerEntity", - "tumbling_window_count": 5, + "summary_type": { + "ExactAggregate": [ + "Rate", + "Rate" + ] + }, "tumbling_window_width_ms": 60000 } } + }, + "retention_ms": 3900000, + "view_coverage": { + "definition_id": "be9c29c7ba723e7271224b1879a6c7b4", + "selection": [ + { + "columns": [ + { + "column": { + "column": "job", + "scan_ordinal": 0, + "source": { + "TimeSeries": { + "metric": "http_request_duration_seconds_bucket" + } + } + }, + "constraint": { + "in": [ + { + "Utf8": "user-service" + } + ] + } + } + ] + } + ] } }, { - "guarantee": null, "id": 2, "lifetime": "None", "output_schema": { @@ -807,9 +751,8 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { @@ -819,39 +762,37 @@ ] }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "le", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 4, - 0 + 0, + 2, + 3, + 4 ] ] }, @@ -860,36 +801,23 @@ "timing": "query_time" }, "payload": { - "ASAP": { - "SummaryMerge": { - "children": [ - 1 - ] + "NonASAP": { + "TimeRange": { + "child": 1, + "kind": "range", + "range": { + "nanos": 0, + "secs": 300 + } } } } }, { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact accumulator over exact input" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 3, - "lifetime": "None", + "lifetime": { + "OneEvaluation": "Memory" + }, "output_schema": { "closed": true, "fields": [ @@ -898,40 +826,38 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { - "Plain": "float64" + "ExactAggregate": [ + "Rate", + "Rate" + ] }, - "name": "value", - "nullable": false, - "table": null + "name": "state", + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "le", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -943,103 +869,86 @@ ] }, "output_state": { - "primitive": "Raw", + "primitive": "SummaryState", "timing": "query_time" }, "payload": { "ASAP": { - "FinalizeExactAccumulator": { - "child": 2 + "SummaryMerge": { + "children": [ + 2 + ] } } } }, { - "guarantee": null, "id": 4, "lifetime": "None", "output_schema": { "closed": true, "fields": [ + { + "dtype": { + "Plain": "timestamp" + }, + "name": "ts", + "nullable": false + }, + { + "dtype": { + "Plain": "float64" + }, + "name": "value", + "nullable": false + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "job", + "nullable": true + }, { "dtype": { "Plain": "utf8" }, "name": "le", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { - "ExactAggregate": [ - "Sum", - "Sum" - ] + "Plain": "utf8" }, - "name": "state", - "nullable": false, - "table": null + "name": "$promql_series_identity", + "nullable": false } ], - "time_index": null, + "time_index": 0, "unique_keys": [ [ + 4, 0 ] ] }, "output_state": { - "primitive": "SummaryState", + "primitive": "Raw", "timing": "query_time" }, "payload": { "ASAP": { - "SummaryAgg": { - "child": 3, - "family": { - "ExactAggregate": [ - "Sum", - "Sum" - ] - }, - "filter": null, - "grouping": "PerSubpopulationInstance", - "input": { - "item": null, - "weight": { - "Column": "SampleValue" - } - }, - "reduction": { - "Reduce": [ - 3 - ] - } + "FinalizeExactAccumulator": { + "child": 3 } } } }, { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact accumulator over exact input" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 5, - "lifetime": "None", + "lifetime": { + "OneEvaluation": "Memory" + }, "output_schema": { "closed": true, "fields": [ @@ -1048,16 +957,14 @@ "Plain": "utf8" }, "name": "le", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "float64" }, "name": "sum", - "nullable": false, - "table": null + "nullable": false } ], "time_index": null, @@ -1072,34 +979,34 @@ "timing": "query_time" }, "payload": { - "ASAP": { - "FinalizeExactAccumulator": { - "child": 4 + "NonASAP": { + "Aggregate": { + "child": 4, + "filters": [], + "having": null, + "measures": [ + { + "col": null, + "kind": "sum" + } + ], + "output_names": [ + "" + ], + "reduction": { + "Reduce": [ + 3 + ] + } } } } }, { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact over exact inputs" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 6, - "lifetime": "None", + "lifetime": { + "OneEvaluation": "Memory" + }, "output_schema": { "closed": false, "fields": [ @@ -1108,8 +1015,7 @@ "Plain": "float64" }, "name": "histogram_quantile", - "nullable": false, - "table": null + "nullable": false } ], "time_index": null, @@ -1147,24 +1053,6 @@ } }, { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact over exact inputs" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 7, "lifetime": "None", "output_schema": { @@ -1175,8 +1063,7 @@ "Plain": "float64" }, "name": "histogram_quantile", - "nullable": false, - "table": null + "nullable": false } ], "time_index": null, @@ -1203,75 +1090,10 @@ } }, { - "guarantee": null, "id": 8, - "lifetime": "None", - "output_schema": { - "closed": false, - "fields": [ - { - "dtype": { - "ExactAggregate": [ - "Max", - "Max" - ] - }, - "name": "state", - "nullable": false, - "table": null - } - ], - "time_index": null, - "unique_keys": [] - }, - "output_state": { - "primitive": "SummaryState", - "timing": "query_time" - }, - "payload": { - "ASAP": { - "SummaryAgg": { - "child": 7, - "family": { - "ExactAggregate": [ - "Max", - "Max" - ] - }, - "filter": null, - "grouping": "PerSubpopulationInstance", - "input": { - "item": null, - "weight": { - "Column": "SampleValue" - } - }, - "reduction": "PerEntity" - } - } - } - }, - { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact accumulator over exact input" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } + "lifetime": { + "OneEvaluation": "Memory" }, - "id": 9, - "lifetime": "None", "output_schema": { "closed": false, "fields": [ @@ -1280,8 +1102,7 @@ "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false } ], "time_index": null, @@ -1292,22 +1113,33 @@ "timing": "query_time" }, "payload": { - "ASAP": { - "FinalizeExactAccumulator": { - "child": 8 + "NonASAP": { + "Aggregate": { + "child": 7, + "filters": [], + "having": null, + "measures": [ + { + "col": null, + "kind": "max" + } + ], + "output_names": [ + "" + ], + "reduction": "PerEntity" } } } } ], "roots": [ - 9 + 8 ], "scalar_roots": [] - }, - "display_name": "Q1 Max acc + Sum acc + Rate acc ×5 tumbling windows · ingestion time: Rate acc for Q1, 5 × 1m tumbling windows" + } }, - "tumbling_window_retention_ms": { - "1": 3900000 - } + "warnings": [ + "assumed data fact: the data arrival declares no max_out_of_order_ms, so the samples of source http_request_duration_seconds_bucket that an ingestion-time rate, increase or irate state reads are taken to arrive in timestamp order; if this does not hold, answers may be wrong by an amount no accuracy target bounds (a late sample can be read as a counter reset), or ingestion may fail" + ] } diff --git a/crates/devtools/tests/corpus_snapshots/o11y_bench/q11.json b/crates/devtools/tests/corpus_snapshots/o11y_bench/q11.json index 657fc6322..779c7b5d5 100644 --- a/crates/devtools/tests/corpus_snapshots/o11y_bench/q11.json +++ b/crates/devtools/tests/corpus_snapshots/o11y_bench/q11.json @@ -1,102 +1,73 @@ { "accuracy_guarantees": "enforced", "materialized_views": [], - "materialized_views_error": "raw-data window of node 1 is unsupported: a subquery or a TimeShift above the range", - "proven_optimal": true, + "materialized_views_error": "the executor cannot compile the plan's query run: invalid DAG: node 3: invalid DAG: value operation has no native implementation", "query": "max_over_time(sum(process_resident_memory_bytes)[6h:1m])", "selected": { "accepted": [ { "accepted_metric": null, - "accepted_regardless_of_guarantee": false, - "guarantee_check_rejection": null, "sketches": [], - "summary": "metric not set; accepted exact", - "target_metric": null + "summary": "accepted exact" } ], "cost": { - "cost_per_second": 0.00630740014507, - "cost_unit": "cost_per_second", - "cpu_core_seconds_per_second": 6.3074e-6, + "cost_per_second": 0.0249703500958, + "cpu_core_seconds_per_second": 0.00002497035, "per_node": { "0": { - "cost_per_second": 0.00427000002523, + "cost_per_second": 0.00427000001708, + "cost_source": "analytical", "cpu_core_seconds_per_second": 4.27e-6, - "detail": "scan 1000 samples; x 1.0167/s; keeps 32000 bytes of raw data on disk; holds 32 bytes for 0.0 ms", + "detail": "scan 1000 samples; x 1.0167/s; holds 32 bytes for 0.004 ms; keeps 115232000 bytes of raw data on disk", "multiplier": 1, "operator": "Scan", - "price_source": "analytical", "timing": "query_time" }, "1": { - "cost_per_second": 0.0010166666919, - "cpu_core_seconds_per_second": 1.01666666667e-6, - "detail": "time range 1s: keep 1000 of 1000 rows; x 1.0167/s; holds 32 bytes for 0.0 ms", + "cost_per_second": 0.0111833333781, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.0000111833333333, + "detail": "time range 1s: keep 1000 of 1000 rows; x 1.0167/s; holds 32 bytes for 0.011 ms", "multiplier": 1, "operator": "TimeRange", - "price_source": "analytical", "timing": "query_time" }, "2": { - "cost_per_second": 0.00101666671082, - "cpu_core_seconds_per_second": 1.01666666667e-6, - "detail": "build exact Sum accumulator into 1 states: 1000 rows x depth 1; x 1.0167/s; holds 56 bytes for 0.0 ms", + "cost_per_second": 0.00678116668701, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 6.78116666667e-6, + "detail": "hash aggregate 1000 rows into 1 groups: 4 ops per row, 1 group visits; x 1.0167/s; holds 24 bytes for 0.007 ms", "multiplier": 1, - "operator": "SummaryAgg", - "price_source": "analytical", + "operator": "Aggregate", "timing": "query_time" }, "3": { - "cost_per_second": 1.01666666667e-6, + "cost_per_second": 1.01666666768e-6, + "cost_source": "analytical", "cpu_core_seconds_per_second": 1.01666666667e-9, - "detail": "finalize 1 accumulators; x 1.0167/s", - "multiplier": 1, - "operator": "FinalizeExactAccumulator", - "price_source": "analytical", - "timing": "query_time" - }, - "4": { - "cost_per_second": 1.01667297407e-6, - "cpu_core_seconds_per_second": 1.01666666667e-9, - "detail": "operator: pass 1 rows; x 1.0167/s; holds 8 bytes for 0.0 ms", + "detail": "operator: pass 1 rows; x 1.0167/s; holds 8 bytes for 0.000 ms", "multiplier": 1, "operator": "PromqlSubquery", - "price_source": "analytical", "timing": "query_time" }, - "5": { - "cost_per_second": 1.01671081847e-6, - "cpu_core_seconds_per_second": 1.01666666667e-9, - "detail": "build exact Max accumulator into 1 states: 1 rows x depth 1; x 1.0167/s; holds 56 bytes for 0.0 ms", - "multiplier": 1, - "operator": "SummaryAgg", - "price_source": "analytical", - "timing": "query_time" - }, - "6": { - "cost_per_second": 1.01666666667e-6, - "cpu_core_seconds_per_second": 1.01666666667e-9, - "detail": "finalize 1 accumulators; x 1.0167/s", + "4": { + "cost_per_second": 0.00273483334701, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 2.73483333333e-6, + "detail": "hash aggregate 1 rows into 1 groups: 20 ops per row, 1 group visits; x 1.0167/s; holds 40 bytes for 0.003 ms", "multiplier": 1, - "operator": "FinalizeExactAccumulator", - "price_source": "analytical", + "operator": "Aggregate", "timing": "query_time" } }, "source": "analytical-cost-v2 (illustrative statistics)", "usage": { - "average_memory_bytes": 0.0011605616, - "cpu_ops_per_second": 3054.06666667, - "disk_bytes": 32000, - "latency_per_query": [ - { - "compaction_ms": 0.0, - "query_ms": 0.006204 - } - ], + "average_memory_bytes": 0.000766656133333, + "cpu_ops_per_second": 21717.0166667, + "disk_bytes": 115232000, "peak_cpu_cores": 1, - "peak_memory_bytes": 184, + "peak_memory_bytes": 136, "scanned_bytes_per_second": 32533.3333333 } }, @@ -117,24 +88,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -164,24 +132,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -198,35 +163,6 @@ }, { "consumer": 3, - "data_state": { - "primitive": "SummaryState", - "timing": "query_time" - }, - "grouping": "NotApplicable", - "intermediate_schema": { - "closed": true, - "fields": [ - { - "dtype": { - "ExactAggregate": [ - "Sum", - "Sum" - ] - }, - "name": "state", - "nullable": false, - "table": null - } - ], - "time_index": null, - "unique_keys": [] - }, - "producer": 2, - "role": "Input", - "window": "NotApplicable" - }, - { - "consumer": 4, "data_state": { "primitive": "Raw", "timing": "query_time" @@ -240,19 +176,18 @@ "Plain": "float64" }, "name": "sum", - "nullable": false, - "table": null + "nullable": false } ], "time_index": null, "unique_keys": [] }, - "producer": 3, + "producer": 2, "role": "Input", "window": "NotApplicable" }, { - "consumer": 5, + "consumer": 4, "data_state": { "primitive": "Raw", "timing": "query_time" @@ -266,67 +201,19 @@ "Plain": "float64" }, "name": "sum", - "nullable": false, - "table": null - } - ], - "time_index": null, - "unique_keys": [] - }, - "producer": 4, - "role": "Input", - "window": "NotApplicable" - }, - { - "consumer": 6, - "data_state": { - "primitive": "SummaryState", - "timing": "query_time" - }, - "grouping": "NotApplicable", - "intermediate_schema": { - "closed": true, - "fields": [ - { - "dtype": { - "ExactAggregate": [ - "Max", - "Max" - ] - }, - "name": "state", - "nullable": false, - "table": null + "nullable": false } ], "time_index": null, "unique_keys": [] }, - "producer": 5, + "producer": 3, "role": "Input", "window": "NotApplicable" } ], "nodes": [ { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact over exact inputs" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 0, "lifetime": "None", "output_schema": { @@ -337,24 +224,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -381,24 +265,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -416,27 +297,10 @@ } } } - } + }, + "retention_ms": 3601000 }, { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact over exact inputs" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 1, "lifetime": "None", "output_schema": { @@ -447,24 +311,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -493,49 +354,43 @@ } }, { - "guarantee": null, "id": 2, - "lifetime": "None", + "lifetime": { + "OneEvaluation": "Memory" + }, "output_schema": { "closed": true, "fields": [ { "dtype": { - "ExactAggregate": [ - "Sum", - "Sum" - ] + "Plain": "float64" }, - "name": "state", - "nullable": false, - "table": null + "name": "sum", + "nullable": false } ], "time_index": null, "unique_keys": [] }, "output_state": { - "primitive": "SummaryState", + "primitive": "Raw", "timing": "query_time" }, "payload": { - "ASAP": { - "SummaryAgg": { + "NonASAP": { + "Aggregate": { "child": 1, - "family": { - "ExactAggregate": [ - "Sum", - "Sum" - ] - }, - "filter": null, - "grouping": "PerSubpopulationInstance", - "input": { - "item": null, - "weight": { - "Column": "SampleValue" + "filters": [], + "having": null, + "measures": [ + { + "col": null, + "kind": "sum" } - }, + ], + "output_names": [ + "" + ], "reduction": { "Reduce": [] } @@ -544,74 +399,10 @@ } }, { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact accumulator over exact input" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 3, - "lifetime": "None", - "output_schema": { - "closed": true, - "fields": [ - { - "dtype": { - "Plain": "float64" - }, - "name": "sum", - "nullable": false, - "table": null - } - ], - "time_index": null, - "unique_keys": [] - }, - "output_state": { - "primitive": "Raw", - "timing": "query_time" - }, - "payload": { - "ASAP": { - "FinalizeExactAccumulator": { - "child": 2 - } - } - } - }, - { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact accumulator over exact input" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } + "lifetime": { + "OneEvaluation": "Memory" }, - "id": 4, - "lifetime": "None", "output_schema": { "closed": true, "fields": [ @@ -620,8 +411,7 @@ "Plain": "float64" }, "name": "sum", - "nullable": false, - "table": null + "nullable": false } ], "time_index": null, @@ -634,7 +424,7 @@ "payload": { "NonASAP": { "PromqlSubquery": { - "child": 3, + "child": 2, "range": { "nanos": 0, "secs": 21600 @@ -648,75 +438,10 @@ } }, { - "guarantee": null, - "id": 5, - "lifetime": "None", - "output_schema": { - "closed": true, - "fields": [ - { - "dtype": { - "ExactAggregate": [ - "Max", - "Max" - ] - }, - "name": "state", - "nullable": false, - "table": null - } - ], - "time_index": null, - "unique_keys": [] - }, - "output_state": { - "primitive": "SummaryState", - "timing": "query_time" - }, - "payload": { - "ASAP": { - "SummaryAgg": { - "child": 4, - "family": { - "ExactAggregate": [ - "Max", - "Max" - ] - }, - "filter": null, - "grouping": "PerSubpopulationInstance", - "input": { - "item": null, - "weight": { - "Column": "SampleValue" - } - }, - "reduction": "PerEntity" - } - } - } - }, - { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact accumulator over exact input" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } + "id": 4, + "lifetime": { + "OneEvaluation": "Memory" }, - "id": 6, - "lifetime": "None", "output_schema": { "closed": true, "fields": [ @@ -725,8 +450,7 @@ "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false } ], "time_index": null, @@ -737,20 +461,30 @@ "timing": "query_time" }, "payload": { - "ASAP": { - "FinalizeExactAccumulator": { - "child": 5 + "NonASAP": { + "Aggregate": { + "child": 3, + "filters": [], + "having": null, + "measures": [ + { + "col": null, + "kind": "max" + } + ], + "output_names": [ + "" + ], + "reduction": "PerEntity" } } } } ], "roots": [ - 6 + 4 ], "scalar_roots": [] - }, - "display_name": "Q1 Max acc + Sum acc" - }, - "tumbling_window_retention_ms": {} + } + } } diff --git a/crates/devtools/tests/corpus_snapshots/o11y_bench/q12.json b/crates/devtools/tests/corpus_snapshots/o11y_bench/q12.json index 8f1b22d92..d7a836ffd 100644 --- a/crates/devtools/tests/corpus_snapshots/o11y_bench/q12.json +++ b/crates/devtools/tests/corpus_snapshots/o11y_bench/q12.json @@ -1,54 +1,43 @@ { "accuracy_guarantees": "enforced", "materialized_views": [], - "proven_optimal": true, "query": "service_cache_refresh_lag_seconds{job=\"user-service\"}", "selected": { "accepted": [ { "accepted_metric": null, - "accepted_regardless_of_guarantee": false, - "guarantee_check_rejection": null, "sketches": [], - "summary": "metric not set; accepted exact", - "target_metric": null + "summary": "accepted exact" } ], "cost": { - "cost_per_second": 0.00691333341629, - "cost_unit": "cost_per_second", - "cpu_core_seconds_per_second": 6.91333333333e-6, + "cost_per_second": 0.0170800001025, + "cpu_core_seconds_per_second": 0.00001708, "per_node": { "0": { - "cost_per_second": 0.00589666670815, + "cost_per_second": 0.00589666670205, + "cost_source": "analytical", "cpu_core_seconds_per_second": 5.89666666667e-6, - "detail": "scan 1000 samples; x 1.0167/s; keeps 48000 bytes of raw data on disk; holds 48 bytes for 0.0 ms", + "detail": "scan 1000 samples; x 1.0167/s; holds 48 bytes for 0.006 ms; keeps 172848000 bytes of raw data on disk", "multiplier": 1, "operator": "Scan", - "price_source": "analytical", "timing": "query_time" }, "1": { - "cost_per_second": 0.00101666670815, - "cpu_core_seconds_per_second": 1.01666666667e-6, - "detail": "time range 1s: keep 1000 of 1000 rows; x 1.0167/s; holds 48 bytes for 0.0 ms", + "cost_per_second": 0.0111833334004, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.0000111833333333, + "detail": "time range 1s: keep 1000 of 1000 rows; x 1.0167/s; holds 48 bytes for 0.011 ms", "multiplier": 1, "operator": "TimeRange", - "price_source": "analytical", "timing": "query_time" } }, "source": "analytical-cost-v2 (illustrative statistics)", "usage": { - "average_memory_bytes": 0.00066368, - "cpu_ops_per_second": 2033.33333333, - "disk_bytes": 48000, - "latency_per_query": [ - { - "compaction_ms": 0.0, - "query_ms": 0.0068 - } - ], + "average_memory_bytes": 0.00081984, + "cpu_ops_per_second": 12200.0, + "disk_bytes": 172848000, "peak_cpu_cores": 1, "peak_memory_bytes": 96, "scanned_bytes_per_second": 48800.0 @@ -71,32 +60,28 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -114,24 +99,6 @@ ], "nodes": [ { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact over exact inputs" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 0, "lifetime": "None", "output_schema": { @@ -142,32 +109,28 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -209,32 +172,28 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -252,27 +211,10 @@ } } } - } + }, + "retention_ms": 3601000 }, { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact over exact inputs" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 1, "lifetime": "None", "output_schema": { @@ -283,32 +225,28 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -341,8 +279,6 @@ 1 ], "scalar_roots": [] - }, - "display_name": "Q1 exact" - }, - "tumbling_window_retention_ms": {} + } + } } diff --git a/crates/devtools/tests/corpus_snapshots/o11y_bench/q18.json b/crates/devtools/tests/corpus_snapshots/o11y_bench/q18.json index d2401d112..58b38367c 100644 --- a/crates/devtools/tests/corpus_snapshots/o11y_bench/q18.json +++ b/crates/devtools/tests/corpus_snapshots/o11y_bench/q18.json @@ -3,6098 +3,19 @@ "materialized_views": [ { "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", + "definition_id": "c861201688f76930d0a62614d5e30961", "selection": [ { "columns": [ { "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -21660000 - }, - { - "included": -21600000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 21600000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 0, - "node_id": 3, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 2 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -21720000 - }, - { - "included": -21660000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 21660000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 1, - "node_id": 6, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 2 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -21780000 - }, - { - "included": -21720000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 21720000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 2, - "node_id": 9, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 2 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -21840000 - }, - { - "included": -21780000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 21780000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 3, - "node_id": 12, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 2 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -21900000 - }, - { - "included": -21840000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 21840000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 4, - "node_id": 15, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 2 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -21960000 - }, - { - "included": -21900000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 21900000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 5, - "node_id": 18, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 2 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -22020000 - }, - { - "included": -21960000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 21960000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 6, - "node_id": 21, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 2 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -22080000 - }, - { - "included": -22020000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 22020000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 7, - "node_id": 24, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 2 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -22140000 - }, - { - "included": -22080000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 22080000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 8, - "node_id": 27, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 2 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -22200000 - }, - { - "included": -22140000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 22140000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 9, - "node_id": 30, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 2 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -22260000 - }, - { - "included": -22200000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 22200000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 10, - "node_id": 33, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 2 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -22320000 - }, - { - "included": -22260000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 22260000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 11, - "node_id": 36, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 2 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -22380000 - }, - { - "included": -22320000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 22320000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 12, - "node_id": 39, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 2 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -22440000 - }, - { - "included": -22380000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 22380000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 13, - "node_id": 42, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 2 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -22500000 - }, - { - "included": -22440000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 22440000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 14, - "node_id": 45, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 2 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -22560000 - }, - { - "included": -22500000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 22500000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 15, - "node_id": 48, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 2 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -22620000 - }, - { - "included": -22560000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 22560000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 16, - "node_id": 51, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 2 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -22680000 - }, - { - "included": -22620000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 22620000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 17, - "node_id": 54, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 2 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -22740000 - }, - { - "included": -22680000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 22680000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 18, - "node_id": 57, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 2 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -22800000 - }, - { - "included": -22740000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 22740000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 19, - "node_id": 60, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 2 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -22860000 - }, - { - "included": -22800000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 22800000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 20, - "node_id": 63, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 2 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -22920000 - }, - { - "included": -22860000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 22860000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 21, - "node_id": 66, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 2 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -22980000 - }, - { - "included": -22920000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 22920000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 22, - "node_id": 69, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 2 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -23040000 - }, - { - "included": -22980000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 22980000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 23, - "node_id": 72, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 2 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -23100000 - }, - { - "included": -23040000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 23040000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 24, - "node_id": 75, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 2 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -23160000 - }, - { - "included": -23100000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 23100000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 25, - "node_id": 78, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 2 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -23220000 - }, - { - "included": -23160000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 23160000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 26, - "node_id": 81, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 2 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -23280000 - }, - { - "included": -23220000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 23220000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 27, - "node_id": 84, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 2 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -23340000 - }, - { - "included": -23280000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 23280000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 28, - "node_id": 87, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 2 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -23400000 - }, - { - "included": -23340000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 23340000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 29, - "node_id": 90, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 2 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -23460000 - }, - { - "included": -23400000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 23400000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 30, - "node_id": 93, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 2 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -23520000 - }, - { - "included": -23460000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 23460000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 31, - "node_id": 96, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 2 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -23580000 - }, - { - "included": -23520000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 23520000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 32, - "node_id": 99, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 2 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -23640000 - }, - { - "included": -23580000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 23580000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 33, - "node_id": 102, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 2 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -23700000 - }, - { - "included": -23640000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 23640000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 34, - "node_id": 105, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 2 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -23760000 - }, - { - "included": -23700000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 23700000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 35, - "node_id": 108, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 2 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -23820000 - }, - { - "included": -23760000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 23760000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 36, - "node_id": 111, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 2 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -23880000 - }, - { - "included": -23820000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 23820000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 37, - "node_id": 114, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 2 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -23940000 - }, - { - "included": -23880000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 23880000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 38, - "node_id": 117, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 2 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -24000000 - }, - { - "included": -23940000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 23940000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 39, - "node_id": 120, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 2 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -24060000 - }, - { - "included": -24000000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 24000000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 40, - "node_id": 123, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 2 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -24120000 - }, - { - "included": -24060000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 24060000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 41, - "node_id": 126, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 2 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -24180000 - }, - { - "included": -24120000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 24120000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 42, - "node_id": 129, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 2 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -24240000 - }, - { - "included": -24180000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 24180000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 43, - "node_id": 132, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 2 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -24300000 - }, - { - "included": -24240000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 24240000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 44, - "node_id": 135, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 2 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -24360000 - }, - { - "included": -24300000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 24300000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 45, - "node_id": 138, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 2 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -24420000 - }, - { - "included": -24360000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 24360000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 46, - "node_id": 141, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 2 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -24480000 - }, - { - "included": -24420000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 24420000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 47, - "node_id": 144, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 2 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -24540000 - }, - { - "included": -24480000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 24480000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 48, - "node_id": 147, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 2 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -24600000 - }, - { - "included": -24540000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 24540000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 49, - "node_id": 150, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 2 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -24660000 - }, - { - "included": -24600000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 24600000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 50, - "node_id": 153, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 2 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -24720000 - }, - { - "included": -24660000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 24660000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 51, - "node_id": 156, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 2 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -24780000 - }, - { - "included": -24720000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 24720000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 52, - "node_id": 159, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 2 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -24840000 - }, - { - "included": -24780000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 24780000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 53, - "node_id": 162, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 2 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -24900000 - }, - { - "included": -24840000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 24840000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 54, - "node_id": 165, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 2 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -24960000 - }, - { - "included": -24900000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 24900000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 55, - "node_id": 168, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 2 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -25020000 - }, - { - "included": -24960000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 24960000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 56, - "node_id": 171, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 2 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -25080000 - }, - { - "included": -25020000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 25020000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 57, - "node_id": 174, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 2 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -25140000 - }, - { - "included": -25080000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 25080000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 58, - "node_id": 177, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 2 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -25200000 - }, - { - "included": -25140000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 25140000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 59, - "node_id": 180, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 2 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -21660000 - }, - { - "included": -21600000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 21600000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 0, - "node_id": 188, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 9 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -21720000 - }, - { - "included": -21660000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 21660000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 1, - "node_id": 191, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 9 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -21780000 - }, - { - "included": -21720000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 21720000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 2, - "node_id": 194, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 9 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -21840000 - }, - { - "included": -21780000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 21780000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 3, - "node_id": 197, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 9 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -21900000 - }, - { - "included": -21840000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 21840000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 4, - "node_id": 200, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 9 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -21960000 - }, - { - "included": -21900000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 21900000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 5, - "node_id": 203, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 9 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -22020000 - }, - { - "included": -21960000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 21960000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 6, - "node_id": 206, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 9 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -22080000 - }, - { - "included": -22020000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 22020000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 7, - "node_id": 209, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 9 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -22140000 - }, - { - "included": -22080000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 22080000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 8, - "node_id": 212, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 9 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -22200000 - }, - { - "included": -22140000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 22140000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 9, - "node_id": 215, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 9 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -22260000 - }, - { - "included": -22200000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 22200000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 10, - "node_id": 218, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 9 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -22320000 - }, - { - "included": -22260000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 22260000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 11, - "node_id": 221, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 9 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -22380000 - }, - { - "included": -22320000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 22320000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 12, - "node_id": 224, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 9 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -22440000 - }, - { - "included": -22380000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 22380000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 13, - "node_id": 227, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 9 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -22500000 - }, - { - "included": -22440000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 22440000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 14, - "node_id": 230, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 9 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -22560000 - }, - { - "included": -22500000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 22500000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 15, - "node_id": 233, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 9 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -22620000 - }, - { - "included": -22560000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 22560000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 16, - "node_id": 236, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 9 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -22680000 - }, - { - "included": -22620000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 22620000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 17, - "node_id": 239, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 9 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -22740000 - }, - { - "included": -22680000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 22680000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 18, - "node_id": 242, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 9 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -22800000 - }, - { - "included": -22740000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 22740000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 19, - "node_id": 245, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 9 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -22860000 - }, - { - "included": -22800000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 22800000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 20, - "node_id": 248, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 9 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -22920000 - }, - { - "included": -22860000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 22860000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 21, - "node_id": 251, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 9 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -22980000 - }, - { - "included": -22920000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 22920000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 22, - "node_id": 254, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 9 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -23040000 - }, - { - "included": -22980000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 22980000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 23, - "node_id": 257, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 9 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -23100000 - }, - { - "included": -23040000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 23040000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 24, - "node_id": 260, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 9 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -23160000 - }, - { - "included": -23100000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 23100000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 25, - "node_id": 263, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 9 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -23220000 - }, - { - "included": -23160000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 23160000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 26, - "node_id": 266, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 9 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -23280000 - }, - { - "included": -23220000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 23220000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 27, - "node_id": 269, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 9 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -23340000 - }, - { - "included": -23280000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 23280000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 28, - "node_id": 272, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 9 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -23400000 - }, - { - "included": -23340000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 23340000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 29, - "node_id": 275, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 9 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -23460000 - }, - { - "included": -23400000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 23400000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 30, - "node_id": 278, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 9 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -23520000 - }, - { - "included": -23460000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 23460000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 31, - "node_id": 281, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 9 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -23580000 - }, - { - "included": -23520000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 23520000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 32, - "node_id": 284, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 9 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -23640000 - }, - { - "included": -23580000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 23580000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 33, - "node_id": 287, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 9 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -23700000 - }, - { - "included": -23640000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 23640000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 34, - "node_id": 290, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 9 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -23760000 - }, - { - "included": -23700000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 23700000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 35, - "node_id": 293, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 9 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -23820000 - }, - { - "included": -23760000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 23760000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 36, - "node_id": 296, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 9 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -23880000 - }, - { - "included": -23820000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 23820000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 37, - "node_id": 299, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 9 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -23940000 - }, - { - "included": -23880000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 23880000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 38, - "node_id": 302, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 9 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -24000000 - }, - { - "included": -23940000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 23940000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 39, - "node_id": 305, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 9 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -24060000 - }, - { - "included": -24000000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 24000000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 40, - "node_id": 308, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 9 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -24120000 - }, - { - "included": -24060000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 24060000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 41, - "node_id": 311, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 9 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -24180000 - }, - { - "included": -24120000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 24120000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 42, - "node_id": 314, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 9 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -24240000 - }, - { - "included": -24180000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 24180000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 43, - "node_id": 317, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 9 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -24300000 - }, - { - "included": -24240000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 24240000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 44, - "node_id": 320, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 9 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -24360000 - }, - { - "included": -24300000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 24300000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 45, - "node_id": 323, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 9 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -24420000 - }, - { - "included": -24360000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 24360000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 46, - "node_id": 326, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 9 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -24480000 - }, - { - "included": -24420000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 24420000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 47, - "node_id": 329, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 9 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -24540000 - }, - { - "included": -24480000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 24480000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 48, - "node_id": 332, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 9 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -24600000 - }, - { - "included": -24540000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 24540000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 49, - "node_id": 335, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 9 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -24660000 - }, - { - "included": -24600000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 24600000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 50, - "node_id": 338, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 9 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -24720000 - }, - { - "included": -24660000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 24660000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 51, - "node_id": 341, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 9 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -24780000 - }, - { - "included": -24720000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 24720000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 52, - "node_id": 344, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 9 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -24840000 - }, - { - "included": -24780000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 24780000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 53, - "node_id": 347, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 9 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -24900000 - }, - { - "included": -24840000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 24840000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 54, - "node_id": 350, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 9 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -24960000 - }, - { - "included": -24900000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 24900000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 55, - "node_id": 353, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 9 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" + "column": "job", + "scan_ordinal": 0, + "source": { + "TimeSeries": { + "metric": "http_requests_total" } - ] - } - } - ], - "relative_time": [ - { - "excluded": -25020000 - }, - { - "included": -24960000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 24960000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 56, - "node_id": 356, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 9 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null + } }, "constraint": { "in": [ @@ -6104,101 +25,32 @@ ] } } - ], - "relative_time": [ - { - "excluded": -25080000 - }, - { - "included": -25020000 - } ] } ] }, "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 25020000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] + "materialized_views": [] }, - "installed_count": 60, - "installed_position": 57, - "node_id": 359, + "node_id": 1, "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 9 + "retention_ms": 28800000 }, { "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", + "definition_id": "87591b6875ec8f0172a29d8ebfa40327", "selection": [ { "columns": [ { "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" + "column": "job", + "scan_ordinal": 0, + "source": { + "TimeSeries": { + "metric": "http_requests_total" } - ] - } - } - ], - "relative_time": [ - { - "excluded": -25140000 - }, - { - "included": -25080000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 25080000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 58, - "node_id": 362, - "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 9 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null + } }, "constraint": { "in": [ @@ -6208,207 +60,174 @@ ] } } - ], - "relative_time": [ - { - "excluded": -25200000 - }, - { - "included": -25140000 - } ] } ] }, "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 25140000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] + "materialized_views": [] }, - "installed_count": 60, - "installed_position": 59, - "node_id": 365, + "node_id": 8, "refresh": "at_ingestion", - "retention_ms": 28800000, - "stage2_node_id": 9 + "retention_ms": 28800000 } ], - "proven_optimal": true, "query": "sum(increase(http_requests_total{job=\"payment-service\",status=~\"5..\"}[1h] offset 6h)) / sum(increase(http_requests_total{job=\"payment-service\"}[1h] offset 6h))", "selected": { "accepted": [ { "accepted_metric": null, - "accepted_regardless_of_guarantee": false, - "guarantee_check_rejection": null, "sketches": [], - "summary": "metric not set; accepted exact", - "target_metric": null + "summary": "accepted exact" } ], "cost": { - "cost_per_second": 6.87727148353, - "cost_unit": "cost_per_second", - "cpu_core_seconds_per_second": 0.000143269716667, + "cost_per_second": 69.9566171722, + "cpu_core_seconds_per_second": 0.05167776335, "per_node": { "0": { "cost_per_second": 0.0074, + "cost_source": "analytical", "cpu_core_seconds_per_second": 7.4e-6, "detail": "scan 1000 samples; x 1.0000/s", "multiplier": 1, "operator": "Scan", - "price_source": "analytical", "timing": "ingestion_time" }, "1": { - "cost_per_second": 0.0, - "cpu_core_seconds_per_second": 0.0, - "detail": "time shift: pass 1000 rows, free; x 1.0000/s", + "cost_per_second": 9.783, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.000163, + "detail": "build exact Increase accumulator in tumbling windows of 60000 ms for 1000 groups: 1000 rows x 107 ops, 100 group visits, 16.7 states emitted; x 1.0000/s; keeps 76960000 bytes", "multiplier": 1, - "operator": "TimeShift", - "price_source": "analytical", + "operator": "SummaryTumblingWindowChain", "timing": "ingestion_time" }, "10": { - "cost_per_second": 0.061000882488, - "cpu_core_seconds_per_second": 0.000061, - "detail": "merge 60000 states into 1000; x 1.0167/s; holds 56000 bytes for 0.1 ms", + "cost_per_second": 0.671000012078, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.000671, + "detail": "time range 3600s: 60 tumbling windows of states; x 1.0167/s; holds 144 bytes for 0.660 ms", "multiplier": 1, - "operator": "SummaryMerge", - "price_source": "analytical", + "operator": "TimeRange", "timing": "query_time" }, "11": { - "cost_per_second": 0.00101666666667, - "cpu_core_seconds_per_second": 1.01666666667e-6, - "detail": "finalize 1000 accumulators; x 1.0167/s", + "cost_per_second": 22.4687377633, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.0224683333333, + "detail": "merge 60000 states into 1000; x 1.0167/s; holds 144000 bytes for 22.100 ms", "multiplier": 1, - "operator": "FinalizeExactAccumulator", - "price_source": "analytical", + "operator": "SummaryMerge", "timing": "query_time" }, "12": { - "cost_per_second": 0.00101666754915, - "cpu_core_seconds_per_second": 1.01666666667e-6, - "detail": "build exact Sum accumulator into 1 states: 1000 rows x depth 1; x 1.0167/s; holds 56 bytes for 0.1 ms", + "cost_per_second": 2.54166666667, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.00254166666667, + "detail": "finalize 1000 accumulators; x 1.0167/s", "multiplier": 1, - "operator": "SummaryAgg", - "price_source": "analytical", + "operator": "FinalizeExactAccumulator", "timing": "query_time" }, "13": { - "cost_per_second": 1.01666666667e-6, - "cpu_core_seconds_per_second": 1.01666666667e-9, - "detail": "finalize 1 accumulators; x 1.0167/s", + "cost_per_second": 0.00678116668701, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 6.78116666667e-6, + "detail": "hash aggregate 1000 rows into 1 groups: 4 ops per row, 1 group visits; x 1.0167/s; holds 24 bytes for 0.007 ms", "multiplier": 1, - "operator": "FinalizeExactAccumulator", - "price_source": "analytical", + "operator": "Aggregate", "timing": "query_time" }, "14": { - "cost_per_second": 1.01679273638e-6, + "cost_per_second": 1.01666666768e-6, + "cost_source": "analytical", "cpu_core_seconds_per_second": 1.01666666667e-9, - "detail": "operator: pass 1 rows; x 1.0167/s; holds 8 bytes for 0.1 ms", + "detail": "operator: pass 1 rows; x 1.0167/s; holds 8 bytes for 0.000 ms", "multiplier": 1, "operator": "BinaryOp", - "price_source": "analytical", "timing": "query_time" }, "2": { - "cost_per_second": 3.369, - "cpu_core_seconds_per_second": 2e-6, - "detail": "build exact Increase accumulator in 60 tumbling windows of 60000 ms for 1000 groups: 1000 rows x (1 + depth 1); x 1.0000/s; keeps 26936000 bytes", + "cost_per_second": 0.0, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.0, + "detail": "time shift: pass 1000 rows, free; x 1.0167/s; holds 160000 bytes for 0.000 ms", "multiplier": 1, - "operator": "SummaryTumblingWindowChain", - "price_source": "analytical", - "timing": "ingestion_time" + "operator": "TimeShift", + "timing": "query_time" }, "3": { - "cost_per_second": 0.061000882488, - "cpu_core_seconds_per_second": 0.000061, - "detail": "merge 60000 states into 1000; x 1.0167/s; holds 56000 bytes for 0.1 ms", + "cost_per_second": 0.67100001342, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.000671, + "detail": "time range 3600s: 60 tumbling windows of states; x 1.0167/s; holds 160 bytes for 0.660 ms", "multiplier": 1, - "operator": "SummaryMerge", - "price_source": "analytical", + "operator": "TimeRange", "timing": "query_time" }, "4": { - "cost_per_second": 0.00101666666667, - "cpu_core_seconds_per_second": 1.01666666667e-6, - "detail": "finalize 1000 accumulators; x 1.0167/s", + "cost_per_second": 22.4687827, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.0224683333333, + "detail": "merge 60000 states into 1000; x 1.0167/s; holds 160000 bytes for 22.100 ms", "multiplier": 1, - "operator": "FinalizeExactAccumulator", - "price_source": "analytical", + "operator": "SummaryMerge", "timing": "query_time" }, "5": { - "cost_per_second": 0.00101666754915, - "cpu_core_seconds_per_second": 1.01666666667e-6, - "detail": "build exact Sum accumulator into 1 states: 1000 rows x depth 1; x 1.0167/s; holds 56 bytes for 0.1 ms", + "cost_per_second": 2.54166666667, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.00254166666667, + "detail": "finalize 1000 accumulators; x 1.0167/s", "multiplier": 1, - "operator": "SummaryAgg", - "price_source": "analytical", + "operator": "FinalizeExactAccumulator", "timing": "query_time" }, "6": { - "cost_per_second": 1.01666666667e-6, - "cpu_core_seconds_per_second": 1.01666666667e-9, - "detail": "finalize 1 accumulators; x 1.0167/s", + "cost_per_second": 0.00678116668701, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 6.78116666667e-6, + "detail": "hash aggregate 1000 rows into 1 groups: 4 ops per row, 1 group visits; x 1.0167/s; holds 24 bytes for 0.007 ms", "multiplier": 1, - "operator": "FinalizeExactAccumulator", - "price_source": "analytical", + "operator": "Aggregate", "timing": "query_time" }, "7": { "cost_per_second": 0.0058, + "cost_source": "analytical", "cpu_core_seconds_per_second": 5.8e-6, "detail": "scan 1000 samples; x 1.0000/s", "multiplier": 1, "operator": "Scan", - "price_source": "analytical", "timing": "ingestion_time" }, "8": { - "cost_per_second": 0.0, - "cpu_core_seconds_per_second": 0.0, - "detail": "time shift: pass 1000 rows, free; x 1.0000/s", + "cost_per_second": 8.784, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.000126, + "detail": "build exact Increase accumulator in tumbling windows of 60000 ms for 1000 groups: 1000 rows x 70 ops, 100 group visits, 16.7 states emitted; x 1.0000/s; keeps 69264000 bytes", "multiplier": 1, - "operator": "TimeShift", - "price_source": "analytical", + "operator": "SummaryTumblingWindowChain", "timing": "ingestion_time" }, "9": { - "cost_per_second": 3.369, - "cpu_core_seconds_per_second": 2e-6, - "detail": "build exact Increase accumulator in 60 tumbling windows of 60000 ms for 1000 groups: 1000 rows x (1 + depth 1); x 1.0000/s; keeps 26936000 bytes", + "cost_per_second": 0.0, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.0, + "detail": "time shift: pass 1000 rows, free; x 1.0167/s; holds 144000 bytes for 0.000 ms", "multiplier": 1, - "operator": "SummaryTumblingWindowChain", - "price_source": "analytical", - "timing": "ingestion_time" + "operator": "TimeShift", + "timing": "query_time" } }, "source": "analytical-cost-v2 (illustrative statistics)", "usage": { - "average_memory_bytes": 53872014.1349, - "cpu_ops_per_second": 132069.716667, + "average_memory_bytes": 146230830.578, + "cpu_ops_per_second": 51666563.35, "disk_bytes": 0, - "latency_per_query": [ - { - "compaction_ms": 0.0, - "query_ms": 0.124003 - } - ], "peak_cpu_cores": 2, - "peak_memory_bytes": 53984232, + "peak_memory_bytes": 146832472, "scanned_bytes_per_second": 112000.0 } }, @@ -6429,40 +248,35 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "status", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -6475,12 +289,12 @@ }, "producer": 0, "role": "Input", - "window": "RequiresAlignedPanePhaseOrExactBoundaryResidual" + "window": "NotApplicable" }, { "consumer": 2, "data_state": { - "primitive": "Raw", + "primitive": "SummaryState", "timing": "ingestion_time" }, "grouping": "NotApplicable", @@ -6491,60 +305,123 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { - "Plain": "float64" + "ExactAggregate": [ + "Increase", + "Increase" + ] }, - "name": "value", - "nullable": false, - "table": null + "name": "state", + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "status", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 4, - 0 + 0, + 2, + 3, + 4 ] ] }, "producer": 1, "role": "Input", - "window": "RequiresAlignedPanePhaseOrExactBoundaryResidual" + "window": "NotApplicable" + }, + { + "consumer": 3, + "data_state": { + "primitive": "SummaryState", + "timing": "query_time" + }, + "grouping": "NotApplicable", + "intermediate_schema": { + "closed": true, + "fields": [ + { + "dtype": { + "Plain": "timestamp" + }, + "name": "window_end", + "nullable": false + }, + { + "dtype": { + "ExactAggregate": [ + "Increase", + "Increase" + ] + }, + "name": "state", + "nullable": false + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "job", + "nullable": true + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "status", + "nullable": true + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false + } + ], + "time_index": 0, + "unique_keys": [ + [ + 0, + 2, + 3, + 4 + ] + ] + }, + "producer": 2, + "role": "Input", + "window": "NotApplicable" }, { - "consumer": 3, + "consumer": 4, "data_state": { "primitive": "SummaryState", - "timing": "ingestion_time" + "timing": "query_time" }, "grouping": "NotApplicable", "intermediate_schema": { @@ -6554,9 +431,8 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { @@ -6566,48 +442,46 @@ ] }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "status", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 4, - 0 + 0, + 2, + 3, + 4 ] ] }, - "producer": 2, + "producer": 3, "role": "Input", "window": "NotApplicable" }, { - "consumer": 4, + "consumer": 5, "data_state": { "primitive": "SummaryState", "timing": "query_time" @@ -6621,8 +495,7 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { @@ -6632,32 +505,28 @@ ] }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "status", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -6668,12 +537,12 @@ ] ] }, - "producer": 3, + "producer": 4, "role": "Input", "window": "NotApplicable" }, { - "consumer": 5, + "consumer": 6, "data_state": { "primitive": "Raw", "timing": "query_time" @@ -6687,40 +556,35 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "status", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -6731,35 +595,6 @@ ] ] }, - "producer": 4, - "role": "Input", - "window": "NotApplicable" - }, - { - "consumer": 6, - "data_state": { - "primitive": "SummaryState", - "timing": "query_time" - }, - "grouping": "NotApplicable", - "intermediate_schema": { - "closed": true, - "fields": [ - { - "dtype": { - "ExactAggregate": [ - "Sum", - "Sum" - ] - }, - "name": "state", - "nullable": false, - "table": null - } - ], - "time_index": null, - "unique_keys": [] - }, "producer": 5, "role": "Input", "window": "NotApplicable" @@ -6779,32 +614,28 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -6817,12 +648,12 @@ }, "producer": 7, "role": "Input", - "window": "RequiresAlignedPanePhaseOrExactBoundaryResidual" + "window": "NotApplicable" }, { "consumer": 9, "data_state": { - "primitive": "Raw", + "primitive": "SummaryState", "timing": "ingestion_time" }, "grouping": "NotApplicable", @@ -6833,52 +664,52 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { - "Plain": "float64" + "ExactAggregate": [ + "Increase", + "Increase" + ] }, - "name": "value", - "nullable": false, - "table": null + "name": "state", + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 3, - 0 + 0, + 2, + 3 ] ] }, "producer": 8, "role": "Input", - "window": "RequiresAlignedPanePhaseOrExactBoundaryResidual" + "window": "NotApplicable" }, { "consumer": 10, "data_state": { "primitive": "SummaryState", - "timing": "ingestion_time" + "timing": "query_time" }, "grouping": "NotApplicable", "intermediate_schema": { @@ -6888,9 +719,8 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { @@ -6900,31 +730,29 @@ ] }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 3, - 0 + 0, + 2, + 3 ] ] }, @@ -6946,9 +774,8 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { @@ -6958,31 +785,29 @@ ] }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 3, - 0 + 0, + 2, + 3 ] ] }, @@ -6993,7 +818,7 @@ { "consumer": 12, "data_state": { - "primitive": "Raw", + "primitive": "SummaryState", "timing": "query_time" }, "grouping": "NotApplicable", @@ -7005,32 +830,31 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { - "Plain": "float64" + "ExactAggregate": [ + "Increase", + "Increase" + ] }, - "name": "value", - "nullable": false, - "table": null + "name": "state", + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -7048,7 +872,7 @@ { "consumer": 13, "data_state": { - "primitive": "SummaryState", + "primitive": "Raw", "timing": "query_time" }, "grouping": "NotApplicable", @@ -7057,18 +881,40 @@ "fields": [ { "dtype": { - "ExactAggregate": [ - "Sum", - "Sum" - ] + "Plain": "timestamp" }, - "name": "state", - "nullable": false, - "table": null + "name": "ts", + "nullable": false + }, + { + "dtype": { + "Plain": "float64" + }, + "name": "value", + "nullable": false + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "job", + "nullable": true + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false } ], - "time_index": null, - "unique_keys": [] + "time_index": 0, + "unique_keys": [ + [ + 3, + 0 + ] + ] }, "producer": 12, "role": "Input", @@ -7089,8 +935,7 @@ "Plain": "float64" }, "name": "sum", - "nullable": false, - "table": null + "nullable": false } ], "time_index": null, @@ -7115,8 +960,7 @@ "Plain": "float64" }, "name": "sum", - "nullable": false, - "table": null + "nullable": false } ], "time_index": null, @@ -7129,24 +973,6 @@ ], "nodes": [ { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact over exact inputs" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 0, "lifetime": "None", "output_schema": { @@ -7157,40 +983,35 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "status", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -7246,40 +1067,35 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "status", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -7300,26 +1116,10 @@ } }, { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact over exact inputs" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 1, - "lifetime": "None", + "lifetime": { + "AcrossEvaluations": "Memory" + }, "output_schema": { "closed": true, "fields": [ @@ -7327,72 +1127,110 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { - "Plain": "float64" + "ExactAggregate": [ + "Increase", + "Increase" + ] }, - "name": "value", - "nullable": false, - "table": null + "name": "state", + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "status", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 4, - 0 + 0, + 2, + 3, + 4 ] ] }, "output_state": { - "primitive": "Raw", + "primitive": "SummaryState", "timing": "ingestion_time" }, "payload": { - "NonASAP": { - "TimeShift": { + "ASAP": { + "SummaryTumblingWindowChain": { "child": 0, - "shift": { - "at": null, - "offset_ms": 21600000 - } + "filter": null, + "input": { + "item": null, + "weight": { + "Column": "SampleValue" + } + }, + "reduction": "PerEntity", + "summary_type": { + "ExactAggregate": [ + "Increase", + "Increase" + ] + }, + "tumbling_window_width_ms": 60000 } } + }, + "retention_ms": 28800000, + "view_coverage": { + "definition_id": "c861201688f76930d0a62614d5e30961", + "selection": [ + { + "columns": [ + { + "column": { + "column": "job", + "scan_ordinal": 0, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + } + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ] + } + ] } }, { - "guarantee": null, "id": 2, "lifetime": { - "AcrossEvaluations": "Memory" + "OneEvaluation": "Memory" }, "output_schema": { "closed": true, @@ -7401,9 +1239,8 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { @@ -7413,73 +1250,57 @@ ] }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "status", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 4, - 0 + 0, + 2, + 3, + 4 ] ] }, "output_state": { "primitive": "SummaryState", - "timing": "ingestion_time" + "timing": "query_time" }, "payload": { - "ASAP": { - "SummaryTumblingWindowChain": { + "NonASAP": { + "TimeShift": { "child": 1, - "family": { - "ExactAggregate": [ - "Increase", - "Increase" - ] - }, - "filter": null, - "grouping": "PerSubpopulationInstance", - "input": { - "item": null, - "weight": { - "Column": "SampleValue" - } - }, - "reduction": "PerEntity", - "tumbling_window_count": 60, - "tumbling_window_width_ms": 60000 + "shift": { + "at": null, + "offset_ms": 21600000 + } } } } }, { - "guarantee": null, "id": 3, "lifetime": "None", "output_schema": { @@ -7489,9 +1310,8 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { @@ -7501,39 +1321,37 @@ ] }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "status", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 4, - 0 + 0, + 2, + 3, + 4 ] ] }, @@ -7542,36 +1360,23 @@ "timing": "query_time" }, "payload": { - "ASAP": { - "SummaryMerge": { - "children": [ - 2 - ] + "NonASAP": { + "TimeRange": { + "child": 2, + "kind": "range", + "range": { + "nanos": 0, + "secs": 3600 + } } } } }, { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact accumulator over exact input" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 4, - "lifetime": "None", + "lifetime": { + "OneEvaluation": "Memory" + }, "output_schema": { "closed": true, "fields": [ @@ -7580,40 +1385,38 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { - "Plain": "float64" + "ExactAggregate": [ + "Increase", + "Increase" + ] }, - "name": "value", - "nullable": false, - "table": null + "name": "state", + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "status", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -7625,19 +1428,20 @@ ] }, "output_state": { - "primitive": "Raw", + "primitive": "SummaryState", "timing": "query_time" }, "payload": { "ASAP": { - "FinalizeExactAccumulator": { - "child": 3 + "SummaryMerge": { + "children": [ + 3 + ] } } } }, { - "guarantee": null, "id": 5, "lifetime": "None", "output_schema": { @@ -7645,69 +1449,65 @@ "fields": [ { "dtype": { - "ExactAggregate": [ - "Sum", - "Sum" - ] + "Plain": "timestamp" + }, + "name": "ts", + "nullable": false + }, + { + "dtype": { + "Plain": "float64" + }, + "name": "value", + "nullable": false + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "job", + "nullable": true + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "status", + "nullable": true + }, + { + "dtype": { + "Plain": "utf8" }, - "name": "state", - "nullable": false, - "table": null + "name": "$promql_series_identity", + "nullable": false } ], - "time_index": null, - "unique_keys": [] + "time_index": 0, + "unique_keys": [ + [ + 4, + 0 + ] + ] }, "output_state": { - "primitive": "SummaryState", + "primitive": "Raw", "timing": "query_time" }, "payload": { "ASAP": { - "SummaryAgg": { - "child": 4, - "family": { - "ExactAggregate": [ - "Sum", - "Sum" - ] - }, - "filter": null, - "grouping": "PerSubpopulationInstance", - "input": { - "item": null, - "weight": { - "Column": "SampleValue" - } - }, - "reduction": { - "Reduce": [] - } + "FinalizeExactAccumulator": { + "child": 4 } } } }, { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact accumulator over exact input" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 6, - "lifetime": "None", + "lifetime": { + "OneEvaluation": "Memory" + }, "output_schema": { "closed": true, "fields": [ @@ -7716,8 +1516,7 @@ "Plain": "float64" }, "name": "sum", - "nullable": false, - "table": null + "nullable": false } ], "time_index": null, @@ -7728,32 +1527,28 @@ "timing": "query_time" }, "payload": { - "ASAP": { - "FinalizeExactAccumulator": { - "child": 5 + "NonASAP": { + "Aggregate": { + "child": 5, + "filters": [], + "having": null, + "measures": [ + { + "col": null, + "kind": "sum" + } + ], + "output_names": [ + "" + ], + "reduction": { + "Reduce": [] + } } } } }, { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact over exact inputs" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 7, "lifetime": "None", "output_schema": { @@ -7764,32 +1559,28 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -7831,32 +1622,28 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -7877,26 +1664,10 @@ } }, { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact over exact inputs" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 8, - "lifetime": "None", + "lifetime": { + "AcrossEvaluations": "Memory" + }, "output_schema": { "closed": true, "fields": [ @@ -7904,64 +1675,102 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { - "Plain": "float64" + "ExactAggregate": [ + "Increase", + "Increase" + ] }, - "name": "value", - "nullable": false, - "table": null + "name": "state", + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 3, - 0 + 0, + 2, + 3 ] ] }, "output_state": { - "primitive": "Raw", + "primitive": "SummaryState", "timing": "ingestion_time" }, "payload": { - "NonASAP": { - "TimeShift": { + "ASAP": { + "SummaryTumblingWindowChain": { "child": 7, - "shift": { - "at": null, - "offset_ms": 21600000 - } + "filter": null, + "input": { + "item": null, + "weight": { + "Column": "SampleValue" + } + }, + "reduction": "PerEntity", + "summary_type": { + "ExactAggregate": [ + "Increase", + "Increase" + ] + }, + "tumbling_window_width_ms": 60000 } } + }, + "retention_ms": 28800000, + "view_coverage": { + "definition_id": "87591b6875ec8f0172a29d8ebfa40327", + "selection": [ + { + "columns": [ + { + "column": { + "column": "job", + "scan_ordinal": 0, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + } + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ] + } + ] } }, { - "guarantee": null, "id": 9, "lifetime": { - "AcrossEvaluations": "Memory" + "OneEvaluation": "Memory" }, "output_schema": { "closed": true, @@ -7970,9 +1779,8 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { @@ -7982,65 +1790,49 @@ ] }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 3, - 0 + 0, + 2, + 3 ] ] }, "output_state": { "primitive": "SummaryState", - "timing": "ingestion_time" + "timing": "query_time" }, "payload": { - "ASAP": { - "SummaryTumblingWindowChain": { + "NonASAP": { + "TimeShift": { "child": 8, - "family": { - "ExactAggregate": [ - "Increase", - "Increase" - ] - }, - "filter": null, - "grouping": "PerSubpopulationInstance", - "input": { - "item": null, - "weight": { - "Column": "SampleValue" - } - }, - "reduction": "PerEntity", - "tumbling_window_count": 60, - "tumbling_window_width_ms": 60000 + "shift": { + "at": null, + "offset_ms": 21600000 + } } } } }, { - "guarantee": null, "id": 10, "lifetime": "None", "output_schema": { @@ -8050,9 +1842,8 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { @@ -8062,31 +1853,29 @@ ] }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 3, - 0 + 0, + 2, + 3 ] ] }, @@ -8095,36 +1884,23 @@ "timing": "query_time" }, "payload": { - "ASAP": { - "SummaryMerge": { - "children": [ - 9 - ] + "NonASAP": { + "TimeRange": { + "child": 9, + "kind": "range", + "range": { + "nanos": 0, + "secs": 3600 + } } } } }, { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact accumulator over exact input" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 11, - "lifetime": "None", + "lifetime": { + "OneEvaluation": "Memory" + }, "output_schema": { "closed": true, "fields": [ @@ -8133,32 +1909,31 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { - "Plain": "float64" + "ExactAggregate": [ + "Increase", + "Increase" + ] }, - "name": "value", - "nullable": false, - "table": null + "name": "state", + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -8170,19 +1945,20 @@ ] }, "output_state": { - "primitive": "Raw", + "primitive": "SummaryState", "timing": "query_time" }, "payload": { "ASAP": { - "FinalizeExactAccumulator": { - "child": 10 + "SummaryMerge": { + "children": [ + 10 + ] } } } }, { - "guarantee": null, "id": 12, "lifetime": "None", "output_schema": { @@ -8190,69 +1966,58 @@ "fields": [ { "dtype": { - "ExactAggregate": [ - "Sum", - "Sum" - ] + "Plain": "timestamp" }, - "name": "state", - "nullable": false, - "table": null + "name": "ts", + "nullable": false + }, + { + "dtype": { + "Plain": "float64" + }, + "name": "value", + "nullable": false + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "job", + "nullable": true + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false } ], - "time_index": null, - "unique_keys": [] + "time_index": 0, + "unique_keys": [ + [ + 3, + 0 + ] + ] }, "output_state": { - "primitive": "SummaryState", + "primitive": "Raw", "timing": "query_time" }, "payload": { "ASAP": { - "SummaryAgg": { - "child": 11, - "family": { - "ExactAggregate": [ - "Sum", - "Sum" - ] - }, - "filter": null, - "grouping": "PerSubpopulationInstance", - "input": { - "item": null, - "weight": { - "Column": "SampleValue" - } - }, - "reduction": { - "Reduce": [] - } + "FinalizeExactAccumulator": { + "child": 11 } } } }, { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact accumulator over exact input" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 13, - "lifetime": "None", + "lifetime": { + "OneEvaluation": "Memory" + }, "output_schema": { "closed": true, "fields": [ @@ -8261,8 +2026,7 @@ "Plain": "float64" }, "name": "sum", - "nullable": false, - "table": null + "nullable": false } ], "time_index": null, @@ -8273,34 +2037,32 @@ "timing": "query_time" }, "payload": { - "ASAP": { - "FinalizeExactAccumulator": { - "child": 12 + "NonASAP": { + "Aggregate": { + "child": 12, + "filters": [], + "having": null, + "measures": [ + { + "col": null, + "kind": "sum" + } + ], + "output_names": [ + "" + ], + "reduction": { + "Reduce": [] + } } } } }, { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact over exact inputs" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 14, - "lifetime": "None", + "lifetime": { + "OneEvaluation": "Memory" + }, "output_schema": { "closed": true, "fields": [ @@ -8309,8 +2071,7 @@ "Plain": "float64" }, "name": "sum", - "nullable": false, - "table": null + "nullable": false } ], "time_index": null, @@ -8343,11 +2104,9 @@ 14 ], "scalar_roots": [] - }, - "display_name": "Q1 Sum acc + Increase acc ×60 tumbling windows · ingestion time: Increase acc for Q1, 60 × 1m tumbling windows, Increase acc for Q1, 60 × 1m tumbling windows" + } }, - "tumbling_window_retention_ms": { - "2": 28800000, - "9": 28800000 - } + "warnings": [ + "assumed data fact: the data arrival declares no max_out_of_order_ms, so the samples of source http_requests_total that an ingestion-time rate, increase or irate state reads are taken to arrive in timestamp order; if this does not hold, answers may be wrong by an amount no accuracy target bounds (a late sample can be read as a counter reset), or ingestion may fail" + ] } diff --git a/crates/devtools/tests/corpus_snapshots/o11y_bench/q19.json b/crates/devtools/tests/corpus_snapshots/o11y_bench/q19.json index 07f84f744..923ea21c8 100644 --- a/crates/devtools/tests/corpus_snapshots/o11y_bench/q19.json +++ b/crates/devtools/tests/corpus_snapshots/o11y_bench/q19.json @@ -3,6150 +3,19 @@ "materialized_views": [ { "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", + "definition_id": "c861201688f76930d0a62614d5e30961", "selection": [ { "columns": [ { "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -60000 - }, - { - "included": 0 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 0, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 0, - "node_id": 2, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -120000 - }, - { - "included": -60000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 60000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 1, - "node_id": 5, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -180000 - }, - { - "included": -120000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 120000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 2, - "node_id": 8, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -240000 - }, - { - "included": -180000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 180000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 3, - "node_id": 11, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -300000 - }, - { - "included": -240000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 240000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 4, - "node_id": 14, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -360000 - }, - { - "included": -300000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 300000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 5, - "node_id": 17, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -420000 - }, - { - "included": -360000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 360000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 6, - "node_id": 20, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -480000 - }, - { - "included": -420000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 420000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 7, - "node_id": 23, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -540000 - }, - { - "included": -480000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 480000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 8, - "node_id": 26, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -600000 - }, - { - "included": -540000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 540000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 9, - "node_id": 29, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -660000 - }, - { - "included": -600000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 600000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 10, - "node_id": 32, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -720000 - }, - { - "included": -660000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 660000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 11, - "node_id": 35, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -780000 - }, - { - "included": -720000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 720000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 12, - "node_id": 38, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -840000 - }, - { - "included": -780000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 780000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 13, - "node_id": 41, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -900000 - }, - { - "included": -840000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 840000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 14, - "node_id": 44, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -960000 - }, - { - "included": -900000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 900000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 15, - "node_id": 47, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -1020000 - }, - { - "included": -960000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 960000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 16, - "node_id": 50, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -1080000 - }, - { - "included": -1020000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1020000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 17, - "node_id": 53, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -1140000 - }, - { - "included": -1080000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1080000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 18, - "node_id": 56, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -1200000 - }, - { - "included": -1140000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1140000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 19, - "node_id": 59, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -1260000 - }, - { - "included": -1200000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1200000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 20, - "node_id": 62, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -1320000 - }, - { - "included": -1260000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1260000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 21, - "node_id": 65, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -1380000 - }, - { - "included": -1320000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1320000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 22, - "node_id": 68, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -1440000 - }, - { - "included": -1380000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1380000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 23, - "node_id": 71, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -1500000 - }, - { - "included": -1440000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1440000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 24, - "node_id": 74, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -1560000 - }, - { - "included": -1500000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1500000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 25, - "node_id": 77, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -1620000 - }, - { - "included": -1560000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1560000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 26, - "node_id": 80, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -1680000 - }, - { - "included": -1620000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1620000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 27, - "node_id": 83, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -1740000 - }, - { - "included": -1680000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1680000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 28, - "node_id": 86, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -1800000 - }, - { - "included": -1740000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1740000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 29, - "node_id": 89, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -1860000 - }, - { - "included": -1800000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1800000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 30, - "node_id": 92, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -1920000 - }, - { - "included": -1860000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1860000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 31, - "node_id": 95, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -1980000 - }, - { - "included": -1920000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1920000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 32, - "node_id": 98, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -2040000 - }, - { - "included": -1980000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1980000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 33, - "node_id": 101, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -2100000 - }, - { - "included": -2040000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2040000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 34, - "node_id": 104, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -2160000 - }, - { - "included": -2100000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2100000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 35, - "node_id": 107, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -2220000 - }, - { - "included": -2160000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2160000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 36, - "node_id": 110, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -2280000 - }, - { - "included": -2220000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2220000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 37, - "node_id": 113, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -2340000 - }, - { - "included": -2280000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2280000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 38, - "node_id": 116, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -2400000 - }, - { - "included": -2340000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2340000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 39, - "node_id": 119, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -2460000 - }, - { - "included": -2400000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2400000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 40, - "node_id": 122, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -2520000 - }, - { - "included": -2460000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2460000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 41, - "node_id": 125, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -2580000 - }, - { - "included": -2520000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2520000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 42, - "node_id": 128, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -2640000 - }, - { - "included": -2580000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2580000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 43, - "node_id": 131, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -2700000 - }, - { - "included": -2640000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2640000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 44, - "node_id": 134, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -2760000 - }, - { - "included": -2700000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2700000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 45, - "node_id": 137, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -2820000 - }, - { - "included": -2760000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2760000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 46, - "node_id": 140, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -2880000 - }, - { - "included": -2820000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2820000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 47, - "node_id": 143, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -2940000 - }, - { - "included": -2880000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2880000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 48, - "node_id": 146, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -3000000 - }, - { - "included": -2940000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2940000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 49, - "node_id": 149, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -3060000 - }, - { - "included": -3000000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3000000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 50, - "node_id": 152, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -3120000 - }, - { - "included": -3060000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3060000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 51, - "node_id": 155, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -3180000 - }, - { - "included": -3120000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3120000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 52, - "node_id": 158, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -3240000 - }, - { - "included": -3180000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3180000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 53, - "node_id": 161, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -3300000 - }, - { - "included": -3240000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3240000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 54, - "node_id": 164, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -3360000 - }, - { - "included": -3300000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3300000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 55, - "node_id": 167, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -3420000 - }, - { - "included": -3360000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3360000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 56, - "node_id": 170, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -3480000 - }, - { - "included": -3420000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3420000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 57, - "node_id": 173, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -3540000 - }, - { - "included": -3480000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3480000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 58, - "node_id": 176, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -3600000 - }, - { - "included": -3540000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3540000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 59, - "node_id": 179, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -60000 - }, - { - "included": 0 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 0, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 0, - "node_id": 186, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -120000 - }, - { - "included": -60000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 60000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 1, - "node_id": 189, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -180000 - }, - { - "included": -120000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 120000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 2, - "node_id": 192, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -240000 - }, - { - "included": -180000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 180000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 3, - "node_id": 195, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -300000 - }, - { - "included": -240000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 240000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 4, - "node_id": 198, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -360000 - }, - { - "included": -300000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 300000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 5, - "node_id": 201, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -420000 - }, - { - "included": -360000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 360000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 6, - "node_id": 204, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -480000 - }, - { - "included": -420000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 420000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 7, - "node_id": 207, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -540000 - }, - { - "included": -480000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 480000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 8, - "node_id": 210, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -600000 - }, - { - "included": -540000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 540000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 9, - "node_id": 213, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -660000 - }, - { - "included": -600000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 600000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 10, - "node_id": 216, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -720000 - }, - { - "included": -660000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 660000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 11, - "node_id": 219, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -780000 - }, - { - "included": -720000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 720000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 12, - "node_id": 222, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -840000 - }, - { - "included": -780000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 780000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 13, - "node_id": 225, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -900000 - }, - { - "included": -840000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 840000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 14, - "node_id": 228, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -960000 - }, - { - "included": -900000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 900000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 15, - "node_id": 231, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -1020000 - }, - { - "included": -960000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 960000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 16, - "node_id": 234, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -1080000 - }, - { - "included": -1020000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1020000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 17, - "node_id": 237, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -1140000 - }, - { - "included": -1080000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1080000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 18, - "node_id": 240, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -1200000 - }, - { - "included": -1140000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1140000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 19, - "node_id": 243, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -1260000 - }, - { - "included": -1200000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1200000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 20, - "node_id": 246, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -1320000 - }, - { - "included": -1260000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1260000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 21, - "node_id": 249, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -1380000 - }, - { - "included": -1320000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1320000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 22, - "node_id": 252, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -1440000 - }, - { - "included": -1380000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1380000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 23, - "node_id": 255, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -1500000 - }, - { - "included": -1440000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1440000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 24, - "node_id": 258, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -1560000 - }, - { - "included": -1500000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1500000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 25, - "node_id": 261, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -1620000 - }, - { - "included": -1560000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1560000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 26, - "node_id": 264, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -1680000 - }, - { - "included": -1620000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1620000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 27, - "node_id": 267, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -1740000 - }, - { - "included": -1680000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1680000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 28, - "node_id": 270, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -1800000 - }, - { - "included": -1740000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1740000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 29, - "node_id": 273, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -1860000 - }, - { - "included": -1800000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1800000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 30, - "node_id": 276, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -1920000 - }, - { - "included": -1860000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1860000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 31, - "node_id": 279, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -1980000 - }, - { - "included": -1920000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1920000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 32, - "node_id": 282, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -2040000 - }, - { - "included": -1980000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1980000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 33, - "node_id": 285, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -2100000 - }, - { - "included": -2040000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2040000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 34, - "node_id": 288, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -2160000 - }, - { - "included": -2100000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2100000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 35, - "node_id": 291, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -2220000 - }, - { - "included": -2160000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2160000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 36, - "node_id": 294, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -2280000 - }, - { - "included": -2220000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2220000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 37, - "node_id": 297, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -2340000 - }, - { - "included": -2280000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2280000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 38, - "node_id": 300, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -2400000 - }, - { - "included": -2340000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2340000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 39, - "node_id": 303, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -2460000 - }, - { - "included": -2400000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2400000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 40, - "node_id": 306, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -2520000 - }, - { - "included": -2460000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2460000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 41, - "node_id": 309, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -2580000 - }, - { - "included": -2520000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2520000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 42, - "node_id": 312, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -2640000 - }, - { - "included": -2580000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2580000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 43, - "node_id": 315, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -2700000 - }, - { - "included": -2640000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2640000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 44, - "node_id": 318, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -2760000 - }, - { - "included": -2700000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2700000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 45, - "node_id": 321, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -2820000 - }, - { - "included": -2760000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2760000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 46, - "node_id": 324, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -2880000 - }, - { - "included": -2820000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2820000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 47, - "node_id": 327, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -2940000 - }, - { - "included": -2880000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2880000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 48, - "node_id": 330, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -3000000 - }, - { - "included": -2940000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2940000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 49, - "node_id": 333, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -3060000 - }, - { - "included": -3000000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3000000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 50, - "node_id": 336, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -3120000 - }, - { - "included": -3060000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3060000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 51, - "node_id": 339, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -3180000 - }, - { - "included": -3120000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3120000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 52, - "node_id": 342, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -3240000 - }, - { - "included": -3180000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3180000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 53, - "node_id": 345, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -3300000 - }, - { - "included": -3240000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3240000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 54, - "node_id": 348, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -3360000 - }, - { - "included": -3300000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3300000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 55, - "node_id": 351, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -3420000 - }, - { - "included": -3360000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3360000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 56, - "node_id": 354, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "payment-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -3480000 - }, - { - "included": -3420000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3420000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 57, - "node_id": 357, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null + "column": "job", + "scan_ordinal": 0, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + } }, "constraint": { "in": [ @@ -6156,49 +25,32 @@ ] } } - ], - "relative_time": [ - { - "excluded": -3540000 - }, - { - "included": -3480000 - } ] } ] }, "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3480000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] + "materialized_views": [] }, - "installed_count": 60, - "installed_position": 58, - "node_id": 360, + "node_id": 1, "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 + "retention_ms": 7200000 }, { "coverage": { - "definition_id": "6e0a4719202a0469f55a3a8469d009cc", + "definition_id": "87591b6875ec8f0172a29d8ebfa40327", "selection": [ { "columns": [ { "column": { - "name": "job", - "table": null + "column": "job", + "scan_ordinal": 0, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + } }, "constraint": { "in": [ @@ -6208,189 +60,156 @@ ] } } - ], - "relative_time": [ - { - "excluded": -3600000 - }, - { - "included": -3540000 - } ] } ] }, "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3540000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] + "materialized_views": [] }, - "installed_count": 60, - "installed_position": 59, - "node_id": 363, + "node_id": 7, "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 + "retention_ms": 7200000 } ], - "proven_optimal": true, "query": "sum(increase(http_requests_total{job=\"payment-service\",status=~\"5..\"}[1h])) / sum(increase(http_requests_total{job=\"payment-service\"}[1h]))", "selected": { "accepted": [ { "accepted_metric": null, - "accepted_regardless_of_guarantee": false, - "guarantee_check_rejection": null, "sketches": [], - "summary": "metric not set; accepted exact", - "target_metric": null + "summary": "accepted exact" } ], "cost": { - "cost_per_second": 1.83727148353, - "cost_unit": "cost_per_second", - "cpu_core_seconds_per_second": 0.000143269716667, + "cost_per_second": 56.2766171722, + "cpu_core_seconds_per_second": 0.05167776335, "per_node": { "0": { "cost_per_second": 0.0074, + "cost_source": "analytical", "cpu_core_seconds_per_second": 7.4e-6, "detail": "scan 1000 samples; x 1.0000/s", "multiplier": 1, "operator": "Scan", - "price_source": "analytical", "timing": "ingestion_time" }, "1": { - "cost_per_second": 0.849, - "cpu_core_seconds_per_second": 2e-6, - "detail": "build exact Increase accumulator in 60 tumbling windows of 60000 ms for 1000 groups: 1000 rows x (1 + depth 1); x 1.0000/s; keeps 6776000 bytes", + "cost_per_second": 2.583, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.000163, + "detail": "build exact Increase accumulator in tumbling windows of 60000 ms for 1000 groups: 1000 rows x 107 ops, 100 group visits, 16.7 states emitted; x 1.0000/s; keeps 19360000 bytes", "multiplier": 1, "operator": "SummaryTumblingWindowChain", - "price_source": "analytical", "timing": "ingestion_time" }, "10": { - "cost_per_second": 0.00101666754915, - "cpu_core_seconds_per_second": 1.01666666667e-6, - "detail": "build exact Sum accumulator into 1 states: 1000 rows x depth 1; x 1.0167/s; holds 56 bytes for 0.1 ms", + "cost_per_second": 2.54166666667, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.00254166666667, + "detail": "finalize 1000 accumulators; x 1.0167/s", "multiplier": 1, - "operator": "SummaryAgg", - "price_source": "analytical", + "operator": "FinalizeExactAccumulator", "timing": "query_time" }, "11": { - "cost_per_second": 1.01666666667e-6, - "cpu_core_seconds_per_second": 1.01666666667e-9, - "detail": "finalize 1 accumulators; x 1.0167/s", + "cost_per_second": 0.00678116668701, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 6.78116666667e-6, + "detail": "hash aggregate 1000 rows into 1 groups: 4 ops per row, 1 group visits; x 1.0167/s; holds 24 bytes for 0.007 ms", "multiplier": 1, - "operator": "FinalizeExactAccumulator", - "price_source": "analytical", + "operator": "Aggregate", "timing": "query_time" }, "12": { - "cost_per_second": 1.01679273638e-6, + "cost_per_second": 1.01666666768e-6, + "cost_source": "analytical", "cpu_core_seconds_per_second": 1.01666666667e-9, - "detail": "operator: pass 1 rows; x 1.0167/s; holds 8 bytes for 0.1 ms", + "detail": "operator: pass 1 rows; x 1.0167/s; holds 8 bytes for 0.000 ms", "multiplier": 1, "operator": "BinaryOp", - "price_source": "analytical", "timing": "query_time" }, "2": { - "cost_per_second": 0.061000882488, - "cpu_core_seconds_per_second": 0.000061, - "detail": "merge 60000 states into 1000; x 1.0167/s; holds 56000 bytes for 0.1 ms", + "cost_per_second": 0.67100001342, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.000671, + "detail": "time range 3600s: 60 tumbling windows of states; x 1.0167/s; holds 160 bytes for 0.660 ms", "multiplier": 1, - "operator": "SummaryMerge", - "price_source": "analytical", + "operator": "TimeRange", "timing": "query_time" }, "3": { - "cost_per_second": 0.00101666666667, - "cpu_core_seconds_per_second": 1.01666666667e-6, - "detail": "finalize 1000 accumulators; x 1.0167/s", + "cost_per_second": 22.4687827, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.0224683333333, + "detail": "merge 60000 states into 1000; x 1.0167/s; holds 160000 bytes for 22.100 ms", "multiplier": 1, - "operator": "FinalizeExactAccumulator", - "price_source": "analytical", + "operator": "SummaryMerge", "timing": "query_time" }, "4": { - "cost_per_second": 0.00101666754915, - "cpu_core_seconds_per_second": 1.01666666667e-6, - "detail": "build exact Sum accumulator into 1 states: 1000 rows x depth 1; x 1.0167/s; holds 56 bytes for 0.1 ms", + "cost_per_second": 2.54166666667, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.00254166666667, + "detail": "finalize 1000 accumulators; x 1.0167/s", "multiplier": 1, - "operator": "SummaryAgg", - "price_source": "analytical", + "operator": "FinalizeExactAccumulator", "timing": "query_time" }, "5": { - "cost_per_second": 1.01666666667e-6, - "cpu_core_seconds_per_second": 1.01666666667e-9, - "detail": "finalize 1 accumulators; x 1.0167/s", + "cost_per_second": 0.00678116668701, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 6.78116666667e-6, + "detail": "hash aggregate 1000 rows into 1 groups: 4 ops per row, 1 group visits; x 1.0167/s; holds 24 bytes for 0.007 ms", "multiplier": 1, - "operator": "FinalizeExactAccumulator", - "price_source": "analytical", + "operator": "Aggregate", "timing": "query_time" }, "6": { "cost_per_second": 0.0058, + "cost_source": "analytical", "cpu_core_seconds_per_second": 5.8e-6, "detail": "scan 1000 samples; x 1.0000/s", "multiplier": 1, "operator": "Scan", - "price_source": "analytical", "timing": "ingestion_time" }, "7": { - "cost_per_second": 0.849, - "cpu_core_seconds_per_second": 2e-6, - "detail": "build exact Increase accumulator in 60 tumbling windows of 60000 ms for 1000 groups: 1000 rows x (1 + depth 1); x 1.0000/s; keeps 6776000 bytes", + "cost_per_second": 2.304, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.000126, + "detail": "build exact Increase accumulator in tumbling windows of 60000 ms for 1000 groups: 1000 rows x 70 ops, 100 group visits, 16.7 states emitted; x 1.0000/s; keeps 17424000 bytes", "multiplier": 1, "operator": "SummaryTumblingWindowChain", - "price_source": "analytical", "timing": "ingestion_time" }, "8": { - "cost_per_second": 0.061000882488, - "cpu_core_seconds_per_second": 0.000061, - "detail": "merge 60000 states into 1000; x 1.0167/s; holds 56000 bytes for 0.1 ms", + "cost_per_second": 0.671000012078, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.000671, + "detail": "time range 3600s: 60 tumbling windows of states; x 1.0167/s; holds 144 bytes for 0.660 ms", "multiplier": 1, - "operator": "SummaryMerge", - "price_source": "analytical", + "operator": "TimeRange", "timing": "query_time" }, "9": { - "cost_per_second": 0.00101666666667, - "cpu_core_seconds_per_second": 1.01666666667e-6, - "detail": "finalize 1000 accumulators; x 1.0167/s", + "cost_per_second": 22.4687377633, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.0224683333333, + "detail": "merge 60000 states into 1000; x 1.0167/s; holds 144000 bytes for 22.100 ms", "multiplier": 1, - "operator": "FinalizeExactAccumulator", - "price_source": "analytical", + "operator": "SummaryMerge", "timing": "query_time" } }, "source": "analytical-cost-v2 (illustrative statistics)", "usage": { - "average_memory_bytes": 13552014.1349, - "cpu_ops_per_second": 132069.716667, + "average_memory_bytes": 36790830.5776, + "cpu_ops_per_second": 51666563.35, "disk_bytes": 0, - "latency_per_query": [ - { - "compaction_ms": 0.0, - "query_ms": 0.124003 - } - ], "peak_cpu_cores": 2, - "peak_memory_bytes": 13664232, + "peak_memory_bytes": 37088472, "scanned_bytes_per_second": 112000.0 } }, @@ -6411,40 +230,35 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "status", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -6457,7 +271,7 @@ }, "producer": 0, "role": "Input", - "window": "RequiresAlignedPanePhaseOrExactBoundaryResidual" + "window": "NotApplicable" }, { "consumer": 2, @@ -6473,9 +287,8 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { @@ -6485,39 +298,37 @@ ] }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "status", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 4, - 0 + 0, + 2, + 3, + 4 ] ] }, @@ -6539,9 +350,8 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { @@ -6551,39 +361,37 @@ ] }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "status", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 4, - 0 + 0, + 2, + 3, + 4 ] ] }, @@ -6594,7 +402,7 @@ { "consumer": 4, "data_state": { - "primitive": "Raw", + "primitive": "SummaryState", "timing": "query_time" }, "grouping": "NotApplicable", @@ -6606,40 +414,38 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { - "Plain": "float64" + "ExactAggregate": [ + "Increase", + "Increase" + ] }, - "name": "value", - "nullable": false, - "table": null + "name": "state", + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "status", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -6657,7 +463,7 @@ { "consumer": 5, "data_state": { - "primitive": "SummaryState", + "primitive": "Raw", "timing": "query_time" }, "grouping": "NotApplicable", @@ -6666,18 +472,47 @@ "fields": [ { "dtype": { - "ExactAggregate": [ - "Sum", - "Sum" - ] + "Plain": "timestamp" }, - "name": "state", - "nullable": false, - "table": null + "name": "ts", + "nullable": false + }, + { + "dtype": { + "Plain": "float64" + }, + "name": "value", + "nullable": false + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "job", + "nullable": true + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "status", + "nullable": true + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false } ], - "time_index": null, - "unique_keys": [] + "time_index": 0, + "unique_keys": [ + [ + 4, + 0 + ] + ] }, "producer": 4, "role": "Input", @@ -6698,32 +533,28 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -6736,7 +567,7 @@ }, "producer": 6, "role": "Input", - "window": "RequiresAlignedPanePhaseOrExactBoundaryResidual" + "window": "NotApplicable" }, { "consumer": 8, @@ -6752,9 +583,8 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { @@ -6764,31 +594,29 @@ ] }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 3, - 0 + 0, + 2, + 3 ] ] }, @@ -6810,9 +638,8 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { @@ -6822,31 +649,29 @@ ] }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 3, - 0 + 0, + 2, + 3 ] ] }, @@ -6857,7 +682,7 @@ { "consumer": 10, "data_state": { - "primitive": "Raw", + "primitive": "SummaryState", "timing": "query_time" }, "grouping": "NotApplicable", @@ -6869,32 +694,31 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { - "Plain": "float64" + "ExactAggregate": [ + "Increase", + "Increase" + ] }, - "name": "value", - "nullable": false, - "table": null + "name": "state", + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -6912,7 +736,7 @@ { "consumer": 11, "data_state": { - "primitive": "SummaryState", + "primitive": "Raw", "timing": "query_time" }, "grouping": "NotApplicable", @@ -6921,18 +745,40 @@ "fields": [ { "dtype": { - "ExactAggregate": [ - "Sum", - "Sum" - ] + "Plain": "timestamp" }, - "name": "state", - "nullable": false, - "table": null + "name": "ts", + "nullable": false + }, + { + "dtype": { + "Plain": "float64" + }, + "name": "value", + "nullable": false + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "job", + "nullable": true + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false } ], - "time_index": null, - "unique_keys": [] + "time_index": 0, + "unique_keys": [ + [ + 3, + 0 + ] + ] }, "producer": 10, "role": "Input", @@ -6953,8 +799,7 @@ "Plain": "float64" }, "name": "sum", - "nullable": false, - "table": null + "nullable": false } ], "time_index": null, @@ -6979,8 +824,7 @@ "Plain": "float64" }, "name": "sum", - "nullable": false, - "table": null + "nullable": false } ], "time_index": null, @@ -6993,24 +837,6 @@ ], "nodes": [ { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact over exact inputs" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 0, "lifetime": "None", "output_schema": { @@ -7021,40 +847,35 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "status", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -7110,40 +931,35 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "status", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -7158,17 +974,126 @@ "TimeSeries": { "metric": "http_requests_total" } - } + } + } + } + } + }, + { + "id": 1, + "lifetime": { + "AcrossEvaluations": "Memory" + }, + "output_schema": { + "closed": true, + "fields": [ + { + "dtype": { + "Plain": "timestamp" + }, + "name": "window_end", + "nullable": false + }, + { + "dtype": { + "ExactAggregate": [ + "Increase", + "Increase" + ] + }, + "name": "state", + "nullable": false + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "job", + "nullable": true + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "status", + "nullable": true + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false + } + ], + "time_index": 0, + "unique_keys": [ + [ + 0, + 2, + 3, + 4 + ] + ] + }, + "output_state": { + "primitive": "SummaryState", + "timing": "ingestion_time" + }, + "payload": { + "ASAP": { + "SummaryTumblingWindowChain": { + "child": 0, + "filter": null, + "input": { + "item": null, + "weight": { + "Column": "SampleValue" + } + }, + "reduction": "PerEntity", + "summary_type": { + "ExactAggregate": [ + "Increase", + "Increase" + ] + }, + "tumbling_window_width_ms": 60000 + } + } + }, + "retention_ms": 7200000, + "view_coverage": { + "definition_id": "c861201688f76930d0a62614d5e30961", + "selection": [ + { + "columns": [ + { + "column": { + "column": "job", + "scan_ordinal": 0, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + } + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ] } - } + ] } }, { - "guarantee": null, - "id": 1, - "lifetime": { - "AcrossEvaluations": "Memory" - }, + "id": 2, + "lifetime": "None", "output_schema": { "closed": true, "fields": [ @@ -7176,9 +1101,8 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { @@ -7188,75 +1112,62 @@ ] }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "status", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 4, - 0 + 0, + 2, + 3, + 4 ] ] }, "output_state": { "primitive": "SummaryState", - "timing": "ingestion_time" + "timing": "query_time" }, "payload": { - "ASAP": { - "SummaryTumblingWindowChain": { - "child": 0, - "family": { - "ExactAggregate": [ - "Increase", - "Increase" - ] - }, - "filter": null, - "grouping": "PerSubpopulationInstance", - "input": { - "item": null, - "weight": { - "Column": "SampleValue" - } - }, - "reduction": "PerEntity", - "tumbling_window_count": 60, - "tumbling_window_width_ms": 60000 + "NonASAP": { + "TimeRange": { + "child": 1, + "kind": "range", + "range": { + "nanos": 0, + "secs": 3600 + } } } } }, { - "guarantee": null, - "id": 2, - "lifetime": "None", + "id": 3, + "lifetime": { + "OneEvaluation": "Memory" + }, "output_schema": { "closed": true, "fields": [ @@ -7265,8 +1176,7 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { @@ -7276,32 +1186,28 @@ ] }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "status", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -7320,32 +1226,14 @@ "ASAP": { "SummaryMerge": { "children": [ - 1 + 2 ] } } } }, { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact accumulator over exact input" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, - "id": 3, + "id": 4, "lifetime": "None", "output_schema": { "closed": true, @@ -7355,40 +1243,35 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "status", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -7406,83 +1289,16 @@ "payload": { "ASAP": { "FinalizeExactAccumulator": { - "child": 2 - } - } - } - }, - { - "guarantee": null, - "id": 4, - "lifetime": "None", - "output_schema": { - "closed": true, - "fields": [ - { - "dtype": { - "ExactAggregate": [ - "Sum", - "Sum" - ] - }, - "name": "state", - "nullable": false, - "table": null - } - ], - "time_index": null, - "unique_keys": [] - }, - "output_state": { - "primitive": "SummaryState", - "timing": "query_time" - }, - "payload": { - "ASAP": { - "SummaryAgg": { - "child": 3, - "family": { - "ExactAggregate": [ - "Sum", - "Sum" - ] - }, - "filter": null, - "grouping": "PerSubpopulationInstance", - "input": { - "item": null, - "weight": { - "Column": "SampleValue" - } - }, - "reduction": { - "Reduce": [] - } + "child": 3 } } } }, { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact accumulator over exact input" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 5, - "lifetime": "None", + "lifetime": { + "OneEvaluation": "Memory" + }, "output_schema": { "closed": true, "fields": [ @@ -7491,8 +1307,7 @@ "Plain": "float64" }, "name": "sum", - "nullable": false, - "table": null + "nullable": false } ], "time_index": null, @@ -7503,32 +1318,28 @@ "timing": "query_time" }, "payload": { - "ASAP": { - "FinalizeExactAccumulator": { - "child": 4 + "NonASAP": { + "Aggregate": { + "child": 4, + "filters": [], + "having": null, + "measures": [ + { + "col": null, + "kind": "sum" + } + ], + "output_names": [ + "" + ], + "reduction": { + "Reduce": [] + } } } } }, { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact over exact inputs" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 6, "lifetime": "None", "output_schema": { @@ -7539,32 +1350,28 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -7606,32 +1413,28 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -7652,7 +1455,6 @@ } }, { - "guarantee": null, "id": 7, "lifetime": { "AcrossEvaluations": "Memory" @@ -7664,9 +1466,8 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { @@ -7676,31 +1477,29 @@ ] }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 3, - 0 + 0, + 2, + 3 ] ] }, @@ -7712,14 +1511,7 @@ "ASAP": { "SummaryTumblingWindowChain": { "child": 6, - "family": { - "ExactAggregate": [ - "Increase", - "Increase" - ] - }, "filter": null, - "grouping": "PerSubpopulationInstance", "input": { "item": null, "weight": { @@ -7727,14 +1519,46 @@ } }, "reduction": "PerEntity", - "tumbling_window_count": 60, + "summary_type": { + "ExactAggregate": [ + "Increase", + "Increase" + ] + }, "tumbling_window_width_ms": 60000 } } + }, + "retention_ms": 7200000, + "view_coverage": { + "definition_id": "87591b6875ec8f0172a29d8ebfa40327", + "selection": [ + { + "columns": [ + { + "column": { + "column": "job", + "scan_ordinal": 0, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + } + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ] + } + ] } }, { - "guarantee": null, "id": 8, "lifetime": "None", "output_schema": { @@ -7744,9 +1568,8 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { @@ -7756,31 +1579,29 @@ ] }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 3, - 0 + 0, + 2, + 3 ] ] }, @@ -7789,36 +1610,23 @@ "timing": "query_time" }, "payload": { - "ASAP": { - "SummaryMerge": { - "children": [ - 7 - ] + "NonASAP": { + "TimeRange": { + "child": 7, + "kind": "range", + "range": { + "nanos": 0, + "secs": 3600 + } } } } }, { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact accumulator over exact input" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 9, - "lifetime": "None", + "lifetime": { + "OneEvaluation": "Memory" + }, "output_schema": { "closed": true, "fields": [ @@ -7827,32 +1635,31 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { - "Plain": "float64" + "ExactAggregate": [ + "Increase", + "Increase" + ] }, - "name": "value", - "nullable": false, - "table": null + "name": "state", + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -7864,19 +1671,20 @@ ] }, "output_state": { - "primitive": "Raw", + "primitive": "SummaryState", "timing": "query_time" }, "payload": { "ASAP": { - "FinalizeExactAccumulator": { - "child": 8 + "SummaryMerge": { + "children": [ + 8 + ] } } } }, { - "guarantee": null, "id": 10, "lifetime": "None", "output_schema": { @@ -7884,69 +1692,58 @@ "fields": [ { "dtype": { - "ExactAggregate": [ - "Sum", - "Sum" - ] + "Plain": "timestamp" }, - "name": "state", - "nullable": false, - "table": null + "name": "ts", + "nullable": false + }, + { + "dtype": { + "Plain": "float64" + }, + "name": "value", + "nullable": false + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "job", + "nullable": true + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false } ], - "time_index": null, - "unique_keys": [] + "time_index": 0, + "unique_keys": [ + [ + 3, + 0 + ] + ] }, "output_state": { - "primitive": "SummaryState", + "primitive": "Raw", "timing": "query_time" }, "payload": { "ASAP": { - "SummaryAgg": { - "child": 9, - "family": { - "ExactAggregate": [ - "Sum", - "Sum" - ] - }, - "filter": null, - "grouping": "PerSubpopulationInstance", - "input": { - "item": null, - "weight": { - "Column": "SampleValue" - } - }, - "reduction": { - "Reduce": [] - } + "FinalizeExactAccumulator": { + "child": 9 } } } }, { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact accumulator over exact input" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 11, - "lifetime": "None", + "lifetime": { + "OneEvaluation": "Memory" + }, "output_schema": { "closed": true, "fields": [ @@ -7955,8 +1752,7 @@ "Plain": "float64" }, "name": "sum", - "nullable": false, - "table": null + "nullable": false } ], "time_index": null, @@ -7967,34 +1763,32 @@ "timing": "query_time" }, "payload": { - "ASAP": { - "FinalizeExactAccumulator": { - "child": 10 + "NonASAP": { + "Aggregate": { + "child": 10, + "filters": [], + "having": null, + "measures": [ + { + "col": null, + "kind": "sum" + } + ], + "output_names": [ + "" + ], + "reduction": { + "Reduce": [] + } } } } }, { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact over exact inputs" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 12, - "lifetime": "None", + "lifetime": { + "OneEvaluation": "Memory" + }, "output_schema": { "closed": true, "fields": [ @@ -8003,8 +1797,7 @@ "Plain": "float64" }, "name": "sum", - "nullable": false, - "table": null + "nullable": false } ], "time_index": null, @@ -8037,11 +1830,9 @@ 12 ], "scalar_roots": [] - }, - "display_name": "Q1 Sum acc + Increase acc ×60 tumbling windows · ingestion time: Increase acc for Q1, 60 × 1m tumbling windows, Increase acc for Q1, 60 × 1m tumbling windows" + } }, - "tumbling_window_retention_ms": { - "1": 7200000, - "7": 7200000 - } + "warnings": [ + "assumed data fact: the data arrival declares no max_out_of_order_ms, so the samples of source http_requests_total that an ingestion-time rate, increase or irate state reads are taken to arrive in timestamp order; if this does not hold, answers may be wrong by an amount no accuracy target bounds (a late sample can be read as a counter reset), or ingestion may fail" + ] } diff --git a/crates/devtools/tests/corpus_snapshots/o11y_bench/q20.json b/crates/devtools/tests/corpus_snapshots/o11y_bench/q20.json index db2d86cae..222b37c07 100644 --- a/crates/devtools/tests/corpus_snapshots/o11y_bench/q20.json +++ b/crates/devtools/tests/corpus_snapshots/o11y_bench/q20.json @@ -1,74 +1,54 @@ { "accuracy_guarantees": "enforced", "materialized_views": [], - "proven_optimal": true, "query": "sum(process_resident_memory_bytes)", "selected": { "accepted": [ { "accepted_metric": null, - "accepted_regardless_of_guarantee": false, - "guarantee_check_rejection": null, "sketches": [], - "summary": "metric not set; accepted exact", - "target_metric": null + "summary": "accepted exact" } ], "cost": { - "cost_per_second": 0.00630435009457, - "cost_unit": "cost_per_second", - "cpu_core_seconds_per_second": 6.30435e-6, + "cost_per_second": 0.0222345000822, + "cpu_core_seconds_per_second": 0.0000222345, "per_node": { "0": { - "cost_per_second": 0.00427000002522, + "cost_per_second": 0.00427000001708, + "cost_source": "analytical", "cpu_core_seconds_per_second": 4.27e-6, - "detail": "scan 1000 samples; x 1.0167/s; keeps 32000 bytes of raw data on disk; holds 32 bytes for 0.0 ms", + "detail": "scan 1000 samples; x 1.0167/s; holds 32 bytes for 0.004 ms; keeps 115232000 bytes of raw data on disk", "multiplier": 1, "operator": "Scan", - "price_source": "analytical", "timing": "query_time" }, "1": { - "cost_per_second": 0.00101666669188, - "cpu_core_seconds_per_second": 1.01666666667e-6, - "detail": "time range 1s: keep 1000 of 1000 rows; x 1.0167/s; holds 32 bytes for 0.0 ms", + "cost_per_second": 0.0111833333781, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.0000111833333333, + "detail": "time range 1s: keep 1000 of 1000 rows; x 1.0167/s; holds 32 bytes for 0.011 ms", "multiplier": 1, "operator": "TimeRange", - "price_source": "analytical", "timing": "query_time" }, "2": { - "cost_per_second": 0.0010166667108, - "cpu_core_seconds_per_second": 1.01666666667e-6, - "detail": "build exact Sum accumulator into 1 states: 1000 rows x depth 1; x 1.0167/s; holds 56 bytes for 0.0 ms", + "cost_per_second": 0.00678116668701, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 6.78116666667e-6, + "detail": "hash aggregate 1000 rows into 1 groups: 4 ops per row, 1 group visits; x 1.0167/s; holds 24 bytes for 0.007 ms", "multiplier": 1, - "operator": "SummaryAgg", - "price_source": "analytical", - "timing": "query_time" - }, - "3": { - "cost_per_second": 1.01666666667e-6, - "cpu_core_seconds_per_second": 1.01666666667e-9, - "detail": "finalize 1 accumulators; x 1.0167/s", - "multiplier": 1, - "operator": "FinalizeExactAccumulator", - "price_source": "analytical", + "operator": "Aggregate", "timing": "query_time" } }, "source": "analytical-cost-v2 (illustrative statistics)", "usage": { - "average_memory_bytes": 0.000756522, - "cpu_ops_per_second": 3051.01666667, - "disk_bytes": 32000, - "latency_per_query": [ - { - "compaction_ms": 0.0, - "query_ms": 0.006201 - } - ], + "average_memory_bytes": 0.000657254666667, + "cpu_ops_per_second": 18981.1666667, + "disk_bytes": 115232000, "peak_cpu_cores": 1, - "peak_memory_bytes": 120, + "peak_memory_bytes": 88, "scanned_bytes_per_second": 32533.3333333 } }, @@ -89,24 +69,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -136,24 +113,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -167,57 +141,10 @@ "producer": 1, "role": "Input", "window": "NotApplicable" - }, - { - "consumer": 3, - "data_state": { - "primitive": "SummaryState", - "timing": "query_time" - }, - "grouping": "NotApplicable", - "intermediate_schema": { - "closed": true, - "fields": [ - { - "dtype": { - "ExactAggregate": [ - "Sum", - "Sum" - ] - }, - "name": "state", - "nullable": false, - "table": null - } - ], - "time_index": null, - "unique_keys": [] - }, - "producer": 2, - "role": "Input", - "window": "NotApplicable" } ], "nodes": [ { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact over exact inputs" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 0, "lifetime": "None", "output_schema": { @@ -228,24 +155,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -272,24 +196,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -307,27 +228,10 @@ } } } - } + }, + "retention_ms": 3601000 }, { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact over exact inputs" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 1, "lifetime": "None", "output_schema": { @@ -338,24 +242,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -384,111 +285,55 @@ } }, { - "guarantee": null, "id": 2, - "lifetime": "None", + "lifetime": { + "OneEvaluation": "Memory" + }, "output_schema": { "closed": true, "fields": [ { "dtype": { - "ExactAggregate": [ - "Sum", - "Sum" - ] + "Plain": "float64" }, - "name": "state", - "nullable": false, - "table": null + "name": "sum", + "nullable": false } ], "time_index": null, "unique_keys": [] }, "output_state": { - "primitive": "SummaryState", + "primitive": "Raw", "timing": "query_time" }, "payload": { - "ASAP": { - "SummaryAgg": { + "NonASAP": { + "Aggregate": { "child": 1, - "family": { - "ExactAggregate": [ - "Sum", - "Sum" - ] - }, - "filter": null, - "grouping": "PerSubpopulationInstance", - "input": { - "item": null, - "weight": { - "Column": "SampleValue" + "filters": [], + "having": null, + "measures": [ + { + "col": null, + "kind": "sum" } - }, + ], + "output_names": [ + "" + ], "reduction": { "Reduce": [] } } } } - }, - { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact accumulator over exact input" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, - "id": 3, - "lifetime": "None", - "output_schema": { - "closed": true, - "fields": [ - { - "dtype": { - "Plain": "float64" - }, - "name": "sum", - "nullable": false, - "table": null - } - ], - "time_index": null, - "unique_keys": [] - }, - "output_state": { - "primitive": "Raw", - "timing": "query_time" - }, - "payload": { - "ASAP": { - "FinalizeExactAccumulator": { - "child": 2 - } - } - } } ], "roots": [ - 3 + 2 ], "scalar_roots": [] - }, - "display_name": "Q1 Sum acc" - }, - "tumbling_window_retention_ms": {} + } + } } diff --git a/crates/devtools/tests/corpus_snapshots/o11y_bench/q21.json b/crates/devtools/tests/corpus_snapshots/o11y_bench/q21.json index 976174aee..c8fb2a475 100644 --- a/crates/devtools/tests/corpus_snapshots/o11y_bench/q21.json +++ b/crates/devtools/tests/corpus_snapshots/o11y_bench/q21.json @@ -3,222 +3,19 @@ "materialized_views": [ { "coverage": { - "definition_id": "b167c0953f55d24c1c15591dddb42cb4", + "definition_id": "58395555aae268f4f68a8a3e82a6c45b", "selection": [ { "columns": [ { "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "order-service" + "column": "job", + "scan_ordinal": 0, + "source": { + "TimeSeries": { + "metric": "http_requests_total" } - ] - } - } - ], - "relative_time": [ - { - "excluded": -3660000 - }, - { - "included": -3600000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3600000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 5, - "installed_position": 0, - "node_id": 3, - "refresh": "at_ingestion", - "retention_ms": 7500000, - "stage2_node_id": 2 - }, - { - "coverage": { - "definition_id": "b167c0953f55d24c1c15591dddb42cb4", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "order-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -3720000 - }, - { - "included": -3660000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3660000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 5, - "installed_position": 1, - "node_id": 6, - "refresh": "at_ingestion", - "retention_ms": 7500000, - "stage2_node_id": 2 - }, - { - "coverage": { - "definition_id": "b167c0953f55d24c1c15591dddb42cb4", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "order-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -3780000 - }, - { - "included": -3720000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3720000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 5, - "installed_position": 2, - "node_id": 9, - "refresh": "at_ingestion", - "retention_ms": 7500000, - "stage2_node_id": 2 - }, - { - "coverage": { - "definition_id": "b167c0953f55d24c1c15591dddb42cb4", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "order-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -3840000 - }, - { - "included": -3780000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3780000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 5, - "installed_position": 3, - "node_id": 12, - "refresh": "at_ingestion", - "retention_ms": 7500000, - "stage2_node_id": 2 - }, - { - "coverage": { - "definition_id": "b167c0953f55d24c1c15591dddb42cb4", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null + } }, "constraint": { "in": [ @@ -228,135 +25,102 @@ ] } } - ], - "relative_time": [ - { - "excluded": -3900000 - }, - { - "included": -3840000 - } ] } ] }, "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3840000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] + "materialized_views": [] }, - "installed_count": 5, - "installed_position": 4, - "node_id": 15, + "node_id": 1, "refresh": "at_ingestion", - "retention_ms": 7500000, - "stage2_node_id": 2 + "retention_ms": 7500000 } ], - "proven_optimal": true, "query": "sum(rate(http_requests_total{job=\"order-service\"}[5m] offset 1h))", "selected": { "accepted": [ { "accepted_metric": null, - "accepted_regardless_of_guarantee": false, - "guarantee_check_rejection": null, "sketches": [], - "summary": "metric not set; accepted exact", - "target_metric": null + "summary": "accepted exact" } ], "cost": { - "cost_per_second": 0.896917733207, - "cost_unit": "cost_per_second", - "cpu_core_seconds_per_second": 0.0000149176833333, + "cost_per_second": 7.90171665603, + "cpu_core_seconds_per_second": 0.0056336645, "per_node": { "0": { "cost_per_second": 0.0058, + "cost_source": "analytical", "cpu_core_seconds_per_second": 5.8e-6, "detail": "scan 1000 samples; x 1.0000/s", "multiplier": 1, "operator": "Scan", - "price_source": "analytical", "timing": "ingestion_time" }, "1": { - "cost_per_second": 0.0, - "cpu_core_seconds_per_second": 0.0, - "detail": "time shift: pass 1000 rows, free; x 1.0000/s", + "cost_per_second": 2.394, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.000126, + "detail": "build exact Rate accumulator in tumbling windows of 60000 ms for 1000 groups: 1000 rows x 70 ops, 100 group visits, 16.7 states emitted; x 1.0000/s; keeps 18144000 bytes", "multiplier": 1, - "operator": "TimeShift", - "price_source": "analytical", + "operator": "SummaryTumblingWindowChain", "timing": "ingestion_time" }, "2": { - "cost_per_second": 0.884, - "cpu_core_seconds_per_second": 2e-6, - "detail": "build exact Rate accumulator in 5 tumbling windows of 60000 ms for 1000 groups: 1000 rows x (1 + depth 1); x 1.0000/s; keeps 7056000 bytes", + "cost_per_second": 0.0, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.0, + "detail": "time shift: pass 1000 rows, free; x 1.0167/s; holds 144000 bytes for 0.000 ms", "multiplier": 1, - "operator": "SummaryTumblingWindowChain", - "price_source": "analytical", - "timing": "ingestion_time" + "operator": "TimeShift", + "timing": "query_time" }, "3": { - "cost_per_second": 0.00508338315712, - "cpu_core_seconds_per_second": 5.08333333333e-6, - "detail": "merge 5000 states into 1000; x 1.0167/s; holds 56000 bytes for 0.0 ms", + "cost_per_second": 0.0559166676732, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.0000559166666667, + "detail": "time range 300s: 5 tumbling windows of states; x 1.0167/s; holds 144 bytes for 0.055 ms", "multiplier": 1, - "operator": "SummaryMerge", - "price_source": "analytical", + "operator": "TimeRange", "timing": "query_time" }, "4": { - "cost_per_second": 0.00101666666667, - "cpu_core_seconds_per_second": 1.01666666667e-6, - "detail": "finalize 1000 accumulators; x 1.0167/s", + "cost_per_second": 2.897552155, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.0028975, + "detail": "merge 5000 states into 1000; x 1.0167/s; holds 144000 bytes for 2.850 ms", "multiplier": 1, - "operator": "FinalizeExactAccumulator", - "price_source": "analytical", + "operator": "SummaryMerge", "timing": "query_time" }, "5": { - "cost_per_second": 0.00101666671649, - "cpu_core_seconds_per_second": 1.01666666667e-6, - "detail": "build exact Sum accumulator into 1 states: 1000 rows x depth 1; x 1.0167/s; holds 56 bytes for 0.0 ms", + "cost_per_second": 2.54166666667, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.00254166666667, + "detail": "finalize 1000 accumulators; x 1.0167/s", "multiplier": 1, - "operator": "SummaryAgg", - "price_source": "analytical", + "operator": "FinalizeExactAccumulator", "timing": "query_time" }, "6": { - "cost_per_second": 1.01666666667e-6, - "cpu_core_seconds_per_second": 1.01666666667e-9, - "detail": "finalize 1 accumulators; x 1.0167/s", + "cost_per_second": 0.00678116668701, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 6.78116666667e-6, + "detail": "hash aggregate 1000 rows into 1 groups: 4 ops per row, 1 group visits; x 1.0167/s; holds 24 bytes for 0.007 ms", "multiplier": 1, - "operator": "FinalizeExactAccumulator", - "price_source": "analytical", + "operator": "Aggregate", "timing": "query_time" } }, "source": "analytical-cost-v2 (illustrative statistics)", "usage": { - "average_memory_bytes": 7056000.39899, - "cpu_ops_per_second": 10117.6833333, + "average_memory_bytes": 18144417.2482, + "cpu_ops_per_second": 5628864.5, "disk_bytes": 0, - "latency_per_query": [ - { - "compaction_ms": 0.0, - "query_ms": 0.007001 - } - ], "peak_cpu_cores": 2, - "peak_memory_bytes": 7112104, + "peak_memory_bytes": 18432216, "scanned_bytes_per_second": 48000.0 } }, @@ -377,32 +141,28 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -415,12 +175,12 @@ }, "producer": 0, "role": "Input", - "window": "RequiresAlignedPanePhaseOrExactBoundaryResidual" + "window": "NotApplicable" }, { "consumer": 2, "data_state": { - "primitive": "Raw", + "primitive": "SummaryState", "timing": "ingestion_time" }, "grouping": "NotApplicable", @@ -431,52 +191,52 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { - "Plain": "float64" + "ExactAggregate": [ + "Rate", + "Rate" + ] }, - "name": "value", - "nullable": false, - "table": null + "name": "state", + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 3, - 0 + 0, + 2, + 3 ] ] }, "producer": 1, "role": "Input", - "window": "RequiresAlignedPanePhaseOrExactBoundaryResidual" + "window": "NotApplicable" }, { "consumer": 3, "data_state": { "primitive": "SummaryState", - "timing": "ingestion_time" + "timing": "query_time" }, "grouping": "NotApplicable", "intermediate_schema": { @@ -486,9 +246,8 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { @@ -498,31 +257,29 @@ ] }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 3, - 0 + 0, + 2, + 3 ] ] }, @@ -544,9 +301,8 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { @@ -556,31 +312,29 @@ ] }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 3, - 0 + 0, + 2, + 3 ] ] }, @@ -591,7 +345,7 @@ { "consumer": 5, "data_state": { - "primitive": "Raw", + "primitive": "SummaryState", "timing": "query_time" }, "grouping": "NotApplicable", @@ -603,32 +357,31 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { - "Plain": "float64" + "ExactAggregate": [ + "Rate", + "Rate" + ] }, - "name": "value", - "nullable": false, - "table": null + "name": "state", + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -646,7 +399,7 @@ { "consumer": 6, "data_state": { - "primitive": "SummaryState", + "primitive": "Raw", "timing": "query_time" }, "grouping": "NotApplicable", @@ -655,18 +408,40 @@ "fields": [ { "dtype": { - "ExactAggregate": [ - "Sum", - "Sum" - ] + "Plain": "timestamp" }, - "name": "state", - "nullable": false, - "table": null + "name": "ts", + "nullable": false + }, + { + "dtype": { + "Plain": "float64" + }, + "name": "value", + "nullable": false + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "job", + "nullable": true + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false } ], - "time_index": null, - "unique_keys": [] + "time_index": 0, + "unique_keys": [ + [ + 3, + 0 + ] + ] }, "producer": 5, "role": "Input", @@ -675,24 +450,6 @@ ], "nodes": [ { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact over exact inputs" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 0, "lifetime": "None", "output_schema": { @@ -703,32 +460,28 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -770,32 +523,28 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -816,26 +565,10 @@ } }, { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact over exact inputs" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 1, - "lifetime": "None", + "lifetime": { + "AcrossEvaluations": "Memory" + }, "output_schema": { "closed": true, "fields": [ @@ -843,64 +576,102 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { - "Plain": "float64" + "ExactAggregate": [ + "Rate", + "Rate" + ] }, - "name": "value", - "nullable": false, - "table": null + "name": "state", + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 3, - 0 + 0, + 2, + 3 ] ] }, "output_state": { - "primitive": "Raw", + "primitive": "SummaryState", "timing": "ingestion_time" }, "payload": { - "NonASAP": { - "TimeShift": { + "ASAP": { + "SummaryTumblingWindowChain": { "child": 0, - "shift": { - "at": null, - "offset_ms": 3600000 - } + "filter": null, + "input": { + "item": null, + "weight": { + "Column": "SampleValue" + } + }, + "reduction": "PerEntity", + "summary_type": { + "ExactAggregate": [ + "Rate", + "Rate" + ] + }, + "tumbling_window_width_ms": 60000 } } + }, + "retention_ms": 7500000, + "view_coverage": { + "definition_id": "58395555aae268f4f68a8a3e82a6c45b", + "selection": [ + { + "columns": [ + { + "column": { + "column": "job", + "scan_ordinal": 0, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + } + }, + "constraint": { + "in": [ + { + "Utf8": "order-service" + } + ] + } + } + ] + } + ] } }, { - "guarantee": null, "id": 2, "lifetime": { - "AcrossEvaluations": "Memory" + "OneEvaluation": "Memory" }, "output_schema": { "closed": true, @@ -909,9 +680,8 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { @@ -921,65 +691,49 @@ ] }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 3, - 0 + 0, + 2, + 3 ] ] }, "output_state": { "primitive": "SummaryState", - "timing": "ingestion_time" + "timing": "query_time" }, "payload": { - "ASAP": { - "SummaryTumblingWindowChain": { + "NonASAP": { + "TimeShift": { "child": 1, - "family": { - "ExactAggregate": [ - "Rate", - "Rate" - ] - }, - "filter": null, - "grouping": "PerSubpopulationInstance", - "input": { - "item": null, - "weight": { - "Column": "SampleValue" - } - }, - "reduction": "PerEntity", - "tumbling_window_count": 5, - "tumbling_window_width_ms": 60000 + "shift": { + "at": null, + "offset_ms": 3600000 + } } } } }, { - "guarantee": null, "id": 3, "lifetime": "None", "output_schema": { @@ -989,9 +743,8 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { @@ -1001,31 +754,29 @@ ] }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 3, - 0 + 0, + 2, + 3 ] ] }, @@ -1034,36 +785,23 @@ "timing": "query_time" }, "payload": { - "ASAP": { - "SummaryMerge": { - "children": [ - 2 - ] + "NonASAP": { + "TimeRange": { + "child": 2, + "kind": "range", + "range": { + "nanos": 0, + "secs": 300 + } } } } }, { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact accumulator over exact input" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 4, - "lifetime": "None", + "lifetime": { + "OneEvaluation": "Memory" + }, "output_schema": { "closed": true, "fields": [ @@ -1072,32 +810,31 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { - "Plain": "float64" + "ExactAggregate": [ + "Rate", + "Rate" + ] }, - "name": "value", - "nullable": false, - "table": null + "name": "state", + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -1109,19 +846,20 @@ ] }, "output_state": { - "primitive": "Raw", + "primitive": "SummaryState", "timing": "query_time" }, "payload": { "ASAP": { - "FinalizeExactAccumulator": { - "child": 3 + "SummaryMerge": { + "children": [ + 3 + ] } } } }, { - "guarantee": null, "id": 5, "lifetime": "None", "output_schema": { @@ -1129,69 +867,58 @@ "fields": [ { "dtype": { - "ExactAggregate": [ - "Sum", - "Sum" - ] + "Plain": "timestamp" }, - "name": "state", - "nullable": false, - "table": null + "name": "ts", + "nullable": false + }, + { + "dtype": { + "Plain": "float64" + }, + "name": "value", + "nullable": false + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "job", + "nullable": true + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false } ], - "time_index": null, - "unique_keys": [] + "time_index": 0, + "unique_keys": [ + [ + 3, + 0 + ] + ] }, "output_state": { - "primitive": "SummaryState", + "primitive": "Raw", "timing": "query_time" }, "payload": { "ASAP": { - "SummaryAgg": { - "child": 4, - "family": { - "ExactAggregate": [ - "Sum", - "Sum" - ] - }, - "filter": null, - "grouping": "PerSubpopulationInstance", - "input": { - "item": null, - "weight": { - "Column": "SampleValue" - } - }, - "reduction": { - "Reduce": [] - } + "FinalizeExactAccumulator": { + "child": 4 } } } }, { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact accumulator over exact input" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 6, - "lifetime": "None", + "lifetime": { + "OneEvaluation": "Memory" + }, "output_schema": { "closed": true, "fields": [ @@ -1200,8 +927,7 @@ "Plain": "float64" }, "name": "sum", - "nullable": false, - "table": null + "nullable": false } ], "time_index": null, @@ -1212,9 +938,23 @@ "timing": "query_time" }, "payload": { - "ASAP": { - "FinalizeExactAccumulator": { - "child": 5 + "NonASAP": { + "Aggregate": { + "child": 5, + "filters": [], + "having": null, + "measures": [ + { + "col": null, + "kind": "sum" + } + ], + "output_names": [ + "" + ], + "reduction": { + "Reduce": [] + } } } } @@ -1224,10 +964,9 @@ 6 ], "scalar_roots": [] - }, - "display_name": "Q1 Sum acc + Rate acc ×5 tumbling windows · ingestion time: Rate acc for Q1, 5 × 1m tumbling windows" + } }, - "tumbling_window_retention_ms": { - "2": 7500000 - } + "warnings": [ + "assumed data fact: the data arrival declares no max_out_of_order_ms, so the samples of source http_requests_total that an ingestion-time rate, increase or irate state reads are taken to arrive in timestamp order; if this does not hold, answers may be wrong by an amount no accuracy target bounds (a late sample can be read as a counter reset), or ingestion may fail" + ] } diff --git a/crates/devtools/tests/corpus_snapshots/o11y_bench/q22.json b/crates/devtools/tests/corpus_snapshots/o11y_bench/q22.json index 2301ac533..ff7ba7f4b 100644 --- a/crates/devtools/tests/corpus_snapshots/o11y_bench/q22.json +++ b/crates/devtools/tests/corpus_snapshots/o11y_bench/q22.json @@ -3,222 +3,19 @@ "materialized_views": [ { "coverage": { - "definition_id": "b167c0953f55d24c1c15591dddb42cb4", + "definition_id": "58395555aae268f4f68a8a3e82a6c45b", "selection": [ { "columns": [ { "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "order-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -60000 - }, - { - "included": 0 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 0, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 5, - "installed_position": 0, - "node_id": 2, - "refresh": "at_ingestion", - "retention_ms": 3900000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "b167c0953f55d24c1c15591dddb42cb4", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "order-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -120000 - }, - { - "included": -60000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 60000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 5, - "installed_position": 1, - "node_id": 5, - "refresh": "at_ingestion", - "retention_ms": 3900000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "b167c0953f55d24c1c15591dddb42cb4", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "order-service" + "column": "job", + "scan_ordinal": 0, + "source": { + "TimeSeries": { + "metric": "http_requests_total" } - ] - } - } - ], - "relative_time": [ - { - "excluded": -180000 - }, - { - "included": -120000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 120000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 5, - "installed_position": 2, - "node_id": 8, - "refresh": "at_ingestion", - "retention_ms": 3900000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "b167c0953f55d24c1c15591dddb42cb4", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null - }, - "constraint": { - "in": [ - { - "Utf8": "order-service" - } - ] - } - } - ], - "relative_time": [ - { - "excluded": -240000 - }, - { - "included": -180000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 180000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 5, - "installed_position": 3, - "node_id": 11, - "refresh": "at_ingestion", - "retention_ms": 3900000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "b167c0953f55d24c1c15591dddb42cb4", - "selection": [ - { - "columns": [ - { - "column": { - "name": "job", - "table": null + } }, "constraint": { "in": [ @@ -228,126 +25,93 @@ ] } } - ], - "relative_time": [ - { - "excluded": -300000 - }, - { - "included": -240000 - } ] } ] }, "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 240000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] + "materialized_views": [] }, - "installed_count": 5, - "installed_position": 4, - "node_id": 14, + "node_id": 1, "refresh": "at_ingestion", - "retention_ms": 3900000, - "stage2_node_id": 1 + "retention_ms": 3900000 } ], - "proven_optimal": true, "query": "sum(rate(http_requests_total{job=\"order-service\"}[5m]))", "selected": { "accepted": [ { "accepted_metric": null, - "accepted_regardless_of_guarantee": false, - "guarantee_check_rejection": null, "sketches": [], - "summary": "metric not set; accepted exact", - "target_metric": null + "summary": "accepted exact" } ], "cost": { - "cost_per_second": 0.476917733207, - "cost_unit": "cost_per_second", - "cpu_core_seconds_per_second": 0.0000149176833333, + "cost_per_second": 6.82171665603, + "cpu_core_seconds_per_second": 0.0056336645, "per_node": { "0": { "cost_per_second": 0.0058, + "cost_source": "analytical", "cpu_core_seconds_per_second": 5.8e-6, "detail": "scan 1000 samples; x 1.0000/s", "multiplier": 1, "operator": "Scan", - "price_source": "analytical", "timing": "ingestion_time" }, "1": { - "cost_per_second": 0.464, - "cpu_core_seconds_per_second": 2e-6, - "detail": "build exact Rate accumulator in 5 tumbling windows of 60000 ms for 1000 groups: 1000 rows x (1 + depth 1); x 1.0000/s; keeps 3696000 bytes", + "cost_per_second": 1.314, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.000126, + "detail": "build exact Rate accumulator in tumbling windows of 60000 ms for 1000 groups: 1000 rows x 70 ops, 100 group visits, 16.7 states emitted; x 1.0000/s; keeps 9504000 bytes", "multiplier": 1, "operator": "SummaryTumblingWindowChain", - "price_source": "analytical", "timing": "ingestion_time" }, "2": { - "cost_per_second": 0.00508338315712, - "cpu_core_seconds_per_second": 5.08333333333e-6, - "detail": "merge 5000 states into 1000; x 1.0167/s; holds 56000 bytes for 0.0 ms", + "cost_per_second": 0.0559166676732, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.0000559166666667, + "detail": "time range 300s: 5 tumbling windows of states; x 1.0167/s; holds 144 bytes for 0.055 ms", "multiplier": 1, - "operator": "SummaryMerge", - "price_source": "analytical", + "operator": "TimeRange", "timing": "query_time" }, "3": { - "cost_per_second": 0.00101666666667, - "cpu_core_seconds_per_second": 1.01666666667e-6, - "detail": "finalize 1000 accumulators; x 1.0167/s", + "cost_per_second": 2.897552155, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.0028975, + "detail": "merge 5000 states into 1000; x 1.0167/s; holds 144000 bytes for 2.850 ms", "multiplier": 1, - "operator": "FinalizeExactAccumulator", - "price_source": "analytical", + "operator": "SummaryMerge", "timing": "query_time" }, "4": { - "cost_per_second": 0.00101666671649, - "cpu_core_seconds_per_second": 1.01666666667e-6, - "detail": "build exact Sum accumulator into 1 states: 1000 rows x depth 1; x 1.0167/s; holds 56 bytes for 0.0 ms", + "cost_per_second": 2.54166666667, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.00254166666667, + "detail": "finalize 1000 accumulators; x 1.0167/s", "multiplier": 1, - "operator": "SummaryAgg", - "price_source": "analytical", + "operator": "FinalizeExactAccumulator", "timing": "query_time" }, "5": { - "cost_per_second": 1.01666666667e-6, - "cpu_core_seconds_per_second": 1.01666666667e-9, - "detail": "finalize 1 accumulators; x 1.0167/s", + "cost_per_second": 0.00678116668701, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 6.78116666667e-6, + "detail": "hash aggregate 1000 rows into 1 groups: 4 ops per row, 1 group visits; x 1.0167/s; holds 24 bytes for 0.007 ms", "multiplier": 1, - "operator": "FinalizeExactAccumulator", - "price_source": "analytical", + "operator": "Aggregate", "timing": "query_time" } }, "source": "analytical-cost-v2 (illustrative statistics)", "usage": { - "average_memory_bytes": 3696000.39899, - "cpu_ops_per_second": 10117.6833333, + "average_memory_bytes": 9504417.24821, + "cpu_ops_per_second": 5628864.5, "disk_bytes": 0, - "latency_per_query": [ - { - "compaction_ms": 0.0, - "query_ms": 0.007001 - } - ], "peak_cpu_cores": 2, - "peak_memory_bytes": 3752104, + "peak_memory_bytes": 9648216, "scanned_bytes_per_second": 48000.0 } }, @@ -368,32 +132,28 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -406,7 +166,7 @@ }, "producer": 0, "role": "Input", - "window": "RequiresAlignedPanePhaseOrExactBoundaryResidual" + "window": "NotApplicable" }, { "consumer": 2, @@ -422,9 +182,8 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { @@ -434,31 +193,29 @@ ] }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 3, - 0 + 0, + 2, + 3 ] ] }, @@ -480,9 +237,8 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { @@ -492,31 +248,29 @@ ] }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 3, - 0 + 0, + 2, + 3 ] ] }, @@ -527,7 +281,7 @@ { "consumer": 4, "data_state": { - "primitive": "Raw", + "primitive": "SummaryState", "timing": "query_time" }, "grouping": "NotApplicable", @@ -539,32 +293,31 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { - "Plain": "float64" + "ExactAggregate": [ + "Rate", + "Rate" + ] }, - "name": "value", - "nullable": false, - "table": null + "name": "state", + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -582,7 +335,7 @@ { "consumer": 5, "data_state": { - "primitive": "SummaryState", + "primitive": "Raw", "timing": "query_time" }, "grouping": "NotApplicable", @@ -591,18 +344,40 @@ "fields": [ { "dtype": { - "ExactAggregate": [ - "Sum", - "Sum" - ] + "Plain": "timestamp" }, - "name": "state", - "nullable": false, - "table": null + "name": "ts", + "nullable": false + }, + { + "dtype": { + "Plain": "float64" + }, + "name": "value", + "nullable": false + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "job", + "nullable": true + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false } ], - "time_index": null, - "unique_keys": [] + "time_index": 0, + "unique_keys": [ + [ + 3, + 0 + ] + ] }, "producer": 4, "role": "Input", @@ -611,24 +386,6 @@ ], "nodes": [ { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact over exact inputs" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 0, "lifetime": "None", "output_schema": { @@ -639,32 +396,28 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -706,32 +459,28 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -752,7 +501,6 @@ } }, { - "guarantee": null, "id": 1, "lifetime": { "AcrossEvaluations": "Memory" @@ -764,9 +512,8 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { @@ -776,31 +523,29 @@ ] }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 3, - 0 + 0, + 2, + 3 ] ] }, @@ -812,14 +557,7 @@ "ASAP": { "SummaryTumblingWindowChain": { "child": 0, - "family": { - "ExactAggregate": [ - "Rate", - "Rate" - ] - }, "filter": null, - "grouping": "PerSubpopulationInstance", "input": { "item": null, "weight": { @@ -827,14 +565,46 @@ } }, "reduction": "PerEntity", - "tumbling_window_count": 5, + "summary_type": { + "ExactAggregate": [ + "Rate", + "Rate" + ] + }, "tumbling_window_width_ms": 60000 } } + }, + "retention_ms": 3900000, + "view_coverage": { + "definition_id": "58395555aae268f4f68a8a3e82a6c45b", + "selection": [ + { + "columns": [ + { + "column": { + "column": "job", + "scan_ordinal": 0, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + } + }, + "constraint": { + "in": [ + { + "Utf8": "order-service" + } + ] + } + } + ] + } + ] } }, { - "guarantee": null, "id": 2, "lifetime": "None", "output_schema": { @@ -844,9 +614,8 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { @@ -856,31 +625,29 @@ ] }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 3, - 0 + 0, + 2, + 3 ] ] }, @@ -889,36 +656,23 @@ "timing": "query_time" }, "payload": { - "ASAP": { - "SummaryMerge": { - "children": [ - 1 - ] + "NonASAP": { + "TimeRange": { + "child": 1, + "kind": "range", + "range": { + "nanos": 0, + "secs": 300 + } } } } }, { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact accumulator over exact input" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 3, - "lifetime": "None", + "lifetime": { + "OneEvaluation": "Memory" + }, "output_schema": { "closed": true, "fields": [ @@ -927,32 +681,31 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { - "Plain": "float64" + "ExactAggregate": [ + "Rate", + "Rate" + ] }, - "name": "value", - "nullable": false, - "table": null + "name": "state", + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -964,19 +717,20 @@ ] }, "output_state": { - "primitive": "Raw", + "primitive": "SummaryState", "timing": "query_time" }, "payload": { "ASAP": { - "FinalizeExactAccumulator": { - "child": 2 + "SummaryMerge": { + "children": [ + 2 + ] } } } }, { - "guarantee": null, "id": 4, "lifetime": "None", "output_schema": { @@ -984,69 +738,58 @@ "fields": [ { "dtype": { - "ExactAggregate": [ - "Sum", - "Sum" - ] + "Plain": "timestamp" }, - "name": "state", - "nullable": false, - "table": null + "name": "ts", + "nullable": false + }, + { + "dtype": { + "Plain": "float64" + }, + "name": "value", + "nullable": false + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "job", + "nullable": true + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false } ], - "time_index": null, - "unique_keys": [] + "time_index": 0, + "unique_keys": [ + [ + 3, + 0 + ] + ] }, "output_state": { - "primitive": "SummaryState", + "primitive": "Raw", "timing": "query_time" }, "payload": { "ASAP": { - "SummaryAgg": { - "child": 3, - "family": { - "ExactAggregate": [ - "Sum", - "Sum" - ] - }, - "filter": null, - "grouping": "PerSubpopulationInstance", - "input": { - "item": null, - "weight": { - "Column": "SampleValue" - } - }, - "reduction": { - "Reduce": [] - } + "FinalizeExactAccumulator": { + "child": 3 } } } }, { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact accumulator over exact input" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 5, - "lifetime": "None", + "lifetime": { + "OneEvaluation": "Memory" + }, "output_schema": { "closed": true, "fields": [ @@ -1055,8 +798,7 @@ "Plain": "float64" }, "name": "sum", - "nullable": false, - "table": null + "nullable": false } ], "time_index": null, @@ -1067,9 +809,23 @@ "timing": "query_time" }, "payload": { - "ASAP": { - "FinalizeExactAccumulator": { - "child": 4 + "NonASAP": { + "Aggregate": { + "child": 4, + "filters": [], + "having": null, + "measures": [ + { + "col": null, + "kind": "sum" + } + ], + "output_names": [ + "" + ], + "reduction": { + "Reduce": [] + } } } } @@ -1079,10 +835,9 @@ 5 ], "scalar_roots": [] - }, - "display_name": "Q1 Sum acc + Rate acc ×5 tumbling windows · ingestion time: Rate acc for Q1, 5 × 1m tumbling windows" + } }, - "tumbling_window_retention_ms": { - "1": 3900000 - } + "warnings": [ + "assumed data fact: the data arrival declares no max_out_of_order_ms, so the samples of source http_requests_total that an ingestion-time rate, increase or irate state reads are taken to arrive in timestamp order; if this does not hold, answers may be wrong by an amount no accuracy target bounds (a late sample can be read as a counter reset), or ingestion may fail" + ] } diff --git a/crates/devtools/tests/corpus_snapshots/o11y_bench/q23.json b/crates/devtools/tests/corpus_snapshots/o11y_bench/q23.json index 25a992051..3f5f02164 100644 --- a/crates/devtools/tests/corpus_snapshots/o11y_bench/q23.json +++ b/crates/devtools/tests/corpus_snapshots/o11y_bench/q23.json @@ -3,281 +3,96 @@ "materialized_views": [ { "coverage": { - "definition_id": "983f5e2904acda1d575c27b0861b6380", + "definition_id": "c040dfa3c5b9e9aff9110d588012a461", "selection": [ { - "columns": [], - "relative_time": [ - { - "excluded": -60000 - }, - { - "included": 0 - } - ] + "columns": [] } ] }, "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 0, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 5, - "installed_position": 0, - "node_id": 2, - "refresh": "at_ingestion", - "retention_ms": 3900000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "983f5e2904acda1d575c27b0861b6380", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -120000 - }, - { - "included": -60000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 60000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 5, - "installed_position": 1, - "node_id": 5, - "refresh": "at_ingestion", - "retention_ms": 3900000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "983f5e2904acda1d575c27b0861b6380", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -180000 - }, - { - "included": -120000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 120000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 5, - "installed_position": 2, - "node_id": 8, - "refresh": "at_ingestion", - "retention_ms": 3900000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "983f5e2904acda1d575c27b0861b6380", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -240000 - }, - { - "included": -180000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 180000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] + "materialized_views": [] }, - "installed_count": 5, - "installed_position": 3, - "node_id": 11, + "node_id": 1, "refresh": "at_ingestion", - "retention_ms": 3900000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "983f5e2904acda1d575c27b0861b6380", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -300000 - }, - { - "included": -240000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 240000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 5, - "installed_position": 4, - "node_id": 14, - "refresh": "at_ingestion", - "retention_ms": 3900000, - "stage2_node_id": 1 + "retention_ms": 3900000 } ], - "proven_optimal": true, "query": "sum(rate(http_requests_total{job=~\"user-service|order-service|payment-service\"}[5m]))", "selected": { "accepted": [ { "accepted_metric": null, - "accepted_regardless_of_guarantee": false, - "guarantee_check_rejection": null, "sketches": [], - "summary": "metric not set; accepted exact", - "target_metric": null + "summary": "accepted exact" } ], "cost": { - "cost_per_second": 0.476917733207, - "cost_unit": "cost_per_second", - "cpu_core_seconds_per_second": 0.0000149176833333, + "cost_per_second": 6.82171665603, + "cpu_core_seconds_per_second": 0.0056336645, "per_node": { "0": { "cost_per_second": 0.0058, + "cost_source": "analytical", "cpu_core_seconds_per_second": 5.8e-6, "detail": "scan 1000 samples; x 1.0000/s", "multiplier": 1, "operator": "Scan", - "price_source": "analytical", "timing": "ingestion_time" }, "1": { - "cost_per_second": 0.464, - "cpu_core_seconds_per_second": 2e-6, - "detail": "build exact Rate accumulator in 5 tumbling windows of 60000 ms for 1000 groups: 1000 rows x (1 + depth 1); x 1.0000/s; keeps 3696000 bytes", + "cost_per_second": 1.314, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.000126, + "detail": "build exact Rate accumulator in tumbling windows of 60000 ms for 1000 groups: 1000 rows x 70 ops, 100 group visits, 16.7 states emitted; x 1.0000/s; keeps 9504000 bytes", "multiplier": 1, "operator": "SummaryTumblingWindowChain", - "price_source": "analytical", "timing": "ingestion_time" }, "2": { - "cost_per_second": 0.00508338315712, - "cpu_core_seconds_per_second": 5.08333333333e-6, - "detail": "merge 5000 states into 1000; x 1.0167/s; holds 56000 bytes for 0.0 ms", + "cost_per_second": 0.0559166676732, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.0000559166666667, + "detail": "time range 300s: 5 tumbling windows of states; x 1.0167/s; holds 144 bytes for 0.055 ms", "multiplier": 1, - "operator": "SummaryMerge", - "price_source": "analytical", + "operator": "TimeRange", "timing": "query_time" }, "3": { - "cost_per_second": 0.00101666666667, - "cpu_core_seconds_per_second": 1.01666666667e-6, - "detail": "finalize 1000 accumulators; x 1.0167/s", + "cost_per_second": 2.897552155, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.0028975, + "detail": "merge 5000 states into 1000; x 1.0167/s; holds 144000 bytes for 2.850 ms", "multiplier": 1, - "operator": "FinalizeExactAccumulator", - "price_source": "analytical", + "operator": "SummaryMerge", "timing": "query_time" }, "4": { - "cost_per_second": 0.00101666671649, - "cpu_core_seconds_per_second": 1.01666666667e-6, - "detail": "build exact Sum accumulator into 1 states: 1000 rows x depth 1; x 1.0167/s; holds 56 bytes for 0.0 ms", + "cost_per_second": 2.54166666667, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.00254166666667, + "detail": "finalize 1000 accumulators; x 1.0167/s", "multiplier": 1, - "operator": "SummaryAgg", - "price_source": "analytical", + "operator": "FinalizeExactAccumulator", "timing": "query_time" }, "5": { - "cost_per_second": 1.01666666667e-6, - "cpu_core_seconds_per_second": 1.01666666667e-9, - "detail": "finalize 1 accumulators; x 1.0167/s", + "cost_per_second": 0.00678116668701, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 6.78116666667e-6, + "detail": "hash aggregate 1000 rows into 1 groups: 4 ops per row, 1 group visits; x 1.0167/s; holds 24 bytes for 0.007 ms", "multiplier": 1, - "operator": "FinalizeExactAccumulator", - "price_source": "analytical", + "operator": "Aggregate", "timing": "query_time" } }, "source": "analytical-cost-v2 (illustrative statistics)", "usage": { - "average_memory_bytes": 3696000.39899, - "cpu_ops_per_second": 10117.6833333, + "average_memory_bytes": 9504417.24821, + "cpu_ops_per_second": 5628864.5, "disk_bytes": 0, - "latency_per_query": [ - { - "compaction_ms": 0.0, - "query_ms": 0.007001 - } - ], "peak_cpu_cores": 2, - "peak_memory_bytes": 3752104, + "peak_memory_bytes": 9648216, "scanned_bytes_per_second": 48000.0 } }, @@ -298,32 +113,28 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -336,7 +147,7 @@ }, "producer": 0, "role": "Input", - "window": "RequiresAlignedPanePhaseOrExactBoundaryResidual" + "window": "NotApplicable" }, { "consumer": 2, @@ -352,9 +163,8 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { @@ -364,31 +174,29 @@ ] }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 3, - 0 + 0, + 2, + 3 ] ] }, @@ -410,9 +218,8 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { @@ -422,31 +229,29 @@ ] }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 3, - 0 + 0, + 2, + 3 ] ] }, @@ -457,7 +262,7 @@ { "consumer": 4, "data_state": { - "primitive": "Raw", + "primitive": "SummaryState", "timing": "query_time" }, "grouping": "NotApplicable", @@ -469,32 +274,31 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { - "Plain": "float64" + "ExactAggregate": [ + "Rate", + "Rate" + ] }, - "name": "value", - "nullable": false, - "table": null + "name": "state", + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -512,7 +316,7 @@ { "consumer": 5, "data_state": { - "primitive": "SummaryState", + "primitive": "Raw", "timing": "query_time" }, "grouping": "NotApplicable", @@ -521,18 +325,40 @@ "fields": [ { "dtype": { - "ExactAggregate": [ - "Sum", - "Sum" - ] + "Plain": "timestamp" }, - "name": "state", - "nullable": false, - "table": null + "name": "ts", + "nullable": false + }, + { + "dtype": { + "Plain": "float64" + }, + "name": "value", + "nullable": false + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "job", + "nullable": true + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false } ], - "time_index": null, - "unique_keys": [] + "time_index": 0, + "unique_keys": [ + [ + 3, + 0 + ] + ] }, "producer": 4, "role": "Input", @@ -541,24 +367,6 @@ ], "nodes": [ { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact over exact inputs" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 0, "lifetime": "None", "output_schema": { @@ -569,32 +377,28 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -636,32 +440,28 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -682,7 +482,6 @@ } }, { - "guarantee": null, "id": 1, "lifetime": { "AcrossEvaluations": "Memory" @@ -694,9 +493,8 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { @@ -706,31 +504,29 @@ ] }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 3, - 0 + 0, + 2, + 3 ] ] }, @@ -742,14 +538,7 @@ "ASAP": { "SummaryTumblingWindowChain": { "child": 0, - "family": { - "ExactAggregate": [ - "Rate", - "Rate" - ] - }, "filter": null, - "grouping": "PerSubpopulationInstance", "input": { "item": null, "weight": { @@ -757,14 +546,27 @@ } }, "reduction": "PerEntity", - "tumbling_window_count": 5, + "summary_type": { + "ExactAggregate": [ + "Rate", + "Rate" + ] + }, "tumbling_window_width_ms": 60000 } } + }, + "retention_ms": 3900000, + "view_coverage": { + "definition_id": "c040dfa3c5b9e9aff9110d588012a461", + "selection": [ + { + "columns": [] + } + ] } }, { - "guarantee": null, "id": 2, "lifetime": "None", "output_schema": { @@ -774,9 +576,8 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { @@ -786,31 +587,29 @@ ] }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 3, - 0 + 0, + 2, + 3 ] ] }, @@ -819,36 +618,23 @@ "timing": "query_time" }, "payload": { - "ASAP": { - "SummaryMerge": { - "children": [ - 1 - ] + "NonASAP": { + "TimeRange": { + "child": 1, + "kind": "range", + "range": { + "nanos": 0, + "secs": 300 + } } } } }, { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact accumulator over exact input" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 3, - "lifetime": "None", + "lifetime": { + "OneEvaluation": "Memory" + }, "output_schema": { "closed": true, "fields": [ @@ -857,32 +643,31 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { - "Plain": "float64" + "ExactAggregate": [ + "Rate", + "Rate" + ] }, - "name": "value", - "nullable": false, - "table": null + "name": "state", + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -894,19 +679,20 @@ ] }, "output_state": { - "primitive": "Raw", + "primitive": "SummaryState", "timing": "query_time" }, "payload": { "ASAP": { - "FinalizeExactAccumulator": { - "child": 2 + "SummaryMerge": { + "children": [ + 2 + ] } } } }, { - "guarantee": null, "id": 4, "lifetime": "None", "output_schema": { @@ -914,69 +700,58 @@ "fields": [ { "dtype": { - "ExactAggregate": [ - "Sum", - "Sum" - ] + "Plain": "timestamp" }, - "name": "state", - "nullable": false, - "table": null + "name": "ts", + "nullable": false + }, + { + "dtype": { + "Plain": "float64" + }, + "name": "value", + "nullable": false + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "job", + "nullable": true + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false } ], - "time_index": null, - "unique_keys": [] + "time_index": 0, + "unique_keys": [ + [ + 3, + 0 + ] + ] }, "output_state": { - "primitive": "SummaryState", + "primitive": "Raw", "timing": "query_time" }, "payload": { "ASAP": { - "SummaryAgg": { - "child": 3, - "family": { - "ExactAggregate": [ - "Sum", - "Sum" - ] - }, - "filter": null, - "grouping": "PerSubpopulationInstance", - "input": { - "item": null, - "weight": { - "Column": "SampleValue" - } - }, - "reduction": { - "Reduce": [] - } + "FinalizeExactAccumulator": { + "child": 3 } } } }, { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact accumulator over exact input" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 5, - "lifetime": "None", + "lifetime": { + "OneEvaluation": "Memory" + }, "output_schema": { "closed": true, "fields": [ @@ -985,8 +760,7 @@ "Plain": "float64" }, "name": "sum", - "nullable": false, - "table": null + "nullable": false } ], "time_index": null, @@ -997,9 +771,23 @@ "timing": "query_time" }, "payload": { - "ASAP": { - "FinalizeExactAccumulator": { - "child": 4 + "NonASAP": { + "Aggregate": { + "child": 4, + "filters": [], + "having": null, + "measures": [ + { + "col": null, + "kind": "sum" + } + ], + "output_names": [ + "" + ], + "reduction": { + "Reduce": [] + } } } } @@ -1009,10 +797,9 @@ 5 ], "scalar_roots": [] - }, - "display_name": "Q1 Sum acc + Rate acc ×5 tumbling windows · ingestion time: Rate acc for Q1, 5 × 1m tumbling windows" + } }, - "tumbling_window_retention_ms": { - "1": 3900000 - } + "warnings": [ + "assumed data fact: the data arrival declares no max_out_of_order_ms, so the samples of source http_requests_total that an ingestion-time rate, increase or irate state reads are taken to arrive in timestamp order; if this does not hold, answers may be wrong by an amount no accuracy target bounds (a late sample can be read as a counter reset), or ingestion may fail" + ] } diff --git a/crates/devtools/tests/corpus_snapshots/o11y_bench/q25.json b/crates/devtools/tests/corpus_snapshots/o11y_bench/q25.json index 7f7397165..0e33b7378 100644 --- a/crates/devtools/tests/corpus_snapshots/o11y_bench/q25.json +++ b/crates/devtools/tests/corpus_snapshots/o11y_bench/q25.json @@ -3,4714 +3,175 @@ "materialized_views": [ { "coverage": { - "definition_id": "0e9e6da224697c324f6268c4637236dd", + "definition_id": "59e8b0f304a546294da7b0f4fcb52e4f", "selection": [ { - "columns": [], - "relative_time": [ - { - "excluded": -60000 - }, - { - "included": 0 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 0, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 0, - "node_id": 2, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "0e9e6da224697c324f6268c4637236dd", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -120000 - }, - { - "included": -60000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 60000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 1, - "node_id": 5, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "0e9e6da224697c324f6268c4637236dd", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -180000 - }, - { - "included": -120000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 120000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 2, - "node_id": 8, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "0e9e6da224697c324f6268c4637236dd", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -240000 - }, - { - "included": -180000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 180000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 3, - "node_id": 11, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "0e9e6da224697c324f6268c4637236dd", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -300000 - }, - { - "included": -240000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 240000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 4, - "node_id": 14, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "0e9e6da224697c324f6268c4637236dd", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -360000 - }, - { - "included": -300000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 300000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 5, - "node_id": 17, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "0e9e6da224697c324f6268c4637236dd", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -420000 - }, - { - "included": -360000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 360000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 6, - "node_id": 20, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "0e9e6da224697c324f6268c4637236dd", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -480000 - }, - { - "included": -420000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 420000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 7, - "node_id": 23, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "0e9e6da224697c324f6268c4637236dd", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -540000 - }, - { - "included": -480000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 480000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 8, - "node_id": 26, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "0e9e6da224697c324f6268c4637236dd", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -600000 - }, - { - "included": -540000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 540000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 9, - "node_id": 29, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "0e9e6da224697c324f6268c4637236dd", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -660000 - }, - { - "included": -600000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 600000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 10, - "node_id": 32, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "0e9e6da224697c324f6268c4637236dd", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -720000 - }, - { - "included": -660000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 660000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 11, - "node_id": 35, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "0e9e6da224697c324f6268c4637236dd", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -780000 - }, - { - "included": -720000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 720000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 12, - "node_id": 38, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "0e9e6da224697c324f6268c4637236dd", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -840000 - }, - { - "included": -780000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 780000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 13, - "node_id": 41, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "0e9e6da224697c324f6268c4637236dd", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -900000 - }, - { - "included": -840000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 840000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 14, - "node_id": 44, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "0e9e6da224697c324f6268c4637236dd", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -960000 - }, - { - "included": -900000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 900000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 15, - "node_id": 47, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "0e9e6da224697c324f6268c4637236dd", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1020000 - }, - { - "included": -960000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 960000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 16, - "node_id": 50, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "0e9e6da224697c324f6268c4637236dd", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1080000 - }, - { - "included": -1020000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1020000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 17, - "node_id": 53, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "0e9e6da224697c324f6268c4637236dd", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1140000 - }, - { - "included": -1080000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1080000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 18, - "node_id": 56, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "0e9e6da224697c324f6268c4637236dd", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1200000 - }, - { - "included": -1140000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1140000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 19, - "node_id": 59, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "0e9e6da224697c324f6268c4637236dd", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1260000 - }, - { - "included": -1200000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1200000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 20, - "node_id": 62, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "0e9e6da224697c324f6268c4637236dd", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1320000 - }, - { - "included": -1260000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1260000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 21, - "node_id": 65, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "0e9e6da224697c324f6268c4637236dd", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1380000 - }, - { - "included": -1320000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1320000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 22, - "node_id": 68, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "0e9e6da224697c324f6268c4637236dd", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1440000 - }, - { - "included": -1380000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1380000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 23, - "node_id": 71, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "0e9e6da224697c324f6268c4637236dd", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1500000 - }, - { - "included": -1440000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1440000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 24, - "node_id": 74, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "0e9e6da224697c324f6268c4637236dd", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1560000 - }, - { - "included": -1500000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1500000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 25, - "node_id": 77, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "0e9e6da224697c324f6268c4637236dd", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1620000 - }, - { - "included": -1560000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1560000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 26, - "node_id": 80, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "0e9e6da224697c324f6268c4637236dd", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1680000 - }, - { - "included": -1620000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1620000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 27, - "node_id": 83, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "0e9e6da224697c324f6268c4637236dd", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1740000 - }, - { - "included": -1680000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1680000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 28, - "node_id": 86, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "0e9e6da224697c324f6268c4637236dd", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1800000 - }, - { - "included": -1740000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1740000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 29, - "node_id": 89, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "0e9e6da224697c324f6268c4637236dd", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1860000 - }, - { - "included": -1800000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1800000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 30, - "node_id": 92, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "0e9e6da224697c324f6268c4637236dd", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1920000 - }, - { - "included": -1860000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1860000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 31, - "node_id": 95, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "0e9e6da224697c324f6268c4637236dd", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1980000 - }, - { - "included": -1920000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1920000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 32, - "node_id": 98, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "0e9e6da224697c324f6268c4637236dd", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2040000 - }, - { - "included": -1980000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1980000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 33, - "node_id": 101, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "0e9e6da224697c324f6268c4637236dd", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2100000 - }, - { - "included": -2040000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2040000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 34, - "node_id": 104, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "0e9e6da224697c324f6268c4637236dd", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2160000 - }, - { - "included": -2100000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2100000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 35, - "node_id": 107, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "0e9e6da224697c324f6268c4637236dd", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2220000 - }, - { - "included": -2160000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2160000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 36, - "node_id": 110, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "0e9e6da224697c324f6268c4637236dd", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2280000 - }, - { - "included": -2220000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2220000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 37, - "node_id": 113, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "0e9e6da224697c324f6268c4637236dd", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2340000 - }, - { - "included": -2280000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2280000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 38, - "node_id": 116, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "0e9e6da224697c324f6268c4637236dd", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2400000 - }, - { - "included": -2340000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2340000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 39, - "node_id": 119, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "0e9e6da224697c324f6268c4637236dd", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2460000 - }, - { - "included": -2400000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2400000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 40, - "node_id": 122, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "0e9e6da224697c324f6268c4637236dd", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2520000 - }, - { - "included": -2460000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2460000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 41, - "node_id": 125, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "0e9e6da224697c324f6268c4637236dd", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2580000 - }, - { - "included": -2520000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2520000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 42, - "node_id": 128, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "0e9e6da224697c324f6268c4637236dd", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2640000 - }, - { - "included": -2580000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2580000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 43, - "node_id": 131, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "0e9e6da224697c324f6268c4637236dd", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2700000 - }, - { - "included": -2640000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2640000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 44, - "node_id": 134, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "0e9e6da224697c324f6268c4637236dd", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2760000 - }, - { - "included": -2700000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2700000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 45, - "node_id": 137, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "0e9e6da224697c324f6268c4637236dd", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2820000 - }, - { - "included": -2760000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2760000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 46, - "node_id": 140, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "0e9e6da224697c324f6268c4637236dd", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2880000 - }, - { - "included": -2820000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2820000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 47, - "node_id": 143, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "0e9e6da224697c324f6268c4637236dd", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2940000 - }, - { - "included": -2880000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2880000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 48, - "node_id": 146, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "0e9e6da224697c324f6268c4637236dd", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -3000000 - }, - { - "included": -2940000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2940000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 49, - "node_id": 149, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "0e9e6da224697c324f6268c4637236dd", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -3060000 - }, - { - "included": -3000000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3000000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 50, - "node_id": 152, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "0e9e6da224697c324f6268c4637236dd", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -3120000 - }, - { - "included": -3060000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3060000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 51, - "node_id": 155, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "0e9e6da224697c324f6268c4637236dd", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -3180000 - }, - { - "included": -3120000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3120000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 52, - "node_id": 158, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "0e9e6da224697c324f6268c4637236dd", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -3240000 - }, - { - "included": -3180000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3180000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 53, - "node_id": 161, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "0e9e6da224697c324f6268c4637236dd", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -3300000 - }, - { - "included": -3240000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3240000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 54, - "node_id": 164, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "0e9e6da224697c324f6268c4637236dd", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -3360000 - }, - { - "included": -3300000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3300000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 55, - "node_id": 167, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "0e9e6da224697c324f6268c4637236dd", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -3420000 - }, - { - "included": -3360000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3360000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 56, - "node_id": 170, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "0e9e6da224697c324f6268c4637236dd", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -3480000 - }, - { - "included": -3420000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3420000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 57, - "node_id": 173, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "0e9e6da224697c324f6268c4637236dd", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -3540000 - }, - { - "included": -3480000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3480000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 58, - "node_id": 176, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "0e9e6da224697c324f6268c4637236dd", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -3600000 - }, - { - "included": -3540000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3540000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 59, - "node_id": 179, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "983f5e2904acda1d575c27b0861b6380", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -60000 - }, - { - "included": 0 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 0, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 0, - "node_id": 186, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "983f5e2904acda1d575c27b0861b6380", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -120000 - }, - { - "included": -60000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 60000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 1, - "node_id": 189, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "983f5e2904acda1d575c27b0861b6380", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -180000 - }, - { - "included": -120000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 120000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 2, - "node_id": 192, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "983f5e2904acda1d575c27b0861b6380", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -240000 - }, - { - "included": -180000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 180000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 3, - "node_id": 195, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "983f5e2904acda1d575c27b0861b6380", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -300000 - }, - { - "included": -240000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 240000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 4, - "node_id": 198, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "983f5e2904acda1d575c27b0861b6380", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -360000 - }, - { - "included": -300000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 300000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 5, - "node_id": 201, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "983f5e2904acda1d575c27b0861b6380", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -420000 - }, - { - "included": -360000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 360000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 6, - "node_id": 204, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "983f5e2904acda1d575c27b0861b6380", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -480000 - }, - { - "included": -420000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 420000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 7, - "node_id": 207, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "983f5e2904acda1d575c27b0861b6380", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -540000 - }, - { - "included": -480000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 480000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 8, - "node_id": 210, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "983f5e2904acda1d575c27b0861b6380", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -600000 - }, - { - "included": -540000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 540000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 9, - "node_id": 213, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "983f5e2904acda1d575c27b0861b6380", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -660000 - }, - { - "included": -600000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 600000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 10, - "node_id": 216, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "983f5e2904acda1d575c27b0861b6380", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -720000 - }, - { - "included": -660000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 660000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 11, - "node_id": 219, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "983f5e2904acda1d575c27b0861b6380", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -780000 - }, - { - "included": -720000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 720000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 12, - "node_id": 222, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "983f5e2904acda1d575c27b0861b6380", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -840000 - }, - { - "included": -780000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 780000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 13, - "node_id": 225, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "983f5e2904acda1d575c27b0861b6380", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -900000 - }, - { - "included": -840000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 840000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 14, - "node_id": 228, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "983f5e2904acda1d575c27b0861b6380", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -960000 - }, - { - "included": -900000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 900000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 15, - "node_id": 231, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "983f5e2904acda1d575c27b0861b6380", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1020000 - }, - { - "included": -960000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 960000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 16, - "node_id": 234, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "983f5e2904acda1d575c27b0861b6380", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1080000 - }, - { - "included": -1020000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1020000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 17, - "node_id": 237, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "983f5e2904acda1d575c27b0861b6380", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1140000 - }, - { - "included": -1080000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1080000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 18, - "node_id": 240, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "983f5e2904acda1d575c27b0861b6380", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1200000 - }, - { - "included": -1140000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1140000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 19, - "node_id": 243, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "983f5e2904acda1d575c27b0861b6380", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1260000 - }, - { - "included": -1200000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1200000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 20, - "node_id": 246, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "983f5e2904acda1d575c27b0861b6380", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1320000 - }, - { - "included": -1260000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1260000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 21, - "node_id": 249, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "983f5e2904acda1d575c27b0861b6380", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1380000 - }, - { - "included": -1320000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1320000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 22, - "node_id": 252, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "983f5e2904acda1d575c27b0861b6380", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1440000 - }, - { - "included": -1380000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1380000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 23, - "node_id": 255, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "983f5e2904acda1d575c27b0861b6380", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1500000 - }, - { - "included": -1440000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1440000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 24, - "node_id": 258, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "983f5e2904acda1d575c27b0861b6380", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1560000 - }, - { - "included": -1500000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1500000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 25, - "node_id": 261, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "983f5e2904acda1d575c27b0861b6380", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1620000 - }, - { - "included": -1560000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1560000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 26, - "node_id": 264, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "983f5e2904acda1d575c27b0861b6380", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1680000 - }, - { - "included": -1620000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1620000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 27, - "node_id": 267, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "983f5e2904acda1d575c27b0861b6380", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1740000 - }, - { - "included": -1680000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1680000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 28, - "node_id": 270, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "983f5e2904acda1d575c27b0861b6380", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1800000 - }, - { - "included": -1740000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1740000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 29, - "node_id": 273, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "983f5e2904acda1d575c27b0861b6380", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1860000 - }, - { - "included": -1800000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1800000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 30, - "node_id": 276, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "983f5e2904acda1d575c27b0861b6380", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1920000 - }, - { - "included": -1860000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1860000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 31, - "node_id": 279, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "983f5e2904acda1d575c27b0861b6380", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1980000 - }, - { - "included": -1920000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1920000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 32, - "node_id": 282, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "983f5e2904acda1d575c27b0861b6380", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2040000 - }, - { - "included": -1980000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1980000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 33, - "node_id": 285, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "983f5e2904acda1d575c27b0861b6380", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2100000 - }, - { - "included": -2040000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2040000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 34, - "node_id": 288, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "983f5e2904acda1d575c27b0861b6380", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2160000 - }, - { - "included": -2100000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2100000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 35, - "node_id": 291, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "983f5e2904acda1d575c27b0861b6380", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2220000 - }, - { - "included": -2160000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2160000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 36, - "node_id": 294, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "983f5e2904acda1d575c27b0861b6380", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2280000 - }, - { - "included": -2220000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2220000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 37, - "node_id": 297, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "983f5e2904acda1d575c27b0861b6380", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2340000 - }, - { - "included": -2280000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2280000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 38, - "node_id": 300, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "983f5e2904acda1d575c27b0861b6380", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2400000 - }, - { - "included": -2340000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2340000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 39, - "node_id": 303, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "983f5e2904acda1d575c27b0861b6380", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2460000 - }, - { - "included": -2400000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2400000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 40, - "node_id": 306, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "983f5e2904acda1d575c27b0861b6380", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2520000 - }, - { - "included": -2460000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2460000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 41, - "node_id": 309, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "983f5e2904acda1d575c27b0861b6380", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2580000 - }, - { - "included": -2520000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2520000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 42, - "node_id": 312, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "983f5e2904acda1d575c27b0861b6380", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2640000 - }, - { - "included": -2580000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2580000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 43, - "node_id": 315, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "983f5e2904acda1d575c27b0861b6380", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2700000 - }, - { - "included": -2640000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2640000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 44, - "node_id": 318, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "983f5e2904acda1d575c27b0861b6380", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2760000 - }, - { - "included": -2700000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2700000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 45, - "node_id": 321, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "983f5e2904acda1d575c27b0861b6380", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2820000 - }, - { - "included": -2760000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2760000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 46, - "node_id": 324, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "983f5e2904acda1d575c27b0861b6380", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2880000 - }, - { - "included": -2820000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2820000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 47, - "node_id": 327, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "983f5e2904acda1d575c27b0861b6380", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2940000 - }, - { - "included": -2880000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2880000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 48, - "node_id": 330, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "983f5e2904acda1d575c27b0861b6380", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -3000000 - }, - { - "included": -2940000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2940000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 49, - "node_id": 333, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "983f5e2904acda1d575c27b0861b6380", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -3060000 - }, - { - "included": -3000000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3000000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 50, - "node_id": 336, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "983f5e2904acda1d575c27b0861b6380", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -3120000 - }, - { - "included": -3060000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3060000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 51, - "node_id": 339, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "983f5e2904acda1d575c27b0861b6380", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -3180000 - }, - { - "included": -3120000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3120000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 52, - "node_id": 342, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "983f5e2904acda1d575c27b0861b6380", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -3240000 - }, - { - "included": -3180000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3180000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 53, - "node_id": 345, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "983f5e2904acda1d575c27b0861b6380", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -3300000 - }, - { - "included": -3240000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3240000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 54, - "node_id": 348, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "983f5e2904acda1d575c27b0861b6380", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -3360000 - }, - { - "included": -3300000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3300000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 55, - "node_id": 351, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "983f5e2904acda1d575c27b0861b6380", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -3420000 - }, - { - "included": -3360000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3360000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 56, - "node_id": 354, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "983f5e2904acda1d575c27b0861b6380", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -3480000 - }, - { - "included": -3420000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3420000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 57, - "node_id": 357, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 - }, - { - "coverage": { - "definition_id": "983f5e2904acda1d575c27b0861b6380", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -3540000 - }, - { - "included": -3480000 - } - ] + "columns": [] } ] }, "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3480000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] + "materialized_views": [] }, - "installed_count": 60, - "installed_position": 58, - "node_id": 360, + "node_id": 1, "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 + "retention_ms": 7200000 }, { "coverage": { - "definition_id": "983f5e2904acda1d575c27b0861b6380", + "definition_id": "c040dfa3c5b9e9aff9110d588012a461", "selection": [ { - "columns": [], - "relative_time": [ - { - "excluded": -3600000 - }, - { - "included": -3540000 - } - ] + "columns": [] } ] }, "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3540000, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - }, - "width_ms": 60000 - } - ] + "materialized_views": [] }, - "installed_count": 60, - "installed_position": 59, - "node_id": 363, + "node_id": 7, "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 7 + "retention_ms": 7200000 } ], - "proven_optimal": true, "query": "sum(rate(http_requests_total{status=~\"5..\",job=~\"user-service|order-service|payment-service\"}[1h])) / sum(rate(http_requests_total{job=~\"user-service|order-service|payment-service\"}[1h]))", "selected": { "accepted": [ { "accepted_metric": null, - "accepted_regardless_of_guarantee": false, - "guarantee_check_rejection": null, "sketches": [], - "summary": "metric not set; accepted exact", - "target_metric": null + "summary": "accepted exact" } ], "cost": { - "cost_per_second": 1.83727148353, - "cost_unit": "cost_per_second", - "cpu_core_seconds_per_second": 0.000143269716667, + "cost_per_second": 56.2766171722, + "cpu_core_seconds_per_second": 0.05167776335, "per_node": { "0": { "cost_per_second": 0.0074, + "cost_source": "analytical", "cpu_core_seconds_per_second": 7.4e-6, "detail": "scan 1000 samples; x 1.0000/s", "multiplier": 1, "operator": "Scan", - "price_source": "analytical", "timing": "ingestion_time" }, "1": { - "cost_per_second": 0.849, - "cpu_core_seconds_per_second": 2e-6, - "detail": "build exact Rate accumulator in 60 tumbling windows of 60000 ms for 1000 groups: 1000 rows x (1 + depth 1); x 1.0000/s; keeps 6776000 bytes", + "cost_per_second": 2.583, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.000163, + "detail": "build exact Rate accumulator in tumbling windows of 60000 ms for 1000 groups: 1000 rows x 107 ops, 100 group visits, 16.7 states emitted; x 1.0000/s; keeps 19360000 bytes", "multiplier": 1, "operator": "SummaryTumblingWindowChain", - "price_source": "analytical", "timing": "ingestion_time" }, "10": { - "cost_per_second": 0.00101666754915, - "cpu_core_seconds_per_second": 1.01666666667e-6, - "detail": "build exact Sum accumulator into 1 states: 1000 rows x depth 1; x 1.0167/s; holds 56 bytes for 0.1 ms", + "cost_per_second": 2.54166666667, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.00254166666667, + "detail": "finalize 1000 accumulators; x 1.0167/s", "multiplier": 1, - "operator": "SummaryAgg", - "price_source": "analytical", + "operator": "FinalizeExactAccumulator", "timing": "query_time" }, "11": { - "cost_per_second": 1.01666666667e-6, - "cpu_core_seconds_per_second": 1.01666666667e-9, - "detail": "finalize 1 accumulators; x 1.0167/s", + "cost_per_second": 0.00678116668701, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 6.78116666667e-6, + "detail": "hash aggregate 1000 rows into 1 groups: 4 ops per row, 1 group visits; x 1.0167/s; holds 24 bytes for 0.007 ms", "multiplier": 1, - "operator": "FinalizeExactAccumulator", - "price_source": "analytical", + "operator": "Aggregate", "timing": "query_time" }, "12": { - "cost_per_second": 1.01679273638e-6, + "cost_per_second": 1.01666666768e-6, + "cost_source": "analytical", "cpu_core_seconds_per_second": 1.01666666667e-9, - "detail": "operator: pass 1 rows; x 1.0167/s; holds 8 bytes for 0.1 ms", + "detail": "operator: pass 1 rows; x 1.0167/s; holds 8 bytes for 0.000 ms", "multiplier": 1, "operator": "BinaryOp", - "price_source": "analytical", "timing": "query_time" }, "2": { - "cost_per_second": 0.061000882488, - "cpu_core_seconds_per_second": 0.000061, - "detail": "merge 60000 states into 1000; x 1.0167/s; holds 56000 bytes for 0.1 ms", + "cost_per_second": 0.67100001342, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.000671, + "detail": "time range 3600s: 60 tumbling windows of states; x 1.0167/s; holds 160 bytes for 0.660 ms", "multiplier": 1, - "operator": "SummaryMerge", - "price_source": "analytical", + "operator": "TimeRange", "timing": "query_time" }, "3": { - "cost_per_second": 0.00101666666667, - "cpu_core_seconds_per_second": 1.01666666667e-6, - "detail": "finalize 1000 accumulators; x 1.0167/s", + "cost_per_second": 22.4687827, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.0224683333333, + "detail": "merge 60000 states into 1000; x 1.0167/s; holds 160000 bytes for 22.100 ms", "multiplier": 1, - "operator": "FinalizeExactAccumulator", - "price_source": "analytical", + "operator": "SummaryMerge", "timing": "query_time" }, "4": { - "cost_per_second": 0.00101666754915, - "cpu_core_seconds_per_second": 1.01666666667e-6, - "detail": "build exact Sum accumulator into 1 states: 1000 rows x depth 1; x 1.0167/s; holds 56 bytes for 0.1 ms", + "cost_per_second": 2.54166666667, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.00254166666667, + "detail": "finalize 1000 accumulators; x 1.0167/s", "multiplier": 1, - "operator": "SummaryAgg", - "price_source": "analytical", + "operator": "FinalizeExactAccumulator", "timing": "query_time" }, "5": { - "cost_per_second": 1.01666666667e-6, - "cpu_core_seconds_per_second": 1.01666666667e-9, - "detail": "finalize 1 accumulators; x 1.0167/s", + "cost_per_second": 0.00678116668701, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 6.78116666667e-6, + "detail": "hash aggregate 1000 rows into 1 groups: 4 ops per row, 1 group visits; x 1.0167/s; holds 24 bytes for 0.007 ms", "multiplier": 1, - "operator": "FinalizeExactAccumulator", - "price_source": "analytical", + "operator": "Aggregate", "timing": "query_time" }, "6": { "cost_per_second": 0.0058, + "cost_source": "analytical", "cpu_core_seconds_per_second": 5.8e-6, "detail": "scan 1000 samples; x 1.0000/s", "multiplier": 1, "operator": "Scan", - "price_source": "analytical", "timing": "ingestion_time" }, "7": { - "cost_per_second": 0.849, - "cpu_core_seconds_per_second": 2e-6, - "detail": "build exact Rate accumulator in 60 tumbling windows of 60000 ms for 1000 groups: 1000 rows x (1 + depth 1); x 1.0000/s; keeps 6776000 bytes", + "cost_per_second": 2.304, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.000126, + "detail": "build exact Rate accumulator in tumbling windows of 60000 ms for 1000 groups: 1000 rows x 70 ops, 100 group visits, 16.7 states emitted; x 1.0000/s; keeps 17424000 bytes", "multiplier": 1, "operator": "SummaryTumblingWindowChain", - "price_source": "analytical", "timing": "ingestion_time" }, "8": { - "cost_per_second": 0.061000882488, - "cpu_core_seconds_per_second": 0.000061, - "detail": "merge 60000 states into 1000; x 1.0167/s; holds 56000 bytes for 0.1 ms", + "cost_per_second": 0.671000012078, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.000671, + "detail": "time range 3600s: 60 tumbling windows of states; x 1.0167/s; holds 144 bytes for 0.660 ms", "multiplier": 1, - "operator": "SummaryMerge", - "price_source": "analytical", + "operator": "TimeRange", "timing": "query_time" }, "9": { - "cost_per_second": 0.00101666666667, - "cpu_core_seconds_per_second": 1.01666666667e-6, - "detail": "finalize 1000 accumulators; x 1.0167/s", + "cost_per_second": 22.4687377633, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.0224683333333, + "detail": "merge 60000 states into 1000; x 1.0167/s; holds 144000 bytes for 22.100 ms", "multiplier": 1, - "operator": "FinalizeExactAccumulator", - "price_source": "analytical", + "operator": "SummaryMerge", "timing": "query_time" } }, "source": "analytical-cost-v2 (illustrative statistics)", "usage": { - "average_memory_bytes": 13552014.1349, - "cpu_ops_per_second": 132069.716667, + "average_memory_bytes": 36790830.5776, + "cpu_ops_per_second": 51666563.35, "disk_bytes": 0, - "latency_per_query": [ - { - "compaction_ms": 0.0, - "query_ms": 0.124003 - } - ], "peak_cpu_cores": 2, - "peak_memory_bytes": 13664232, + "peak_memory_bytes": 37088472, "scanned_bytes_per_second": 112000.0 } }, @@ -4731,40 +192,35 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "status", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -4777,7 +233,7 @@ }, "producer": 0, "role": "Input", - "window": "RequiresAlignedPanePhaseOrExactBoundaryResidual" + "window": "NotApplicable" }, { "consumer": 2, @@ -4793,9 +249,8 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { @@ -4805,39 +260,37 @@ ] }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "status", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 4, - 0 + 0, + 2, + 3, + 4 ] ] }, @@ -4859,9 +312,8 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { @@ -4871,39 +323,37 @@ ] }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "status", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 4, - 0 + 0, + 2, + 3, + 4 ] ] }, @@ -4914,7 +364,7 @@ { "consumer": 4, "data_state": { - "primitive": "Raw", + "primitive": "SummaryState", "timing": "query_time" }, "grouping": "NotApplicable", @@ -4926,40 +376,38 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { - "Plain": "float64" + "ExactAggregate": [ + "Rate", + "Rate" + ] }, - "name": "value", - "nullable": false, - "table": null + "name": "state", + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "status", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -4977,7 +425,7 @@ { "consumer": 5, "data_state": { - "primitive": "SummaryState", + "primitive": "Raw", "timing": "query_time" }, "grouping": "NotApplicable", @@ -4986,18 +434,47 @@ "fields": [ { "dtype": { - "ExactAggregate": [ - "Sum", - "Sum" - ] + "Plain": "timestamp" }, - "name": "state", - "nullable": false, - "table": null + "name": "ts", + "nullable": false + }, + { + "dtype": { + "Plain": "float64" + }, + "name": "value", + "nullable": false + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "job", + "nullable": true + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "status", + "nullable": true + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false } ], - "time_index": null, - "unique_keys": [] + "time_index": 0, + "unique_keys": [ + [ + 4, + 0 + ] + ] }, "producer": 4, "role": "Input", @@ -5018,32 +495,28 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -5056,7 +529,7 @@ }, "producer": 6, "role": "Input", - "window": "RequiresAlignedPanePhaseOrExactBoundaryResidual" + "window": "NotApplicable" }, { "consumer": 8, @@ -5072,9 +545,8 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { @@ -5084,31 +556,29 @@ ] }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 3, - 0 + 0, + 2, + 3 ] ] }, @@ -5130,9 +600,8 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { @@ -5142,31 +611,29 @@ ] }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 3, - 0 + 0, + 2, + 3 ] ] }, @@ -5177,7 +644,7 @@ { "consumer": 10, "data_state": { - "primitive": "Raw", + "primitive": "SummaryState", "timing": "query_time" }, "grouping": "NotApplicable", @@ -5189,32 +656,31 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { - "Plain": "float64" + "ExactAggregate": [ + "Rate", + "Rate" + ] }, - "name": "value", - "nullable": false, - "table": null + "name": "state", + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -5232,7 +698,7 @@ { "consumer": 11, "data_state": { - "primitive": "SummaryState", + "primitive": "Raw", "timing": "query_time" }, "grouping": "NotApplicable", @@ -5241,18 +707,40 @@ "fields": [ { "dtype": { - "ExactAggregate": [ - "Sum", - "Sum" - ] + "Plain": "timestamp" }, - "name": "state", - "nullable": false, - "table": null + "name": "ts", + "nullable": false + }, + { + "dtype": { + "Plain": "float64" + }, + "name": "value", + "nullable": false + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "job", + "nullable": true + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false } ], - "time_index": null, - "unique_keys": [] + "time_index": 0, + "unique_keys": [ + [ + 3, + 0 + ] + ] }, "producer": 10, "role": "Input", @@ -5273,8 +761,7 @@ "Plain": "float64" }, "name": "sum", - "nullable": false, - "table": null + "nullable": false } ], "time_index": null, @@ -5299,8 +786,7 @@ "Plain": "float64" }, "name": "sum", - "nullable": false, - "table": null + "nullable": false } ], "time_index": null, @@ -5313,24 +799,6 @@ ], "nodes": [ { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact over exact inputs" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 0, "lifetime": "None", "output_schema": { @@ -5341,40 +809,35 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "status", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -5430,40 +893,35 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "status", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -5474,21 +932,111 @@ ] ] }, - "source": { - "TimeSeries": { - "metric": "http_requests_total" - } - } + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + } + } + } + } + }, + { + "id": 1, + "lifetime": { + "AcrossEvaluations": "Memory" + }, + "output_schema": { + "closed": true, + "fields": [ + { + "dtype": { + "Plain": "timestamp" + }, + "name": "window_end", + "nullable": false + }, + { + "dtype": { + "ExactAggregate": [ + "Rate", + "Rate" + ] + }, + "name": "state", + "nullable": false + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "job", + "nullable": true + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "status", + "nullable": true + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false + } + ], + "time_index": 0, + "unique_keys": [ + [ + 0, + 2, + 3, + 4 + ] + ] + }, + "output_state": { + "primitive": "SummaryState", + "timing": "ingestion_time" + }, + "payload": { + "ASAP": { + "SummaryTumblingWindowChain": { + "child": 0, + "filter": null, + "input": { + "item": null, + "weight": { + "Column": "SampleValue" + } + }, + "reduction": "PerEntity", + "summary_type": { + "ExactAggregate": [ + "Rate", + "Rate" + ] + }, + "tumbling_window_width_ms": 60000 } } + }, + "retention_ms": 7200000, + "view_coverage": { + "definition_id": "59e8b0f304a546294da7b0f4fcb52e4f", + "selection": [ + { + "columns": [] + } + ] } }, { - "guarantee": null, - "id": 1, - "lifetime": { - "AcrossEvaluations": "Memory" - }, + "id": 2, + "lifetime": "None", "output_schema": { "closed": true, "fields": [ @@ -5496,9 +1044,8 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { @@ -5508,75 +1055,62 @@ ] }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "status", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 4, - 0 + 0, + 2, + 3, + 4 ] ] }, "output_state": { "primitive": "SummaryState", - "timing": "ingestion_time" + "timing": "query_time" }, "payload": { - "ASAP": { - "SummaryTumblingWindowChain": { - "child": 0, - "family": { - "ExactAggregate": [ - "Rate", - "Rate" - ] - }, - "filter": null, - "grouping": "PerSubpopulationInstance", - "input": { - "item": null, - "weight": { - "Column": "SampleValue" - } - }, - "reduction": "PerEntity", - "tumbling_window_count": 60, - "tumbling_window_width_ms": 60000 + "NonASAP": { + "TimeRange": { + "child": 1, + "kind": "range", + "range": { + "nanos": 0, + "secs": 3600 + } } } } }, { - "guarantee": null, - "id": 2, - "lifetime": "None", + "id": 3, + "lifetime": { + "OneEvaluation": "Memory" + }, "output_schema": { "closed": true, "fields": [ @@ -5585,8 +1119,7 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { @@ -5596,32 +1129,28 @@ ] }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "status", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -5640,32 +1169,14 @@ "ASAP": { "SummaryMerge": { "children": [ - 1 + 2 ] } } } }, { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact accumulator over exact input" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, - "id": 3, + "id": 4, "lifetime": "None", "output_schema": { "closed": true, @@ -5675,40 +1186,35 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "status", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -5726,83 +1232,16 @@ "payload": { "ASAP": { "FinalizeExactAccumulator": { - "child": 2 - } - } - } - }, - { - "guarantee": null, - "id": 4, - "lifetime": "None", - "output_schema": { - "closed": true, - "fields": [ - { - "dtype": { - "ExactAggregate": [ - "Sum", - "Sum" - ] - }, - "name": "state", - "nullable": false, - "table": null - } - ], - "time_index": null, - "unique_keys": [] - }, - "output_state": { - "primitive": "SummaryState", - "timing": "query_time" - }, - "payload": { - "ASAP": { - "SummaryAgg": { - "child": 3, - "family": { - "ExactAggregate": [ - "Sum", - "Sum" - ] - }, - "filter": null, - "grouping": "PerSubpopulationInstance", - "input": { - "item": null, - "weight": { - "Column": "SampleValue" - } - }, - "reduction": { - "Reduce": [] - } + "child": 3 } } } }, { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact accumulator over exact input" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 5, - "lifetime": "None", + "lifetime": { + "OneEvaluation": "Memory" + }, "output_schema": { "closed": true, "fields": [ @@ -5811,8 +1250,7 @@ "Plain": "float64" }, "name": "sum", - "nullable": false, - "table": null + "nullable": false } ], "time_index": null, @@ -5823,32 +1261,28 @@ "timing": "query_time" }, "payload": { - "ASAP": { - "FinalizeExactAccumulator": { - "child": 4 + "NonASAP": { + "Aggregate": { + "child": 4, + "filters": [], + "having": null, + "measures": [ + { + "col": null, + "kind": "sum" + } + ], + "output_names": [ + "" + ], + "reduction": { + "Reduce": [] + } } } } }, { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact over exact inputs" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 6, "lifetime": "None", "output_schema": { @@ -5859,32 +1293,28 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -5926,32 +1356,28 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -5972,7 +1398,6 @@ } }, { - "guarantee": null, "id": 7, "lifetime": { "AcrossEvaluations": "Memory" @@ -5984,9 +1409,8 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { @@ -5996,31 +1420,29 @@ ] }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 3, - 0 + 0, + 2, + 3 ] ] }, @@ -6032,14 +1454,7 @@ "ASAP": { "SummaryTumblingWindowChain": { "child": 6, - "family": { - "ExactAggregate": [ - "Rate", - "Rate" - ] - }, "filter": null, - "grouping": "PerSubpopulationInstance", "input": { "item": null, "weight": { @@ -6047,14 +1462,27 @@ } }, "reduction": "PerEntity", - "tumbling_window_count": 60, + "summary_type": { + "ExactAggregate": [ + "Rate", + "Rate" + ] + }, "tumbling_window_width_ms": 60000 } } + }, + "retention_ms": 7200000, + "view_coverage": { + "definition_id": "c040dfa3c5b9e9aff9110d588012a461", + "selection": [ + { + "columns": [] + } + ] } }, { - "guarantee": null, "id": 8, "lifetime": "None", "output_schema": { @@ -6064,9 +1492,8 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { @@ -6076,31 +1503,29 @@ ] }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 3, - 0 + 0, + 2, + 3 ] ] }, @@ -6109,36 +1534,23 @@ "timing": "query_time" }, "payload": { - "ASAP": { - "SummaryMerge": { - "children": [ - 7 - ] + "NonASAP": { + "TimeRange": { + "child": 7, + "kind": "range", + "range": { + "nanos": 0, + "secs": 3600 + } } } } }, { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact accumulator over exact input" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 9, - "lifetime": "None", + "lifetime": { + "OneEvaluation": "Memory" + }, "output_schema": { "closed": true, "fields": [ @@ -6147,32 +1559,31 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { - "Plain": "float64" + "ExactAggregate": [ + "Rate", + "Rate" + ] }, - "name": "value", - "nullable": false, - "table": null + "name": "state", + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -6184,19 +1595,20 @@ ] }, "output_state": { - "primitive": "Raw", + "primitive": "SummaryState", "timing": "query_time" }, "payload": { "ASAP": { - "FinalizeExactAccumulator": { - "child": 8 + "SummaryMerge": { + "children": [ + 8 + ] } } } }, { - "guarantee": null, "id": 10, "lifetime": "None", "output_schema": { @@ -6204,69 +1616,58 @@ "fields": [ { "dtype": { - "ExactAggregate": [ - "Sum", - "Sum" - ] + "Plain": "timestamp" }, - "name": "state", - "nullable": false, - "table": null + "name": "ts", + "nullable": false + }, + { + "dtype": { + "Plain": "float64" + }, + "name": "value", + "nullable": false + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "job", + "nullable": true + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false } ], - "time_index": null, - "unique_keys": [] + "time_index": 0, + "unique_keys": [ + [ + 3, + 0 + ] + ] }, "output_state": { - "primitive": "SummaryState", + "primitive": "Raw", "timing": "query_time" }, "payload": { "ASAP": { - "SummaryAgg": { - "child": 9, - "family": { - "ExactAggregate": [ - "Sum", - "Sum" - ] - }, - "filter": null, - "grouping": "PerSubpopulationInstance", - "input": { - "item": null, - "weight": { - "Column": "SampleValue" - } - }, - "reduction": { - "Reduce": [] - } + "FinalizeExactAccumulator": { + "child": 9 } } } }, { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact accumulator over exact input" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 11, - "lifetime": "None", + "lifetime": { + "OneEvaluation": "Memory" + }, "output_schema": { "closed": true, "fields": [ @@ -6275,8 +1676,7 @@ "Plain": "float64" }, "name": "sum", - "nullable": false, - "table": null + "nullable": false } ], "time_index": null, @@ -6287,34 +1687,32 @@ "timing": "query_time" }, "payload": { - "ASAP": { - "FinalizeExactAccumulator": { - "child": 10 + "NonASAP": { + "Aggregate": { + "child": 10, + "filters": [], + "having": null, + "measures": [ + { + "col": null, + "kind": "sum" + } + ], + "output_names": [ + "" + ], + "reduction": { + "Reduce": [] + } } } } }, { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact over exact inputs" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 12, - "lifetime": "None", + "lifetime": { + "OneEvaluation": "Memory" + }, "output_schema": { "closed": true, "fields": [ @@ -6323,8 +1721,7 @@ "Plain": "float64" }, "name": "sum", - "nullable": false, - "table": null + "nullable": false } ], "time_index": null, @@ -6357,11 +1754,9 @@ 12 ], "scalar_roots": [] - }, - "display_name": "Q1 Sum acc + Rate acc ×60 tumbling windows · ingestion time: Rate acc for Q1, 60 × 1m tumbling windows, Rate acc for Q1, 60 × 1m tumbling windows" + } }, - "tumbling_window_retention_ms": { - "1": 7200000, - "7": 7200000 - } + "warnings": [ + "assumed data fact: the data arrival declares no max_out_of_order_ms, so the samples of source http_requests_total that an ingestion-time rate, increase or irate state reads are taken to arrive in timestamp order; if this does not hold, answers may be wrong by an amount no accuracy target bounds (a late sample can be read as a counter reset), or ingestion may fail" + ] } diff --git a/crates/devtools/tests/corpus_snapshots/o11y_bench/q26.json b/crates/devtools/tests/corpus_snapshots/o11y_bench/q26.json index 4cd80d8e8..b1657dfa9 100644 --- a/crates/devtools/tests/corpus_snapshots/o11y_bench/q26.json +++ b/crates/devtools/tests/corpus_snapshots/o11y_bench/q26.json @@ -3,2371 +3,96 @@ "materialized_views": [ { "coverage": { - "definition_id": "c2bdb67952bf1d139fefe57a0d6c2c78", + "definition_id": "8cd15d6e9455f508688b3e8edc1367e2", "selection": [ { - "columns": [], - "relative_time": [ - { - "excluded": -60000 - }, - { - "included": 0 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 0, - "source": { - "TimeSeries": { - "metric": "process_cpu_seconds_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 0, - "node_id": 2, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c2bdb67952bf1d139fefe57a0d6c2c78", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -120000 - }, - { - "included": -60000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 60000, - "source": { - "TimeSeries": { - "metric": "process_cpu_seconds_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 1, - "node_id": 5, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c2bdb67952bf1d139fefe57a0d6c2c78", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -180000 - }, - { - "included": -120000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 120000, - "source": { - "TimeSeries": { - "metric": "process_cpu_seconds_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 2, - "node_id": 8, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c2bdb67952bf1d139fefe57a0d6c2c78", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -240000 - }, - { - "included": -180000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 180000, - "source": { - "TimeSeries": { - "metric": "process_cpu_seconds_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 3, - "node_id": 11, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c2bdb67952bf1d139fefe57a0d6c2c78", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -300000 - }, - { - "included": -240000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 240000, - "source": { - "TimeSeries": { - "metric": "process_cpu_seconds_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 4, - "node_id": 14, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c2bdb67952bf1d139fefe57a0d6c2c78", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -360000 - }, - { - "included": -300000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 300000, - "source": { - "TimeSeries": { - "metric": "process_cpu_seconds_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 5, - "node_id": 17, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c2bdb67952bf1d139fefe57a0d6c2c78", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -420000 - }, - { - "included": -360000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 360000, - "source": { - "TimeSeries": { - "metric": "process_cpu_seconds_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 6, - "node_id": 20, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c2bdb67952bf1d139fefe57a0d6c2c78", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -480000 - }, - { - "included": -420000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 420000, - "source": { - "TimeSeries": { - "metric": "process_cpu_seconds_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 7, - "node_id": 23, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c2bdb67952bf1d139fefe57a0d6c2c78", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -540000 - }, - { - "included": -480000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 480000, - "source": { - "TimeSeries": { - "metric": "process_cpu_seconds_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 8, - "node_id": 26, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c2bdb67952bf1d139fefe57a0d6c2c78", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -600000 - }, - { - "included": -540000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 540000, - "source": { - "TimeSeries": { - "metric": "process_cpu_seconds_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 9, - "node_id": 29, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c2bdb67952bf1d139fefe57a0d6c2c78", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -660000 - }, - { - "included": -600000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 600000, - "source": { - "TimeSeries": { - "metric": "process_cpu_seconds_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 10, - "node_id": 32, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c2bdb67952bf1d139fefe57a0d6c2c78", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -720000 - }, - { - "included": -660000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 660000, - "source": { - "TimeSeries": { - "metric": "process_cpu_seconds_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 11, - "node_id": 35, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c2bdb67952bf1d139fefe57a0d6c2c78", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -780000 - }, - { - "included": -720000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 720000, - "source": { - "TimeSeries": { - "metric": "process_cpu_seconds_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 12, - "node_id": 38, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c2bdb67952bf1d139fefe57a0d6c2c78", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -840000 - }, - { - "included": -780000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 780000, - "source": { - "TimeSeries": { - "metric": "process_cpu_seconds_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 13, - "node_id": 41, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c2bdb67952bf1d139fefe57a0d6c2c78", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -900000 - }, - { - "included": -840000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 840000, - "source": { - "TimeSeries": { - "metric": "process_cpu_seconds_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 14, - "node_id": 44, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c2bdb67952bf1d139fefe57a0d6c2c78", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -960000 - }, - { - "included": -900000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 900000, - "source": { - "TimeSeries": { - "metric": "process_cpu_seconds_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 15, - "node_id": 47, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c2bdb67952bf1d139fefe57a0d6c2c78", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1020000 - }, - { - "included": -960000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 960000, - "source": { - "TimeSeries": { - "metric": "process_cpu_seconds_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 16, - "node_id": 50, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c2bdb67952bf1d139fefe57a0d6c2c78", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1080000 - }, - { - "included": -1020000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1020000, - "source": { - "TimeSeries": { - "metric": "process_cpu_seconds_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 17, - "node_id": 53, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c2bdb67952bf1d139fefe57a0d6c2c78", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1140000 - }, - { - "included": -1080000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1080000, - "source": { - "TimeSeries": { - "metric": "process_cpu_seconds_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 18, - "node_id": 56, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c2bdb67952bf1d139fefe57a0d6c2c78", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1200000 - }, - { - "included": -1140000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1140000, - "source": { - "TimeSeries": { - "metric": "process_cpu_seconds_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 19, - "node_id": 59, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c2bdb67952bf1d139fefe57a0d6c2c78", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1260000 - }, - { - "included": -1200000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1200000, - "source": { - "TimeSeries": { - "metric": "process_cpu_seconds_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 20, - "node_id": 62, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c2bdb67952bf1d139fefe57a0d6c2c78", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1320000 - }, - { - "included": -1260000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1260000, - "source": { - "TimeSeries": { - "metric": "process_cpu_seconds_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 21, - "node_id": 65, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c2bdb67952bf1d139fefe57a0d6c2c78", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1380000 - }, - { - "included": -1320000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1320000, - "source": { - "TimeSeries": { - "metric": "process_cpu_seconds_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 22, - "node_id": 68, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c2bdb67952bf1d139fefe57a0d6c2c78", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1440000 - }, - { - "included": -1380000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1380000, - "source": { - "TimeSeries": { - "metric": "process_cpu_seconds_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 23, - "node_id": 71, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c2bdb67952bf1d139fefe57a0d6c2c78", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1500000 - }, - { - "included": -1440000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1440000, - "source": { - "TimeSeries": { - "metric": "process_cpu_seconds_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 24, - "node_id": 74, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c2bdb67952bf1d139fefe57a0d6c2c78", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1560000 - }, - { - "included": -1500000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1500000, - "source": { - "TimeSeries": { - "metric": "process_cpu_seconds_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 25, - "node_id": 77, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c2bdb67952bf1d139fefe57a0d6c2c78", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1620000 - }, - { - "included": -1560000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1560000, - "source": { - "TimeSeries": { - "metric": "process_cpu_seconds_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 26, - "node_id": 80, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c2bdb67952bf1d139fefe57a0d6c2c78", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1680000 - }, - { - "included": -1620000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1620000, - "source": { - "TimeSeries": { - "metric": "process_cpu_seconds_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 27, - "node_id": 83, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c2bdb67952bf1d139fefe57a0d6c2c78", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1740000 - }, - { - "included": -1680000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1680000, - "source": { - "TimeSeries": { - "metric": "process_cpu_seconds_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 28, - "node_id": 86, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c2bdb67952bf1d139fefe57a0d6c2c78", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1800000 - }, - { - "included": -1740000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1740000, - "source": { - "TimeSeries": { - "metric": "process_cpu_seconds_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 29, - "node_id": 89, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c2bdb67952bf1d139fefe57a0d6c2c78", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1860000 - }, - { - "included": -1800000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1800000, - "source": { - "TimeSeries": { - "metric": "process_cpu_seconds_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 30, - "node_id": 92, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c2bdb67952bf1d139fefe57a0d6c2c78", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1920000 - }, - { - "included": -1860000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1860000, - "source": { - "TimeSeries": { - "metric": "process_cpu_seconds_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 31, - "node_id": 95, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c2bdb67952bf1d139fefe57a0d6c2c78", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1980000 - }, - { - "included": -1920000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1920000, - "source": { - "TimeSeries": { - "metric": "process_cpu_seconds_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 32, - "node_id": 98, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c2bdb67952bf1d139fefe57a0d6c2c78", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2040000 - }, - { - "included": -1980000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1980000, - "source": { - "TimeSeries": { - "metric": "process_cpu_seconds_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 33, - "node_id": 101, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c2bdb67952bf1d139fefe57a0d6c2c78", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2100000 - }, - { - "included": -2040000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2040000, - "source": { - "TimeSeries": { - "metric": "process_cpu_seconds_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 34, - "node_id": 104, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c2bdb67952bf1d139fefe57a0d6c2c78", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2160000 - }, - { - "included": -2100000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2100000, - "source": { - "TimeSeries": { - "metric": "process_cpu_seconds_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 35, - "node_id": 107, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c2bdb67952bf1d139fefe57a0d6c2c78", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2220000 - }, - { - "included": -2160000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2160000, - "source": { - "TimeSeries": { - "metric": "process_cpu_seconds_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 36, - "node_id": 110, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c2bdb67952bf1d139fefe57a0d6c2c78", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2280000 - }, - { - "included": -2220000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2220000, - "source": { - "TimeSeries": { - "metric": "process_cpu_seconds_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 37, - "node_id": 113, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c2bdb67952bf1d139fefe57a0d6c2c78", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2340000 - }, - { - "included": -2280000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2280000, - "source": { - "TimeSeries": { - "metric": "process_cpu_seconds_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 38, - "node_id": 116, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c2bdb67952bf1d139fefe57a0d6c2c78", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2400000 - }, - { - "included": -2340000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2340000, - "source": { - "TimeSeries": { - "metric": "process_cpu_seconds_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 39, - "node_id": 119, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c2bdb67952bf1d139fefe57a0d6c2c78", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2460000 - }, - { - "included": -2400000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2400000, - "source": { - "TimeSeries": { - "metric": "process_cpu_seconds_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 40, - "node_id": 122, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c2bdb67952bf1d139fefe57a0d6c2c78", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2520000 - }, - { - "included": -2460000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2460000, - "source": { - "TimeSeries": { - "metric": "process_cpu_seconds_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 41, - "node_id": 125, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c2bdb67952bf1d139fefe57a0d6c2c78", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2580000 - }, - { - "included": -2520000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2520000, - "source": { - "TimeSeries": { - "metric": "process_cpu_seconds_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 42, - "node_id": 128, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c2bdb67952bf1d139fefe57a0d6c2c78", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2640000 - }, - { - "included": -2580000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2580000, - "source": { - "TimeSeries": { - "metric": "process_cpu_seconds_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 43, - "node_id": 131, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c2bdb67952bf1d139fefe57a0d6c2c78", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2700000 - }, - { - "included": -2640000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2640000, - "source": { - "TimeSeries": { - "metric": "process_cpu_seconds_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 44, - "node_id": 134, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c2bdb67952bf1d139fefe57a0d6c2c78", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2760000 - }, - { - "included": -2700000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2700000, - "source": { - "TimeSeries": { - "metric": "process_cpu_seconds_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 45, - "node_id": 137, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c2bdb67952bf1d139fefe57a0d6c2c78", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2820000 - }, - { - "included": -2760000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2760000, - "source": { - "TimeSeries": { - "metric": "process_cpu_seconds_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 46, - "node_id": 140, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c2bdb67952bf1d139fefe57a0d6c2c78", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2880000 - }, - { - "included": -2820000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2820000, - "source": { - "TimeSeries": { - "metric": "process_cpu_seconds_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 47, - "node_id": 143, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c2bdb67952bf1d139fefe57a0d6c2c78", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2940000 - }, - { - "included": -2880000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2880000, - "source": { - "TimeSeries": { - "metric": "process_cpu_seconds_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 48, - "node_id": 146, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c2bdb67952bf1d139fefe57a0d6c2c78", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -3000000 - }, - { - "included": -2940000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2940000, - "source": { - "TimeSeries": { - "metric": "process_cpu_seconds_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 49, - "node_id": 149, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c2bdb67952bf1d139fefe57a0d6c2c78", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -3060000 - }, - { - "included": -3000000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3000000, - "source": { - "TimeSeries": { - "metric": "process_cpu_seconds_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 50, - "node_id": 152, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c2bdb67952bf1d139fefe57a0d6c2c78", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -3120000 - }, - { - "included": -3060000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3060000, - "source": { - "TimeSeries": { - "metric": "process_cpu_seconds_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 51, - "node_id": 155, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c2bdb67952bf1d139fefe57a0d6c2c78", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -3180000 - }, - { - "included": -3120000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3120000, - "source": { - "TimeSeries": { - "metric": "process_cpu_seconds_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 52, - "node_id": 158, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c2bdb67952bf1d139fefe57a0d6c2c78", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -3240000 - }, - { - "included": -3180000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3180000, - "source": { - "TimeSeries": { - "metric": "process_cpu_seconds_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 53, - "node_id": 161, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c2bdb67952bf1d139fefe57a0d6c2c78", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -3300000 - }, - { - "included": -3240000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3240000, - "source": { - "TimeSeries": { - "metric": "process_cpu_seconds_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 54, - "node_id": 164, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c2bdb67952bf1d139fefe57a0d6c2c78", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -3360000 - }, - { - "included": -3300000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3300000, - "source": { - "TimeSeries": { - "metric": "process_cpu_seconds_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 55, - "node_id": 167, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c2bdb67952bf1d139fefe57a0d6c2c78", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -3420000 - }, - { - "included": -3360000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3360000, - "source": { - "TimeSeries": { - "metric": "process_cpu_seconds_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 56, - "node_id": 170, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c2bdb67952bf1d139fefe57a0d6c2c78", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -3480000 - }, - { - "included": -3420000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3420000, - "source": { - "TimeSeries": { - "metric": "process_cpu_seconds_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 57, - "node_id": 173, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c2bdb67952bf1d139fefe57a0d6c2c78", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -3540000 - }, - { - "included": -3480000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3480000, - "source": { - "TimeSeries": { - "metric": "process_cpu_seconds_total" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 58, - "node_id": 176, - "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c2bdb67952bf1d139fefe57a0d6c2c78", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -3600000 - }, - { - "included": -3540000 - } - ] + "columns": [] } ] }, "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3540000, - "source": { - "TimeSeries": { - "metric": "process_cpu_seconds_total" - } - }, - "width_ms": 60000 - } - ] + "materialized_views": [] }, - "installed_count": 60, - "installed_position": 59, - "node_id": 179, + "node_id": 1, "refresh": "at_ingestion", - "retention_ms": 7200000, - "stage2_node_id": 1 + "retention_ms": 7200000 } ], - "proven_optimal": true, "query": "sum(rate(process_cpu_seconds_total[1h]))", "selected": { "accepted": [ { "accepted_metric": null, - "accepted_regardless_of_guarantee": false, - "guarantee_check_rejection": null, "sketches": [], - "summary": "metric not set; accepted exact", - "target_metric": null + "summary": "accepted exact" } ], "cost": { - "cost_per_second": 0.916234791682, - "cost_unit": "cost_per_second", - "cpu_core_seconds_per_second": 0.00006923435, + "cost_per_second": 27.7173406708, + "cpu_core_seconds_per_second": 0.0257809811667, "per_node": { "0": { "cost_per_second": 0.0042, + "cost_source": "analytical", "cpu_core_seconds_per_second": 4.2e-6, "detail": "scan 1000 samples; x 1.0000/s", "multiplier": 1, "operator": "Scan", - "price_source": "analytical", "timing": "ingestion_time" }, "1": { - "cost_per_second": 0.849, - "cpu_core_seconds_per_second": 2e-6, - "detail": "build exact Rate accumulator in 60 tumbling windows of 60000 ms for 1000 groups: 1000 rows x (1 + depth 1); x 1.0000/s; keeps 6776000 bytes", + "cost_per_second": 2.025, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.000089, + "detail": "build exact Rate accumulator in tumbling windows of 60000 ms for 1000 groups: 1000 rows x 33 ops, 100 group visits, 16.7 states emitted; x 1.0000/s; keeps 15488000 bytes", "multiplier": 1, "operator": "SummaryTumblingWindowChain", - "price_source": "analytical", "timing": "ingestion_time" }, "2": { - "cost_per_second": 0.0610004412404, - "cpu_core_seconds_per_second": 0.000061, - "detail": "merge 60000 states into 1000; x 1.0167/s; holds 56000 bytes for 0.1 ms", + "cost_per_second": 0.671000010736, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.000671, + "detail": "time range 3600s: 60 tumbling windows of states; x 1.0167/s; holds 128 bytes for 0.660 ms", "multiplier": 1, - "operator": "SummaryMerge", - "price_source": "analytical", + "operator": "TimeRange", "timing": "query_time" }, "3": { - "cost_per_second": 0.00101666666667, - "cpu_core_seconds_per_second": 1.01666666667e-6, - "detail": "finalize 1000 accumulators; x 1.0167/s", + "cost_per_second": 22.4686928267, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.0224683333333, + "detail": "merge 60000 states into 1000; x 1.0167/s; holds 128000 bytes for 22.100 ms", "multiplier": 1, - "operator": "FinalizeExactAccumulator", - "price_source": "analytical", + "operator": "SummaryMerge", "timing": "query_time" }, "4": { - "cost_per_second": 0.00101666710791, - "cpu_core_seconds_per_second": 1.01666666667e-6, - "detail": "build exact Sum accumulator into 1 states: 1000 rows x depth 1; x 1.0167/s; holds 56 bytes for 0.1 ms", + "cost_per_second": 2.54166666667, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.00254166666667, + "detail": "finalize 1000 accumulators; x 1.0167/s", "multiplier": 1, - "operator": "SummaryAgg", - "price_source": "analytical", + "operator": "FinalizeExactAccumulator", "timing": "query_time" }, "5": { - "cost_per_second": 1.01666666667e-6, - "cpu_core_seconds_per_second": 1.01666666667e-9, - "detail": "finalize 1 accumulators; x 1.0167/s", + "cost_per_second": 0.00678116668701, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 6.78116666667e-6, + "detail": "hash aggregate 1000 rows into 1 groups: 4 ops per row, 1 group visits; x 1.0167/s; holds 24 bytes for 0.007 ms", "multiplier": 1, - "operator": "FinalizeExactAccumulator", - "price_source": "analytical", + "operator": "Aggregate", "timing": "query_time" } }, "source": "analytical-cost-v2 (illustrative statistics)", "usage": { - "average_memory_bytes": 6776003.53345, - "cpu_ops_per_second": 66034.35, + "average_memory_bytes": 15490876.0327, + "cpu_ops_per_second": 25777781.1667, "disk_bytes": 0, - "latency_per_query": [ - { - "compaction_ms": 0.0, - "query_ms": 0.062001 - } - ], "peak_cpu_cores": 2, - "peak_memory_bytes": 6832088, + "peak_memory_bytes": 15616184, "scanned_bytes_per_second": 32000.0 } }, @@ -2388,24 +113,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -2418,7 +140,7 @@ }, "producer": 0, "role": "Input", - "window": "RequiresAlignedPanePhaseOrExactBoundaryResidual" + "window": "NotApplicable" }, { "consumer": 2, @@ -2434,9 +156,8 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { @@ -2446,23 +167,21 @@ ] }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 2, - 0 + 0, + 2 ] ] }, @@ -2484,9 +203,8 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { @@ -2496,23 +214,21 @@ ] }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 2, - 0 + 0, + 2 ] ] }, @@ -2523,7 +239,7 @@ { "consumer": 4, "data_state": { - "primitive": "Raw", + "primitive": "SummaryState", "timing": "query_time" }, "grouping": "NotApplicable", @@ -2535,24 +251,24 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { - "Plain": "float64" + "ExactAggregate": [ + "Rate", + "Rate" + ] }, - "name": "value", - "nullable": false, - "table": null + "name": "state", + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -2570,7 +286,7 @@ { "consumer": 5, "data_state": { - "primitive": "SummaryState", + "primitive": "Raw", "timing": "query_time" }, "grouping": "NotApplicable", @@ -2579,18 +295,33 @@ "fields": [ { "dtype": { - "ExactAggregate": [ - "Sum", - "Sum" - ] + "Plain": "timestamp" }, - "name": "state", - "nullable": false, - "table": null + "name": "ts", + "nullable": false + }, + { + "dtype": { + "Plain": "float64" + }, + "name": "value", + "nullable": false + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false } ], - "time_index": null, - "unique_keys": [] + "time_index": 0, + "unique_keys": [ + [ + 2, + 0 + ] + ] }, "producer": 4, "role": "Input", @@ -2599,24 +330,6 @@ ], "nodes": [ { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact over exact inputs" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 0, "lifetime": "None", "output_schema": { @@ -2627,24 +340,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -2671,24 +381,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -2709,7 +416,6 @@ } }, { - "guarantee": null, "id": 1, "lifetime": { "AcrossEvaluations": "Memory" @@ -2721,9 +427,8 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { @@ -2733,23 +438,21 @@ ] }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 2, - 0 + 0, + 2 ] ] }, @@ -2761,14 +464,7 @@ "ASAP": { "SummaryTumblingWindowChain": { "child": 0, - "family": { - "ExactAggregate": [ - "Rate", - "Rate" - ] - }, "filter": null, - "grouping": "PerSubpopulationInstance", "input": { "item": null, "weight": { @@ -2776,14 +472,27 @@ } }, "reduction": "PerEntity", - "tumbling_window_count": 60, + "summary_type": { + "ExactAggregate": [ + "Rate", + "Rate" + ] + }, "tumbling_window_width_ms": 60000 } } + }, + "retention_ms": 7200000, + "view_coverage": { + "definition_id": "8cd15d6e9455f508688b3e8edc1367e2", + "selection": [ + { + "columns": [] + } + ] } }, { - "guarantee": null, "id": 2, "lifetime": "None", "output_schema": { @@ -2793,9 +502,8 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { @@ -2805,23 +513,21 @@ ] }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 2, - 0 + 0, + 2 ] ] }, @@ -2830,36 +536,23 @@ "timing": "query_time" }, "payload": { - "ASAP": { - "SummaryMerge": { - "children": [ - 1 - ] + "NonASAP": { + "TimeRange": { + "child": 1, + "kind": "range", + "range": { + "nanos": 0, + "secs": 3600 + } } } } }, { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact accumulator over exact input" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 3, - "lifetime": "None", + "lifetime": { + "OneEvaluation": "Memory" + }, "output_schema": { "closed": true, "fields": [ @@ -2868,24 +561,24 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { - "Plain": "float64" + "ExactAggregate": [ + "Rate", + "Rate" + ] }, - "name": "value", - "nullable": false, - "table": null + "name": "state", + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -2897,19 +590,20 @@ ] }, "output_state": { - "primitive": "Raw", + "primitive": "SummaryState", "timing": "query_time" }, "payload": { "ASAP": { - "FinalizeExactAccumulator": { - "child": 2 + "SummaryMerge": { + "children": [ + 2 + ] } } } }, { - "guarantee": null, "id": 4, "lifetime": "None", "output_schema": { @@ -2917,69 +611,51 @@ "fields": [ { "dtype": { - "ExactAggregate": [ - "Sum", - "Sum" - ] + "Plain": "timestamp" }, - "name": "state", - "nullable": false, - "table": null + "name": "ts", + "nullable": false + }, + { + "dtype": { + "Plain": "float64" + }, + "name": "value", + "nullable": false + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false } ], - "time_index": null, - "unique_keys": [] + "time_index": 0, + "unique_keys": [ + [ + 2, + 0 + ] + ] }, "output_state": { - "primitive": "SummaryState", + "primitive": "Raw", "timing": "query_time" }, "payload": { "ASAP": { - "SummaryAgg": { - "child": 3, - "family": { - "ExactAggregate": [ - "Sum", - "Sum" - ] - }, - "filter": null, - "grouping": "PerSubpopulationInstance", - "input": { - "item": null, - "weight": { - "Column": "SampleValue" - } - }, - "reduction": { - "Reduce": [] - } + "FinalizeExactAccumulator": { + "child": 3 } } } }, { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact accumulator over exact input" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 5, - "lifetime": "None", + "lifetime": { + "OneEvaluation": "Memory" + }, "output_schema": { "closed": true, "fields": [ @@ -2988,8 +664,7 @@ "Plain": "float64" }, "name": "sum", - "nullable": false, - "table": null + "nullable": false } ], "time_index": null, @@ -3000,9 +675,23 @@ "timing": "query_time" }, "payload": { - "ASAP": { - "FinalizeExactAccumulator": { - "child": 4 + "NonASAP": { + "Aggregate": { + "child": 4, + "filters": [], + "having": null, + "measures": [ + { + "col": null, + "kind": "sum" + } + ], + "output_names": [ + "" + ], + "reduction": { + "Reduce": [] + } } } } @@ -3012,10 +701,9 @@ 5 ], "scalar_roots": [] - }, - "display_name": "Q1 Sum acc + Rate acc ×60 tumbling windows · ingestion time: Rate acc for Q1, 60 × 1m tumbling windows" + } }, - "tumbling_window_retention_ms": { - "1": 7200000 - } + "warnings": [ + "assumed data fact: the data arrival declares no max_out_of_order_ms, so the samples of source process_cpu_seconds_total that an ingestion-time rate, increase or irate state reads are taken to arrive in timestamp order; if this does not hold, answers may be wrong by an amount no accuracy target bounds (a late sample can be read as a counter reset), or ingestion may fail" + ] } diff --git a/crates/devtools/tests/corpus_snapshots/o11y_bench/q27.json b/crates/devtools/tests/corpus_snapshots/o11y_bench/q27.json index 8d553aae8..ae2097b8f 100644 --- a/crates/devtools/tests/corpus_snapshots/o11y_bench/q27.json +++ b/crates/devtools/tests/corpus_snapshots/o11y_bench/q27.json @@ -1,74 +1,54 @@ { "accuracy_guarantees": "enforced", "materialized_views": [], - "proven_optimal": true, "query": "sum(up)", "selected": { "accepted": [ { "accepted_metric": null, - "accepted_regardless_of_guarantee": false, - "guarantee_check_rejection": null, "sketches": [], - "summary": "metric not set; accepted exact", - "target_metric": null + "summary": "accepted exact" } ], "cost": { - "cost_per_second": 0.00630435009457, - "cost_unit": "cost_per_second", - "cpu_core_seconds_per_second": 6.30435e-6, + "cost_per_second": 0.0222345000822, + "cpu_core_seconds_per_second": 0.0000222345, "per_node": { "0": { - "cost_per_second": 0.00427000002522, + "cost_per_second": 0.00427000001708, + "cost_source": "analytical", "cpu_core_seconds_per_second": 4.27e-6, - "detail": "scan 1000 samples; x 1.0167/s; keeps 32000 bytes of raw data on disk; holds 32 bytes for 0.0 ms", + "detail": "scan 1000 samples; x 1.0167/s; holds 32 bytes for 0.004 ms; keeps 115232000 bytes of raw data on disk", "multiplier": 1, "operator": "Scan", - "price_source": "analytical", "timing": "query_time" }, "1": { - "cost_per_second": 0.00101666669188, - "cpu_core_seconds_per_second": 1.01666666667e-6, - "detail": "time range 1s: keep 1000 of 1000 rows; x 1.0167/s; holds 32 bytes for 0.0 ms", + "cost_per_second": 0.0111833333781, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.0000111833333333, + "detail": "time range 1s: keep 1000 of 1000 rows; x 1.0167/s; holds 32 bytes for 0.011 ms", "multiplier": 1, "operator": "TimeRange", - "price_source": "analytical", "timing": "query_time" }, "2": { - "cost_per_second": 0.0010166667108, - "cpu_core_seconds_per_second": 1.01666666667e-6, - "detail": "build exact Sum accumulator into 1 states: 1000 rows x depth 1; x 1.0167/s; holds 56 bytes for 0.0 ms", + "cost_per_second": 0.00678116668701, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 6.78116666667e-6, + "detail": "hash aggregate 1000 rows into 1 groups: 4 ops per row, 1 group visits; x 1.0167/s; holds 24 bytes for 0.007 ms", "multiplier": 1, - "operator": "SummaryAgg", - "price_source": "analytical", - "timing": "query_time" - }, - "3": { - "cost_per_second": 1.01666666667e-6, - "cpu_core_seconds_per_second": 1.01666666667e-9, - "detail": "finalize 1 accumulators; x 1.0167/s", - "multiplier": 1, - "operator": "FinalizeExactAccumulator", - "price_source": "analytical", + "operator": "Aggregate", "timing": "query_time" } }, "source": "analytical-cost-v2 (illustrative statistics)", "usage": { - "average_memory_bytes": 0.000756522, - "cpu_ops_per_second": 3051.01666667, - "disk_bytes": 32000, - "latency_per_query": [ - { - "compaction_ms": 0.0, - "query_ms": 0.006201 - } - ], + "average_memory_bytes": 0.000657254666667, + "cpu_ops_per_second": 18981.1666667, + "disk_bytes": 115232000, "peak_cpu_cores": 1, - "peak_memory_bytes": 120, + "peak_memory_bytes": 88, "scanned_bytes_per_second": 32533.3333333 } }, @@ -89,24 +69,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -136,24 +113,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -167,57 +141,10 @@ "producer": 1, "role": "Input", "window": "NotApplicable" - }, - { - "consumer": 3, - "data_state": { - "primitive": "SummaryState", - "timing": "query_time" - }, - "grouping": "NotApplicable", - "intermediate_schema": { - "closed": true, - "fields": [ - { - "dtype": { - "ExactAggregate": [ - "Sum", - "Sum" - ] - }, - "name": "state", - "nullable": false, - "table": null - } - ], - "time_index": null, - "unique_keys": [] - }, - "producer": 2, - "role": "Input", - "window": "NotApplicable" } ], "nodes": [ { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact over exact inputs" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 0, "lifetime": "None", "output_schema": { @@ -228,24 +155,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -272,24 +196,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -307,27 +228,10 @@ } } } - } + }, + "retention_ms": 3601000 }, { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact over exact inputs" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 1, "lifetime": "None", "output_schema": { @@ -338,24 +242,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -384,111 +285,55 @@ } }, { - "guarantee": null, "id": 2, - "lifetime": "None", + "lifetime": { + "OneEvaluation": "Memory" + }, "output_schema": { "closed": true, "fields": [ { "dtype": { - "ExactAggregate": [ - "Sum", - "Sum" - ] + "Plain": "float64" }, - "name": "state", - "nullable": false, - "table": null + "name": "sum", + "nullable": false } ], "time_index": null, "unique_keys": [] }, "output_state": { - "primitive": "SummaryState", + "primitive": "Raw", "timing": "query_time" }, "payload": { - "ASAP": { - "SummaryAgg": { + "NonASAP": { + "Aggregate": { "child": 1, - "family": { - "ExactAggregate": [ - "Sum", - "Sum" - ] - }, - "filter": null, - "grouping": "PerSubpopulationInstance", - "input": { - "item": null, - "weight": { - "Column": "SampleValue" + "filters": [], + "having": null, + "measures": [ + { + "col": null, + "kind": "sum" } - }, + ], + "output_names": [ + "" + ], "reduction": { "Reduce": [] } } } } - }, - { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact accumulator over exact input" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, - "id": 3, - "lifetime": "None", - "output_schema": { - "closed": true, - "fields": [ - { - "dtype": { - "Plain": "float64" - }, - "name": "sum", - "nullable": false, - "table": null - } - ], - "time_index": null, - "unique_keys": [] - }, - "output_state": { - "primitive": "Raw", - "timing": "query_time" - }, - "payload": { - "ASAP": { - "FinalizeExactAccumulator": { - "child": 2 - } - } - } } ], "roots": [ - 3 + 2 ], "scalar_roots": [] - }, - "display_name": "Q1 Sum acc" - }, - "tumbling_window_retention_ms": {} + } + } } diff --git a/crates/devtools/tests/planner_fuzz.rs b/crates/devtools/tests/planner_fuzz.rs index 232d75332..b16317a64 100644 --- a/crates/devtools/tests/planner_fuzz.rs +++ b/crates/devtools/tests/planner_fuzz.rs @@ -34,38 +34,14 @@ fn check(relation: Relation, workload: FuzzWorkload, target: usize) -> Outcome { ) } -/// #903: Q1 = `quantile_over_time(0.9, data[5m])` exact; Q2 = Q1 at -/// ε = 0.01, δ = 0.001 builds a KLL instead of reading Q1's exact result -/// (0.166 per second against the witness's 0.144). Open bug: flip to -/// `Holds` when #903 is fixed. -#[test] -fn finds_issue_903_looser_accuracy_not_reading_the_exact_result() { - let mut workload = seed("issue903.workload.json"); - workload.queries.truncate(1); - match check(Relation::LooserAccuracy, workload, 0) { - Outcome::Violated(m) => { - let witness = m.witness.expect("a marginal relation has a witness"); - assert_eq!(witness.derivation, Some(Witness::Same)); - assert!( - (m.related_cost_per_second - 0.166).abs() < 0.005, - "{}", - m.summary - ); - assert!( - (witness.cost_per_second - 0.144).abs() < 0.005, - "{}", - m.summary - ); - } - other => panic!("expected #903's violation, got {other:?}"), - } -} - /// #902: Q3 = p90 per city; the same query for `city = 'city1'` built its /// own KLL instead of selecting the group from Q3's result. #943 fixed the /// issue's ungrouped case and closed #902; this grouped variant /// (`… AND city = 'city1' GROUP BY city`) holds only since #952's /// finer-aggregate reuse (#950), and is now its regression test. +/// After the current cost refits it fails again: the planner's plan costs +/// about 0.080/s more than the witness's (which adds about 0.00001/s), past +/// the tolerance. That gap is the one #891 tracks. #[test] fn finds_issue_902_filter_on_group_key_not_reading_the_grouped_result() { let mut workload = seed("issue902.json"); @@ -87,10 +63,19 @@ fn finds_issue_902_filter_on_group_key_not_reading_the_grouped_result() { ); } -/// #891: the per-pod CPU rate query filtered on a group key does not read -/// the grouped rate result. Open bug: flip to `Holds` when #891 is fixed. +/// #891: the per-pod CPU rate query filtered on a group key now reads the +/// grouped rate: the related plan filters the grouped sum's exact states, +/// then merges and finalizes them (`Filter`, `SummaryMerge`, +/// `FinalizeExactAccumulator`), adding 0.000100/s to W's 0.008500/s. The +/// witness filters the finished result instead, adding 0.000035/s. The +/// gap that remains, open in #891, is pinned here: the planner's plan +/// (0.008600/s) costs 0.000065/s more than the witness's (0.008535/s), at +/// most the tolerance (1% of W, 0.000086/s). After the current cost refits +/// it fails: the planner's plan costs about 0.0925/s more than the +/// witness's (which adds about 0.00004/s), past the tolerance. Tracked in +/// #891. #[test] -fn finds_issue_891_filtered_rate_not_reading_the_grouped_rate() { +fn issue_891_filtered_rate_reads_the_grouped_rate() { let mut workload = seed("issue891.workload.json"); let grouped = PromqlSpec { metric: "container_cpu_usage_seconds_total".into(), @@ -110,16 +95,19 @@ fn finds_issue_891_filtered_rate_not_reading_the_grouped_rate() { }; workload.queries = vec![workload.queries[0].with_spec(QuerySpec::Promql(grouped))]; match check(Relation::FilterOnGroupKey, workload, 0) { - Outcome::Violated(m) => { + Outcome::Holds(m) => { let witness = m.witness.expect("a marginal relation has a witness"); assert!(matches!(witness.derivation, Some(Witness::Filter { .. }))); + assert!(m.related_plan.contains("Filter"), "{}", m.related_plan); + let gap_per_second = m.related_cost_per_second - witness.cost_per_second; assert!( - m.related_cost_per_second > witness.cost_per_second * 1.1, - "{}", - m.summary + gap_per_second <= m.tolerance_per_second, + "the planner's plan costs {gap_per_second}/s more than the witness's, over the \ + tolerance {}/s", + m.tolerance_per_second ); } - other => panic!("expected #891's violation, got {other:?}"), + other => panic!("expected the filtered rate to read the grouped rate, got {other:?}"), } } @@ -149,7 +137,7 @@ fn campaign_writes_summary_coverage_and_drafts() { seed: 7, cases: 2, time_budget_s: 600.0, - relations: vec![Relation::LooserAccuracy, Relation::ExactDuplicate], + relations: vec![Relation::ExactDuplicate, Relation::OtherRepeatInterval], seeds: vec![seed("issue903.workload.json")], out_dir: out.clone(), max_shrink_attempts: 5, @@ -170,7 +158,7 @@ fn campaign_writes_summary_coverage_and_drafts() { ); let markdown = std::fs::read_to_string(out.join("summary.md")).expect("summary.md"); assert!(markdown.contains("## Design-doc reuse patterns")); - assert!(markdown.contains("`a-looser-accuracy`")); + assert!(markdown.contains("`f-exact-duplicate`")); for finding in &summary.findings { let draft = std::fs::read_to_string(&finding.draft).expect("the draft"); assert!(draft.starts_with("# ")); @@ -219,7 +207,7 @@ fn promql( } /// The witness `relation` builds for `workload`'s query, which must build -/// and price (the check is conclusive). +/// and cost (the check is conclusive). fn witness_of(relation: Relation, workload: FuzzWorkload) -> Witness { match check(relation, workload, 0) { Outcome::Holds(m) | Outcome::Violated(m) => m @@ -230,10 +218,10 @@ fn witness_of(relation: Relation, workload: FuzzWorkload) -> Witness { } } -/// The sketch rollup witness builds and prices for a grouped quantile: +/// The sketch rollup witness builds and costs for a grouped quantile: /// a `SummaryMerge` with `group_by` over the finer per-group KLL. #[test] -fn sketch_rollup_witness_prices_for_a_grouped_quantile() { +fn sketch_rollup_witness_costs_for_a_grouped_quantile() { let q = promql( "m_gauge", None, @@ -247,10 +235,10 @@ fn sketch_rollup_witness_prices_for_a_grouped_quantile() { ); } -/// The exact rollup witness builds and prices for a grouped count (a sum +/// The exact rollup witness builds and costs for a grouped count (a sum /// of the finer counts). #[test] -fn exact_rollup_witness_prices_for_a_grouped_count() { +fn exact_rollup_witness_costs_for_a_grouped_count() { let q = promql( "m_gauge", None, @@ -262,9 +250,9 @@ fn exact_rollup_witness_prices_for_a_grouped_count() { )); } -/// The quantile-rank witness builds and prices from the same KLL. +/// The quantile-rank witness builds and costs from the same KLL. #[test] -fn quantile_rank_witness_prices_from_the_same_sketch() { +fn quantile_rank_witness_costs_from_the_same_sketch() { let q = promql("m_gauge", Some(RangeFunction::QuantileOverTime(0.9)), None); assert_eq!( witness_of(Relation::OtherQuantileRank, generated(q, true)), @@ -272,9 +260,9 @@ fn quantile_rank_witness_prices_from_the_same_sketch() { ); } -/// The smaller top-k witness builds and prices over the same input. +/// The smaller top-k witness builds and costs over the same input. #[test] -fn smaller_topk_witness_prices_over_the_same_input() { +fn smaller_topk_witness_costs_over_the_same_input() { let q = promql( "m_counter", Some(RangeFunction::Rate), @@ -287,10 +275,10 @@ fn smaller_topk_witness_prices_over_the_same_input() { } /// (l): a filtered share of a grouped rate reads the rate from q's result -/// (ASAPWorkloads #12); the plain graft cannot price (its numerator is an +/// (ASAPWorkloads #12); the plain graft cannot cost (its numerator is an /// unplanned raw rate), so the witness planned with the operand must. #[test] -fn ratio_witness_prices_over_the_shared_rate() { +fn ratio_witness_costs_over_the_shared_rate() { let q = promql( "m_counter", Some(RangeFunction::Rate), @@ -304,7 +292,7 @@ fn ratio_witness_prices_over_the_shared_rate() { /// (l), avg as sum / count: the sum is read from q's result. #[test] -fn sum_over_count_witness_prices_over_the_shared_sum() { +fn sum_over_count_witness_costs_over_the_shared_sum() { let q = promql( "m_gauge", Some(RangeFunction::SumOverTime), @@ -318,7 +306,7 @@ fn sum_over_count_witness_prices_over_the_shared_sum() { /// (o): `increase` is q's `rate` times the window. #[test] -fn increase_witness_prices_as_the_scaled_rate() { +fn increase_witness_costs_as_the_scaled_rate() { let q = promql( "m_counter", Some(RangeFunction::Rate), @@ -332,7 +320,7 @@ fn increase_witness_prices_as_the_scaled_rate() { /// (g): a threshold over a grouped minimum reads q's result. #[test] -fn threshold_witness_prices_over_the_result() { +fn threshold_witness_costs_over_the_result() { let q = promql( "m_gauge", Some(RangeFunction::MaxOverTime), @@ -344,30 +332,54 @@ fn threshold_witness_prices_over_the_result() { )); } -/// (m): a longer window lengthens q's tumbling-window chain of exact sums. -#[test] -fn longer_window_witness_prices_over_a_lengthened_chain() { - let q = promql( +/// An exact `sum by (labels) (sum_over_time(m_gauge[30m]))` every minute at +/// phase 0 over 1,000 series sampled every second: reading a chain of +/// tumbling windows is cheaper than scanning 30 minutes of raw data, so +/// `W`'s plan reads one. +fn summed_over_a_chain(labels: &[&str]) -> FuzzWorkload { + use asap_devtools::planner_fuzz::generate::generated_workload; + use asap_devtools::planner_fuzz::{FuzzQuery, Language}; + use asap_types::accuracy::AccuracyTarget; + use asap_types::workload::{RepeatedDemand, RepetitionInterval, TimestampMs}; + let QuerySpec::Promql(mut spec) = promql( "m_gauge", Some(RangeFunction::SumOverTime), - Some((OuterOp::Sum, &["job"])), + Some((OuterOp::Sum, labels)), + ) else { + unreachable!("a PromQL spec") + }; + spec.window_ms = 1_800_000; + let mut query = FuzzQuery::generated( + QuerySpec::Promql(spec), + AccuracyTarget::exact(), + 60_000, + None, ); + query.entry.demand = RepeatedDemand::FixedIntervalAt { + interval: RepetitionInterval(60_000), + evaluation_phase: TimestampMs(0), + }; + generated_workload(Language::Promql, vec![query], 1_000, 1_000) +} + +/// (m): a longer window is a longer `TimeRange` over the chain `q`'s plan +/// reads, the same chain. +#[test] +fn longer_window_witness_costs_over_the_same_chain() { assert!(matches!( - witness_of(Relation::LongerWindow, generated(q, false)), + witness_of(Relation::LongerWindow, summed_over_a_chain(&["job"])), Witness::LongerWindow { rollup: None, .. } )); } -/// (n): the lengthened chain rolled up to one key fewer. +/// (n): the longer range over q's chain rolled up to one key fewer. #[test] -fn longer_window_rollup_witness_prices() { - let q = promql( - "m_gauge", - Some(RangeFunction::SumOverTime), - Some((OuterOp::Sum, &["job", "region"])), - ); +fn longer_window_rollup_witness_costs() { assert!(matches!( - witness_of(Relation::LongerWindowRollup, generated(q, false)), + witness_of( + Relation::LongerWindowRollup, + summed_over_a_chain(&["job", "region"]) + ), Witness::LongerWindow { rollup: Some(_), .. @@ -379,7 +391,7 @@ fn longer_window_rollup_witness_prices() { /// frequency L2 reads one UnivMon, from q's own state or from `W ∪ {q'}` /// planned with UnivMon only (planner-layering Example 2). #[test] -fn other_frequency_statistic_witness_prices_from_one_univmon() { +fn other_frequency_statistic_witness_costs_from_one_univmon() { use asap_devtools::planner_fuzz::generate::generated_workload; use asap_devtools::planner_fuzz::{FuzzQuery, Language}; use asap_types::accuracy::AccuracyTarget; @@ -406,3 +418,23 @@ fn other_frequency_statistic_witness_prices_from_one_univmon() { Witness::OtherStatistic { .. } | Witness::UnivMonOnly )); } + +/// `W`'s selected plan, costed outside the search, costs what the search +/// selected it for: the witness's reference point is on the same footing. +#[test] +fn the_selected_plan_costs_what_the_search_selected_it_for() { + let mut workload = seed("issue903.workload.json"); + workload.queries.truncate(1); + let mut planner = Planner::new(FuzzSettings::default()); + let planned = planner.plan(&workload).expect("W plans"); + let costed = planner + .reference_cost(&workload, &planned) + .expect("W's plan costs"); + assert!( + (costed.cost_per_second - planned.cost_per_second).abs() + <= 1e-9 * planned.cost_per_second.abs(), + "{} vs {}", + costed.cost_per_second, + planned.cost_per_second + ); +} diff --git a/crates/devtools/tests/sketch_plan_snapshots.rs b/crates/devtools/tests/sketch_plan_snapshots.rs index 1ab0097d1..278a223ae 100644 --- a/crates/devtools/tests/sketch_plan_snapshots.rs +++ b/crates/devtools/tests/sketch_plan_snapshots.rs @@ -14,23 +14,19 @@ //! `deployment` (the `asap_planner` arguments of its deployment //! capabilities and cost inputs, files in `deployments/`), and whether //! accuracy guarantees are `ignored` (enforced otherwise). `.json` beside it is its snapshot, -//! the corpus snapshot of the selected plan plus every query's text and the -//! search status. Workloads copied from elsewhere live in `workloads/`. +//! the corpus snapshot of the selected plan plus every query's text. Workloads copied from elsewhere live in `workloads/`. //! Regenerate with //! `UPDATE_SNAPSHOTS=1 cargo test --release -p asap-devtools --test corpus_snapshots -- --ignored sketch_plans_match_snapshots`. //! -//! Each workload is planned by `asap_planner --run-until stage3`. Planning -//! has no time limit, so every workload must end with status `optimal`; any -//! other ending fails instead of being recorded. +//! Each workload is planned by `asap_planner --run-until stage3`, with no +//! time limit. A workload whose selection fails records its error, as a +//! corpus query does. use std::path::PathBuf; use serde_json::{json, Value}; -use super::{ - asap_planner, first_difference, first_differing_path, plan_snapshot, proven_optimal, rounded, - search_status, -}; +use super::{asap_planner, first_difference, first_differing_path, plan_snapshot}; fn repository_root() -> PathBuf { PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("../..") @@ -70,8 +66,8 @@ fn arguments(manifest: &Value, entry: &Value) -> Vec { args } -/// The stable part of an `asap_planner` document: every query's text, the -/// selected plan (see [`plan_snapshot`]) and how its search ended. +/// The stable part of an `asap_planner` document: every query's text and +/// the selected plan (see [`plan_snapshot`]). fn snapshot_of(document: &Value) -> Value { let mut snapshot = plan_snapshot(document); let queries: Vec = document["workload"]["queries"] @@ -81,11 +77,7 @@ fn snapshot_of(document: &Value) -> Value { .map(|query| query["text"].clone()) .collect(); snapshot.insert("queries".into(), Value::Array(queries)); - let status = search_status(document); - if !status.is_null() { - snapshot.insert("search_status".into(), status.clone()); - } - rounded(&Value::Object(snapshot)) + Value::Object(snapshot) } /// Every manifest workload's plan matches its committed snapshot. @@ -124,13 +116,6 @@ fn sketch_plans_match_snapshots() { for entry in entries { let id = entry["id"].as_str().unwrap(); let document = asap_planner(&arguments(&manifest, entry), &scratch); - if !proven_optimal(&document) { - failures.push(format!( - "{id}: the plan is not proven optimal (status {})", - search_status(&document) - )); - continue; - } let actual_text = serde_json::to_string_pretty(&snapshot_of(&document)).unwrap() + "\n"; let path = dir.join(format!("{id}.json")); if update { @@ -198,25 +183,20 @@ fn manifest_entries_build_their_arguments() { } } -/// A snapshot keeps every query's text and the search status. +/// A snapshot keeps every query's text. #[test] -fn snapshot_keeps_the_queries_and_search_status() { +fn snapshot_keeps_the_queries() { let document = json!({ "accuracy_guarantees": "enforced", "workload": {"queries": [{"text": "q1"}, {"text": "q2"}]}, - "stage2_physical_asap_optimization": {"candidates": [ - {"id": "P1", "display_name": "plan A", "dag": {"nodes": [1]}}, - ]}, "stage3_plan_selection": { - "cost": {"cost_per_second": 0.1 + 0.2}, + "plan": {"nodes": [1]}, + "cost": {"cost_per_second": 0.3}, "accepted_accuracy_per_query": [], - "search_report": {"status": "optimal", "solver_ms": 1}, }, }); let snapshot = snapshot_of(&document); assert_eq!(snapshot["queries"], json!(["q1", "q2"])); - assert_eq!(snapshot["search_status"], json!("optimal")); - assert_eq!(snapshot["proven_optimal"], json!(true)); assert_eq!( snapshot["selected"]["cost"], json!({"cost_per_second": 0.3}) diff --git a/crates/devtools/tests/sketch_plan_snapshots/fuzz-issue903-raw-retained.json b/crates/devtools/tests/sketch_plan_snapshots/fuzz-issue903-raw-retained.json index 8b2451a16..c7f7ed322 100644 --- a/crates/devtools/tests/sketch_plan_snapshots/fuzz-issue903-raw-retained.json +++ b/crates/devtools/tests/sketch_plan_snapshots/fuzz-issue903-raw-retained.json @@ -1,110 +1,62 @@ { "accuracy_guarantees": "enforced", "materialized_views": [], - "proven_optimal": true, "queries": [ "quantile_over_time(0.9, data[5m])", "quantile_over_time(0.9, data[5m])" ], - "search_status": "optimal", "selected": { "accepted": [ { "accepted_metric": null, - "accepted_regardless_of_guarantee": false, - "guarantee_check_rejection": null, "sketches": [], - "summary": "metric not set; accepted exact", - "target_metric": null + "summary": "accepted exact" }, { - "accepted_metric": "rank", - "accepted_regardless_of_guarantee": false, - "guarantee_check_rejection": null, - "sketches": [ - "Kll" - ], - "summary": "metric not set; accepted rank (Kll)", - "target_metric": null + "accepted_metric": null, + "sketches": [], + "summary": "accepted exact" } ], "cost": { - "cost_per_second": 1.8693312527e-9, - "cost_unit": "cost_per_second", - "cpu_core_seconds_per_second": 0.000166, + "cost_per_second": 7.96674686384e-8, + "cpu_core_seconds_per_second": 0.007084, "per_node": { "0": { - "cost_per_second": 9.444960053e-10, + "cost_per_second": 9.44496003092e-10, + "cost_source": "analytical", "cpu_core_seconds_per_second": 0.000084, - "detail": "scan 200000 samples; x 0.1000/s; keeps 6400000 bytes of raw data on disk; holds 32 bytes for 1.4 ms", + "detail": "scan 200000 samples; x 0.1000/s; holds 32 bytes for 0.840 ms; keeps 6400000 bytes of raw data on disk", "multiplier": 1, "operator": "Scan", - "price_source": "analytical", "timing": "query_time" }, "1": { - "cost_per_second": 2.25940009003e-10, - "cpu_core_seconds_per_second": 0.00002, - "detail": "time range 300s: keep 200000 of 200000 rows; x 0.1000/s; holds 6400000 bytes for 1.4 ms", + "cost_per_second": 2.4736800081e-9, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.00022, + "detail": "time range 300s: keep 200000 of 200000 rows; x 0.1000/s; holds 32 bytes for 2.200 ms", "multiplier": 1, "operator": "TimeRange", - "price_source": "analytical", "timing": "query_time" }, "2": { - "cost_per_second": 4.50078002701e-10, - "cpu_core_seconds_per_second": 0.00004, - "detail": "hash aggregate 200000 rows into 10000 groups; x 0.1000/s; holds 1920000 bytes for 1.4 ms", + "cost_per_second": 7.62492926272e-8, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.00678, + "detail": "hash aggregate 200000 rows into 10000 groups: 44 ops per row, 200000 group visits; x 0.1000/s; holds 1920000 bytes for 67.800 ms", "multiplier": 1, "operator": "Aggregate", - "price_source": "analytical", - "timing": "query_time" - }, - "3": { - "cost_per_second": 2.26329231058e-10, - "cpu_core_seconds_per_second": 0.00002, - "detail": "build Kll into 10000 states: 200000 rows x depth 1; x 0.1000/s; holds 10000000 bytes for 1.3 ms", - "multiplier": 1, - "operator": "SummaryAgg", - "price_source": "analytical", - "timing": "query_time" - }, - "4": { - "cost_per_second": 1.1244e-11, - "cpu_core_seconds_per_second": 1e-6, - "detail": "estimate 10000 groups x 1; x 0.1000/s", - "multiplier": 1, - "operator": "SummaryEstimate", - "price_source": "analytical", - "timing": "query_time" - }, - "5": { - "cost_per_second": 1.12440046375e-11, - "cpu_core_seconds_per_second": 1e-6, - "detail": "operator: pass 10000 rows; x 0.1000/s; holds 32 bytes for 1.3 ms", - "multiplier": 1, - "operator": "Project", - "price_source": "analytical", "timing": "query_time" } }, "source": "analytical-cost-v2 (illustrative statistics)", "usage": { - "average_memory_bytes": 2458.08864, - "cpu_ops_per_second": 102000.0, + "average_memory_bytes": 13017.609728, + "cpu_ops_per_second": 7020000.0, "disk_bytes": 6400000, - "latency_per_query": [ - { - "compaction_ms": 0.0, - "query_ms": 1.44 - }, - { - "compaction_ms": 0.0, - "query_ms": 1.26 - } - ], "peak_cpu_cores": 2, - "peak_memory_bytes": 18320064, + "peak_memory_bytes": 1920064, "scanned_bytes_per_second": 640000.0 } }, @@ -125,24 +77,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -172,24 +121,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -203,180 +149,10 @@ "producer": 1, "role": "Input", "window": "NotApplicable" - }, - { - "consumer": 3, - "data_state": { - "primitive": "Raw", - "timing": "query_time" - }, - "grouping": "NotApplicable", - "intermediate_schema": { - "closed": true, - "fields": [ - { - "dtype": { - "Plain": "timestamp" - }, - "name": "ts", - "nullable": false, - "table": null - }, - { - "dtype": { - "Plain": "float64" - }, - "name": "value", - "nullable": false, - "table": null - }, - { - "dtype": { - "Plain": "utf8" - }, - "name": "$promql_series_identity", - "nullable": false, - "table": null - } - ], - "time_index": 0, - "unique_keys": [ - [ - 2, - 0 - ] - ] - }, - "producer": 1, - "role": "Input", - "window": "NotApplicable" - }, - { - "consumer": 4, - "data_state": { - "primitive": "SummaryState", - "timing": "query_time" - }, - "grouping": "NotApplicable", - "intermediate_schema": { - "closed": true, - "fields": [ - { - "dtype": { - "Plain": "timestamp" - }, - "name": "ts", - "nullable": false, - "table": null - }, - { - "dtype": { - "Sketch": [ - { - "algorithm": "Kll", - "params": { - "Kll": { - "failure_probability": 1e-7, - "k": 567 - } - } - }, - "PerSubpopulationInstance" - ] - }, - "name": "state", - "nullable": false, - "table": null - }, - { - "dtype": { - "Plain": "utf8" - }, - "name": "$promql_series_identity", - "nullable": false, - "table": null - } - ], - "time_index": 0, - "unique_keys": [ - [ - 2, - 0 - ] - ] - }, - "producer": 3, - "role": "Input", - "window": "NotApplicable" - }, - { - "consumer": 5, - "data_state": { - "primitive": "Raw", - "timing": "query_time" - }, - "grouping": "NotApplicable", - "intermediate_schema": { - "closed": true, - "fields": [ - { - "dtype": { - "Plain": "timestamp" - }, - "name": "ts", - "nullable": false, - "table": null - }, - { - "dtype": { - "Plain": "float64" - }, - "name": "quantile", - "nullable": false, - "table": null - }, - { - "dtype": { - "Plain": "utf8" - }, - "name": "$promql_series_identity", - "nullable": false, - "table": null - } - ], - "time_index": 0, - "unique_keys": [ - [ - 2, - 0 - ] - ] - }, - "producer": 4, - "role": "Input", - "window": "NotApplicable" } ], "nodes": [ { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact over exact inputs" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 0, "lifetime": "None", "output_schema": { @@ -387,24 +163,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -431,24 +204,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -466,31 +236,12 @@ } } } - } + }, + "retention_ms": 300000 }, { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact over exact inputs" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 1, - "lifetime": { - "OneEvaluation": "Memory" - }, + "lifetime": "None", "output_schema": { "closed": true, "fields": [ @@ -499,24 +250,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -545,26 +293,10 @@ } }, { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact over exact inputs" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 2, - "lifetime": "None", + "lifetime": { + "OneEvaluation": "Memory" + }, "output_schema": { "closed": true, "fields": [ @@ -573,24 +305,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -613,11 +342,6 @@ "having": null, "measures": [ { - "accuracy": { - "accuracy_scope": "whole_result", - "failure_probability": 0.0, - "max_error": 0.0 - }, "col": null, "kind": "quantile", "q": 0.9 @@ -630,285 +354,13 @@ } } } - }, - { - "guarantee": null, - "id": 3, - "lifetime": "None", - "output_schema": { - "closed": true, - "fields": [ - { - "dtype": { - "Plain": "timestamp" - }, - "name": "ts", - "nullable": false, - "table": null - }, - { - "dtype": { - "Sketch": [ - { - "algorithm": "Kll", - "params": { - "Kll": { - "failure_probability": 1e-7, - "k": 567 - } - } - }, - "PerSubpopulationInstance" - ] - }, - "name": "state", - "nullable": false, - "table": null - }, - { - "dtype": { - "Plain": "utf8" - }, - "name": "$promql_series_identity", - "nullable": false, - "table": null - } - ], - "time_index": 0, - "unique_keys": [ - [ - 2, - 0 - ] - ] - }, - "output_state": { - "primitive": "SummaryState", - "timing": "query_time" - }, - "payload": { - "ASAP": { - "SummaryAgg": { - "child": 1, - "family": { - "Sketch": [ - { - "algorithm": "Kll", - "params": { - "Kll": { - "failure_probability": 1e-7, - "k": 567 - } - } - }, - "PerSubpopulationInstance" - ] - }, - "filter": null, - "grouping": "PerSubpopulationInstance", - "input": { - "item": null, - "weight": { - "Column": "SampleValue" - } - }, - "reduction": "PerEntity" - } - } - } - }, - { - "guarantee": { - "provenance": [ - { - "algorithm": "Kll", - "contract": "apache_datasketches_kll_empirical_99_delta_scaled_a9b42755072b", - "kind": "sketch_evaluation", - "params": { - "Kll": { - "failure_probability": 1e-7, - "k": 567 - } - }, - "query": "Quantile { q: 0.9 }" - } - ], - "spec": { - "accuracy_metric": "rank", - "accuracy_scope": "per_value", - "error": { - "op": "constant", - "value": 0.00998173834561 - }, - "failure_probability": { - "op": "constant", - "value": 1e-7 - } - } - }, - "id": 4, - "lifetime": "None", - "output_schema": { - "closed": true, - "fields": [ - { - "dtype": { - "Plain": "timestamp" - }, - "name": "ts", - "nullable": false, - "table": null - }, - { - "dtype": { - "Plain": "float64" - }, - "name": "quantile", - "nullable": false, - "table": null - }, - { - "dtype": { - "Plain": "utf8" - }, - "name": "$promql_series_identity", - "nullable": false, - "table": null - } - ], - "time_index": 0, - "unique_keys": [ - [ - 2, - 0 - ] - ] - }, - "output_state": { - "primitive": "Raw", - "timing": "query_time" - }, - "payload": { - "ASAP": { - "SummaryEstimate": { - "query": { - "Quantile": { - "q": 0.9 - } - }, - "summary_input": 3 - } - } - } - }, - { - "guarantee": { - "provenance": [ - { - "algorithm": "Kll", - "contract": "apache_datasketches_kll_empirical_99_delta_scaled_a9b42755072b", - "kind": "sketch_evaluation", - "params": { - "Kll": { - "failure_probability": 1e-7, - "k": 567 - } - }, - "query": "Quantile { q: 0.9 }" - } - ], - "spec": { - "accuracy_metric": "rank", - "accuracy_scope": "per_value", - "error": { - "op": "constant", - "value": 0.00998173834561 - }, - "failure_probability": { - "op": "constant", - "value": 1e-7 - } - } - }, - "id": 5, - "lifetime": "None", - "output_schema": { - "closed": true, - "fields": [ - { - "dtype": { - "Plain": "timestamp" - }, - "name": "ts", - "nullable": false, - "table": null - }, - { - "dtype": { - "Plain": "float64" - }, - "name": "value", - "nullable": false, - "table": null - }, - { - "dtype": { - "Plain": "utf8" - }, - "name": "$promql_series_identity", - "nullable": false, - "table": null - } - ], - "time_index": 0, - "unique_keys": [ - [ - 2, - 0 - ] - ] - }, - "output_state": { - "primitive": "Raw", - "timing": "query_time" - }, - "payload": { - "NonASAP": { - "Project": { - "child": 4, - "cols": [ - { - "alias": "ts", - "expr": { - "Column": 0 - } - }, - { - "alias": "value", - "expr": { - "Column": 1 - } - }, - { - "alias": "$promql_series_identity", - "expr": { - "Column": 2 - } - } - ], - "qualifier": null - } - } - } } ], "roots": [ 2, - 5 + 2 ], "scalar_roots": [] - }, - "display_name": "Q1 exact · Q2 Kll · shared Scan (Q1, Q2) · query time, shared: TimeRange" - }, - "tumbling_window_retention_ms": {} + } + } } diff --git a/crates/devtools/tests/sketch_plan_snapshots/nested-windows-15m-1h-mvp.json b/crates/devtools/tests/sketch_plan_snapshots/nested-windows-15m-1h-mvp.json index edf25adfe..a21cb446d 100644 --- a/crates/devtools/tests/sketch_plan_snapshots/nested-windows-15m-1h-mvp.json +++ b/crates/devtools/tests/sketch_plan_snapshots/nested-windows-15m-1h-mvp.json @@ -3,2419 +3,242 @@ "materialized_views": [ { "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", + "definition_id": "5597d603bdf9f1a148d7c7e5344c1a97", "selection": [ { - "columns": [], - "relative_time": [ - { - "excluded": -60000 - }, - { - "included": 0 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 0, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 0, - "node_id": 2, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -120000 - }, - { - "included": -60000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 60000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 1, - "node_id": 5, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -180000 - }, - { - "included": -120000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 120000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 2, - "node_id": 8, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -240000 - }, - { - "included": -180000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 180000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 3, - "node_id": 11, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -300000 - }, - { - "included": -240000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 240000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 4, - "node_id": 14, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -360000 - }, - { - "included": -300000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 300000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 5, - "node_id": 17, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -420000 - }, - { - "included": -360000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 360000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 6, - "node_id": 20, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -480000 - }, - { - "included": -420000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 420000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 7, - "node_id": 23, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -540000 - }, - { - "included": -480000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 480000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 8, - "node_id": 26, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -600000 - }, - { - "included": -540000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 540000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 9, - "node_id": 29, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -660000 - }, - { - "included": -600000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 600000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 10, - "node_id": 32, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -720000 - }, - { - "included": -660000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 660000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 11, - "node_id": 35, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -780000 - }, - { - "included": -720000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 720000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 12, - "node_id": 38, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -840000 - }, - { - "included": -780000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 780000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 13, - "node_id": 41, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -900000 - }, - { - "included": -840000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 840000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 14, - "node_id": 44, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -960000 - }, - { - "included": -900000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 900000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 15, - "node_id": 50, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1020000 - }, - { - "included": -960000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 960000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 16, - "node_id": 53, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1080000 - }, - { - "included": -1020000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1020000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 17, - "node_id": 56, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1140000 - }, - { - "included": -1080000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1080000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 18, - "node_id": 59, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1200000 - }, - { - "included": -1140000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1140000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 19, - "node_id": 62, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1260000 - }, - { - "included": -1200000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1200000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 20, - "node_id": 65, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1320000 - }, - { - "included": -1260000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1260000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 21, - "node_id": 68, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1380000 - }, - { - "included": -1320000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1320000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 22, - "node_id": 71, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1440000 - }, - { - "included": -1380000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1380000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 23, - "node_id": 74, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1500000 - }, - { - "included": -1440000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1440000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 24, - "node_id": 77, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1560000 - }, - { - "included": -1500000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1500000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 25, - "node_id": 80, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1620000 - }, - { - "included": -1560000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1560000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 26, - "node_id": 83, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1680000 - }, - { - "included": -1620000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1620000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 27, - "node_id": 86, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1740000 - }, - { - "included": -1680000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1680000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 28, - "node_id": 89, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1800000 - }, - { - "included": -1740000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1740000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 29, - "node_id": 92, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1860000 - }, - { - "included": -1800000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1800000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 30, - "node_id": 95, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1920000 - }, - { - "included": -1860000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1860000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 31, - "node_id": 98, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1980000 - }, - { - "included": -1920000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1920000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 32, - "node_id": 101, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2040000 - }, - { - "included": -1980000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1980000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 33, - "node_id": 104, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2100000 - }, - { - "included": -2040000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2040000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 34, - "node_id": 107, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2160000 - }, - { - "included": -2100000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2100000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 35, - "node_id": 110, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2220000 - }, - { - "included": -2160000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2160000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 36, - "node_id": 113, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2280000 - }, - { - "included": -2220000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2220000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 37, - "node_id": 116, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2340000 - }, - { - "included": -2280000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2280000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 38, - "node_id": 119, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2400000 - }, - { - "included": -2340000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2340000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 39, - "node_id": 122, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2460000 - }, - { - "included": -2400000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2400000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 40, - "node_id": 125, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2520000 - }, - { - "included": -2460000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2460000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 41, - "node_id": 128, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2580000 - }, - { - "included": -2520000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2520000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 42, - "node_id": 131, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2640000 - }, - { - "included": -2580000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2580000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 43, - "node_id": 134, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2700000 - }, - { - "included": -2640000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2640000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 44, - "node_id": 137, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2760000 - }, - { - "included": -2700000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2700000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 45, - "node_id": 140, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2820000 - }, - { - "included": -2760000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2760000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 46, - "node_id": 143, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2880000 - }, - { - "included": -2820000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2820000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 47, - "node_id": 146, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2940000 - }, - { - "included": -2880000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2880000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 48, - "node_id": 149, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -3000000 - }, - { - "included": -2940000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2940000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 49, - "node_id": 152, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -3060000 - }, - { - "included": -3000000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3000000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 50, - "node_id": 155, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -3120000 - }, - { - "included": -3060000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3060000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 51, - "node_id": 158, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -3180000 - }, - { - "included": -3120000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3120000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 52, - "node_id": 161, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -3240000 - }, - { - "included": -3180000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3180000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 53, - "node_id": 164, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -3300000 - }, - { - "included": -3240000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3240000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 54, - "node_id": 167, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -3360000 - }, - { - "included": -3300000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3300000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 55, - "node_id": 170, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -3420000 - }, - { - "included": -3360000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3360000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 56, - "node_id": 173, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -3480000 - }, - { - "included": -3420000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3420000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 57, - "node_id": 176, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -3540000 - }, - { - "included": -3480000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3480000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 58, - "node_id": 179, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -3600000 - }, - { - "included": -3540000 - } - ] + "columns": [] } ] }, "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3540000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] + "materialized_views": [] }, - "installed_count": 60, - "installed_position": 59, - "node_id": 182, + "node_id": 1, "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 + "retention_ms": 3600000 } ], - "proven_optimal": true, "queries": [ "quantile_over_time(0.9, data[15m])", "quantile_over_time(0.9, data[1h])" ], - "search_status": "optimal", "selected": { "accepted": [ { "accepted_metric": "relative", - "accepted_regardless_of_guarantee": false, - "guarantee_check_rejection": null, "sketches": [ "DDSketch" ], - "summary": "metric not set; accepted relative (DDSketch)", - "target_metric": null + "summary": "accepted relative (DDSketch)" }, { "accepted_metric": "relative", - "accepted_regardless_of_guarantee": false, - "guarantee_check_rejection": null, "sketches": [ "DDSketch" ], - "summary": "metric not set; accepted relative (DDSketch)", - "target_metric": null + "summary": "accepted relative (DDSketch)" } ], "cost": { - "cost_per_second": 3.99002037349e-6, - "cost_unit": "cost_per_second", - "cpu_core_seconds_per_second": 0.0124131666667, + "cost_per_second": 7.06422112846e-6, + "cpu_core_seconds_per_second": 0.489907833333, "per_node": { "0": { "cost_per_second": 9.44496e-8, + "cost_source": "analytical", "cpu_core_seconds_per_second": 0.0084, "detail": "scan 2000000 samples; x 1.0000/s", "multiplier": 1, "operator": "Scan", - "price_source": "analytical", "timing": "ingestion_time" }, "1": { - "cost_per_second": 3.89542189639e-6, - "cpu_core_seconds_per_second": 0.004, - "detail": "build DDSketch in 60 tumbling windows of 60000 ms for 10000 groups: 2000000 rows x (1 + depth 1); x 1.0000/s; keeps 3347680000 bytes", + "cost_per_second": 6.59868036528e-6, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.448586666667, + "detail": "build DDSketch in tumbling windows of 60000 ms for 10000 groups: 2000000 rows x 210 ops, 1000 group visits, 166.7 states emitted; x 1.0000/s; keeps 1351760000 bytes", "multiplier": 1, "operator": "SummaryTumblingWindowChain", - "price_source": "analytical", "timing": "ingestion_time" }, "2": { - "cost_per_second": 2.82888458476e-11, - "cpu_core_seconds_per_second": 2.5e-6, - "detail": "merge 150000 states into 10000; x 0.0167/s; holds 54880000 bytes for 0.2 ms", + "cost_per_second": 1.23684028037e-9, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.00011, + "detail": "time range 3600s: 60 tumbling windows of states; x 0.0167/s; holds 2216 bytes for 6.600 ms", "multiplier": 1, - "operator": "SummaryMerge", - "price_source": "analytical", + "operator": "TimeRange", "timing": "query_time" }, "3": { - "cost_per_second": 1.874e-12, - "cpu_core_seconds_per_second": 1.66666666667e-7, + "cost_per_second": 2.83905232793e-7, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.0251833333333, + "detail": "merge 600000 states into 10000; x 0.0167/s; holds 25680000 bytes for 1511.000 ms", + "multiplier": 1, + "operator": "SummaryMerge", + "timing": "query_time" + }, + "4": { + "cost_per_second": 6.559e-9, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.000583333333333, "detail": "estimate 10000 groups x 1; x 0.0167/s", "multiplier": 1, "operator": "SummaryEstimate", - "price_source": "analytical", "timing": "query_time" }, - "4": { - "cost_per_second": 1.87400010428e-12, + "5": { + "cost_per_second": 1.87400000613e-12, + "cost_source": "analytical", "cpu_core_seconds_per_second": 1.66666666667e-7, - "detail": "operator: pass 10000 rows; x 0.0167/s; holds 32 bytes for 0.2 ms", + "detail": "operator: pass 10000 rows; x 0.0167/s; holds 32 bytes for 0.010 ms", "multiplier": 1, "operator": "Project", - "price_source": "analytical", "timing": "query_time" }, - "5": { - "cost_per_second": 1.13092261327e-10, - "cpu_core_seconds_per_second": 0.00001, - "detail": "merge 600000 states into 10000; x 0.0167/s; holds 54880000 bytes for 0.6 ms", + "6": { + "cost_per_second": 3.09210070092e-10, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.0000275, + "detail": "time range 900s: 15 tumbling windows of states; x 0.0167/s; holds 2216 bytes for 1.650 ms", + "multiplier": 1, + "operator": "TimeRange", + "timing": "query_time" + }, + "7": { + "cost_per_second": 7.25181320411e-8, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.00643333333333, + "detail": "merge 150000 states into 10000; x 0.0167/s; holds 24560000 bytes for 386.000 ms", "multiplier": 1, "operator": "SummaryMerge", - "price_source": "analytical", "timing": "query_time" }, - "6": { - "cost_per_second": 1.874e-12, - "cpu_core_seconds_per_second": 1.66666666667e-7, + "8": { + "cost_per_second": 6.559e-9, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.000583333333333, "detail": "estimate 10000 groups x 1; x 0.0167/s", "multiplier": 1, "operator": "SummaryEstimate", - "price_source": "analytical", "timing": "query_time" }, - "7": { - "cost_per_second": 1.87400038033e-12, + "9": { + "cost_per_second": 1.87400000613e-12, + "cost_source": "analytical", "cpu_core_seconds_per_second": 1.66666666667e-7, - "detail": "operator: pass 10000 rows; x 0.0167/s; holds 32 bytes for 0.6 ms", + "detail": "operator: pass 10000 rows; x 0.0167/s; holds 32 bytes for 0.010 ms", "multiplier": 1, "operator": "Project", - "price_source": "analytical", "timing": "query_time" } }, "source": "analytical-cost-v2 (illustrative statistics)", "usage": { - "average_memory_bytes": 3347680722.59, - "cpu_ops_per_second": 6013166.66667, + "average_memory_bytes": 1352564710.97, + "cpu_ops_per_second": 483507833.333, "disk_bytes": 0, - "latency_per_query": [ - { - "compaction_ms": 0.0, - "query_ms": 0.17 - }, - { - "compaction_ms": 0.0, - "query_ms": 0.62 - } - ], "peak_cpu_cores": 3, - "peak_memory_bytes": 3457440096, + "peak_memory_bytes": 1402004528, "scanned_bytes_per_second": 64000000.0 } }, "dag": { "edges": [ { - "consumer": 1, + "consumer": 1, + "data_state": { + "primitive": "Raw", + "timing": "ingestion_time" + }, + "grouping": "NotApplicable", + "intermediate_schema": { + "closed": true, + "fields": [ + { + "dtype": { + "Plain": "timestamp" + }, + "name": "ts", + "nullable": false + }, + { + "dtype": { + "Plain": "float64" + }, + "name": "value", + "nullable": false + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false + } + ], + "time_index": 0, + "unique_keys": [ + [ + 2, + 0 + ] + ] + }, + "producer": 0, + "role": "Input", + "window": "NotApplicable" + }, + { + "consumer": 2, + "data_state": { + "primitive": "SummaryState", + "timing": "ingestion_time" + }, + "grouping": "NotApplicable", + "intermediate_schema": { + "closed": true, + "fields": [ + { + "dtype": { + "Plain": "timestamp" + }, + "name": "window_end", + "nullable": false + }, + { + "dtype": { + "Sketch": "DDSketch" + }, + "name": "state", + "nullable": false + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false + } + ], + "time_index": 0, + "unique_keys": [ + [ + 0, + 2 + ] + ] + }, + "producer": 1, + "role": "Input", + "window": "NotApplicable" + }, + { + "consumer": 3, "data_state": { - "primitive": "Raw", - "timing": "ingestion_time" + "primitive": "SummaryState", + "timing": "query_time" }, "grouping": "NotApplicable", "intermediate_schema": { @@ -2425,44 +248,41 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { - "Plain": "float64" + "Sketch": "DDSketch" }, - "name": "value", - "nullable": false, - "table": null + "name": "state", + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 2, - 0 + 0, + 2 ] ] }, - "producer": 0, + "producer": 2, "role": "Input", - "window": "RequiresAlignedPanePhaseOrExactBoundaryResidual" + "window": "NotApplicable" }, { - "consumer": 2, + "consumer": 4, "data_state": { "primitive": "SummaryState", - "timing": "ingestion_time" + "timing": "query_time" }, "grouping": "NotApplicable", "intermediate_schema": { @@ -2473,34 +293,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { - "Sketch": [ - { - "algorithm": "DDSketch", - "params": { - "DDSketch": { - "alpha": 0.05 - } - } - }, - "PerSubpopulationInstance" - ] + "Sketch": "DDSketch" }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -2511,14 +318,14 @@ ] ] }, - "producer": 1, + "producer": 3, "role": "Input", "window": "NotApplicable" }, { - "consumer": 3, + "consumer": 5, "data_state": { - "primitive": "SummaryState", + "primitive": "Raw", "timing": "query_time" }, "grouping": "NotApplicable", @@ -2530,34 +337,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { - "Sketch": [ - { - "algorithm": "DDSketch", - "params": { - "DDSketch": { - "alpha": 0.05 - } - } - }, - "PerSubpopulationInstance" - ] + "Plain": "float64" }, - "name": "state", - "nullable": false, - "table": null + "name": "quantile", + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -2568,15 +362,15 @@ ] ] }, - "producer": 2, + "producer": 4, "role": "Input", "window": "NotApplicable" }, { - "consumer": 4, + "consumer": 6, "data_state": { - "primitive": "Raw", - "timing": "query_time" + "primitive": "SummaryState", + "timing": "ingestion_time" }, "grouping": "NotApplicable", "intermediate_schema": { @@ -2586,44 +380,41 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { - "Plain": "float64" + "Sketch": "DDSketch" }, - "name": "quantile", - "nullable": false, - "table": null + "name": "state", + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 2, - 0 + 0, + 2 ] ] }, - "producer": 3, + "producer": 1, "role": "Input", "window": "NotApplicable" }, { - "consumer": 5, + "consumer": 7, "data_state": { "primitive": "SummaryState", - "timing": "ingestion_time" + "timing": "query_time" }, "grouping": "NotApplicable", "intermediate_schema": { @@ -2633,51 +424,38 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { - "Sketch": [ - { - "algorithm": "DDSketch", - "params": { - "DDSketch": { - "alpha": 0.05 - } - } - }, - "PerSubpopulationInstance" - ] + "Sketch": "DDSketch" }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 2, - 0 + 0, + 2 ] ] }, - "producer": 1, + "producer": 6, "role": "Input", "window": "NotApplicable" }, { - "consumer": 6, + "consumer": 8, "data_state": { "primitive": "SummaryState", "timing": "query_time" @@ -2691,34 +469,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { - "Sketch": [ - { - "algorithm": "DDSketch", - "params": { - "DDSketch": { - "alpha": 0.05 - } - } - }, - "PerSubpopulationInstance" - ] + "Sketch": "DDSketch" }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -2729,12 +494,12 @@ ] ] }, - "producer": 5, + "producer": 7, "role": "Input", "window": "NotApplicable" }, { - "consumer": 7, + "consumer": 9, "data_state": { "primitive": "Raw", "timing": "query_time" @@ -2748,24 +513,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "quantile", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -2776,31 +538,13 @@ ] ] }, - "producer": 6, + "producer": 8, "role": "Input", "window": "NotApplicable" } ], "nodes": [ { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact over exact inputs" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 0, "lifetime": "None", "output_schema": { @@ -2811,24 +555,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -2855,24 +596,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -2893,7 +631,6 @@ } }, { - "guarantee": null, "id": 1, "lifetime": { "AcrossEvaluations": "Memory" @@ -2905,42 +642,29 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { - "Sketch": [ - { - "algorithm": "DDSketch", - "params": { - "DDSketch": { - "alpha": 0.05 - } - } - }, - "PerSubpopulationInstance" - ] + "Sketch": "DDSketch" }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 2, - 0 + 0, + 2 ] ] }, @@ -2948,25 +672,16 @@ "primitive": "SummaryState", "timing": "ingestion_time" }, + "parameters": { + "DDSketch": { + "alpha": 0.05 + } + }, "payload": { "ASAP": { "SummaryTumblingWindowChain": { "child": 0, - "family": { - "Sketch": [ - { - "algorithm": "DDSketch", - "params": { - "DDSketch": { - "alpha": 0.05 - } - } - }, - "PerSubpopulationInstance" - ] - }, "filter": null, - "grouping": "PerSubpopulationInstance", "input": { "item": null, "weight": { @@ -2974,14 +689,24 @@ } }, "reduction": "PerEntity", - "tumbling_window_count": 60, + "summary_type": { + "Sketch": "DDSketch" + }, "tumbling_window_width_ms": 60000 } } + }, + "retention_ms": 3600000, + "view_coverage": { + "definition_id": "5597d603bdf9f1a148d7c7e5344c1a97", + "selection": [ + { + "columns": [] + } + ] } }, { - "guarantee": null, "id": 2, "lifetime": "None", "output_schema": { @@ -2991,42 +716,29 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { - "Sketch": [ - { - "algorithm": "DDSketch", - "params": { - "DDSketch": { - "alpha": 0.05 - } - } - }, - "PerSubpopulationInstance" - ] + "Sketch": "DDSketch" }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 2, - 0 + 0, + 2 ] ] }, @@ -3034,54 +746,83 @@ "primitive": "SummaryState", "timing": "query_time" }, + "parameters": { + "DDSketch": { + "alpha": 0.05 + } + }, "payload": { - "ASAP": { - "SummaryMerge": { - "children": [ - 1 - ], - "newest_tumbling_windows": 15 + "NonASAP": { + "TimeRange": { + "child": 1, + "kind": "range", + "range": { + "nanos": 0, + "secs": 3600 + } } } } }, { - "guarantee": { - "provenance": [ + "id": 3, + "lifetime": { + "OneEvaluation": "Memory" + }, + "output_schema": { + "closed": true, + "fields": [ { - "algorithm": "DDSketch", - "contract": "ddsketch_relative_error_alpha_v1", - "kind": "sketch_evaluation", - "params": { - "DDSketch": { - "alpha": 0.05 - } + "dtype": { + "Plain": "timestamp" }, - "query": "Quantile { q: 0.9 }" - } - ], - "spec": { - "accuracy_metric": "relative", - "accuracy_scope": "per_value", - "error": { - "factors": [ - { - "op": "constant", - "value": 0.05 - }, - { - "op": "unknown", - "statistic": "values_non_negative[data]" - } - ], - "op": "product" + "name": "ts", + "nullable": false + }, + { + "dtype": { + "Sketch": "DDSketch" + }, + "name": "state", + "nullable": false }, - "failure_probability": { - "op": "zero" + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false } + ], + "time_index": 0, + "unique_keys": [ + [ + 2, + 0 + ] + ] + }, + "output_state": { + "primitive": "SummaryState", + "timing": "query_time" + }, + "parameters": { + "DDSketch": { + "alpha": 0.05 } }, - "id": 3, + "payload": { + "ASAP": { + "SummaryMerge": { + "children": [ + 2 + ] + } + } + } + }, + { + "id": 4, "lifetime": "None", "output_schema": { "closed": true, @@ -3091,24 +832,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "quantile", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -3131,48 +869,13 @@ "q": 0.9 } }, - "summary_input": 2 + "summary_input": 3 } } } }, { - "guarantee": { - "provenance": [ - { - "algorithm": "DDSketch", - "contract": "ddsketch_relative_error_alpha_v1", - "kind": "sketch_evaluation", - "params": { - "DDSketch": { - "alpha": 0.05 - } - }, - "query": "Quantile { q: 0.9 }" - } - ], - "spec": { - "accuracy_metric": "relative", - "accuracy_scope": "per_value", - "error": { - "factors": [ - { - "op": "constant", - "value": 0.05 - }, - { - "op": "unknown", - "statistic": "values_non_negative[data]" - } - ], - "op": "product" - }, - "failure_probability": { - "op": "zero" - } - } - }, - "id": 4, + "id": 5, "lifetime": "None", "output_schema": { "closed": true, @@ -3182,24 +885,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -3217,7 +917,7 @@ "payload": { "NonASAP": { "Project": { - "child": 3, + "child": 4, "cols": [ { "alias": "ts", @@ -3237,16 +937,74 @@ "Column": 2 } } - ], - "qualifier": null + ] } } } }, { - "guarantee": null, - "id": 5, + "id": 6, "lifetime": "None", + "output_schema": { + "closed": true, + "fields": [ + { + "dtype": { + "Plain": "timestamp" + }, + "name": "window_end", + "nullable": false + }, + { + "dtype": { + "Sketch": "DDSketch" + }, + "name": "state", + "nullable": false + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false + } + ], + "time_index": 0, + "unique_keys": [ + [ + 0, + 2 + ] + ] + }, + "output_state": { + "primitive": "SummaryState", + "timing": "query_time" + }, + "parameters": { + "DDSketch": { + "alpha": 0.05 + } + }, + "payload": { + "NonASAP": { + "TimeRange": { + "child": 1, + "kind": "range", + "range": { + "nanos": 0, + "secs": 900 + } + } + } + } + }, + { + "id": 7, + "lifetime": { + "OneEvaluation": "Memory" + }, "output_schema": { "closed": true, "fields": [ @@ -3255,34 +1013,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { - "Sketch": [ - { - "algorithm": "DDSketch", - "params": { - "DDSketch": { - "alpha": 0.05 - } - } - }, - "PerSubpopulationInstance" - ] + "Sketch": "DDSketch" }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -3297,53 +1042,23 @@ "primitive": "SummaryState", "timing": "query_time" }, + "parameters": { + "DDSketch": { + "alpha": 0.05 + } + }, "payload": { "ASAP": { "SummaryMerge": { "children": [ - 1 + 6 ] } } } }, { - "guarantee": { - "provenance": [ - { - "algorithm": "DDSketch", - "contract": "ddsketch_relative_error_alpha_v1", - "kind": "sketch_evaluation", - "params": { - "DDSketch": { - "alpha": 0.05 - } - }, - "query": "Quantile { q: 0.9 }" - } - ], - "spec": { - "accuracy_metric": "relative", - "accuracy_scope": "per_value", - "error": { - "factors": [ - { - "op": "constant", - "value": 0.05 - }, - { - "op": "unknown", - "statistic": "values_non_negative[data]" - } - ], - "op": "product" - }, - "failure_probability": { - "op": "zero" - } - } - }, - "id": 6, + "id": 8, "lifetime": "None", "output_schema": { "closed": true, @@ -3353,24 +1068,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "quantile", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -3393,48 +1105,13 @@ "q": 0.9 } }, - "summary_input": 5 + "summary_input": 7 } } } }, { - "guarantee": { - "provenance": [ - { - "algorithm": "DDSketch", - "contract": "ddsketch_relative_error_alpha_v1", - "kind": "sketch_evaluation", - "params": { - "DDSketch": { - "alpha": 0.05 - } - }, - "query": "Quantile { q: 0.9 }" - } - ], - "spec": { - "accuracy_metric": "relative", - "accuracy_scope": "per_value", - "error": { - "factors": [ - { - "op": "constant", - "value": 0.05 - }, - { - "op": "unknown", - "statistic": "values_non_negative[data]" - } - ], - "op": "product" - }, - "failure_probability": { - "op": "zero" - } - } - }, - "id": 7, + "id": 9, "lifetime": "None", "output_schema": { "closed": true, @@ -3444,24 +1121,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -3479,7 +1153,7 @@ "payload": { "NonASAP": { "Project": { - "child": 6, + "child": 8, "cols": [ { "alias": "ts", @@ -3499,22 +1173,20 @@ "Column": 2 } } - ], - "qualifier": null + ] } } } } ], "roots": [ - 4, - 7 + 9, + 5 ], "scalar_roots": [] - }, - "display_name": "Q1 DDSketch ×60 tumbling windows · Q2 DDSketch ×60 tumbling windows · shared DDSketch ×60 tumbling windows (Q1, Q2) · shared Scan (Q1, Q2) · ingestion time: DDSketch for Q1, Q2, 60 × 1m tumbling windows" + } }, - "tumbling_window_retention_ms": { - "1": 3600000 - } + "warnings": [ + "assumed data fact: source data declares no column_magnitudes for value, so a DDSketch state over it is costed for magnitudes in [1, 1000000000]; this affects cost only, the accuracy guarantee does not depend on it" + ] } diff --git a/crates/devtools/tests/sketch_plan_snapshots/quantile-grouped-mvp.json b/crates/devtools/tests/sketch_plan_snapshots/quantile-grouped-mvp.json index 37a58c734..bf1b885b4 100644 --- a/crates/devtools/tests/sketch_plan_snapshots/quantile-grouped-mvp.json +++ b/crates/devtools/tests/sketch_plan_snapshots/quantile-grouped-mvp.json @@ -3,1770 +3,119 @@ "materialized_views": [ { "coverage": { - "definition_id": "ef56d9daf3c36b6588ca5f3bbb46d4a6", + "definition_id": "0c7f962d8fde713c1de7a58752019151", "selection": [ { - "columns": [], - "relative_time": [ - { - "excluded": -10000 - }, - { - "included": 0 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 0, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 0, - "node_id": 2, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef56d9daf3c36b6588ca5f3bbb46d4a6", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -20000 - }, - { - "included": -10000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 10000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 1, - "node_id": 5, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef56d9daf3c36b6588ca5f3bbb46d4a6", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -30000 - }, - { - "included": -20000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 20000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 2, - "node_id": 8, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef56d9daf3c36b6588ca5f3bbb46d4a6", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -40000 - }, - { - "included": -30000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 30000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 3, - "node_id": 11, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef56d9daf3c36b6588ca5f3bbb46d4a6", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -50000 - }, - { - "included": -40000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 40000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 4, - "node_id": 14, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef56d9daf3c36b6588ca5f3bbb46d4a6", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -60000 - }, - { - "included": -50000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 50000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 5, - "node_id": 17, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef56d9daf3c36b6588ca5f3bbb46d4a6", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -70000 - }, - { - "included": -60000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 60000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 6, - "node_id": 20, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef56d9daf3c36b6588ca5f3bbb46d4a6", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -80000 - }, - { - "included": -70000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 70000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 7, - "node_id": 23, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef56d9daf3c36b6588ca5f3bbb46d4a6", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -90000 - }, - { - "included": -80000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 80000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 8, - "node_id": 26, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef56d9daf3c36b6588ca5f3bbb46d4a6", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -100000 - }, - { - "included": -90000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 90000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 9, - "node_id": 29, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef56d9daf3c36b6588ca5f3bbb46d4a6", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -110000 - }, - { - "included": -100000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 100000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 10, - "node_id": 32, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef56d9daf3c36b6588ca5f3bbb46d4a6", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -120000 - }, - { - "included": -110000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 110000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 11, - "node_id": 35, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef56d9daf3c36b6588ca5f3bbb46d4a6", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -130000 - }, - { - "included": -120000 - } - ] + "columns": [] } ] }, "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 120000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 10000 - } - ] + "materialized_views": [] }, - "installed_count": 30, - "installed_position": 12, - "node_id": 38, + "node_id": 1, "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef56d9daf3c36b6588ca5f3bbb46d4a6", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -140000 - }, - { - "included": -130000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 130000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 13, - "node_id": 41, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef56d9daf3c36b6588ca5f3bbb46d4a6", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -150000 - }, - { - "included": -140000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 140000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 14, - "node_id": 44, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef56d9daf3c36b6588ca5f3bbb46d4a6", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -160000 - }, - { - "included": -150000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 150000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 15, - "node_id": 47, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef56d9daf3c36b6588ca5f3bbb46d4a6", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -170000 - }, - { - "included": -160000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 160000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 16, - "node_id": 50, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef56d9daf3c36b6588ca5f3bbb46d4a6", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -180000 - }, - { - "included": -170000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 170000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 17, - "node_id": 53, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef56d9daf3c36b6588ca5f3bbb46d4a6", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -190000 - }, - { - "included": -180000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 180000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 18, - "node_id": 56, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef56d9daf3c36b6588ca5f3bbb46d4a6", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -200000 - }, - { - "included": -190000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 190000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 19, - "node_id": 59, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef56d9daf3c36b6588ca5f3bbb46d4a6", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -210000 - }, - { - "included": -200000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 200000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 20, - "node_id": 62, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef56d9daf3c36b6588ca5f3bbb46d4a6", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -220000 - }, - { - "included": -210000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 210000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 21, - "node_id": 65, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef56d9daf3c36b6588ca5f3bbb46d4a6", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -230000 - }, - { - "included": -220000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 220000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 22, - "node_id": 68, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef56d9daf3c36b6588ca5f3bbb46d4a6", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -240000 - }, - { - "included": -230000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 230000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 23, - "node_id": 71, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef56d9daf3c36b6588ca5f3bbb46d4a6", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -250000 - }, - { - "included": -240000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 240000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 24, - "node_id": 74, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef56d9daf3c36b6588ca5f3bbb46d4a6", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -260000 - }, - { - "included": -250000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 250000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 25, - "node_id": 77, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef56d9daf3c36b6588ca5f3bbb46d4a6", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -270000 - }, - { - "included": -260000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 260000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 26, - "node_id": 80, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef56d9daf3c36b6588ca5f3bbb46d4a6", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -280000 - }, - { - "included": -270000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 270000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 27, - "node_id": 83, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef56d9daf3c36b6588ca5f3bbb46d4a6", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -290000 - }, - { - "included": -280000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 280000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 28, - "node_id": 86, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef56d9daf3c36b6588ca5f3bbb46d4a6", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -300000 - }, - { - "included": -290000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 290000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 29, - "node_id": 89, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - } - ], - "proven_optimal": true, - "queries": [ - "avg by (region) (quantile_over_time(0.9, data[5m]))" - ], - "search_status": "optimal", - "selected": { - "accepted": [ - { - "accepted_metric": "relative", - "accepted_regardless_of_guarantee": false, - "guarantee_check_rejection": null, - "sketches": [ - "DDSketch" - ], - "summary": "metric not set; accepted relative (DDSketch)", - "target_metric": null - } - ], - "cost": { - "cost_per_second": 1.96599532814e-6, - "cost_unit": "cost_per_second", - "cpu_core_seconds_per_second": 0.000819, - "per_node": { - "0": { - "cost_per_second": 6.52152e-9, - "cpu_core_seconds_per_second": 0.00058, - "detail": "scan 100000 samples; x 1.0000/s", - "multiplier": 1, - "operator": "Scan", - "price_source": "analytical", - "timing": "ingestion_time" - }, - "1": { - "cost_per_second": 1.95903278013e-6, - "cpu_core_seconds_per_second": 0.0002, - "detail": "build DDSketch in 30 tumbling windows of 10000 ms for 10000 groups: 100000 rows x (1 + depth 1); x 1.0000/s; keeps 1701280000 bytes", - "multiplier": 1, - "operator": "SummaryTumblingWindowChain", - "price_source": "analytical", - "timing": "ingestion_time" - }, - "10": { - "cost_per_second": 1.12440010766e-11, - "cpu_core_seconds_per_second": 1e-6, - "detail": "operator: pass 10000 rows; x 0.1000/s; holds 24 bytes for 0.4 ms", - "multiplier": 1, - "operator": "BinaryOp", - "price_source": "analytical", - "timing": "query_time" - }, - "11": { - "cost_per_second": 1.12440010766e-11, - "cpu_core_seconds_per_second": 1e-6, - "detail": "operator: pass 10000 rows; x 0.1000/s; holds 24 bytes for 0.4 ms", - "multiplier": 1, - "operator": "Project", - "price_source": "analytical", - "timing": "query_time" - }, - "2": { - "cost_per_second": 3.39781760491e-10, - "cpu_core_seconds_per_second": 0.00003, - "detail": "merge 300000 states into 10000; x 0.1000/s; holds 54880000 bytes for 0.4 ms", - "multiplier": 1, - "operator": "SummaryMerge", - "price_source": "analytical", - "timing": "query_time" - }, - "3": { - "cost_per_second": 1.1244e-11, - "cpu_core_seconds_per_second": 1e-6, - "detail": "estimate 10000 groups x 1; x 0.1000/s", - "multiplier": 1, - "operator": "SummaryEstimate", - "price_source": "analytical", - "timing": "query_time" - }, - "4": { - "cost_per_second": 1.12440021531e-11, - "cpu_core_seconds_per_second": 1e-6, - "detail": "operator: pass 10000 rows; x 0.1000/s; holds 48 bytes for 0.4 ms", - "multiplier": 1, - "operator": "Project", - "price_source": "analytical", - "timing": "query_time" - }, - "5": { - "cost_per_second": 1.1269120005e-11, - "cpu_core_seconds_per_second": 1e-6, - "detail": "build exact Sum accumulator into 10000 states: 10000 rows x depth 1; x 0.1000/s; holds 560000 bytes for 0.4 ms", - "multiplier": 1, - "operator": "SummaryAgg", - "price_source": "analytical", - "timing": "query_time" - }, - "6": { - "cost_per_second": 1.1244e-11, - "cpu_core_seconds_per_second": 1e-6, - "detail": "finalize 10000 accumulators; x 0.1000/s", - "multiplier": 1, - "operator": "FinalizeExactAccumulator", - "price_source": "analytical", - "timing": "query_time" - }, - "7": { - "cost_per_second": 1.1269120005e-11, - "cpu_core_seconds_per_second": 1e-6, - "detail": "build exact Count accumulator into 10000 states: 10000 rows x depth 1; x 0.1000/s; holds 560000 bytes for 0.4 ms", - "multiplier": 1, - "operator": "SummaryAgg", - "price_source": "analytical", - "timing": "query_time" - }, - "8": { - "cost_per_second": 1.1244e-11, - "cpu_core_seconds_per_second": 1e-6, - "detail": "finalize 10000 accumulators; x 0.1000/s", - "multiplier": 1, - "operator": "FinalizeExactAccumulator", - "price_source": "analytical", - "timing": "query_time" - }, - "9": { - "cost_per_second": 1.12440010766e-11, - "cpu_core_seconds_per_second": 1e-6, - "detail": "operator: pass 10000 rows; x 0.1000/s; holds 24 bytes for 0.4 ms", - "multiplier": 1, - "operator": "Project", - "price_source": "analytical", - "timing": "query_time" - } - }, - "source": "analytical-cost-v2 (illustrative statistics)", - "usage": { - "average_memory_bytes": 1701282184.0, - "cpu_ops_per_second": 339000.0, - "disk_bytes": 0, - "latency_per_query": [ - { - "compaction_ms": 0.0, - "query_ms": 0.39 - } - ], - "peak_cpu_cores": 2, - "peak_memory_bytes": 1757280168, - "scanned_bytes_per_second": 4800000.0 - } - }, - "dag": { - "edges": [ - { - "consumer": 1, - "data_state": { - "primitive": "Raw", - "timing": "ingestion_time" - }, - "grouping": "NotApplicable", - "intermediate_schema": { - "closed": true, - "fields": [ - { - "dtype": { - "Plain": "timestamp" - }, - "name": "ts", - "nullable": false, - "table": null - }, - { - "dtype": { - "Plain": "float64" - }, - "name": "value", - "nullable": false, - "table": null - }, - { - "dtype": { - "Plain": "utf8" - }, - "name": "region", - "nullable": true, - "table": null - }, - { - "dtype": { - "Plain": "utf8" - }, - "name": "$promql_series_identity", - "nullable": false, - "table": null - } - ], - "time_index": 0, - "unique_keys": [ - [ - 3, - 0 - ] - ] - }, - "producer": 0, - "role": "Input", - "window": "RequiresAlignedPanePhaseOrExactBoundaryResidual" - }, - { - "consumer": 2, - "data_state": { - "primitive": "SummaryState", - "timing": "ingestion_time" - }, - "grouping": "NotApplicable", - "intermediate_schema": { - "closed": true, - "fields": [ - { - "dtype": { - "Plain": "timestamp" - }, - "name": "ts", - "nullable": false, - "table": null - }, - { - "dtype": { - "Sketch": [ - { - "algorithm": "DDSketch", - "params": { - "DDSketch": { - "alpha": 0.05 - } - } - }, - "PerSubpopulationInstance" - ] - }, - "name": "state", - "nullable": false, - "table": null - }, - { - "dtype": { - "Plain": "utf8" - }, - "name": "region", - "nullable": true, - "table": null - }, - { - "dtype": { - "Plain": "utf8" - }, - "name": "$promql_series_identity", - "nullable": false, - "table": null - } - ], - "time_index": 0, - "unique_keys": [ - [ - 3, - 0 - ] - ] - }, - "producer": 1, - "role": "Input", - "window": "NotApplicable" - }, - { - "consumer": 3, - "data_state": { - "primitive": "SummaryState", - "timing": "query_time" - }, - "grouping": "NotApplicable", - "intermediate_schema": { - "closed": true, - "fields": [ - { - "dtype": { - "Plain": "timestamp" - }, - "name": "ts", - "nullable": false, - "table": null - }, - { - "dtype": { - "Sketch": [ - { - "algorithm": "DDSketch", - "params": { - "DDSketch": { - "alpha": 0.05 - } - } - }, - "PerSubpopulationInstance" - ] - }, - "name": "state", - "nullable": false, - "table": null - }, - { - "dtype": { - "Plain": "utf8" - }, - "name": "region", - "nullable": true, - "table": null - }, - { - "dtype": { - "Plain": "utf8" - }, - "name": "$promql_series_identity", - "nullable": false, - "table": null - } - ], - "time_index": 0, - "unique_keys": [ - [ - 3, - 0 - ] - ] - }, - "producer": 2, - "role": "Input", - "window": "NotApplicable" - }, - { - "consumer": 4, - "data_state": { - "primitive": "Raw", - "timing": "query_time" - }, - "grouping": "NotApplicable", - "intermediate_schema": { - "closed": true, - "fields": [ - { - "dtype": { - "Plain": "timestamp" - }, - "name": "ts", - "nullable": false, - "table": null - }, - { - "dtype": { - "Plain": "float64" - }, - "name": "quantile", - "nullable": false, - "table": null - }, - { - "dtype": { - "Plain": "utf8" - }, - "name": "region", - "nullable": true, - "table": null - }, - { - "dtype": { - "Plain": "utf8" - }, - "name": "$promql_series_identity", - "nullable": false, - "table": null - } - ], - "time_index": 0, - "unique_keys": [ - [ - 3, - 0 - ] - ] - }, - "producer": 3, - "role": "Input", - "window": "NotApplicable" - }, - { - "consumer": 5, - "data_state": { - "primitive": "Raw", - "timing": "query_time" - }, - "grouping": "NotApplicable", - "intermediate_schema": { - "closed": true, - "fields": [ - { - "dtype": { - "Plain": "timestamp" - }, - "name": "ts", - "nullable": false, - "table": null - }, - { - "dtype": { - "Plain": "float64" - }, - "name": "value", - "nullable": false, - "table": null - }, - { - "dtype": { - "Plain": "utf8" - }, - "name": "region", - "nullable": true, - "table": null - }, - { - "dtype": { - "Plain": "utf8" - }, - "name": "$promql_series_identity", - "nullable": false, - "table": null - } - ], - "time_index": 0, - "unique_keys": [ - [ - 3, - 0 - ] - ] - }, - "producer": 4, - "role": "Input", - "window": "NotApplicable" - }, - { - "consumer": 6, - "data_state": { - "primitive": "SummaryState", - "timing": "query_time" - }, - "grouping": "NotApplicable", - "intermediate_schema": { - "closed": true, - "fields": [ - { - "dtype": { - "Plain": "utf8" - }, - "name": "region", - "nullable": true, - "table": null - }, - { - "dtype": { - "ExactAggregate": [ - "Sum", - "Sum" - ] - }, - "name": "state", - "nullable": false, - "table": null - } - ], - "time_index": null, - "unique_keys": [ - [ - 0 - ] - ] - }, - "producer": 5, - "role": "Input", - "window": "NotApplicable" + "retention_ms": 300000 + } + ], + "queries": [ + "avg by (region) (quantile_over_time(0.9, data[5m]))" + ], + "selected": { + "accepted": [ + { + "accepted_metric": "relative", + "sketches": [ + "DDSketch" + ], + "summary": "accepted relative (DDSketch)" + } + ], + "cost": { + "cost_per_second": 1.90265409966e-6, + "cpu_core_seconds_per_second": 0.111971, + "per_node": { + "0": { + "cost_per_second": 6.52152e-9, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.00058, + "detail": "scan 100000 samples; x 1.0000/s", + "multiplier": 1, + "operator": "Scan", + "timing": "ingestion_time" }, - { - "consumer": 7, - "data_state": { - "primitive": "Raw", - "timing": "query_time" - }, - "grouping": "NotApplicable", - "intermediate_schema": { - "closed": true, - "fields": [ - { - "dtype": { - "Plain": "timestamp" - }, - "name": "ts", - "nullable": false, - "table": null - }, - { - "dtype": { - "Plain": "float64" - }, - "name": "value", - "nullable": false, - "table": null - }, - { - "dtype": { - "Plain": "utf8" - }, - "name": "region", - "nullable": true, - "table": null - }, - { - "dtype": { - "Plain": "utf8" - }, - "name": "$promql_series_identity", - "nullable": false, - "table": null - } - ], - "time_index": 0, - "unique_keys": [ - [ - 3, - 0 - ] - ] - }, - "producer": 4, - "role": "Input", - "window": "NotApplicable" + "1": { + "cost_per_second": 9.65292205845e-7, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.02877, + "detail": "build DDSketch in tumbling windows of 10000 ms for 10000 groups: 100000 rows x 247 ops, 1000 group visits, 1000.0 states emitted; x 1.0000/s; keeps 558000000 bytes", + "multiplier": 1, + "operator": "SummaryTumblingWindowChain", + "timing": "ingestion_time" }, - { - "consumer": 8, - "data_state": { - "primitive": "SummaryState", - "timing": "query_time" - }, - "grouping": "NotApplicable", - "intermediate_schema": { - "closed": true, - "fields": [ - { - "dtype": { - "Plain": "utf8" - }, - "name": "region", - "nullable": true, - "table": null - }, - { - "dtype": { - "ExactAggregate": [ - "Count", - "Count" - ] - }, - "name": "state", - "nullable": false, - "table": null - } - ], - "time_index": null, - "unique_keys": [ - [ - 0 - ] - ] - }, - "producer": 7, - "role": "Input", - "window": "NotApplicable" + "2": { + "cost_per_second": 3.71052068321e-9, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.00033, + "detail": "time range 300s: 30 tumbling windows of states; x 0.1000/s; holds 1800 bytes for 3.300 ms", + "multiplier": 1, + "operator": "TimeRange", + "timing": "query_time" }, - { - "consumer": 9, - "data_state": { - "primitive": "Raw", - "timing": "query_time" - }, - "grouping": "NotApplicable", - "intermediate_schema": { - "closed": true, - "fields": [ - { - "dtype": { - "Plain": "utf8" - }, - "name": "region", - "nullable": true, - "table": null - }, - { - "dtype": { - "Plain": "int64" - }, - "name": "count", - "nullable": false, - "table": null - } - ], - "time_index": null, - "unique_keys": [ - [ - 0 - ] - ] - }, - "producer": 8, - "role": "Input", - "window": "NotApplicable" + "3": { + "cost_per_second": 8.57517011534e-7, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.0761, + "detail": "merge 300000 states into 10000; x 0.1000/s; holds 21120000 bytes for 761.000 ms", + "multiplier": 1, + "operator": "SummaryMerge", + "timing": "query_time" + }, + "4": { + "cost_per_second": 3.9354e-8, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.0035, + "detail": "estimate 10000 groups x 1; x 0.1000/s", + "multiplier": 1, + "operator": "SummaryEstimate", + "timing": "query_time" }, + "5": { + "cost_per_second": 1.12440000552e-11, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 1e-6, + "detail": "operator: pass 10000 rows; x 0.1000/s; holds 48 bytes for 0.010 ms", + "multiplier": 1, + "operator": "Project", + "timing": "query_time" + }, + "6": { + "cost_per_second": 3.02475975973e-8, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.00269, + "detail": "hash aggregate 10000 rows into 10000 groups: 20 ops per row, 10000 group visits; x 0.1000/s; holds 400000 bytes for 26.900 ms", + "multiplier": 1, + "operator": "Aggregate", + "timing": "query_time" + } + }, + "source": "analytical-cost-v2 (illustrative statistics)", + "usage": { + "average_memory_bytes": 559608308.594, + "cpu_ops_per_second": 111491000.0, + "disk_bytes": 0, + "peak_cpu_cores": 2, + "peak_memory_bytes": 579521896, + "scanned_bytes_per_second": 4800000.0 + } + }, + "dag": { + "edges": [ { - "consumer": 10, + "consumer": 1, "data_state": { "primitive": "Raw", - "timing": "query_time" + "timing": "ingestion_time" }, "grouping": "NotApplicable", "intermediate_schema": { @@ -1774,337 +123,157 @@ "fields": [ { "dtype": { - "Plain": "utf8" + "Plain": "timestamp" }, - "name": "region", - "nullable": true, - "table": null + "name": "ts", + "nullable": false }, { "dtype": { "Plain": "float64" }, - "name": "sum", - "nullable": false, - "table": null - } - ], - "time_index": null, - "unique_keys": [ - [ - 0 - ] - ] - }, - "producer": 6, - "role": "Left", - "window": "NotApplicable" - }, - { - "consumer": 10, - "data_state": { - "primitive": "Raw", - "timing": "query_time" - }, - "grouping": "NotApplicable", - "intermediate_schema": { - "closed": true, - "fields": [ + "name": "value", + "nullable": false + }, { "dtype": { "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { - "Plain": "float64" + "Plain": "utf8" }, - "name": "col_1", - "nullable": false, - "table": null + "name": "$promql_series_identity", + "nullable": false } ], - "time_index": null, + "time_index": 0, "unique_keys": [ [ + 3, 0 ] ] }, - "producer": 9, - "role": "Right", + "producer": 0, + "role": "Input", "window": "NotApplicable" }, { - "consumer": 11, + "consumer": 2, "data_state": { - "primitive": "Raw", - "timing": "query_time" + "primitive": "SummaryState", + "timing": "ingestion_time" }, "grouping": "NotApplicable", "intermediate_schema": { - "closed": true, - "fields": [ - { - "dtype": { - "Plain": "utf8" - }, - "name": "region", - "nullable": true, - "table": null - }, - { - "dtype": { - "Plain": "float64" - }, - "name": "sum", - "nullable": false, - "table": null - } - ], - "time_index": null, - "unique_keys": [ - [ - 0 - ] - ] - }, - "producer": 10, - "role": "Input", - "window": "NotApplicable" - } - ], - "nodes": [ - { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact over exact inputs" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, - "id": 0, - "lifetime": "None", - "output_schema": { "closed": true, "fields": [ { "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { - "Plain": "float64" + "Sketch": "DDSketch" }, - "name": "value", - "nullable": false, - "table": null + "name": "state", + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 3, - 0 + 0, + 2, + 3 ] ] }, - "output_state": { - "primitive": "Raw", - "timing": "ingestion_time" - }, - "payload": { - "NonASAP": { - "Scan": { - "predicates": [], - "schema": { - "closed": true, - "fields": [ - { - "dtype": { - "Plain": "timestamp" - }, - "name": "ts", - "nullable": false, - "table": null - }, - { - "dtype": { - "Plain": "float64" - }, - "name": "value", - "nullable": false, - "table": null - }, - { - "dtype": { - "Plain": "utf8" - }, - "name": "region", - "nullable": true, - "table": null - }, - { - "dtype": { - "Plain": "utf8" - }, - "name": "$promql_series_identity", - "nullable": false, - "table": null - } - ], - "time_index": 0, - "unique_keys": [ - [ - 3, - 0 - ] - ] - }, - "source": { - "TimeSeries": { - "metric": "data" - } - } - } - } - } + "producer": 1, + "role": "Input", + "window": "NotApplicable" }, { - "guarantee": null, - "id": 1, - "lifetime": { - "AcrossEvaluations": "Memory" + "consumer": 3, + "data_state": { + "primitive": "SummaryState", + "timing": "query_time" }, - "output_schema": { + "grouping": "NotApplicable", + "intermediate_schema": { "closed": true, "fields": [ { "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { - "Sketch": [ - { - "algorithm": "DDSketch", - "params": { - "DDSketch": { - "alpha": 0.05 - } - } - }, - "PerSubpopulationInstance" - ] + "Sketch": "DDSketch" }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 3, - 0 + 0, + 2, + 3 ] ] }, - "output_state": { - "primitive": "SummaryState", - "timing": "ingestion_time" - }, - "payload": { - "ASAP": { - "SummaryTumblingWindowChain": { - "child": 0, - "family": { - "Sketch": [ - { - "algorithm": "DDSketch", - "params": { - "DDSketch": { - "alpha": 0.05 - } - } - }, - "PerSubpopulationInstance" - ] - }, - "filter": null, - "grouping": "PerSubpopulationInstance", - "input": { - "item": null, - "weight": { - "Column": "SampleValue" - } - }, - "reduction": "PerEntity", - "tumbling_window_count": 30, - "tumbling_window_width_ms": 10000 - } - } - } + "producer": 2, + "role": "Input", + "window": "NotApplicable" }, { - "guarantee": null, - "id": 2, - "lifetime": "None", - "output_schema": { + "consumer": 4, + "data_state": { + "primitive": "SummaryState", + "timing": "query_time" + }, + "grouping": "NotApplicable", + "intermediate_schema": { "closed": true, "fields": [ { @@ -2112,42 +281,28 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { - "Sketch": [ - { - "algorithm": "DDSketch", - "params": { - "DDSketch": { - "alpha": 0.05 - } - } - }, - "PerSubpopulationInstance" - ] + "Sketch": "DDSketch" }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -2158,59 +313,18 @@ ] ] }, - "output_state": { - "primitive": "SummaryState", - "timing": "query_time" - }, - "payload": { - "ASAP": { - "SummaryMerge": { - "children": [ - 1 - ] - } - } - } + "producer": 3, + "role": "Input", + "window": "NotApplicable" }, { - "guarantee": { - "provenance": [ - { - "algorithm": "DDSketch", - "contract": "ddsketch_relative_error_alpha_v1", - "kind": "sketch_evaluation", - "params": { - "DDSketch": { - "alpha": 0.05 - } - }, - "query": "Quantile { q: 0.9 }" - } - ], - "spec": { - "accuracy_metric": "relative", - "accuracy_scope": "per_value", - "error": { - "factors": [ - { - "op": "constant", - "value": 0.05 - }, - { - "op": "unknown", - "statistic": "values_non_negative[data]" - } - ], - "op": "product" - }, - "failure_probability": { - "op": "zero" - } - } + "consumer": 5, + "data_state": { + "primitive": "Raw", + "timing": "query_time" }, - "id": 3, - "lifetime": "None", - "output_schema": { + "grouping": "NotApplicable", + "intermediate_schema": { "closed": true, "fields": [ { @@ -2218,32 +332,28 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "quantile", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -2254,62 +364,18 @@ ] ] }, - "output_state": { - "primitive": "Raw", - "timing": "query_time" - }, - "payload": { - "ASAP": { - "SummaryEstimate": { - "query": { - "Quantile": { - "q": 0.9 - } - }, - "summary_input": 2 - } - } - } + "producer": 4, + "role": "Input", + "window": "NotApplicable" }, { - "guarantee": { - "provenance": [ - { - "algorithm": "DDSketch", - "contract": "ddsketch_relative_error_alpha_v1", - "kind": "sketch_evaluation", - "params": { - "DDSketch": { - "alpha": 0.05 - } - }, - "query": "Quantile { q: 0.9 }" - } - ], - "spec": { - "accuracy_metric": "relative", - "accuracy_scope": "per_value", - "error": { - "factors": [ - { - "op": "constant", - "value": 0.05 - }, - { - "op": "unknown", - "statistic": "values_non_negative[data]" - } - ], - "op": "product" - }, - "failure_probability": { - "op": "zero" - } - } + "consumer": 6, + "data_state": { + "primitive": "Raw", + "timing": "query_time" }, - "id": 4, - "lifetime": "None", - "output_schema": { + "grouping": "NotApplicable", + "intermediate_schema": { "closed": true, "fields": [ { @@ -2317,32 +383,28 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -2353,291 +415,237 @@ ] ] }, - "output_state": { - "primitive": "Raw", - "timing": "query_time" - }, - "payload": { - "NonASAP": { - "Project": { - "child": 3, - "cols": [ - { - "alias": "ts", - "expr": { - "Column": 0 - } - }, - { - "alias": "value", - "expr": { - "Column": 1 - } - }, - { - "alias": "region", - "expr": { - "Column": 2 - } - }, - { - "alias": "$promql_series_identity", - "expr": { - "Column": 3 - } - } - ], - "qualifier": null - } - } - } - }, + "producer": 5, + "role": "Input", + "window": "NotApplicable" + } + ], + "nodes": [ { - "guarantee": null, - "id": 5, + "id": 0, "lifetime": "None", "output_schema": { "closed": true, "fields": [ + { + "dtype": { + "Plain": "timestamp" + }, + "name": "ts", + "nullable": false + }, + { + "dtype": { + "Plain": "float64" + }, + "name": "value", + "nullable": false + }, { "dtype": { "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { - "ExactAggregate": [ - "Sum", - "Sum" - ] + "Plain": "utf8" }, - "name": "state", - "nullable": false, - "table": null + "name": "$promql_series_identity", + "nullable": false } ], - "time_index": null, + "time_index": 0, "unique_keys": [ [ + 3, 0 ] ] }, "output_state": { - "primitive": "SummaryState", - "timing": "query_time" + "primitive": "Raw", + "timing": "ingestion_time" }, "payload": { - "ASAP": { - "SummaryAgg": { - "child": 4, - "family": { - "ExactAggregate": [ - "Sum", - "Sum" + "NonASAP": { + "Scan": { + "predicates": [], + "schema": { + "closed": true, + "fields": [ + { + "dtype": { + "Plain": "timestamp" + }, + "name": "ts", + "nullable": false + }, + { + "dtype": { + "Plain": "float64" + }, + "name": "value", + "nullable": false + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "region", + "nullable": true + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false + } + ], + "time_index": 0, + "unique_keys": [ + [ + 3, + 0 + ] ] }, - "filter": null, - "grouping": "PerSubpopulationInstance", - "input": { - "item": null, - "weight": { - "Column": "SampleValue" + "source": { + "TimeSeries": { + "metric": "data" } - }, - "reduction": { - "Reduce": [ - 2 - ] } } } } }, { - "guarantee": { - "provenance": [ - { - "kind": "unavailable_statistic", - "statistic": "input_row_count" - }, - { - "kind": "unavailable_statistic", - "statistic": "values_non_negative[data]" - }, + "id": 1, + "lifetime": { + "AcrossEvaluations": "Memory" + }, + "output_schema": { + "closed": true, + "fields": [ { - "guarantee": { - "provenance": [ - { - "algorithm": "DDSketch", - "contract": "ddsketch_relative_error_alpha_v1", - "kind": "sketch_evaluation", - "params": { - "DDSketch": { - "alpha": 0.05 - } - }, - "query": "Quantile { q: 0.9 }" - } - ], - "spec": { - "accuracy_metric": "relative", - "accuracy_scope": "per_value", - "error": { - "factors": [ - { - "op": "constant", - "value": 0.05 - }, - { - "op": "unknown", - "statistic": "values_non_negative[data]" - } - ], - "op": "product" - }, - "failure_probability": { - "op": "zero" - } - } + "dtype": { + "Plain": "timestamp" }, - "input_index": 0, - "kind": "child_guarantee" - }, - { - "kind": "exact", - "reason": "exact aggregation" + "name": "window_end", + "nullable": false }, { - "kind": "composition_step", - "operator": { - "op": "exact_sum" + "dtype": { + "Sketch": "DDSketch" }, - "rule": "exact_sum_relative" - } - ], - "spec": { - "accuracy_metric": "relative", - "accuracy_scope": "per_value", - "error": { - "factors": [ - { - "op": "max", - "terms": [ - { - "factors": [ - { - "op": "constant", - "value": 0.05 - }, - { - "op": "unknown", - "statistic": "values_non_negative[data]" - } - ], - "op": "product" - } - ] - }, - { - "op": "unknown", - "statistic": "values_non_negative[data]" - } - ], - "op": "product" + "name": "state", + "nullable": false }, - "failure_probability": { - "count": { - "op": "unknown", - "statistic": "input_row_count" - }, - "inner": { - "op": "union_bound", - "terms": [ - { - "op": "zero" - } - ] - }, - "op": "scaled" - } - } - }, - "id": 6, - "lifetime": "None", - "output_schema": { - "closed": true, - "fields": [ { "dtype": { "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { - "Plain": "float64" + "Plain": "utf8" }, - "name": "sum", - "nullable": false, - "table": null + "name": "$promql_series_identity", + "nullable": false } ], - "time_index": null, + "time_index": 0, "unique_keys": [ [ - 0 + 0, + 2, + 3 ] ] }, "output_state": { - "primitive": "Raw", - "timing": "query_time" + "primitive": "SummaryState", + "timing": "ingestion_time" + }, + "parameters": { + "DDSketch": { + "alpha": 0.05 + } }, "payload": { "ASAP": { - "FinalizeExactAccumulator": { - "child": 5 + "SummaryTumblingWindowChain": { + "child": 0, + "filter": null, + "input": { + "item": null, + "weight": { + "Column": "SampleValue" + } + }, + "reduction": "PerEntity", + "summary_type": { + "Sketch": "DDSketch" + }, + "tumbling_window_width_ms": 10000 } } + }, + "retention_ms": 300000, + "view_coverage": { + "definition_id": "0c7f962d8fde713c1de7a58752019151", + "selection": [ + { + "columns": [] + } + ] } }, { - "guarantee": null, - "id": 7, + "id": 2, "lifetime": "None", "output_schema": { "closed": true, "fields": [ + { + "dtype": { + "Plain": "timestamp" + }, + "name": "window_end", + "nullable": false + }, + { + "dtype": { + "Sketch": "DDSketch" + }, + "name": "state", + "nullable": false + }, { "dtype": { "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { - "ExactAggregate": [ - "Count", - "Count" - ] + "Plain": "utf8" }, - "name": "state", - "nullable": false, - "table": null + "name": "$promql_series_identity", + "nullable": false } ], - "time_index": null, + "time_index": 0, "unique_keys": [ [ - 0 + 0, + 2, + 3 ] ] }, @@ -2645,137 +653,127 @@ "primitive": "SummaryState", "timing": "query_time" }, + "parameters": { + "DDSketch": { + "alpha": 0.05 + } + }, "payload": { - "ASAP": { - "SummaryAgg": { - "child": 4, - "family": { - "ExactAggregate": [ - "Count", - "Count" - ] - }, - "filter": null, - "grouping": "PerSubpopulationInstance", - "input": { - "item": null, - "weight": { - "Column": "SampleValue" - } - }, - "reduction": { - "Reduce": [ - 2 - ] + "NonASAP": { + "TimeRange": { + "child": 1, + "kind": "range", + "range": { + "nanos": 0, + "secs": 300 } } } } }, { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "count of rows" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } + "id": 3, + "lifetime": { + "OneEvaluation": "Memory" }, - "id": 8, - "lifetime": "None", "output_schema": { "closed": true, "fields": [ + { + "dtype": { + "Plain": "timestamp" + }, + "name": "ts", + "nullable": false + }, + { + "dtype": { + "Sketch": "DDSketch" + }, + "name": "state", + "nullable": false + }, { "dtype": { "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { - "Plain": "int64" + "Plain": "utf8" }, - "name": "count", - "nullable": false, - "table": null + "name": "$promql_series_identity", + "nullable": false } ], - "time_index": null, + "time_index": 0, "unique_keys": [ [ + 3, 0 ] ] }, "output_state": { - "primitive": "Raw", + "primitive": "SummaryState", "timing": "query_time" }, + "parameters": { + "DDSketch": { + "alpha": 0.05 + } + }, "payload": { "ASAP": { - "FinalizeExactAccumulator": { - "child": 7 + "SummaryMerge": { + "children": [ + 2 + ] } } } }, { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact over exact inputs" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, - "id": 9, + "id": 4, "lifetime": "None", "output_schema": { "closed": true, "fields": [ + { + "dtype": { + "Plain": "timestamp" + }, + "name": "ts", + "nullable": false + }, + { + "dtype": { + "Plain": "float64" + }, + "name": "quantile", + "nullable": false + }, { "dtype": { "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { - "Plain": "float64" + "Plain": "utf8" }, - "name": "col_1", - "nullable": false, - "table": null + "name": "$promql_series_identity", + "nullable": false } ], - "time_index": null, + "time_index": 0, "unique_keys": [ [ + 3, 0 ] ] @@ -2785,168 +783,57 @@ "timing": "query_time" }, "payload": { - "NonASAP": { - "Project": { - "child": 8, - "cols": [ - { - "alias": null, - "expr": { - "Column": 0 - } - }, - { - "alias": null, - "expr": { - "Cast": { - "expr": { - "Column": 1 - }, - "to": "float64", - "try_cast": false - } - } + "ASAP": { + "SummaryEstimate": { + "query": { + "Quantile": { + "q": 0.9 } - ], - "qualifier": null + }, + "summary_input": 3 } } } }, { - "guarantee": { - "provenance": [ - { - "kind": "unavailable_statistic", - "statistic": "input_row_count" - }, - { - "kind": "unavailable_statistic", - "statistic": "values_non_negative[data]" - }, + "id": 5, + "lifetime": "None", + "output_schema": { + "closed": true, + "fields": [ { - "guarantee": { - "provenance": [ - { - "algorithm": "DDSketch", - "contract": "ddsketch_relative_error_alpha_v1", - "kind": "sketch_evaluation", - "params": { - "DDSketch": { - "alpha": 0.05 - } - }, - "query": "Quantile { q: 0.9 }" - } - ], - "spec": { - "accuracy_metric": "relative", - "accuracy_scope": "per_value", - "error": { - "factors": [ - { - "op": "constant", - "value": 0.05 - }, - { - "op": "unknown", - "statistic": "values_non_negative[data]" - } - ], - "op": "product" - }, - "failure_probability": { - "op": "zero" - } - } + "dtype": { + "Plain": "timestamp" }, - "input_index": 0, - "kind": "child_guarantee" - }, - { - "kind": "exact", - "reason": "exact aggregation" + "name": "ts", + "nullable": false }, { - "kind": "composition_step", - "operator": { - "op": "exact_sum" + "dtype": { + "Plain": "float64" }, - "rule": "exact_sum_relative" - } - ], - "spec": { - "accuracy_metric": "relative", - "accuracy_scope": "per_value", - "error": { - "factors": [ - { - "op": "max", - "terms": [ - { - "factors": [ - { - "op": "constant", - "value": 0.05 - }, - { - "op": "unknown", - "statistic": "values_non_negative[data]" - } - ], - "op": "product" - } - ] - }, - { - "op": "unknown", - "statistic": "values_non_negative[data]" - } - ], - "op": "product" + "name": "value", + "nullable": false }, - "failure_probability": { - "count": { - "op": "unknown", - "statistic": "input_row_count" - }, - "inner": { - "op": "union_bound", - "terms": [ - { - "op": "zero" - } - ] - }, - "op": "scaled" - } - } - }, - "id": 10, - "lifetime": "None", - "output_schema": { - "closed": true, - "fields": [ { "dtype": { "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { - "Plain": "float64" + "Plain": "utf8" }, - "name": "sum", - "nullable": false, - "table": null + "name": "$promql_series_identity", + "nullable": false } ], - "time_index": null, + "time_index": 0, "unique_keys": [ [ + 3, 0 ] ] @@ -2957,133 +844,43 @@ }, "payload": { "NonASAP": { - "BinaryOp": { - "lhs": 6, - "operator": { - "checked_finite_division": false, - "checked_relative_division": false, - "kind": { - "Arithmetic": "Div" - }, - "vector_match": null - }, - "return_bool": false, - "rhs": 9 - } - } - } - }, - { - "guarantee": { - "provenance": [ - { - "kind": "unavailable_statistic", - "statistic": "input_row_count" - }, - { - "kind": "unavailable_statistic", - "statistic": "values_non_negative[data]" - }, - { - "guarantee": { - "provenance": [ - { - "algorithm": "DDSketch", - "contract": "ddsketch_relative_error_alpha_v1", - "kind": "sketch_evaluation", - "params": { - "DDSketch": { - "alpha": 0.05 - } - }, - "query": "Quantile { q: 0.9 }" + "Project": { + "child": 4, + "cols": [ + { + "alias": "ts", + "expr": { + "Column": 0 } - ], - "spec": { - "accuracy_metric": "relative", - "accuracy_scope": "per_value", - "error": { - "factors": [ - { - "op": "constant", - "value": 0.05 - }, - { - "op": "unknown", - "statistic": "values_non_negative[data]" - } - ], - "op": "product" - }, - "failure_probability": { - "op": "zero" + }, + { + "alias": "value", + "expr": { + "Column": 1 } - } - }, - "input_index": 0, - "kind": "child_guarantee" - }, - { - "kind": "exact", - "reason": "exact aggregation" - }, - { - "kind": "composition_step", - "operator": { - "op": "exact_sum" - }, - "rule": "exact_sum_relative" - } - ], - "spec": { - "accuracy_metric": "relative", - "accuracy_scope": "per_value", - "error": { - "factors": [ + }, { - "op": "max", - "terms": [ - { - "factors": [ - { - "op": "constant", - "value": 0.05 - }, - { - "op": "unknown", - "statistic": "values_non_negative[data]" - } - ], - "op": "product" - } - ] + "alias": "region", + "expr": { + "Column": 2 + } }, { - "op": "unknown", - "statistic": "values_non_negative[data]" - } - ], - "op": "product" - }, - "failure_probability": { - "count": { - "op": "unknown", - "statistic": "input_row_count" - }, - "inner": { - "op": "union_bound", - "terms": [ - { - "op": "zero" + "alias": "$promql_series_identity", + "expr": { + "Column": 3 } - ] - }, - "op": "scaled" + } + ] } } + } + }, + { + "id": 6, + "lifetime": { + "OneEvaluation": "Memory" }, - "id": 11, - "lifetime": "None", "output_schema": { "closed": true, "fields": [ @@ -3092,16 +889,14 @@ "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "float64" }, "name": "avg", - "nullable": false, - "table": null + "nullable": false } ], "time_index": null, @@ -3117,36 +912,36 @@ }, "payload": { "NonASAP": { - "Project": { - "child": 10, - "cols": [ - { - "alias": "region", - "expr": { - "Column": 0 - } - }, + "Aggregate": { + "child": 5, + "filters": [], + "having": null, + "measures": [ { - "alias": "avg", - "expr": { - "Column": 1 - } + "col": null, + "kind": "avg" } ], - "qualifier": null + "output_names": [ + "" + ], + "reduction": { + "Reduce": [ + 2 + ] + } } } } } ], "roots": [ - 11 + 6 ], "scalar_roots": [] - }, - "display_name": "Q1 Sum acc + DDSketch ×30 tumbling windows + Count acc · ingestion time: DDSketch for Q1, 30 × 10s tumbling windows" + } }, - "tumbling_window_retention_ms": { - "1": 300000 - } + "warnings": [ + "assumed data fact: source data declares no column_magnitudes for value, so a DDSketch state over it is costed for magnitudes in [1, 1000000000]; this affects cost only, the accuracy guarantee does not depend on it" + ] } diff --git a/crates/devtools/tests/sketch_plan_snapshots/quantile-single-mvp.json b/crates/devtools/tests/sketch_plan_snapshots/quantile-single-mvp.json index cd00c9c3c..35201fb9d 100644 --- a/crates/devtools/tests/sketch_plan_snapshots/quantile-single-mvp.json +++ b/crates/devtools/tests/sketch_plan_snapshots/quantile-single-mvp.json @@ -3,1227 +3,100 @@ "materialized_views": [ { "coverage": { - "definition_id": "c71f714cc266c54fc993b110840fdf3d", + "definition_id": "5597d603bdf9f1a148d7c7e5344c1a97", "selection": [ { - "columns": [], - "relative_time": [ - { - "excluded": -10000 - }, - { - "included": 0 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 0, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 0, - "node_id": 2, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c71f714cc266c54fc993b110840fdf3d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -20000 - }, - { - "included": -10000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 10000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 1, - "node_id": 5, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c71f714cc266c54fc993b110840fdf3d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -30000 - }, - { - "included": -20000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 20000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 2, - "node_id": 8, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c71f714cc266c54fc993b110840fdf3d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -40000 - }, - { - "included": -30000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 30000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 3, - "node_id": 11, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c71f714cc266c54fc993b110840fdf3d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -50000 - }, - { - "included": -40000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 40000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 4, - "node_id": 14, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c71f714cc266c54fc993b110840fdf3d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -60000 - }, - { - "included": -50000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 50000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 5, - "node_id": 17, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c71f714cc266c54fc993b110840fdf3d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -70000 - }, - { - "included": -60000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 60000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 6, - "node_id": 20, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c71f714cc266c54fc993b110840fdf3d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -80000 - }, - { - "included": -70000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 70000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 7, - "node_id": 23, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c71f714cc266c54fc993b110840fdf3d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -90000 - }, - { - "included": -80000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 80000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 8, - "node_id": 26, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c71f714cc266c54fc993b110840fdf3d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -100000 - }, - { - "included": -90000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 90000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 9, - "node_id": 29, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c71f714cc266c54fc993b110840fdf3d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -110000 - }, - { - "included": -100000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 100000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 10, - "node_id": 32, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c71f714cc266c54fc993b110840fdf3d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -120000 - }, - { - "included": -110000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 110000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 11, - "node_id": 35, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c71f714cc266c54fc993b110840fdf3d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -130000 - }, - { - "included": -120000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 120000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 12, - "node_id": 38, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c71f714cc266c54fc993b110840fdf3d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -140000 - }, - { - "included": -130000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 130000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 13, - "node_id": 41, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c71f714cc266c54fc993b110840fdf3d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -150000 - }, - { - "included": -140000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 140000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 14, - "node_id": 44, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c71f714cc266c54fc993b110840fdf3d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -160000 - }, - { - "included": -150000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 150000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 15, - "node_id": 47, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c71f714cc266c54fc993b110840fdf3d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -170000 - }, - { - "included": -160000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 160000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 16, - "node_id": 50, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c71f714cc266c54fc993b110840fdf3d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -180000 - }, - { - "included": -170000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 170000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 17, - "node_id": 53, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c71f714cc266c54fc993b110840fdf3d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -190000 - }, - { - "included": -180000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 180000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 18, - "node_id": 56, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c71f714cc266c54fc993b110840fdf3d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -200000 - }, - { - "included": -190000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 190000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 19, - "node_id": 59, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c71f714cc266c54fc993b110840fdf3d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -210000 - }, - { - "included": -200000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 200000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 20, - "node_id": 62, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c71f714cc266c54fc993b110840fdf3d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -220000 - }, - { - "included": -210000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 210000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 21, - "node_id": 65, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c71f714cc266c54fc993b110840fdf3d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -230000 - }, - { - "included": -220000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 220000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 22, - "node_id": 68, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c71f714cc266c54fc993b110840fdf3d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -240000 - }, - { - "included": -230000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 230000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 23, - "node_id": 71, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c71f714cc266c54fc993b110840fdf3d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -250000 - }, - { - "included": -240000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 240000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 24, - "node_id": 74, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c71f714cc266c54fc993b110840fdf3d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -260000 - }, - { - "included": -250000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 250000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 25, - "node_id": 77, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c71f714cc266c54fc993b110840fdf3d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -270000 - }, - { - "included": -260000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 260000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 26, - "node_id": 80, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c71f714cc266c54fc993b110840fdf3d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -280000 - }, - { - "included": -270000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 270000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 27, - "node_id": 83, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c71f714cc266c54fc993b110840fdf3d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -290000 - }, - { - "included": -280000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 280000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 28, - "node_id": 86, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "c71f714cc266c54fc993b110840fdf3d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -300000 - }, - { - "included": -290000 - } - ] + "columns": [] } ] }, "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 290000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 10000 - } - ] + "materialized_views": [] }, - "installed_count": 30, - "installed_position": 29, - "node_id": 89, + "node_id": 1, "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 + "retention_ms": 300000 } ], - "proven_optimal": true, "queries": [ "quantile_over_time(0.99, data[5m])" ], - "search_status": "optimal", "selected": { "accepted": [ { - "accepted_metric": "rank", - "accepted_regardless_of_guarantee": false, - "guarantee_check_rejection": null, + "accepted_metric": "relative", "sketches": [ - "Kll" + "DDSketch" ], - "summary": "metric not set; accepted rank (Kll)", - "target_metric": null + "summary": "accepted relative (DDSketch)" } ], "cost": { - "cost_per_second": 6.37724245448e-7, - "cost_unit": "cost_per_second", - "cpu_core_seconds_per_second": 0.000652, + "cost_per_second": 1.82328574786e-6, + "cpu_core_seconds_per_second": 0.105421, "per_node": { "0": { "cost_per_second": 4.72248e-9, + "cost_source": "analytical", "cpu_core_seconds_per_second": 0.00042, "detail": "scan 100000 samples; x 1.0000/s", "multiplier": 1, "operator": "Scan", - "price_source": "analytical", "timing": "ingestion_time" }, "1": { - "cost_per_second": 6.32641306719e-7, - "cpu_core_seconds_per_second": 0.0002, - "detail": "build Kll in 30 tumbling windows of 10000 ms for 10000 groups: 100000 rows x (1 + depth 1); x 1.0000/s; keeps 548080000 bytes", + "cost_per_second": 9.17984496282e-7, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.02507, + "detail": "build DDSketch in tumbling windows of 10000 ms for 10000 groups: 100000 rows x 210 ops, 1000 group visits, 1000.0 states emitted; x 1.0000/s; keeps 553040000 bytes", "multiplier": 1, "operator": "SummaryTumblingWindowChain", - "price_source": "analytical", "timing": "ingestion_time" }, "2": { - "cost_per_second": 3.37970727749e-10, - "cpu_core_seconds_per_second": 0.00003, - "detail": "merge 300000 states into 10000; x 0.1000/s; holds 17680000 bytes for 0.3 ms", + "cost_per_second": 3.71052067714e-9, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.00033, + "detail": "time range 300s: 30 tumbling windows of states; x 0.1000/s; holds 1784 bytes for 3.300 ms", "multiplier": 1, - "operator": "SummaryMerge", - "price_source": "analytical", + "operator": "TimeRange", "timing": "query_time" }, "3": { - "cost_per_second": 1.1244e-11, - "cpu_core_seconds_per_second": 1e-6, + "cost_per_second": 8.57503006901e-7, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.0761, + "detail": "merge 300000 states into 10000; x 0.1000/s; holds 20960000 bytes for 761.000 ms", + "multiplier": 1, + "operator": "SummaryMerge", + "timing": "query_time" + }, + "4": { + "cost_per_second": 3.9354e-8, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.0035, "detail": "estimate 10000 groups x 1; x 0.1000/s", "multiplier": 1, "operator": "SummaryEstimate", - "price_source": "analytical", "timing": "query_time" }, - "4": { - "cost_per_second": 1.12440011778e-11, + "5": { + "cost_per_second": 1.12440000368e-11, + "cost_source": "analytical", "cpu_core_seconds_per_second": 1e-6, - "detail": "operator: pass 10000 rows; x 0.1000/s; holds 32 bytes for 0.3 ms", + "detail": "operator: pass 10000 rows; x 0.1000/s; holds 32 bytes for 0.010 ms", "multiplier": 1, "operator": "Project", - "price_source": "analytical", "timing": "query_time" } }, "source": "analytical-cost-v2 (illustrative statistics)", "usage": { - "average_memory_bytes": 548080565.761, - "cpu_ops_per_second": 332000.0, + "average_memory_bytes": 554635056.589, + "cpu_ops_per_second": 105101000.0, "disk_bytes": 0, - "latency_per_query": [ - { - "compaction_ms": 0.0, - "query_ms": 0.32 - } - ], "peak_cpu_cores": 2, - "peak_memory_bytes": 565760064, + "peak_memory_bytes": 574001848, "scanned_bytes_per_second": 3200000.0 } }, @@ -1244,24 +117,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -1274,7 +144,7 @@ }, "producer": 0, "role": "Input", - "window": "RequiresAlignedPanePhaseOrExactBoundaryResidual" + "window": "NotApplicable" }, { "consumer": 2, @@ -1290,43 +160,29 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { - "Sketch": [ - { - "algorithm": "Kll", - "params": { - "Kll": { - "failure_probability": 0.00003, - "k": 86 - } - } - }, - "PerSubpopulationInstance" - ] + "Sketch": "DDSketch" }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 2, - 0 + 0, + 2 ] ] }, @@ -1341,6 +197,50 @@ "timing": "query_time" }, "grouping": "NotApplicable", + "intermediate_schema": { + "closed": true, + "fields": [ + { + "dtype": { + "Plain": "timestamp" + }, + "name": "window_end", + "nullable": false + }, + { + "dtype": { + "Sketch": "DDSketch" + }, + "name": "state", + "nullable": false + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false + } + ], + "time_index": 0, + "unique_keys": [ + [ + 0, + 2 + ] + ] + }, + "producer": 2, + "role": "Input", + "window": "NotApplicable" + }, + { + "consumer": 4, + "data_state": { + "primitive": "SummaryState", + "timing": "query_time" + }, + "grouping": "NotApplicable", "intermediate_schema": { "closed": true, "fields": [ @@ -1349,35 +249,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { - "Sketch": [ - { - "algorithm": "Kll", - "params": { - "Kll": { - "failure_probability": 0.00003, - "k": 86 - } - } - }, - "PerSubpopulationInstance" - ] + "Sketch": "DDSketch" }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -1388,12 +274,12 @@ ] ] }, - "producer": 2, + "producer": 3, "role": "Input", "window": "NotApplicable" }, { - "consumer": 4, + "consumer": 5, "data_state": { "primitive": "Raw", "timing": "query_time" @@ -1407,24 +293,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "quantile", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -1435,31 +318,13 @@ ] ] }, - "producer": 3, + "producer": 4, "role": "Input", "window": "NotApplicable" } ], "nodes": [ { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact over exact inputs" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 0, "lifetime": "None", "output_schema": { @@ -1470,24 +335,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -1514,24 +376,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -1552,7 +411,6 @@ } }, { - "guarantee": null, "id": 1, "lifetime": { "AcrossEvaluations": "Memory" @@ -1564,43 +422,29 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { - "Sketch": [ - { - "algorithm": "Kll", - "params": { - "Kll": { - "failure_probability": 0.00003, - "k": 86 - } - } - }, - "PerSubpopulationInstance" - ] + "Sketch": "DDSketch" }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 2, - 0 + 0, + 2 ] ] }, @@ -1608,26 +452,16 @@ "primitive": "SummaryState", "timing": "ingestion_time" }, + "parameters": { + "DDSketch": { + "alpha": 0.05 + } + }, "payload": { "ASAP": { "SummaryTumblingWindowChain": { "child": 0, - "family": { - "Sketch": [ - { - "algorithm": "Kll", - "params": { - "Kll": { - "failure_probability": 0.00003, - "k": 86 - } - } - }, - "PerSubpopulationInstance" - ] - }, "filter": null, - "grouping": "PerSubpopulationInstance", "input": { "item": null, "weight": { @@ -1635,16 +469,86 @@ } }, "reduction": "PerEntity", - "tumbling_window_count": 30, + "summary_type": { + "Sketch": "DDSketch" + }, "tumbling_window_width_ms": 10000 } } + }, + "retention_ms": 300000, + "view_coverage": { + "definition_id": "5597d603bdf9f1a148d7c7e5344c1a97", + "selection": [ + { + "columns": [] + } + ] } }, { - "guarantee": null, "id": 2, "lifetime": "None", + "output_schema": { + "closed": true, + "fields": [ + { + "dtype": { + "Plain": "timestamp" + }, + "name": "window_end", + "nullable": false + }, + { + "dtype": { + "Sketch": "DDSketch" + }, + "name": "state", + "nullable": false + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false + } + ], + "time_index": 0, + "unique_keys": [ + [ + 0, + 2 + ] + ] + }, + "output_state": { + "primitive": "SummaryState", + "timing": "query_time" + }, + "parameters": { + "DDSketch": { + "alpha": 0.05 + } + }, + "payload": { + "NonASAP": { + "TimeRange": { + "child": 1, + "kind": "range", + "range": { + "nanos": 0, + "secs": 300 + } + } + } + } + }, + { + "id": 3, + "lifetime": { + "OneEvaluation": "Memory" + }, "output_schema": { "closed": true, "fields": [ @@ -1653,35 +557,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { - "Sketch": [ - { - "algorithm": "Kll", - "params": { - "Kll": { - "failure_probability": 0.00003, - "k": 86 - } - } - }, - "PerSubpopulationInstance" - ] + "Sketch": "DDSketch" }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -1696,46 +586,23 @@ "primitive": "SummaryState", "timing": "query_time" }, + "parameters": { + "DDSketch": { + "alpha": 0.05 + } + }, "payload": { "ASAP": { "SummaryMerge": { "children": [ - 1 + 2 ] } } } }, { - "guarantee": { - "provenance": [ - { - "algorithm": "Kll", - "contract": "apache_datasketches_kll_empirical_99_delta_scaled_a9b42755072b", - "kind": "sketch_evaluation", - "params": { - "Kll": { - "failure_probability": 0.00003, - "k": 86 - } - }, - "query": "Quantile { q: 0.99 }" - } - ], - "spec": { - "accuracy_metric": "rank", - "accuracy_scope": "per_value", - "error": { - "op": "constant", - "value": 0.0489371726995 - }, - "failure_probability": { - "op": "constant", - "value": 0.00003 - } - } - }, - "id": 3, + "id": 4, "lifetime": "None", "output_schema": { "closed": true, @@ -1745,24 +612,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "quantile", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -1785,41 +649,13 @@ "q": 0.99 } }, - "summary_input": 2 + "summary_input": 3 } } } }, { - "guarantee": { - "provenance": [ - { - "algorithm": "Kll", - "contract": "apache_datasketches_kll_empirical_99_delta_scaled_a9b42755072b", - "kind": "sketch_evaluation", - "params": { - "Kll": { - "failure_probability": 0.00003, - "k": 86 - } - }, - "query": "Quantile { q: 0.99 }" - } - ], - "spec": { - "accuracy_metric": "rank", - "accuracy_scope": "per_value", - "error": { - "op": "constant", - "value": 0.0489371726995 - }, - "failure_probability": { - "op": "constant", - "value": 0.00003 - } - } - }, - "id": 4, + "id": 5, "lifetime": "None", "output_schema": { "closed": true, @@ -1829,24 +665,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -1864,7 +697,7 @@ "payload": { "NonASAP": { "Project": { - "child": 3, + "child": 4, "cols": [ { "alias": "ts", @@ -1884,21 +717,19 @@ "Column": 2 } } - ], - "qualifier": null + ] } } } } ], "roots": [ - 4 + 5 ], "scalar_roots": [] - }, - "display_name": "Q1 Kll ×30 tumbling windows · ingestion time: Kll for Q1, 30 × 10s tumbling windows" + } }, - "tumbling_window_retention_ms": { - "1": 300000 - } + "warnings": [ + "assumed data fact: source data declares no column_magnitudes for value, so a DDSketch state over it is costed for magnitudes in [1, 1000000000]; this affects cost only, the accuracy guarantee does not depend on it" + ] } diff --git a/crates/devtools/tests/sketch_plan_snapshots/random-mixed-seed-862-long-w01-q3-mvp.json b/crates/devtools/tests/sketch_plan_snapshots/random-mixed-seed-862-long-w01-q3-mvp.json index 24b68f9ce..aa1d3b019 100644 --- a/crates/devtools/tests/sketch_plan_snapshots/random-mixed-seed-862-long-w01-q3-mvp.json +++ b/crates/devtools/tests/sketch_plan_snapshots/random-mixed-seed-862-long-w01-q3-mvp.json @@ -3,12030 +3,119 @@ "materialized_views": [ { "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", + "definition_id": "0c7f962d8fde713c1de7a58752019151", "selection": [ { - "columns": [], - "relative_time": [ - { - "excluded": -1000 - }, - { - "included": 0 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 0, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 0, - "node_id": 2, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2000 - }, - { - "included": -1000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 1, - "node_id": 5, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -3000 - }, - { - "included": -2000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 2, - "node_id": 8, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -4000 - }, - { - "included": -3000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 3, - "node_id": 11, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -5000 - }, - { - "included": -4000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 4000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 4, - "node_id": 14, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -6000 - }, - { - "included": -5000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 5000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 5, - "node_id": 17, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -7000 - }, - { - "included": -6000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 6000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 6, - "node_id": 20, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -8000 - }, - { - "included": -7000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 7000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 7, - "node_id": 23, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -9000 - }, - { - "included": -8000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 8000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 8, - "node_id": 26, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -10000 - }, - { - "included": -9000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 9000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 9, - "node_id": 29, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -11000 - }, - { - "included": -10000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 10000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 10, - "node_id": 32, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -12000 - }, - { - "included": -11000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 11000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 11, - "node_id": 35, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -13000 - }, - { - "included": -12000 - } - ] + "columns": [] } ] }, "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 12000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] + "materialized_views": [] }, - "installed_count": 300, - "installed_position": 12, - "node_id": 38, + "node_id": 1, "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -14000 - }, - { - "included": -13000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 13000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 13, - "node_id": 41, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -15000 - }, - { - "included": -14000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 14000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 14, - "node_id": 44, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -16000 - }, - { - "included": -15000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 15000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 15, - "node_id": 47, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -17000 - }, - { - "included": -16000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 16000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 16, - "node_id": 50, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -18000 - }, - { - "included": -17000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 17000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 17, - "node_id": 53, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -19000 - }, - { - "included": -18000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 18000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 18, - "node_id": 56, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -20000 - }, - { - "included": -19000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 19000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 19, - "node_id": 59, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -21000 - }, - { - "included": -20000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 20000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 20, - "node_id": 62, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -22000 - }, - { - "included": -21000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 21000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 21, - "node_id": 65, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -23000 - }, - { - "included": -22000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 22000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 22, - "node_id": 68, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -24000 - }, - { - "included": -23000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 23000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 23, - "node_id": 71, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -25000 - }, - { - "included": -24000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 24000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 24, - "node_id": 74, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -26000 - }, - { - "included": -25000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 25000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 25, - "node_id": 77, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -27000 - }, - { - "included": -26000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 26000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 26, - "node_id": 80, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -28000 - }, - { - "included": -27000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 27000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 27, - "node_id": 83, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -29000 - }, - { - "included": -28000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 28000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 28, - "node_id": 86, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -30000 - }, - { - "included": -29000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 29000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 29, - "node_id": 89, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -31000 - }, - { - "included": -30000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 30000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 30, - "node_id": 92, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -32000 - }, - { - "included": -31000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 31000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 31, - "node_id": 95, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -33000 - }, - { - "included": -32000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 32000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 32, - "node_id": 98, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -34000 - }, - { - "included": -33000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 33000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 33, - "node_id": 101, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -35000 - }, - { - "included": -34000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 34000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 34, - "node_id": 104, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -36000 - }, - { - "included": -35000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 35000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 35, - "node_id": 107, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -37000 - }, - { - "included": -36000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 36000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 36, - "node_id": 110, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -38000 - }, - { - "included": -37000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 37000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 37, - "node_id": 113, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -39000 - }, - { - "included": -38000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 38000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 38, - "node_id": 116, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -40000 - }, - { - "included": -39000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 39000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 39, - "node_id": 119, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -41000 - }, - { - "included": -40000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 40000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 40, - "node_id": 122, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -42000 - }, - { - "included": -41000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 41000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 41, - "node_id": 125, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -43000 - }, - { - "included": -42000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 42000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 42, - "node_id": 128, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -44000 - }, - { - "included": -43000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 43000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 43, - "node_id": 131, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -45000 - }, - { - "included": -44000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 44000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 44, - "node_id": 134, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -46000 - }, - { - "included": -45000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 45000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 45, - "node_id": 137, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -47000 - }, - { - "included": -46000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 46000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 46, - "node_id": 140, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -48000 - }, - { - "included": -47000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 47000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 47, - "node_id": 143, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -49000 - }, - { - "included": -48000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 48000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 48, - "node_id": 146, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -50000 - }, - { - "included": -49000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 49000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 49, - "node_id": 149, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -51000 - }, - { - "included": -50000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 50000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 50, - "node_id": 152, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -52000 - }, - { - "included": -51000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 51000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 51, - "node_id": 155, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -53000 - }, - { - "included": -52000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 52000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 52, - "node_id": 158, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -54000 - }, - { - "included": -53000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 53000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 53, - "node_id": 161, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -55000 - }, - { - "included": -54000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 54000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 54, - "node_id": 164, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -56000 - }, - { - "included": -55000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 55000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 55, - "node_id": 167, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -57000 - }, - { - "included": -56000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 56000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 56, - "node_id": 170, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -58000 - }, - { - "included": -57000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 57000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 57, - "node_id": 173, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -59000 - }, - { - "included": -58000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 58000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 58, - "node_id": 176, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -60000 - }, - { - "included": -59000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 59000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 59, - "node_id": 179, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -61000 - }, - { - "included": -60000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 60000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 60, - "node_id": 182, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -62000 - }, - { - "included": -61000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 61000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 61, - "node_id": 185, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -63000 - }, - { - "included": -62000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 62000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 62, - "node_id": 188, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -64000 - }, - { - "included": -63000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 63000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 63, - "node_id": 191, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -65000 - }, - { - "included": -64000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 64000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 64, - "node_id": 194, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -66000 - }, - { - "included": -65000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 65000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 65, - "node_id": 197, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -67000 - }, - { - "included": -66000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 66000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 66, - "node_id": 200, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -68000 - }, - { - "included": -67000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 67000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 67, - "node_id": 203, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -69000 - }, - { - "included": -68000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 68000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 68, - "node_id": 206, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -70000 - }, - { - "included": -69000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 69000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 69, - "node_id": 209, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -71000 - }, - { - "included": -70000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 70000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 70, - "node_id": 212, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -72000 - }, - { - "included": -71000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 71000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 71, - "node_id": 215, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -73000 - }, - { - "included": -72000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 72000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 72, - "node_id": 218, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -74000 - }, - { - "included": -73000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 73000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 73, - "node_id": 221, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -75000 - }, - { - "included": -74000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 74000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 74, - "node_id": 224, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -76000 - }, - { - "included": -75000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 75000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 75, - "node_id": 227, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -77000 - }, - { - "included": -76000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 76000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 76, - "node_id": 230, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -78000 - }, - { - "included": -77000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 77000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 77, - "node_id": 233, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -79000 - }, - { - "included": -78000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 78000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 78, - "node_id": 236, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -80000 - }, - { - "included": -79000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 79000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 79, - "node_id": 239, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -81000 - }, - { - "included": -80000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 80000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 80, - "node_id": 242, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -82000 - }, - { - "included": -81000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 81000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 81, - "node_id": 245, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -83000 - }, - { - "included": -82000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 82000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 82, - "node_id": 248, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -84000 - }, - { - "included": -83000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 83000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 83, - "node_id": 251, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -85000 - }, - { - "included": -84000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 84000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 84, - "node_id": 254, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -86000 - }, - { - "included": -85000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 85000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 85, - "node_id": 257, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -87000 - }, - { - "included": -86000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 86000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 86, - "node_id": 260, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -88000 - }, - { - "included": -87000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 87000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 87, - "node_id": 263, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -89000 - }, - { - "included": -88000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 88000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 88, - "node_id": 266, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -90000 - }, - { - "included": -89000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 89000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 89, - "node_id": 269, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -91000 - }, - { - "included": -90000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 90000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 90, - "node_id": 272, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -92000 - }, - { - "included": -91000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 91000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 91, - "node_id": 275, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -93000 - }, - { - "included": -92000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 92000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 92, - "node_id": 278, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -94000 - }, - { - "included": -93000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 93000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 93, - "node_id": 281, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -95000 - }, - { - "included": -94000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 94000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 94, - "node_id": 284, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -96000 - }, - { - "included": -95000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 95000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 95, - "node_id": 287, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -97000 - }, - { - "included": -96000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 96000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 96, - "node_id": 290, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -98000 - }, - { - "included": -97000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 97000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 97, - "node_id": 293, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -99000 - }, - { - "included": -98000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 98000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 98, - "node_id": 296, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -100000 - }, - { - "included": -99000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 99000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 99, - "node_id": 299, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -101000 - }, - { - "included": -100000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 100000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 100, - "node_id": 302, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -102000 - }, - { - "included": -101000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 101000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 101, - "node_id": 305, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -103000 - }, - { - "included": -102000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 102000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 102, - "node_id": 308, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -104000 - }, - { - "included": -103000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 103000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 103, - "node_id": 311, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -105000 - }, - { - "included": -104000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 104000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 104, - "node_id": 314, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -106000 - }, - { - "included": -105000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 105000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 105, - "node_id": 317, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -107000 - }, - { - "included": -106000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 106000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 106, - "node_id": 320, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -108000 - }, - { - "included": -107000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 107000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 107, - "node_id": 323, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -109000 - }, - { - "included": -108000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 108000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 108, - "node_id": 326, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -110000 - }, - { - "included": -109000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 109000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 109, - "node_id": 329, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -111000 - }, - { - "included": -110000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 110000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 110, - "node_id": 332, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -112000 - }, - { - "included": -111000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 111000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 111, - "node_id": 335, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -113000 - }, - { - "included": -112000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 112000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 112, - "node_id": 338, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -114000 - }, - { - "included": -113000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 113000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 113, - "node_id": 341, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -115000 - }, - { - "included": -114000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 114000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 114, - "node_id": 344, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -116000 - }, - { - "included": -115000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 115000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 115, - "node_id": 347, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -117000 - }, - { - "included": -116000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 116000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 116, - "node_id": 350, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -118000 - }, - { - "included": -117000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 117000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 117, - "node_id": 353, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -119000 - }, - { - "included": -118000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 118000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 118, - "node_id": 356, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -120000 - }, - { - "included": -119000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 119000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 119, - "node_id": 359, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -121000 - }, - { - "included": -120000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 120000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 120, - "node_id": 362, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -122000 - }, - { - "included": -121000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 121000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 121, - "node_id": 365, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -123000 - }, - { - "included": -122000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 122000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 122, - "node_id": 368, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -124000 - }, - { - "included": -123000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 123000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 123, - "node_id": 371, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -125000 - }, - { - "included": -124000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 124000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 124, - "node_id": 374, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -126000 - }, - { - "included": -125000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 125000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 125, - "node_id": 377, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -127000 - }, - { - "included": -126000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 126000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 126, - "node_id": 380, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -128000 - }, - { - "included": -127000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 127000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 127, - "node_id": 383, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -129000 - }, - { - "included": -128000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 128000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 128, - "node_id": 386, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -130000 - }, - { - "included": -129000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 129000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 129, - "node_id": 389, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -131000 - }, - { - "included": -130000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 130000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 130, - "node_id": 392, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -132000 - }, - { - "included": -131000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 131000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 131, - "node_id": 395, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -133000 - }, - { - "included": -132000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 132000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 132, - "node_id": 398, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -134000 - }, - { - "included": -133000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 133000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 133, - "node_id": 401, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -135000 - }, - { - "included": -134000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 134000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 134, - "node_id": 404, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -136000 - }, - { - "included": -135000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 135000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 135, - "node_id": 407, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -137000 - }, - { - "included": -136000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 136000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 136, - "node_id": 410, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -138000 - }, - { - "included": -137000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 137000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 137, - "node_id": 413, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -139000 - }, - { - "included": -138000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 138000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 138, - "node_id": 416, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -140000 - }, - { - "included": -139000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 139000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 139, - "node_id": 419, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -141000 - }, - { - "included": -140000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 140000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 140, - "node_id": 422, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -142000 - }, - { - "included": -141000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 141000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 141, - "node_id": 425, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -143000 - }, - { - "included": -142000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 142000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 142, - "node_id": 428, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -144000 - }, - { - "included": -143000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 143000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 143, - "node_id": 431, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -145000 - }, - { - "included": -144000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 144000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 144, - "node_id": 434, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -146000 - }, - { - "included": -145000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 145000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 145, - "node_id": 437, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -147000 - }, - { - "included": -146000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 146000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 146, - "node_id": 440, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -148000 - }, - { - "included": -147000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 147000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 147, - "node_id": 443, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -149000 - }, - { - "included": -148000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 148000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 148, - "node_id": 446, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -150000 - }, - { - "included": -149000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 149000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 149, - "node_id": 449, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -151000 - }, - { - "included": -150000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 150000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 150, - "node_id": 452, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -152000 - }, - { - "included": -151000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 151000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 151, - "node_id": 455, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -153000 - }, - { - "included": -152000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 152000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 152, - "node_id": 458, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -154000 - }, - { - "included": -153000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 153000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 153, - "node_id": 461, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -155000 - }, - { - "included": -154000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 154000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 154, - "node_id": 464, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -156000 - }, - { - "included": -155000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 155000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 155, - "node_id": 467, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -157000 - }, - { - "included": -156000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 156000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 156, - "node_id": 470, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -158000 - }, - { - "included": -157000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 157000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 157, - "node_id": 473, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -159000 - }, - { - "included": -158000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 158000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 158, - "node_id": 476, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -160000 - }, - { - "included": -159000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 159000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 159, - "node_id": 479, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -161000 - }, - { - "included": -160000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 160000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 160, - "node_id": 482, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -162000 - }, - { - "included": -161000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 161000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 161, - "node_id": 485, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -163000 - }, - { - "included": -162000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 162000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 162, - "node_id": 488, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -164000 - }, - { - "included": -163000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 163000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 163, - "node_id": 491, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -165000 - }, - { - "included": -164000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 164000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 164, - "node_id": 494, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -166000 - }, - { - "included": -165000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 165000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 165, - "node_id": 497, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -167000 - }, - { - "included": -166000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 166000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 166, - "node_id": 500, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -168000 - }, - { - "included": -167000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 167000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 167, - "node_id": 503, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -169000 - }, - { - "included": -168000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 168000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 168, - "node_id": 506, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -170000 - }, - { - "included": -169000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 169000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 169, - "node_id": 509, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -171000 - }, - { - "included": -170000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 170000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 170, - "node_id": 512, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -172000 - }, - { - "included": -171000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 171000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 171, - "node_id": 515, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -173000 - }, - { - "included": -172000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 172000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 172, - "node_id": 518, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -174000 - }, - { - "included": -173000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 173000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 173, - "node_id": 521, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -175000 - }, - { - "included": -174000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 174000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 174, - "node_id": 524, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -176000 - }, - { - "included": -175000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 175000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 175, - "node_id": 527, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -177000 - }, - { - "included": -176000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 176000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 176, - "node_id": 530, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -178000 - }, - { - "included": -177000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 177000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 177, - "node_id": 533, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -179000 - }, - { - "included": -178000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 178000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 178, - "node_id": 536, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -180000 - }, - { - "included": -179000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 179000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 179, - "node_id": 539, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -181000 - }, - { - "included": -180000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 180000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 180, - "node_id": 542, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -182000 - }, - { - "included": -181000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 181000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 181, - "node_id": 545, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -183000 - }, - { - "included": -182000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 182000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 182, - "node_id": 548, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -184000 - }, - { - "included": -183000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 183000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 183, - "node_id": 551, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -185000 - }, - { - "included": -184000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 184000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 184, - "node_id": 554, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -186000 - }, - { - "included": -185000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 185000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 185, - "node_id": 557, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -187000 - }, - { - "included": -186000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 186000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 186, - "node_id": 560, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -188000 - }, - { - "included": -187000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 187000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 187, - "node_id": 563, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -189000 - }, - { - "included": -188000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 188000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 188, - "node_id": 566, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -190000 - }, - { - "included": -189000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 189000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 189, - "node_id": 569, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -191000 - }, - { - "included": -190000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 190000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 190, - "node_id": 572, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -192000 - }, - { - "included": -191000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 191000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 191, - "node_id": 575, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -193000 - }, - { - "included": -192000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 192000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 192, - "node_id": 578, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -194000 - }, - { - "included": -193000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 193000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 193, - "node_id": 581, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -195000 - }, - { - "included": -194000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 194000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 194, - "node_id": 584, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -196000 - }, - { - "included": -195000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 195000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 195, - "node_id": 587, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -197000 - }, - { - "included": -196000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 196000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 196, - "node_id": 590, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -198000 - }, - { - "included": -197000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 197000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 197, - "node_id": 593, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -199000 - }, - { - "included": -198000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 198000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 198, - "node_id": 596, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -200000 - }, - { - "included": -199000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 199000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 199, - "node_id": 599, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -201000 - }, - { - "included": -200000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 200000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 200, - "node_id": 602, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -202000 - }, - { - "included": -201000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 201000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 201, - "node_id": 605, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -203000 - }, - { - "included": -202000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 202000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 202, - "node_id": 608, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -204000 - }, - { - "included": -203000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 203000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 203, - "node_id": 611, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -205000 - }, - { - "included": -204000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 204000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 204, - "node_id": 614, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -206000 - }, - { - "included": -205000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 205000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 205, - "node_id": 617, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -207000 - }, - { - "included": -206000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 206000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 206, - "node_id": 620, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -208000 - }, - { - "included": -207000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 207000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 207, - "node_id": 623, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -209000 - }, - { - "included": -208000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 208000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 208, - "node_id": 626, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -210000 - }, - { - "included": -209000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 209000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 209, - "node_id": 629, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -211000 - }, - { - "included": -210000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 210000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 210, - "node_id": 632, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -212000 - }, - { - "included": -211000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 211000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 211, - "node_id": 635, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -213000 - }, - { - "included": -212000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 212000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 212, - "node_id": 638, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -214000 - }, - { - "included": -213000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 213000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 213, - "node_id": 641, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -215000 - }, - { - "included": -214000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 214000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 214, - "node_id": 644, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -216000 - }, - { - "included": -215000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 215000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 215, - "node_id": 647, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -217000 - }, - { - "included": -216000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 216000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 216, - "node_id": 650, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -218000 - }, - { - "included": -217000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 217000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 217, - "node_id": 653, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -219000 - }, - { - "included": -218000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 218000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 218, - "node_id": 656, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -220000 - }, - { - "included": -219000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 219000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 219, - "node_id": 659, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -221000 - }, - { - "included": -220000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 220000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 220, - "node_id": 662, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -222000 - }, - { - "included": -221000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 221000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 221, - "node_id": 665, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -223000 - }, - { - "included": -222000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 222000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 222, - "node_id": 668, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -224000 - }, - { - "included": -223000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 223000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 223, - "node_id": 671, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -225000 - }, - { - "included": -224000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 224000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 224, - "node_id": 674, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -226000 - }, - { - "included": -225000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 225000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 225, - "node_id": 677, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -227000 - }, - { - "included": -226000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 226000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 226, - "node_id": 680, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -228000 - }, - { - "included": -227000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 227000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 227, - "node_id": 683, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -229000 - }, - { - "included": -228000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 228000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 228, - "node_id": 686, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -230000 - }, - { - "included": -229000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 229000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 229, - "node_id": 689, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -231000 - }, - { - "included": -230000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 230000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 230, - "node_id": 692, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -232000 - }, - { - "included": -231000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 231000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 231, - "node_id": 695, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -233000 - }, - { - "included": -232000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 232000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 232, - "node_id": 698, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -234000 - }, - { - "included": -233000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 233000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 233, - "node_id": 701, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -235000 - }, - { - "included": -234000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 234000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 234, - "node_id": 704, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -236000 - }, - { - "included": -235000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 235000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 235, - "node_id": 707, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -237000 - }, - { - "included": -236000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 236000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 236, - "node_id": 710, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -238000 - }, - { - "included": -237000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 237000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 237, - "node_id": 713, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -239000 - }, - { - "included": -238000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 238000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 238, - "node_id": 716, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -240000 - }, - { - "included": -239000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 239000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 239, - "node_id": 719, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -241000 - }, - { - "included": -240000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 240000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 240, - "node_id": 722, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -242000 - }, - { - "included": -241000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 241000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 241, - "node_id": 725, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -243000 - }, - { - "included": -242000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 242000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 242, - "node_id": 728, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -244000 - }, - { - "included": -243000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 243000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 243, - "node_id": 731, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -245000 - }, - { - "included": -244000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 244000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 244, - "node_id": 734, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -246000 - }, - { - "included": -245000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 245000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 245, - "node_id": 737, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -247000 - }, - { - "included": -246000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 246000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 246, - "node_id": 740, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -248000 - }, - { - "included": -247000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 247000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 247, - "node_id": 743, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -249000 - }, - { - "included": -248000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 248000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 248, - "node_id": 746, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -250000 - }, - { - "included": -249000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 249000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 249, - "node_id": 749, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -251000 - }, - { - "included": -250000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 250000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 250, - "node_id": 752, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -252000 - }, - { - "included": -251000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 251000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 251, - "node_id": 755, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -253000 - }, - { - "included": -252000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 252000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 252, - "node_id": 758, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -254000 - }, - { - "included": -253000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 253000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 253, - "node_id": 761, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -255000 - }, - { - "included": -254000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 254000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 254, - "node_id": 764, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -256000 - }, - { - "included": -255000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 255000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 255, - "node_id": 767, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -257000 - }, - { - "included": -256000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 256000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 256, - "node_id": 770, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -258000 - }, - { - "included": -257000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 257000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 257, - "node_id": 773, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -259000 - }, - { - "included": -258000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 258000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 258, - "node_id": 776, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -260000 - }, - { - "included": -259000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 259000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 259, - "node_id": 779, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -261000 - }, - { - "included": -260000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 260000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 260, - "node_id": 782, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -262000 - }, - { - "included": -261000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 261000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 261, - "node_id": 785, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -263000 - }, - { - "included": -262000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 262000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 262, - "node_id": 788, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -264000 - }, - { - "included": -263000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 263000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 263, - "node_id": 791, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -265000 - }, - { - "included": -264000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 264000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 264, - "node_id": 794, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -266000 - }, - { - "included": -265000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 265000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 265, - "node_id": 797, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -267000 - }, - { - "included": -266000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 266000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 266, - "node_id": 800, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -268000 - }, - { - "included": -267000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 267000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 267, - "node_id": 803, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -269000 - }, - { - "included": -268000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 268000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 268, - "node_id": 806, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -270000 - }, - { - "included": -269000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 269000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 269, - "node_id": 809, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -271000 - }, - { - "included": -270000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 270000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 270, - "node_id": 812, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -272000 - }, - { - "included": -271000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 271000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 271, - "node_id": 815, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -273000 - }, - { - "included": -272000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 272000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 272, - "node_id": 818, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -274000 - }, - { - "included": -273000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 273000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 273, - "node_id": 821, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -275000 - }, - { - "included": -274000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 274000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 274, - "node_id": 824, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -276000 - }, - { - "included": -275000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 275000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 275, - "node_id": 827, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -277000 - }, - { - "included": -276000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 276000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 276, - "node_id": 830, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -278000 - }, - { - "included": -277000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 277000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 277, - "node_id": 833, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -279000 - }, - { - "included": -278000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 278000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 278, - "node_id": 836, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -280000 - }, - { - "included": -279000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 279000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 279, - "node_id": 839, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -281000 - }, - { - "included": -280000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 280000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 280, - "node_id": 842, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -282000 - }, - { - "included": -281000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 281000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 281, - "node_id": 845, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -283000 - }, - { - "included": -282000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 282000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 282, - "node_id": 848, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -284000 - }, - { - "included": -283000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 283000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 283, - "node_id": 851, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -285000 - }, - { - "included": -284000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 284000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 284, - "node_id": 854, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -286000 - }, - { - "included": -285000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 285000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 285, - "node_id": 857, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -287000 - }, - { - "included": -286000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 286000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 286, - "node_id": 860, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -288000 - }, - { - "included": -287000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 287000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 287, - "node_id": 863, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -289000 - }, - { - "included": -288000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 288000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 288, - "node_id": 866, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -290000 - }, - { - "included": -289000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 289000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 289, - "node_id": 869, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -291000 - }, - { - "included": -290000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 290000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 290, - "node_id": 872, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -292000 - }, - { - "included": -291000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 291000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 291, - "node_id": 875, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -293000 - }, - { - "included": -292000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 292000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 292, - "node_id": 878, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -294000 - }, - { - "included": -293000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 293000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 293, - "node_id": 881, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -295000 - }, - { - "included": -294000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 294000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 294, - "node_id": 884, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -296000 - }, - { - "included": -295000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 295000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 295, - "node_id": 887, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -297000 - }, - { - "included": -296000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 296000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 296, - "node_id": 890, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -298000 - }, - { - "included": -297000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 297000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 297, - "node_id": 893, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -299000 - }, - { - "included": -298000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 298000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 298, - "node_id": 896, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -300000 - }, - { - "included": -299000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 299000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 300, - "installed_position": 299, - "node_id": 899, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - } - ], - "proven_optimal": true, - "queries": [ - "avg by (region) (quantile_over_time(0.95, data[300s]))" - ], - "search_status": "optimal", - "selected": { - "accepted": [ - { - "accepted_metric": "relative", - "accepted_regardless_of_guarantee": false, - "guarantee_check_rejection": null, - "sketches": [ - "DDSketch" - ], - "summary": "metric not set; accepted relative (DDSketch)", - "target_metric": null - } - ], - "cost": { - "cost_per_second": 0.0000190434545346, - "cost_unit": "cost_per_second", - "cpu_core_seconds_per_second": 0.00387, - "per_node": { - "0": { - "cost_per_second": 6.52152e-9, - "cpu_core_seconds_per_second": 0.00058, - "detail": "scan 100000 samples; x 1.0000/s", - "multiplier": 1, - "operator": "Scan", - "price_source": "analytical", - "timing": "ingestion_time" - }, - "1": { - "cost_per_second": 0.0000190019900264, - "cpu_core_seconds_per_second": 0.0002, - "detail": "build DDSketch in 300 tumbling windows of 1000 ms for 10000 groups: 100000 rows x (1 + depth 1); x 1.0000/s; keeps 16518880000 bytes", - "multiplier": 1, - "operator": "SummaryTumblingWindowChain", - "price_source": "analytical", - "timing": "ingestion_time" - }, - "10": { - "cost_per_second": 1.12440085298e-10, - "cpu_core_seconds_per_second": 0.00001, - "detail": "operator: pass 10000 rows; x 1.0000/s; holds 24 bytes for 3.1 ms", - "multiplier": 1, - "operator": "BinaryOp", - "price_source": "analytical", - "timing": "query_time" - }, - "11": { - "cost_per_second": 1.12440085298e-10, - "cpu_core_seconds_per_second": 0.00001, - "detail": "operator: pass 10000 rows; x 1.0000/s; holds 24 bytes for 3.1 ms", - "multiplier": 1, - "operator": "Project", - "price_source": "analytical", - "timing": "query_time" - }, - "2": { - "cost_per_second": 3.39270471774e-8, - "cpu_core_seconds_per_second": 0.003, - "detail": "merge 3000000 states into 10000; x 1.0000/s; holds 54880000 bytes for 3.1 ms", - "multiplier": 1, - "operator": "SummaryMerge", - "price_source": "analytical", - "timing": "query_time" - }, - "3": { - "cost_per_second": 1.1244e-10, - "cpu_core_seconds_per_second": 0.00001, - "detail": "estimate 10000 groups x 1; x 1.0000/s", - "multiplier": 1, - "operator": "SummaryEstimate", - "price_source": "analytical", - "timing": "query_time" - }, - "4": { - "cost_per_second": 1.12440170595e-10, - "cpu_core_seconds_per_second": 0.00001, - "detail": "operator: pass 10000 rows; x 1.0000/s; holds 48 bytes for 3.1 ms", - "multiplier": 1, - "operator": "Project", - "price_source": "analytical", - "timing": "query_time" - }, - "5": { - "cost_per_second": 1.1443027732e-10, - "cpu_core_seconds_per_second": 0.00001, - "detail": "build exact Sum accumulator into 10000 states: 10000 rows x depth 1; x 1.0000/s; holds 560000 bytes for 3.1 ms", - "multiplier": 1, - "operator": "SummaryAgg", - "price_source": "analytical", - "timing": "query_time" - }, - "6": { - "cost_per_second": 1.1244e-10, - "cpu_core_seconds_per_second": 0.00001, - "detail": "finalize 10000 accumulators; x 1.0000/s", - "multiplier": 1, - "operator": "FinalizeExactAccumulator", - "price_source": "analytical", - "timing": "query_time" - }, - "7": { - "cost_per_second": 1.1443027732e-10, - "cpu_core_seconds_per_second": 0.00001, - "detail": "build exact Count accumulator into 10000 states: 10000 rows x depth 1; x 1.0000/s; holds 560000 bytes for 3.1 ms", - "multiplier": 1, - "operator": "SummaryAgg", - "price_source": "analytical", - "timing": "query_time" - }, - "8": { - "cost_per_second": 1.1244e-10, - "cpu_core_seconds_per_second": 0.00001, - "detail": "finalize 10000 accumulators; x 1.0000/s", - "multiplier": 1, - "operator": "FinalizeExactAccumulator", - "price_source": "analytical", - "timing": "query_time" - }, - "9": { - "cost_per_second": 1.12440085298e-10, - "cpu_core_seconds_per_second": 0.00001, - "detail": "operator: pass 10000 rows; x 1.0000/s; holds 24 bytes for 3.1 ms", - "multiplier": 1, - "operator": "Project", - "price_source": "analytical", - "timing": "query_time" - } - }, - "source": "analytical-cost-v2 (illustrative statistics)", - "usage": { - "average_memory_bytes": 16519053040.4, - "cpu_ops_per_second": 3390000.0, - "disk_bytes": 0, - "latency_per_query": [ - { - "compaction_ms": 0.0, - "query_ms": 3.09 - } - ], - "peak_cpu_cores": 2, - "peak_memory_bytes": 16574880168, - "scanned_bytes_per_second": 4800000.0 - } - }, - "dag": { - "edges": [ - { - "consumer": 1, - "data_state": { - "primitive": "Raw", - "timing": "ingestion_time" - }, - "grouping": "NotApplicable", - "intermediate_schema": { - "closed": true, - "fields": [ - { - "dtype": { - "Plain": "timestamp" - }, - "name": "ts", - "nullable": false, - "table": null - }, - { - "dtype": { - "Plain": "float64" - }, - "name": "value", - "nullable": false, - "table": null - }, - { - "dtype": { - "Plain": "utf8" - }, - "name": "region", - "nullable": true, - "table": null - }, - { - "dtype": { - "Plain": "utf8" - }, - "name": "$promql_series_identity", - "nullable": false, - "table": null - } - ], - "time_index": 0, - "unique_keys": [ - [ - 3, - 0 - ] - ] - }, - "producer": 0, - "role": "Input", - "window": "RequiresAlignedPanePhaseOrExactBoundaryResidual" - }, - { - "consumer": 2, - "data_state": { - "primitive": "SummaryState", - "timing": "ingestion_time" - }, - "grouping": "NotApplicable", - "intermediate_schema": { - "closed": true, - "fields": [ - { - "dtype": { - "Plain": "timestamp" - }, - "name": "ts", - "nullable": false, - "table": null - }, - { - "dtype": { - "Sketch": [ - { - "algorithm": "DDSketch", - "params": { - "DDSketch": { - "alpha": 0.05 - } - } - }, - "PerSubpopulationInstance" - ] - }, - "name": "state", - "nullable": false, - "table": null - }, - { - "dtype": { - "Plain": "utf8" - }, - "name": "region", - "nullable": true, - "table": null - }, - { - "dtype": { - "Plain": "utf8" - }, - "name": "$promql_series_identity", - "nullable": false, - "table": null - } - ], - "time_index": 0, - "unique_keys": [ - [ - 3, - 0 - ] - ] - }, - "producer": 1, - "role": "Input", - "window": "NotApplicable" - }, - { - "consumer": 3, - "data_state": { - "primitive": "SummaryState", - "timing": "query_time" - }, - "grouping": "NotApplicable", - "intermediate_schema": { - "closed": true, - "fields": [ - { - "dtype": { - "Plain": "timestamp" - }, - "name": "ts", - "nullable": false, - "table": null - }, - { - "dtype": { - "Sketch": [ - { - "algorithm": "DDSketch", - "params": { - "DDSketch": { - "alpha": 0.05 - } - } - }, - "PerSubpopulationInstance" - ] - }, - "name": "state", - "nullable": false, - "table": null - }, - { - "dtype": { - "Plain": "utf8" - }, - "name": "region", - "nullable": true, - "table": null - }, - { - "dtype": { - "Plain": "utf8" - }, - "name": "$promql_series_identity", - "nullable": false, - "table": null - } - ], - "time_index": 0, - "unique_keys": [ - [ - 3, - 0 - ] - ] - }, - "producer": 2, - "role": "Input", - "window": "NotApplicable" - }, - { - "consumer": 4, - "data_state": { - "primitive": "Raw", - "timing": "query_time" - }, - "grouping": "NotApplicable", - "intermediate_schema": { - "closed": true, - "fields": [ - { - "dtype": { - "Plain": "timestamp" - }, - "name": "ts", - "nullable": false, - "table": null - }, - { - "dtype": { - "Plain": "float64" - }, - "name": "quantile", - "nullable": false, - "table": null - }, - { - "dtype": { - "Plain": "utf8" - }, - "name": "region", - "nullable": true, - "table": null - }, - { - "dtype": { - "Plain": "utf8" - }, - "name": "$promql_series_identity", - "nullable": false, - "table": null - } - ], - "time_index": 0, - "unique_keys": [ - [ - 3, - 0 - ] - ] - }, - "producer": 3, - "role": "Input", - "window": "NotApplicable" - }, - { - "consumer": 5, - "data_state": { - "primitive": "Raw", - "timing": "query_time" - }, - "grouping": "NotApplicable", - "intermediate_schema": { - "closed": true, - "fields": [ - { - "dtype": { - "Plain": "timestamp" - }, - "name": "ts", - "nullable": false, - "table": null - }, - { - "dtype": { - "Plain": "float64" - }, - "name": "value", - "nullable": false, - "table": null - }, - { - "dtype": { - "Plain": "utf8" - }, - "name": "region", - "nullable": true, - "table": null - }, - { - "dtype": { - "Plain": "utf8" - }, - "name": "$promql_series_identity", - "nullable": false, - "table": null - } - ], - "time_index": 0, - "unique_keys": [ - [ - 3, - 0 - ] - ] - }, - "producer": 4, - "role": "Input", - "window": "NotApplicable" - }, - { - "consumer": 6, - "data_state": { - "primitive": "SummaryState", - "timing": "query_time" - }, - "grouping": "NotApplicable", - "intermediate_schema": { - "closed": true, - "fields": [ - { - "dtype": { - "Plain": "utf8" - }, - "name": "region", - "nullable": true, - "table": null - }, - { - "dtype": { - "ExactAggregate": [ - "Sum", - "Sum" - ] - }, - "name": "state", - "nullable": false, - "table": null - } - ], - "time_index": null, - "unique_keys": [ - [ - 0 - ] - ] - }, - "producer": 5, - "role": "Input", - "window": "NotApplicable" + "retention_ms": 300000 + } + ], + "queries": [ + "avg by (region) (quantile_over_time(0.95, data[300s]))" + ], + "selected": { + "accepted": [ + { + "accepted_metric": "relative", + "sketches": [ + "DDSketch" + ], + "summary": "accepted relative (DDSketch)" + } + ], + "cost": { + "cost_per_second": 0.0000917148196267, + "cpu_core_seconds_per_second": 7.65776, + "per_node": { + "0": { + "cost_per_second": 6.52152e-9, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.00058, + "detail": "scan 100000 samples; x 1.0000/s", + "multiplier": 1, + "operator": "Scan", + "timing": "ingestion_time" }, - { - "consumer": 7, - "data_state": { - "primitive": "Raw", - "timing": "query_time" - }, - "grouping": "NotApplicable", - "intermediate_schema": { - "closed": true, - "fields": [ - { - "dtype": { - "Plain": "timestamp" - }, - "name": "ts", - "nullable": false, - "table": null - }, - { - "dtype": { - "Plain": "float64" - }, - "name": "value", - "nullable": false, - "table": null - }, - { - "dtype": { - "Plain": "utf8" - }, - "name": "region", - "nullable": true, - "table": null - }, - { - "dtype": { - "Plain": "utf8" - }, - "name": "$promql_series_identity", - "nullable": false, - "table": null - } - ], - "time_index": 0, - "unique_keys": [ - [ - 3, - 0 - ] - ] - }, - "producer": 4, - "role": "Input", - "window": "NotApplicable" + "1": { + "cost_per_second": 6.00497737327e-6, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.05127, + "detail": "build DDSketch in tumbling windows of 1000 ms for 10000 groups: 100000 rows x 247 ops, 1000 group visits, 10000.0 states emitted; x 1.0000/s; keeps 4719680000 bytes", + "multiplier": 1, + "operator": "SummaryTumblingWindowChain", + "timing": "ingestion_time" }, - { - "consumer": 8, - "data_state": { - "primitive": "SummaryState", - "timing": "query_time" - }, - "grouping": "NotApplicable", - "intermediate_schema": { - "closed": true, - "fields": [ - { - "dtype": { - "Plain": "utf8" - }, - "name": "region", - "nullable": true, - "table": null - }, - { - "dtype": { - "ExactAggregate": [ - "Count", - "Count" - ] - }, - "name": "state", - "nullable": false, - "table": null - } - ], - "time_index": null, - "unique_keys": [ - [ - 0 - ] - ] - }, - "producer": 7, - "role": "Input", - "window": "NotApplicable" + "2": { + "cost_per_second": 3.71052059515e-7, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.033, + "detail": "time range 300s: 300 tumbling windows of states; x 1.0000/s; holds 1568 bytes for 33.000 ms", + "multiplier": 1, + "operator": "TimeRange", + "timing": "query_time" }, - { - "consumer": 9, - "data_state": { - "primitive": "Raw", - "timing": "query_time" - }, - "grouping": "NotApplicable", - "intermediate_schema": { - "closed": true, - "fields": [ - { - "dtype": { - "Plain": "utf8" - }, - "name": "region", - "nullable": true, - "table": null - }, - { - "dtype": { - "Plain": "int64" - }, - "name": "count", - "nullable": false, - "table": null - } - ], - "time_index": null, - "unique_keys": [ - [ - 0 - ] - ] - }, - "producer": 8, - "role": "Input", - "window": "NotApplicable" + "3": { + "cost_per_second": 0.0000846361402579, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 7.511, + "detail": "merge 3000000 states into 10000; x 1.0000/s; holds 21120000 bytes for 7511.000 ms", + "multiplier": 1, + "operator": "SummaryMerge", + "timing": "query_time" + }, + "4": { + "cost_per_second": 3.9354e-7, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.035, + "detail": "estimate 10000 groups x 1; x 1.0000/s", + "multiplier": 1, + "operator": "SummaryEstimate", + "timing": "query_time" + }, + "5": { + "cost_per_second": 1.12440000552e-10, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.00001, + "detail": "operator: pass 10000 rows; x 1.0000/s; holds 48 bytes for 0.010 ms", + "multiplier": 1, + "operator": "Project", + "timing": "query_time" }, + "6": { + "cost_per_second": 3.02475975973e-7, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.0269, + "detail": "hash aggregate 10000 rows into 10000 groups: 20 ops per row, 10000 group visits; x 1.0000/s; holds 400000 bytes for 26.900 ms", + "multiplier": 1, + "operator": "Aggregate", + "timing": "query_time" + } + }, + "source": "analytical-cost-v2 (illustrative statistics)", + "usage": { + "average_memory_bytes": 4878323131.74, + "cpu_ops_per_second": 7657280000.0, + "disk_bytes": 0, + "peak_cpu_cores": 9, + "peak_memory_bytes": 4741201664, + "scanned_bytes_per_second": 4800000.0 + } + }, + "dag": { + "edges": [ { - "consumer": 10, + "consumer": 1, "data_state": { "primitive": "Raw", - "timing": "query_time" + "timing": "ingestion_time" }, "grouping": "NotApplicable", "intermediate_schema": { @@ -12034,337 +123,157 @@ "fields": [ { "dtype": { - "Plain": "utf8" + "Plain": "timestamp" }, - "name": "region", - "nullable": true, - "table": null + "name": "ts", + "nullable": false }, { "dtype": { "Plain": "float64" }, - "name": "sum", - "nullable": false, - "table": null - } - ], - "time_index": null, - "unique_keys": [ - [ - 0 - ] - ] - }, - "producer": 6, - "role": "Left", - "window": "NotApplicable" - }, - { - "consumer": 10, - "data_state": { - "primitive": "Raw", - "timing": "query_time" - }, - "grouping": "NotApplicable", - "intermediate_schema": { - "closed": true, - "fields": [ + "name": "value", + "nullable": false + }, { "dtype": { "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { - "Plain": "float64" + "Plain": "utf8" }, - "name": "col_1", - "nullable": false, - "table": null + "name": "$promql_series_identity", + "nullable": false } ], - "time_index": null, + "time_index": 0, "unique_keys": [ [ + 3, 0 ] ] }, - "producer": 9, - "role": "Right", + "producer": 0, + "role": "Input", "window": "NotApplicable" }, { - "consumer": 11, + "consumer": 2, "data_state": { - "primitive": "Raw", - "timing": "query_time" + "primitive": "SummaryState", + "timing": "ingestion_time" }, "grouping": "NotApplicable", "intermediate_schema": { - "closed": true, - "fields": [ - { - "dtype": { - "Plain": "utf8" - }, - "name": "region", - "nullable": true, - "table": null - }, - { - "dtype": { - "Plain": "float64" - }, - "name": "sum", - "nullable": false, - "table": null - } - ], - "time_index": null, - "unique_keys": [ - [ - 0 - ] - ] - }, - "producer": 10, - "role": "Input", - "window": "NotApplicable" - } - ], - "nodes": [ - { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact over exact inputs" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, - "id": 0, - "lifetime": "None", - "output_schema": { "closed": true, "fields": [ { "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { - "Plain": "float64" + "Sketch": "DDSketch" }, - "name": "value", - "nullable": false, - "table": null + "name": "state", + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 3, - 0 + 0, + 2, + 3 ] ] }, - "output_state": { - "primitive": "Raw", - "timing": "ingestion_time" - }, - "payload": { - "NonASAP": { - "Scan": { - "predicates": [], - "schema": { - "closed": true, - "fields": [ - { - "dtype": { - "Plain": "timestamp" - }, - "name": "ts", - "nullable": false, - "table": null - }, - { - "dtype": { - "Plain": "float64" - }, - "name": "value", - "nullable": false, - "table": null - }, - { - "dtype": { - "Plain": "utf8" - }, - "name": "region", - "nullable": true, - "table": null - }, - { - "dtype": { - "Plain": "utf8" - }, - "name": "$promql_series_identity", - "nullable": false, - "table": null - } - ], - "time_index": 0, - "unique_keys": [ - [ - 3, - 0 - ] - ] - }, - "source": { - "TimeSeries": { - "metric": "data" - } - } - } - } - } + "producer": 1, + "role": "Input", + "window": "NotApplicable" }, { - "guarantee": null, - "id": 1, - "lifetime": { - "AcrossEvaluations": "Memory" + "consumer": 3, + "data_state": { + "primitive": "SummaryState", + "timing": "query_time" }, - "output_schema": { + "grouping": "NotApplicable", + "intermediate_schema": { "closed": true, "fields": [ { "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { - "Sketch": [ - { - "algorithm": "DDSketch", - "params": { - "DDSketch": { - "alpha": 0.05 - } - } - }, - "PerSubpopulationInstance" - ] + "Sketch": "DDSketch" }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 3, - 0 + 0, + 2, + 3 ] ] }, - "output_state": { - "primitive": "SummaryState", - "timing": "ingestion_time" - }, - "payload": { - "ASAP": { - "SummaryTumblingWindowChain": { - "child": 0, - "family": { - "Sketch": [ - { - "algorithm": "DDSketch", - "params": { - "DDSketch": { - "alpha": 0.05 - } - } - }, - "PerSubpopulationInstance" - ] - }, - "filter": null, - "grouping": "PerSubpopulationInstance", - "input": { - "item": null, - "weight": { - "Column": "SampleValue" - } - }, - "reduction": "PerEntity", - "tumbling_window_count": 300, - "tumbling_window_width_ms": 1000 - } - } - } + "producer": 2, + "role": "Input", + "window": "NotApplicable" }, { - "guarantee": null, - "id": 2, - "lifetime": "None", - "output_schema": { + "consumer": 4, + "data_state": { + "primitive": "SummaryState", + "timing": "query_time" + }, + "grouping": "NotApplicable", + "intermediate_schema": { "closed": true, "fields": [ { @@ -12372,42 +281,28 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { - "Sketch": [ - { - "algorithm": "DDSketch", - "params": { - "DDSketch": { - "alpha": 0.05 - } - } - }, - "PerSubpopulationInstance" - ] + "Sketch": "DDSketch" }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -12418,59 +313,18 @@ ] ] }, - "output_state": { - "primitive": "SummaryState", - "timing": "query_time" - }, - "payload": { - "ASAP": { - "SummaryMerge": { - "children": [ - 1 - ] - } - } - } + "producer": 3, + "role": "Input", + "window": "NotApplicable" }, { - "guarantee": { - "provenance": [ - { - "algorithm": "DDSketch", - "contract": "ddsketch_relative_error_alpha_v1", - "kind": "sketch_evaluation", - "params": { - "DDSketch": { - "alpha": 0.05 - } - }, - "query": "Quantile { q: 0.95 }" - } - ], - "spec": { - "accuracy_metric": "relative", - "accuracy_scope": "per_value", - "error": { - "factors": [ - { - "op": "constant", - "value": 0.05 - }, - { - "op": "unknown", - "statistic": "values_non_negative[data]" - } - ], - "op": "product" - }, - "failure_probability": { - "op": "zero" - } - } + "consumer": 5, + "data_state": { + "primitive": "Raw", + "timing": "query_time" }, - "id": 3, - "lifetime": "None", - "output_schema": { + "grouping": "NotApplicable", + "intermediate_schema": { "closed": true, "fields": [ { @@ -12478,32 +332,28 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "quantile", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -12514,62 +364,18 @@ ] ] }, - "output_state": { - "primitive": "Raw", - "timing": "query_time" - }, - "payload": { - "ASAP": { - "SummaryEstimate": { - "query": { - "Quantile": { - "q": 0.95 - } - }, - "summary_input": 2 - } - } - } + "producer": 4, + "role": "Input", + "window": "NotApplicable" }, { - "guarantee": { - "provenance": [ - { - "algorithm": "DDSketch", - "contract": "ddsketch_relative_error_alpha_v1", - "kind": "sketch_evaluation", - "params": { - "DDSketch": { - "alpha": 0.05 - } - }, - "query": "Quantile { q: 0.95 }" - } - ], - "spec": { - "accuracy_metric": "relative", - "accuracy_scope": "per_value", - "error": { - "factors": [ - { - "op": "constant", - "value": 0.05 - }, - { - "op": "unknown", - "statistic": "values_non_negative[data]" - } - ], - "op": "product" - }, - "failure_probability": { - "op": "zero" - } - } + "consumer": 6, + "data_state": { + "primitive": "Raw", + "timing": "query_time" }, - "id": 4, - "lifetime": "None", - "output_schema": { + "grouping": "NotApplicable", + "intermediate_schema": { "closed": true, "fields": [ { @@ -12577,32 +383,28 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -12613,291 +415,237 @@ ] ] }, - "output_state": { - "primitive": "Raw", - "timing": "query_time" - }, - "payload": { - "NonASAP": { - "Project": { - "child": 3, - "cols": [ - { - "alias": "ts", - "expr": { - "Column": 0 - } - }, - { - "alias": "value", - "expr": { - "Column": 1 - } - }, - { - "alias": "region", - "expr": { - "Column": 2 - } - }, - { - "alias": "$promql_series_identity", - "expr": { - "Column": 3 - } - } - ], - "qualifier": null - } - } - } - }, + "producer": 5, + "role": "Input", + "window": "NotApplicable" + } + ], + "nodes": [ { - "guarantee": null, - "id": 5, + "id": 0, "lifetime": "None", "output_schema": { "closed": true, "fields": [ + { + "dtype": { + "Plain": "timestamp" + }, + "name": "ts", + "nullable": false + }, + { + "dtype": { + "Plain": "float64" + }, + "name": "value", + "nullable": false + }, { "dtype": { "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { - "ExactAggregate": [ - "Sum", - "Sum" - ] + "Plain": "utf8" }, - "name": "state", - "nullable": false, - "table": null + "name": "$promql_series_identity", + "nullable": false } ], - "time_index": null, + "time_index": 0, "unique_keys": [ [ + 3, 0 ] ] }, "output_state": { - "primitive": "SummaryState", - "timing": "query_time" + "primitive": "Raw", + "timing": "ingestion_time" }, "payload": { - "ASAP": { - "SummaryAgg": { - "child": 4, - "family": { - "ExactAggregate": [ - "Sum", - "Sum" + "NonASAP": { + "Scan": { + "predicates": [], + "schema": { + "closed": true, + "fields": [ + { + "dtype": { + "Plain": "timestamp" + }, + "name": "ts", + "nullable": false + }, + { + "dtype": { + "Plain": "float64" + }, + "name": "value", + "nullable": false + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "region", + "nullable": true + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false + } + ], + "time_index": 0, + "unique_keys": [ + [ + 3, + 0 + ] ] }, - "filter": null, - "grouping": "PerSubpopulationInstance", - "input": { - "item": null, - "weight": { - "Column": "SampleValue" + "source": { + "TimeSeries": { + "metric": "data" } - }, - "reduction": { - "Reduce": [ - 2 - ] } } } } }, { - "guarantee": { - "provenance": [ - { - "kind": "unavailable_statistic", - "statistic": "input_row_count" - }, - { - "kind": "unavailable_statistic", - "statistic": "values_non_negative[data]" - }, + "id": 1, + "lifetime": { + "AcrossEvaluations": "Memory" + }, + "output_schema": { + "closed": true, + "fields": [ { - "guarantee": { - "provenance": [ - { - "algorithm": "DDSketch", - "contract": "ddsketch_relative_error_alpha_v1", - "kind": "sketch_evaluation", - "params": { - "DDSketch": { - "alpha": 0.05 - } - }, - "query": "Quantile { q: 0.95 }" - } - ], - "spec": { - "accuracy_metric": "relative", - "accuracy_scope": "per_value", - "error": { - "factors": [ - { - "op": "constant", - "value": 0.05 - }, - { - "op": "unknown", - "statistic": "values_non_negative[data]" - } - ], - "op": "product" - }, - "failure_probability": { - "op": "zero" - } - } + "dtype": { + "Plain": "timestamp" }, - "input_index": 0, - "kind": "child_guarantee" - }, - { - "kind": "exact", - "reason": "exact aggregation" + "name": "window_end", + "nullable": false }, { - "kind": "composition_step", - "operator": { - "op": "exact_sum" + "dtype": { + "Sketch": "DDSketch" }, - "rule": "exact_sum_relative" - } - ], - "spec": { - "accuracy_metric": "relative", - "accuracy_scope": "per_value", - "error": { - "factors": [ - { - "op": "max", - "terms": [ - { - "factors": [ - { - "op": "constant", - "value": 0.05 - }, - { - "op": "unknown", - "statistic": "values_non_negative[data]" - } - ], - "op": "product" - } - ] - }, - { - "op": "unknown", - "statistic": "values_non_negative[data]" - } - ], - "op": "product" + "name": "state", + "nullable": false }, - "failure_probability": { - "count": { - "op": "unknown", - "statistic": "input_row_count" - }, - "inner": { - "op": "union_bound", - "terms": [ - { - "op": "zero" - } - ] - }, - "op": "scaled" - } - } - }, - "id": 6, - "lifetime": "None", - "output_schema": { - "closed": true, - "fields": [ { "dtype": { "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { - "Plain": "float64" + "Plain": "utf8" }, - "name": "sum", - "nullable": false, - "table": null + "name": "$promql_series_identity", + "nullable": false } ], - "time_index": null, + "time_index": 0, "unique_keys": [ [ - 0 + 0, + 2, + 3 ] ] }, "output_state": { - "primitive": "Raw", - "timing": "query_time" + "primitive": "SummaryState", + "timing": "ingestion_time" + }, + "parameters": { + "DDSketch": { + "alpha": 0.05 + } }, "payload": { "ASAP": { - "FinalizeExactAccumulator": { - "child": 5 + "SummaryTumblingWindowChain": { + "child": 0, + "filter": null, + "input": { + "item": null, + "weight": { + "Column": "SampleValue" + } + }, + "reduction": "PerEntity", + "summary_type": { + "Sketch": "DDSketch" + }, + "tumbling_window_width_ms": 1000 } } + }, + "retention_ms": 300000, + "view_coverage": { + "definition_id": "0c7f962d8fde713c1de7a58752019151", + "selection": [ + { + "columns": [] + } + ] } }, { - "guarantee": null, - "id": 7, + "id": 2, "lifetime": "None", "output_schema": { "closed": true, "fields": [ + { + "dtype": { + "Plain": "timestamp" + }, + "name": "window_end", + "nullable": false + }, + { + "dtype": { + "Sketch": "DDSketch" + }, + "name": "state", + "nullable": false + }, { "dtype": { "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { - "ExactAggregate": [ - "Count", - "Count" - ] + "Plain": "utf8" }, - "name": "state", - "nullable": false, - "table": null + "name": "$promql_series_identity", + "nullable": false } ], - "time_index": null, + "time_index": 0, "unique_keys": [ [ - 0 + 0, + 2, + 3 ] ] }, @@ -12905,137 +653,127 @@ "primitive": "SummaryState", "timing": "query_time" }, + "parameters": { + "DDSketch": { + "alpha": 0.05 + } + }, "payload": { - "ASAP": { - "SummaryAgg": { - "child": 4, - "family": { - "ExactAggregate": [ - "Count", - "Count" - ] - }, - "filter": null, - "grouping": "PerSubpopulationInstance", - "input": { - "item": null, - "weight": { - "Column": "SampleValue" - } - }, - "reduction": { - "Reduce": [ - 2 - ] + "NonASAP": { + "TimeRange": { + "child": 1, + "kind": "range", + "range": { + "nanos": 0, + "secs": 300 } } } } }, { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "count of rows" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } + "id": 3, + "lifetime": { + "OneEvaluation": "Memory" }, - "id": 8, - "lifetime": "None", "output_schema": { "closed": true, "fields": [ + { + "dtype": { + "Plain": "timestamp" + }, + "name": "ts", + "nullable": false + }, + { + "dtype": { + "Sketch": "DDSketch" + }, + "name": "state", + "nullable": false + }, { "dtype": { "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { - "Plain": "int64" + "Plain": "utf8" }, - "name": "count", - "nullable": false, - "table": null + "name": "$promql_series_identity", + "nullable": false } ], - "time_index": null, + "time_index": 0, "unique_keys": [ [ + 3, 0 ] ] }, "output_state": { - "primitive": "Raw", + "primitive": "SummaryState", "timing": "query_time" }, + "parameters": { + "DDSketch": { + "alpha": 0.05 + } + }, "payload": { "ASAP": { - "FinalizeExactAccumulator": { - "child": 7 + "SummaryMerge": { + "children": [ + 2 + ] } } } }, { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact over exact inputs" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, - "id": 9, + "id": 4, "lifetime": "None", "output_schema": { "closed": true, "fields": [ + { + "dtype": { + "Plain": "timestamp" + }, + "name": "ts", + "nullable": false + }, + { + "dtype": { + "Plain": "float64" + }, + "name": "quantile", + "nullable": false + }, { "dtype": { "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { - "Plain": "float64" + "Plain": "utf8" }, - "name": "col_1", - "nullable": false, - "table": null + "name": "$promql_series_identity", + "nullable": false } ], - "time_index": null, + "time_index": 0, "unique_keys": [ [ + 3, 0 ] ] @@ -13045,168 +783,57 @@ "timing": "query_time" }, "payload": { - "NonASAP": { - "Project": { - "child": 8, - "cols": [ - { - "alias": null, - "expr": { - "Column": 0 - } - }, - { - "alias": null, - "expr": { - "Cast": { - "expr": { - "Column": 1 - }, - "to": "float64", - "try_cast": false - } - } + "ASAP": { + "SummaryEstimate": { + "query": { + "Quantile": { + "q": 0.95 } - ], - "qualifier": null + }, + "summary_input": 3 } } } }, { - "guarantee": { - "provenance": [ - { - "kind": "unavailable_statistic", - "statistic": "input_row_count" - }, - { - "kind": "unavailable_statistic", - "statistic": "values_non_negative[data]" - }, + "id": 5, + "lifetime": "None", + "output_schema": { + "closed": true, + "fields": [ { - "guarantee": { - "provenance": [ - { - "algorithm": "DDSketch", - "contract": "ddsketch_relative_error_alpha_v1", - "kind": "sketch_evaluation", - "params": { - "DDSketch": { - "alpha": 0.05 - } - }, - "query": "Quantile { q: 0.95 }" - } - ], - "spec": { - "accuracy_metric": "relative", - "accuracy_scope": "per_value", - "error": { - "factors": [ - { - "op": "constant", - "value": 0.05 - }, - { - "op": "unknown", - "statistic": "values_non_negative[data]" - } - ], - "op": "product" - }, - "failure_probability": { - "op": "zero" - } - } + "dtype": { + "Plain": "timestamp" }, - "input_index": 0, - "kind": "child_guarantee" - }, - { - "kind": "exact", - "reason": "exact aggregation" + "name": "ts", + "nullable": false }, { - "kind": "composition_step", - "operator": { - "op": "exact_sum" + "dtype": { + "Plain": "float64" }, - "rule": "exact_sum_relative" - } - ], - "spec": { - "accuracy_metric": "relative", - "accuracy_scope": "per_value", - "error": { - "factors": [ - { - "op": "max", - "terms": [ - { - "factors": [ - { - "op": "constant", - "value": 0.05 - }, - { - "op": "unknown", - "statistic": "values_non_negative[data]" - } - ], - "op": "product" - } - ] - }, - { - "op": "unknown", - "statistic": "values_non_negative[data]" - } - ], - "op": "product" + "name": "value", + "nullable": false }, - "failure_probability": { - "count": { - "op": "unknown", - "statistic": "input_row_count" - }, - "inner": { - "op": "union_bound", - "terms": [ - { - "op": "zero" - } - ] - }, - "op": "scaled" - } - } - }, - "id": 10, - "lifetime": "None", - "output_schema": { - "closed": true, - "fields": [ { "dtype": { "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { - "Plain": "float64" + "Plain": "utf8" }, - "name": "sum", - "nullable": false, - "table": null + "name": "$promql_series_identity", + "nullable": false } ], - "time_index": null, + "time_index": 0, "unique_keys": [ [ + 3, 0 ] ] @@ -13217,133 +844,43 @@ }, "payload": { "NonASAP": { - "BinaryOp": { - "lhs": 6, - "operator": { - "checked_finite_division": false, - "checked_relative_division": false, - "kind": { - "Arithmetic": "Div" - }, - "vector_match": null - }, - "return_bool": false, - "rhs": 9 - } - } - } - }, - { - "guarantee": { - "provenance": [ - { - "kind": "unavailable_statistic", - "statistic": "input_row_count" - }, - { - "kind": "unavailable_statistic", - "statistic": "values_non_negative[data]" - }, - { - "guarantee": { - "provenance": [ - { - "algorithm": "DDSketch", - "contract": "ddsketch_relative_error_alpha_v1", - "kind": "sketch_evaluation", - "params": { - "DDSketch": { - "alpha": 0.05 - } - }, - "query": "Quantile { q: 0.95 }" + "Project": { + "child": 4, + "cols": [ + { + "alias": "ts", + "expr": { + "Column": 0 } - ], - "spec": { - "accuracy_metric": "relative", - "accuracy_scope": "per_value", - "error": { - "factors": [ - { - "op": "constant", - "value": 0.05 - }, - { - "op": "unknown", - "statistic": "values_non_negative[data]" - } - ], - "op": "product" - }, - "failure_probability": { - "op": "zero" + }, + { + "alias": "value", + "expr": { + "Column": 1 } - } - }, - "input_index": 0, - "kind": "child_guarantee" - }, - { - "kind": "exact", - "reason": "exact aggregation" - }, - { - "kind": "composition_step", - "operator": { - "op": "exact_sum" - }, - "rule": "exact_sum_relative" - } - ], - "spec": { - "accuracy_metric": "relative", - "accuracy_scope": "per_value", - "error": { - "factors": [ + }, { - "op": "max", - "terms": [ - { - "factors": [ - { - "op": "constant", - "value": 0.05 - }, - { - "op": "unknown", - "statistic": "values_non_negative[data]" - } - ], - "op": "product" - } - ] + "alias": "region", + "expr": { + "Column": 2 + } }, { - "op": "unknown", - "statistic": "values_non_negative[data]" - } - ], - "op": "product" - }, - "failure_probability": { - "count": { - "op": "unknown", - "statistic": "input_row_count" - }, - "inner": { - "op": "union_bound", - "terms": [ - { - "op": "zero" + "alias": "$promql_series_identity", + "expr": { + "Column": 3 } - ] - }, - "op": "scaled" + } + ] } } + } + }, + { + "id": 6, + "lifetime": { + "OneEvaluation": "Memory" }, - "id": 11, - "lifetime": "None", "output_schema": { "closed": true, "fields": [ @@ -13352,16 +889,14 @@ "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "float64" }, "name": "avg", - "nullable": false, - "table": null + "nullable": false } ], "time_index": null, @@ -13377,36 +912,36 @@ }, "payload": { "NonASAP": { - "Project": { - "child": 10, - "cols": [ - { - "alias": "region", - "expr": { - "Column": 0 - } - }, + "Aggregate": { + "child": 5, + "filters": [], + "having": null, + "measures": [ { - "alias": "avg", - "expr": { - "Column": 1 - } + "col": null, + "kind": "avg" } ], - "qualifier": null + "output_names": [ + "" + ], + "reduction": { + "Reduce": [ + 2 + ] + } } } } } ], "roots": [ - 11 + 6 ], "scalar_roots": [] - }, - "display_name": "Q1 Sum acc + DDSketch ×300 tumbling windows + Count acc · ingestion time: DDSketch for Q1, 300 × 1s tumbling windows" + } }, - "tumbling_window_retention_ms": { - "1": 300000 - } + "warnings": [ + "assumed data fact: source data declares no column_magnitudes for value, so a DDSketch state over it is costed for magnitudes in [1, 1000000000]; this affects cost only, the accuracy guarantee does not depend on it" + ] } diff --git a/crates/devtools/tests/sketch_plan_snapshots/random-mixed-seed-862-w01-q3-q8-mvp.json b/crates/devtools/tests/sketch_plan_snapshots/random-mixed-seed-862-w01-q3-q8-mvp.json index 767b3e159..1ee0b5db6 100644 --- a/crates/devtools/tests/sketch_plan_snapshots/random-mixed-seed-862-w01-q3-q8-mvp.json +++ b/crates/devtools/tests/sketch_plan_snapshots/random-mixed-seed-862-w01-q3-q8-mvp.json @@ -3,6137 +3,192 @@ "materialized_views": [ { "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", + "definition_id": "0c7f962d8fde713c1de7a58752019151", "selection": [ { - "columns": [], - "relative_time": [ - { - "excluded": -1000 - }, - { - "included": 0 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 0, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 0, - "node_id": 2, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2000 - }, - { - "included": -1000 - } - ] + "columns": [] } ] }, "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] + "materialized_views": [] }, - "installed_count": 100, - "installed_position": 1, - "node_id": 5, + "node_id": 1, "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 + "retention_ms": 100000 }, { "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", + "definition_id": "96705e10b00e7f916542fec1da8dcdb6", "selection": [ { - "columns": [], - "relative_time": [ - { - "excluded": -3000 - }, - { - "included": -2000 - } - ] + "columns": [] } ] }, "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] + "materialized_views": [] }, - "installed_count": 100, - "installed_position": 2, "node_id": 8, "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -4000 - }, - { - "included": -3000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 3, - "node_id": 11, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -5000 - }, - { - "included": -4000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 4000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 4, - "node_id": 14, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -6000 - }, - { - "included": -5000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 5000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 5, - "node_id": 17, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -7000 - }, - { - "included": -6000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 6000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 6, - "node_id": 20, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -8000 - }, - { - "included": -7000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 7000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 7, - "node_id": 23, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -9000 - }, - { - "included": -8000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 8000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 8, - "node_id": 26, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -10000 - }, - { - "included": -9000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 9000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 9, - "node_id": 29, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -11000 - }, - { - "included": -10000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 10000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 10, - "node_id": 32, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -12000 - }, - { - "included": -11000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 11000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 11, - "node_id": 35, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -13000 - }, - { - "included": -12000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 12000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] + "retention_ms": 1200000 + } + ], + "queries": [ + "avg by (region) (quantile_over_time(0.95, data[100s]))", + "topk by (region) (4, rate(data_total[20m]))" + ], + "selected": { + "accepted": [ + { + "accepted_metric": "relative", + "sketches": [ + "DDSketch" + ], + "summary": "accepted relative (DDSketch)" }, - "installed_count": 100, - "installed_position": 12, - "node_id": 38, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -14000 - }, - { - "included": -13000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 13000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 13, - "node_id": 41, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -15000 - }, - { - "included": -14000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 14000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 14, - "node_id": 44, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -16000 - }, - { - "included": -15000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 15000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 15, - "node_id": 47, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -17000 - }, - { - "included": -16000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 16000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 16, - "node_id": 50, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -18000 - }, - { - "included": -17000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 17000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 17, - "node_id": 53, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -19000 - }, - { - "included": -18000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 18000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 18, - "node_id": 56, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -20000 - }, - { - "included": -19000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 19000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 19, - "node_id": 59, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -21000 - }, - { - "included": -20000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 20000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 20, - "node_id": 62, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -22000 - }, - { - "included": -21000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 21000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 21, - "node_id": 65, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -23000 - }, - { - "included": -22000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 22000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 22, - "node_id": 68, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -24000 - }, - { - "included": -23000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 23000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 23, - "node_id": 71, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -25000 - }, - { - "included": -24000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 24000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 24, - "node_id": 74, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -26000 - }, - { - "included": -25000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 25000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 25, - "node_id": 77, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -27000 - }, - { - "included": -26000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 26000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 26, - "node_id": 80, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -28000 - }, - { - "included": -27000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 27000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 27, - "node_id": 83, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -29000 - }, - { - "included": -28000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 28000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 28, - "node_id": 86, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -30000 - }, - { - "included": -29000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 29000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 29, - "node_id": 89, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -31000 - }, - { - "included": -30000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 30000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 30, - "node_id": 92, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -32000 - }, - { - "included": -31000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 31000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 31, - "node_id": 95, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -33000 - }, - { - "included": -32000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 32000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 32, - "node_id": 98, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -34000 - }, - { - "included": -33000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 33000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 33, - "node_id": 101, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -35000 - }, - { - "included": -34000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 34000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 34, - "node_id": 104, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -36000 - }, - { - "included": -35000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 35000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 35, - "node_id": 107, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -37000 - }, - { - "included": -36000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 36000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 36, - "node_id": 110, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -38000 - }, - { - "included": -37000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 37000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 37, - "node_id": 113, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -39000 - }, - { - "included": -38000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 38000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 38, - "node_id": 116, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -40000 - }, - { - "included": -39000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 39000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 39, - "node_id": 119, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -41000 - }, - { - "included": -40000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 40000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 40, - "node_id": 122, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -42000 - }, - { - "included": -41000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 41000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 41, - "node_id": 125, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -43000 - }, - { - "included": -42000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 42000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 42, - "node_id": 128, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -44000 - }, - { - "included": -43000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 43000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 43, - "node_id": 131, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -45000 - }, - { - "included": -44000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 44000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 44, - "node_id": 134, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -46000 - }, - { - "included": -45000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 45000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 45, - "node_id": 137, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -47000 - }, - { - "included": -46000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 46000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 46, - "node_id": 140, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -48000 - }, - { - "included": -47000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 47000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 47, - "node_id": 143, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -49000 - }, - { - "included": -48000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 48000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 48, - "node_id": 146, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -50000 - }, - { - "included": -49000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 49000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 49, - "node_id": 149, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -51000 - }, - { - "included": -50000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 50000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 50, - "node_id": 152, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -52000 - }, - { - "included": -51000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 51000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 51, - "node_id": 155, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -53000 - }, - { - "included": -52000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 52000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 52, - "node_id": 158, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -54000 - }, - { - "included": -53000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 53000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 53, - "node_id": 161, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -55000 - }, - { - "included": -54000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 54000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 54, - "node_id": 164, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -56000 - }, - { - "included": -55000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 55000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 55, - "node_id": 167, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -57000 - }, - { - "included": -56000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 56000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 56, - "node_id": 170, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -58000 - }, - { - "included": -57000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 57000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 57, - "node_id": 173, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -59000 - }, - { - "included": -58000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 58000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 58, - "node_id": 176, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -60000 - }, - { - "included": -59000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 59000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 59, - "node_id": 179, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -61000 - }, - { - "included": -60000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 60000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 60, - "node_id": 182, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -62000 - }, - { - "included": -61000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 61000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 61, - "node_id": 185, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -63000 - }, - { - "included": -62000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 62000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 62, - "node_id": 188, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -64000 - }, - { - "included": -63000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 63000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 63, - "node_id": 191, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -65000 - }, - { - "included": -64000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 64000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 64, - "node_id": 194, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -66000 - }, - { - "included": -65000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 65000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 65, - "node_id": 197, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -67000 - }, - { - "included": -66000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 66000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 66, - "node_id": 200, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -68000 - }, - { - "included": -67000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 67000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 67, - "node_id": 203, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -69000 - }, - { - "included": -68000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 68000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 68, - "node_id": 206, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -70000 - }, - { - "included": -69000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 69000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 69, - "node_id": 209, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -71000 - }, - { - "included": -70000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 70000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 70, - "node_id": 212, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -72000 - }, - { - "included": -71000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 71000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 71, - "node_id": 215, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -73000 - }, - { - "included": -72000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 72000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 72, - "node_id": 218, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -74000 - }, - { - "included": -73000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 73000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 73, - "node_id": 221, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -75000 - }, - { - "included": -74000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 74000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 74, - "node_id": 224, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -76000 - }, - { - "included": -75000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 75000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 75, - "node_id": 227, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -77000 - }, - { - "included": -76000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 76000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 76, - "node_id": 230, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -78000 - }, - { - "included": -77000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 77000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 77, - "node_id": 233, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -79000 - }, - { - "included": -78000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 78000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 78, - "node_id": 236, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -80000 - }, - { - "included": -79000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 79000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 79, - "node_id": 239, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -81000 - }, - { - "included": -80000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 80000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 80, - "node_id": 242, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -82000 - }, - { - "included": -81000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 81000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 81, - "node_id": 245, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -83000 - }, - { - "included": -82000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 82000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 82, - "node_id": 248, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -84000 - }, - { - "included": -83000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 83000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 83, - "node_id": 251, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -85000 - }, - { - "included": -84000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 84000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 84, - "node_id": 254, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -86000 - }, - { - "included": -85000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 85000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 85, - "node_id": 257, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -87000 - }, - { - "included": -86000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 86000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 86, - "node_id": 260, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -88000 - }, - { - "included": -87000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 87000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 87, - "node_id": 263, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -89000 - }, - { - "included": -88000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 88000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 88, - "node_id": 266, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -90000 - }, - { - "included": -89000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 89000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 89, - "node_id": 269, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -91000 - }, - { - "included": -90000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 90000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 90, - "node_id": 272, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -92000 - }, - { - "included": -91000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 91000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 91, - "node_id": 275, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -93000 - }, - { - "included": -92000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 92000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 92, - "node_id": 278, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -94000 - }, - { - "included": -93000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 93000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 93, - "node_id": 281, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -95000 - }, - { - "included": -94000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 94000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 94, - "node_id": 284, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -96000 - }, - { - "included": -95000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 95000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 95, - "node_id": 287, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -97000 - }, - { - "included": -96000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 96000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 96, - "node_id": 290, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -98000 - }, - { - "included": -97000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 97000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 97, - "node_id": 293, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -99000 - }, - { - "included": -98000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 98000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 98, - "node_id": 296, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -100000 - }, - { - "included": -99000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 99000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 99, - "node_id": 299, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "6525fb249a1d74cec1417d0b3fc3ae5a", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -30000 - }, - { - "included": 0 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 0, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 30000 - } - ] - }, - "installed_count": 40, - "installed_position": 0, - "node_id": 312, - "refresh": "at_ingestion", - "retention_ms": 1200000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "6525fb249a1d74cec1417d0b3fc3ae5a", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -60000 - }, - { - "included": -30000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 30000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 30000 - } - ] - }, - "installed_count": 40, - "installed_position": 1, - "node_id": 315, - "refresh": "at_ingestion", - "retention_ms": 1200000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "6525fb249a1d74cec1417d0b3fc3ae5a", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -90000 - }, - { - "included": -60000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 60000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 30000 - } - ] - }, - "installed_count": 40, - "installed_position": 2, - "node_id": 318, - "refresh": "at_ingestion", - "retention_ms": 1200000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "6525fb249a1d74cec1417d0b3fc3ae5a", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -120000 - }, - { - "included": -90000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 90000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 30000 - } - ] - }, - "installed_count": 40, - "installed_position": 3, - "node_id": 321, - "refresh": "at_ingestion", - "retention_ms": 1200000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "6525fb249a1d74cec1417d0b3fc3ae5a", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -150000 - }, - { - "included": -120000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 120000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 30000 - } - ] - }, - "installed_count": 40, - "installed_position": 4, - "node_id": 324, - "refresh": "at_ingestion", - "retention_ms": 1200000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "6525fb249a1d74cec1417d0b3fc3ae5a", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -180000 - }, - { - "included": -150000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 150000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 30000 - } - ] - }, - "installed_count": 40, - "installed_position": 5, - "node_id": 327, - "refresh": "at_ingestion", - "retention_ms": 1200000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "6525fb249a1d74cec1417d0b3fc3ae5a", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -210000 - }, - { - "included": -180000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 180000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 30000 - } - ] - }, - "installed_count": 40, - "installed_position": 6, - "node_id": 330, - "refresh": "at_ingestion", - "retention_ms": 1200000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "6525fb249a1d74cec1417d0b3fc3ae5a", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -240000 - }, - { - "included": -210000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 210000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 30000 - } - ] - }, - "installed_count": 40, - "installed_position": 7, - "node_id": 333, - "refresh": "at_ingestion", - "retention_ms": 1200000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "6525fb249a1d74cec1417d0b3fc3ae5a", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -270000 - }, - { - "included": -240000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 240000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 30000 - } - ] - }, - "installed_count": 40, - "installed_position": 8, - "node_id": 336, - "refresh": "at_ingestion", - "retention_ms": 1200000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "6525fb249a1d74cec1417d0b3fc3ae5a", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -300000 - }, - { - "included": -270000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 270000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 30000 - } - ] - }, - "installed_count": 40, - "installed_position": 9, - "node_id": 339, - "refresh": "at_ingestion", - "retention_ms": 1200000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "6525fb249a1d74cec1417d0b3fc3ae5a", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -330000 - }, - { - "included": -300000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 300000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 30000 - } - ] - }, - "installed_count": 40, - "installed_position": 10, - "node_id": 342, - "refresh": "at_ingestion", - "retention_ms": 1200000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "6525fb249a1d74cec1417d0b3fc3ae5a", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -360000 - }, - { - "included": -330000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 330000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 30000 - } - ] - }, - "installed_count": 40, - "installed_position": 11, - "node_id": 345, - "refresh": "at_ingestion", - "retention_ms": 1200000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "6525fb249a1d74cec1417d0b3fc3ae5a", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -390000 - }, - { - "included": -360000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 360000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 30000 - } - ] - }, - "installed_count": 40, - "installed_position": 12, - "node_id": 348, - "refresh": "at_ingestion", - "retention_ms": 1200000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "6525fb249a1d74cec1417d0b3fc3ae5a", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -420000 - }, - { - "included": -390000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 390000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 30000 - } - ] - }, - "installed_count": 40, - "installed_position": 13, - "node_id": 351, - "refresh": "at_ingestion", - "retention_ms": 1200000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "6525fb249a1d74cec1417d0b3fc3ae5a", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -450000 - }, - { - "included": -420000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 420000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 30000 - } - ] - }, - "installed_count": 40, - "installed_position": 14, - "node_id": 354, - "refresh": "at_ingestion", - "retention_ms": 1200000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "6525fb249a1d74cec1417d0b3fc3ae5a", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -480000 - }, - { - "included": -450000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 450000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 30000 - } - ] - }, - "installed_count": 40, - "installed_position": 15, - "node_id": 357, - "refresh": "at_ingestion", - "retention_ms": 1200000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "6525fb249a1d74cec1417d0b3fc3ae5a", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -510000 - }, - { - "included": -480000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 480000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 30000 - } - ] - }, - "installed_count": 40, - "installed_position": 16, - "node_id": 360, - "refresh": "at_ingestion", - "retention_ms": 1200000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "6525fb249a1d74cec1417d0b3fc3ae5a", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -540000 - }, - { - "included": -510000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 510000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 30000 - } - ] - }, - "installed_count": 40, - "installed_position": 17, - "node_id": 363, - "refresh": "at_ingestion", - "retention_ms": 1200000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "6525fb249a1d74cec1417d0b3fc3ae5a", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -570000 - }, - { - "included": -540000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 540000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 30000 - } - ] - }, - "installed_count": 40, - "installed_position": 18, - "node_id": 366, - "refresh": "at_ingestion", - "retention_ms": 1200000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "6525fb249a1d74cec1417d0b3fc3ae5a", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -600000 - }, - { - "included": -570000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 570000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 30000 - } - ] - }, - "installed_count": 40, - "installed_position": 19, - "node_id": 369, - "refresh": "at_ingestion", - "retention_ms": 1200000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "6525fb249a1d74cec1417d0b3fc3ae5a", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -630000 - }, - { - "included": -600000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 600000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 30000 - } - ] - }, - "installed_count": 40, - "installed_position": 20, - "node_id": 372, - "refresh": "at_ingestion", - "retention_ms": 1200000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "6525fb249a1d74cec1417d0b3fc3ae5a", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -660000 - }, - { - "included": -630000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 630000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 30000 - } - ] - }, - "installed_count": 40, - "installed_position": 21, - "node_id": 375, - "refresh": "at_ingestion", - "retention_ms": 1200000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "6525fb249a1d74cec1417d0b3fc3ae5a", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -690000 - }, - { - "included": -660000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 660000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 30000 - } - ] - }, - "installed_count": 40, - "installed_position": 22, - "node_id": 378, - "refresh": "at_ingestion", - "retention_ms": 1200000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "6525fb249a1d74cec1417d0b3fc3ae5a", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -720000 - }, - { - "included": -690000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 690000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 30000 - } - ] - }, - "installed_count": 40, - "installed_position": 23, - "node_id": 381, - "refresh": "at_ingestion", - "retention_ms": 1200000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "6525fb249a1d74cec1417d0b3fc3ae5a", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -750000 - }, - { - "included": -720000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 720000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 30000 - } - ] - }, - "installed_count": 40, - "installed_position": 24, - "node_id": 384, - "refresh": "at_ingestion", - "retention_ms": 1200000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "6525fb249a1d74cec1417d0b3fc3ae5a", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -780000 - }, - { - "included": -750000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 750000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 30000 - } - ] - }, - "installed_count": 40, - "installed_position": 25, - "node_id": 387, - "refresh": "at_ingestion", - "retention_ms": 1200000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "6525fb249a1d74cec1417d0b3fc3ae5a", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -810000 - }, - { - "included": -780000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 780000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 30000 - } - ] - }, - "installed_count": 40, - "installed_position": 26, - "node_id": 390, - "refresh": "at_ingestion", - "retention_ms": 1200000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "6525fb249a1d74cec1417d0b3fc3ae5a", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -840000 - }, - { - "included": -810000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 810000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 30000 - } - ] - }, - "installed_count": 40, - "installed_position": 27, - "node_id": 393, - "refresh": "at_ingestion", - "retention_ms": 1200000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "6525fb249a1d74cec1417d0b3fc3ae5a", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -870000 - }, - { - "included": -840000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 840000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 30000 - } - ] - }, - "installed_count": 40, - "installed_position": 28, - "node_id": 396, - "refresh": "at_ingestion", - "retention_ms": 1200000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "6525fb249a1d74cec1417d0b3fc3ae5a", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -900000 - }, - { - "included": -870000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 870000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 30000 - } - ] - }, - "installed_count": 40, - "installed_position": 29, - "node_id": 399, - "refresh": "at_ingestion", - "retention_ms": 1200000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "6525fb249a1d74cec1417d0b3fc3ae5a", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -930000 - }, - { - "included": -900000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 900000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 30000 - } - ] - }, - "installed_count": 40, - "installed_position": 30, - "node_id": 402, - "refresh": "at_ingestion", - "retention_ms": 1200000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "6525fb249a1d74cec1417d0b3fc3ae5a", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -960000 - }, - { - "included": -930000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 930000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 30000 - } - ] - }, - "installed_count": 40, - "installed_position": 31, - "node_id": 405, - "refresh": "at_ingestion", - "retention_ms": 1200000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "6525fb249a1d74cec1417d0b3fc3ae5a", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -990000 - }, - { - "included": -960000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 960000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 30000 - } - ] - }, - "installed_count": 40, - "installed_position": 32, - "node_id": 408, - "refresh": "at_ingestion", - "retention_ms": 1200000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "6525fb249a1d74cec1417d0b3fc3ae5a", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1020000 - }, - { - "included": -990000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 990000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 30000 - } - ] - }, - "installed_count": 40, - "installed_position": 33, - "node_id": 411, - "refresh": "at_ingestion", - "retention_ms": 1200000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "6525fb249a1d74cec1417d0b3fc3ae5a", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1050000 - }, - { - "included": -1020000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1020000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 30000 - } - ] - }, - "installed_count": 40, - "installed_position": 34, - "node_id": 414, - "refresh": "at_ingestion", - "retention_ms": 1200000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "6525fb249a1d74cec1417d0b3fc3ae5a", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1080000 - }, - { - "included": -1050000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1050000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 30000 - } - ] - }, - "installed_count": 40, - "installed_position": 35, - "node_id": 417, - "refresh": "at_ingestion", - "retention_ms": 1200000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "6525fb249a1d74cec1417d0b3fc3ae5a", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1110000 - }, - { - "included": -1080000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1080000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 30000 - } - ] - }, - "installed_count": 40, - "installed_position": 36, - "node_id": 420, - "refresh": "at_ingestion", - "retention_ms": 1200000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "6525fb249a1d74cec1417d0b3fc3ae5a", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1140000 - }, - { - "included": -1110000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1110000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 30000 - } - ] - }, - "installed_count": 40, - "installed_position": 37, - "node_id": 423, - "refresh": "at_ingestion", - "retention_ms": 1200000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "6525fb249a1d74cec1417d0b3fc3ae5a", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1170000 - }, - { - "included": -1140000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1140000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 30000 - } - ] - }, - "installed_count": 40, - "installed_position": 38, - "node_id": 426, - "refresh": "at_ingestion", - "retention_ms": 1200000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "6525fb249a1d74cec1417d0b3fc3ae5a", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1200000 - }, - { - "included": -1170000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1170000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 30000 - } - ] - }, - "installed_count": 40, - "installed_position": 39, - "node_id": 429, - "refresh": "at_ingestion", - "retention_ms": 1200000, - "stage2_node_id": 13 - } - ], - "proven_optimal": true, - "queries": [ - "avg by (region) (quantile_over_time(0.95, data[100s]))", - "topk by (region) (4, rate(data_total[20m]))" - ], - "search_status": "optimal", - "selected": { - "accepted": [ - { - "accepted_metric": "relative", - "accepted_regardless_of_guarantee": false, - "guarantee_check_rejection": null, - "sketches": [ - "DDSketch" - ], - "summary": "metric not set; accepted relative (DDSketch)", - "target_metric": null - }, - { - "accepted_metric": null, - "accepted_regardless_of_guarantee": false, - "guarantee_check_rejection": null, - "sketches": [], - "summary": "metric not set; accepted exact", - "target_metric": null - } - ], - "cost": { - "cost_per_second": 6.43176839059e-6, - "cost_unit": "cost_per_second", - "cpu_core_seconds_per_second": 0.00266466666667, - "per_node": { - "0": { - "cost_per_second": 6.52152e-9, - "cpu_core_seconds_per_second": 0.00058, - "detail": "scan 100000 samples; x 1.0000/s", - "multiplier": 1, - "operator": "Scan", - "price_source": "analytical", - "timing": "ingestion_time" - }, - "1": { - "cost_per_second": 6.3775772514e-6, - "cpu_core_seconds_per_second": 0.0002, - "detail": "build DDSketch in 100 tumbling windows of 1000 ms for 10000 groups: 100000 rows x (1 + depth 1); x 1.0000/s; keeps 5542880000 bytes", - "multiplier": 1, - "operator": "SummaryTumblingWindowChain", - "price_source": "analytical", - "timing": "ingestion_time" - }, - "10": { - "cost_per_second": 1.12440030089e-10, - "cpu_core_seconds_per_second": 0.00001, - "detail": "operator: pass 10000 rows; x 1.0000/s; holds 24 bytes for 1.1 ms", - "multiplier": 1, - "operator": "BinaryOp", - "price_source": "analytical", - "timing": "query_time" - }, - "11": { - "cost_per_second": 1.12440030089e-10, - "cpu_core_seconds_per_second": 0.00001, - "detail": "operator: pass 10000 rows; x 1.0000/s; holds 24 bytes for 1.1 ms", - "multiplier": 1, - "operator": "Project", - "price_source": "analytical", - "timing": "query_time" - }, - "12": { - "cost_per_second": 6.52152e-9, - "cpu_core_seconds_per_second": 0.00058, - "detail": "scan 100000 samples; x 1.0000/s", - "multiplier": 1, - "operator": "Scan", - "price_source": "analytical", - "timing": "ingestion_time" - }, - "13": { - "cost_per_second": 2.86570103968e-8, - "cpu_core_seconds_per_second": 0.0002, - "detail": "build exact Rate accumulator in 40 tumbling windows of 30000 ms for 10000 groups: 100000 rows x (1 + depth 1); x 1.0000/s; keeps 22960000 bytes", - "multiplier": 1, - "operator": "SummaryTumblingWindowChain", - "price_source": "analytical", - "timing": "ingestion_time" - }, - "14": { - "cost_per_second": 1.49929446839e-10, - "cpu_core_seconds_per_second": 0.0000133333333333, - "detail": "merge 400000 states into 10000; x 0.0333/s; holds 560000 bytes for 0.4 ms", - "multiplier": 1, - "operator": "SummaryMerge", - "price_source": "analytical", - "timing": "query_time" - }, - "15": { - "cost_per_second": 3.748e-12, - "cpu_core_seconds_per_second": 3.33333333333e-7, - "detail": "finalize 10000 accumulators; x 0.0333/s", - "multiplier": 1, - "operator": "FinalizeExactAccumulator", - "price_source": "analytical", - "timing": "query_time" - }, - "16": { - "cost_per_second": 3.74800080973e-12, - "cpu_core_seconds_per_second": 3.33333333333e-7, - "detail": "sort 10000 rows in 10000 partitions; x 0.0333/s; holds 48 bytes for 0.4 ms", - "multiplier": 1, - "operator": "Sort", - "price_source": "analytical", - "timing": "query_time" - }, - "17": { - "cost_per_second": 3.74800080973e-12, - "cpu_core_seconds_per_second": 3.33333333333e-7, - "detail": "limit to 10000 rows; x 0.0333/s; holds 48 bytes for 0.4 ms", - "multiplier": 1, - "operator": "Limit", - "price_source": "analytical", - "timing": "query_time" - }, - "18": { - "cost_per_second": 3.74800080973e-12, - "cpu_core_seconds_per_second": 3.33333333333e-7, - "detail": "operator: pass 10000 rows; x 0.0333/s; holds 48 bytes for 0.4 ms", - "multiplier": 1, - "operator": "Project", - "price_source": "analytical", - "timing": "query_time" - }, - "2": { - "cost_per_second": 1.13128030496e-8, - "cpu_core_seconds_per_second": 0.001, - "detail": "merge 1000000 states into 10000; x 1.0000/s; holds 54880000 bytes for 1.1 ms", - "multiplier": 1, - "operator": "SummaryMerge", - "price_source": "analytical", - "timing": "query_time" - }, - "3": { - "cost_per_second": 1.1244e-10, - "cpu_core_seconds_per_second": 0.00001, - "detail": "estimate 10000 groups x 1; x 1.0000/s", - "multiplier": 1, - "operator": "SummaryEstimate", - "price_source": "analytical", - "timing": "query_time" - }, - "4": { - "cost_per_second": 1.12440060178e-10, - "cpu_core_seconds_per_second": 0.00001, - "detail": "operator: pass 10000 rows; x 1.0000/s; holds 48 bytes for 1.1 ms", - "multiplier": 1, - "operator": "Project", - "price_source": "analytical", - "timing": "query_time" - }, - "5": { - "cost_per_second": 1.13142071935e-10, - "cpu_core_seconds_per_second": 0.00001, - "detail": "build exact Sum accumulator into 10000 states: 10000 rows x depth 1; x 1.0000/s; holds 560000 bytes for 1.1 ms", - "multiplier": 1, - "operator": "SummaryAgg", - "price_source": "analytical", - "timing": "query_time" - }, - "6": { - "cost_per_second": 1.1244e-10, - "cpu_core_seconds_per_second": 0.00001, - "detail": "finalize 10000 accumulators; x 1.0000/s", - "multiplier": 1, - "operator": "FinalizeExactAccumulator", - "price_source": "analytical", - "timing": "query_time" - }, - "7": { - "cost_per_second": 1.13142071935e-10, - "cpu_core_seconds_per_second": 0.00001, - "detail": "build exact Count accumulator into 10000 states: 10000 rows x depth 1; x 1.0000/s; holds 560000 bytes for 1.1 ms", - "multiplier": 1, - "operator": "SummaryAgg", - "price_source": "analytical", - "timing": "query_time" - }, - "8": { - "cost_per_second": 1.1244e-10, - "cpu_core_seconds_per_second": 0.00001, - "detail": "finalize 10000 accumulators; x 1.0000/s", - "multiplier": 1, - "operator": "FinalizeExactAccumulator", - "price_source": "analytical", - "timing": "query_time" - }, - "9": { - "cost_per_second": 1.12440030089e-10, - "cpu_core_seconds_per_second": 0.00001, - "detail": "operator: pass 10000 rows; x 1.0000/s; holds 24 bytes for 1.1 ms", - "multiplier": 1, - "operator": "Project", - "price_source": "analytical", - "timing": "query_time" - } - }, - "source": "analytical-cost-v2 (illustrative statistics)", - "usage": { - "average_memory_bytes": 5565901048.35, - "cpu_ops_per_second": 1704666.66667, - "disk_bytes": 0, - "latency_per_query": [ - { - "compaction_ms": 0.0, - "query_ms": 1.09 - }, - { - "compaction_ms": 0.0, - "query_ms": 0.44 - } - ], - "peak_cpu_cores": 3, - "peak_memory_bytes": 5622400360, - "scanned_bytes_per_second": 9600000.0 - } - }, - "dag": { - "edges": [ - { - "consumer": 1, - "data_state": { - "primitive": "Raw", - "timing": "ingestion_time" - }, - "grouping": "NotApplicable", - "intermediate_schema": { - "closed": true, - "fields": [ - { - "dtype": { - "Plain": "timestamp" - }, - "name": "ts", - "nullable": false, - "table": null - }, - { - "dtype": { - "Plain": "float64" - }, - "name": "value", - "nullable": false, - "table": null - }, - { - "dtype": { - "Plain": "utf8" - }, - "name": "region", - "nullable": true, - "table": null - }, - { - "dtype": { - "Plain": "utf8" - }, - "name": "$promql_series_identity", - "nullable": false, - "table": null - } - ], - "time_index": 0, - "unique_keys": [ - [ - 3, - 0 - ] - ] - }, - "producer": 0, - "role": "Input", - "window": "RequiresAlignedPanePhaseOrExactBoundaryResidual" - }, - { - "consumer": 2, - "data_state": { - "primitive": "SummaryState", - "timing": "ingestion_time" - }, - "grouping": "NotApplicable", - "intermediate_schema": { - "closed": true, - "fields": [ - { - "dtype": { - "Plain": "timestamp" - }, - "name": "ts", - "nullable": false, - "table": null - }, - { - "dtype": { - "Sketch": [ - { - "algorithm": "DDSketch", - "params": { - "DDSketch": { - "alpha": 0.05 - } - } - }, - "PerSubpopulationInstance" - ] - }, - "name": "state", - "nullable": false, - "table": null - }, - { - "dtype": { - "Plain": "utf8" - }, - "name": "region", - "nullable": true, - "table": null - }, - { - "dtype": { - "Plain": "utf8" - }, - "name": "$promql_series_identity", - "nullable": false, - "table": null - } - ], - "time_index": 0, - "unique_keys": [ - [ - 3, - 0 - ] - ] - }, - "producer": 1, - "role": "Input", - "window": "NotApplicable" - }, - { - "consumer": 3, - "data_state": { - "primitive": "SummaryState", - "timing": "query_time" - }, - "grouping": "NotApplicable", - "intermediate_schema": { - "closed": true, - "fields": [ - { - "dtype": { - "Plain": "timestamp" - }, - "name": "ts", - "nullable": false, - "table": null - }, - { - "dtype": { - "Sketch": [ - { - "algorithm": "DDSketch", - "params": { - "DDSketch": { - "alpha": 0.05 - } - } - }, - "PerSubpopulationInstance" - ] - }, - "name": "state", - "nullable": false, - "table": null - }, - { - "dtype": { - "Plain": "utf8" - }, - "name": "region", - "nullable": true, - "table": null - }, - { - "dtype": { - "Plain": "utf8" - }, - "name": "$promql_series_identity", - "nullable": false, - "table": null - } - ], - "time_index": 0, - "unique_keys": [ - [ - 3, - 0 - ] - ] - }, - "producer": 2, - "role": "Input", - "window": "NotApplicable" - }, - { - "consumer": 4, - "data_state": { - "primitive": "Raw", - "timing": "query_time" - }, - "grouping": "NotApplicable", - "intermediate_schema": { - "closed": true, - "fields": [ - { - "dtype": { - "Plain": "timestamp" - }, - "name": "ts", - "nullable": false, - "table": null - }, - { - "dtype": { - "Plain": "float64" - }, - "name": "quantile", - "nullable": false, - "table": null - }, - { - "dtype": { - "Plain": "utf8" - }, - "name": "region", - "nullable": true, - "table": null - }, - { - "dtype": { - "Plain": "utf8" - }, - "name": "$promql_series_identity", - "nullable": false, - "table": null - } - ], - "time_index": 0, - "unique_keys": [ - [ - 3, - 0 - ] - ] - }, - "producer": 3, - "role": "Input", - "window": "NotApplicable" - }, - { - "consumer": 5, - "data_state": { - "primitive": "Raw", - "timing": "query_time" - }, - "grouping": "NotApplicable", - "intermediate_schema": { - "closed": true, - "fields": [ - { - "dtype": { - "Plain": "timestamp" - }, - "name": "ts", - "nullable": false, - "table": null - }, - { - "dtype": { - "Plain": "float64" - }, - "name": "value", - "nullable": false, - "table": null - }, - { - "dtype": { - "Plain": "utf8" - }, - "name": "region", - "nullable": true, - "table": null - }, - { - "dtype": { - "Plain": "utf8" - }, - "name": "$promql_series_identity", - "nullable": false, - "table": null - } - ], - "time_index": 0, - "unique_keys": [ - [ - 3, - 0 - ] - ] - }, - "producer": 4, - "role": "Input", - "window": "NotApplicable" - }, - { - "consumer": 6, - "data_state": { - "primitive": "SummaryState", - "timing": "query_time" - }, - "grouping": "NotApplicable", - "intermediate_schema": { - "closed": true, - "fields": [ - { - "dtype": { - "Plain": "utf8" - }, - "name": "region", - "nullable": true, - "table": null - }, - { - "dtype": { - "ExactAggregate": [ - "Sum", - "Sum" - ] - }, - "name": "state", - "nullable": false, - "table": null - } - ], - "time_index": null, - "unique_keys": [ - [ - 0 - ] - ] - }, - "producer": 5, - "role": "Input", - "window": "NotApplicable" + { + "accepted_metric": null, + "sketches": [], + "summary": "accepted exact" + } + ], + "cost": { + "cost_per_second": 0.0000317703733544, + "cpu_core_seconds_per_second": 2.65234166667, + "per_node": { + "0": { + "cost_per_second": 6.52152e-9, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.00058, + "detail": "scan 100000 samples; x 1.0000/s", + "multiplier": 1, + "operator": "Scan", + "timing": "ingestion_time" }, - { - "consumer": 7, - "data_state": { - "primitive": "Raw", - "timing": "query_time" - }, - "grouping": "NotApplicable", - "intermediate_schema": { - "closed": true, - "fields": [ - { - "dtype": { - "Plain": "timestamp" - }, - "name": "ts", - "nullable": false, - "table": null - }, - { - "dtype": { - "Plain": "float64" - }, - "name": "value", - "nullable": false, - "table": null - }, - { - "dtype": { - "Plain": "utf8" - }, - "name": "region", - "nullable": true, - "table": null - }, - { - "dtype": { - "Plain": "utf8" - }, - "name": "$promql_series_identity", - "nullable": false, - "table": null - } - ], - "time_index": 0, - "unique_keys": [ - [ - 3, - 0 - ] - ] - }, - "producer": 4, - "role": "Input", - "window": "NotApplicable" + "1": { + "cost_per_second": 2.39800229468e-6, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.05127, + "detail": "build DDSketch in tumbling windows of 1000 ms for 10000 groups: 100000 rows x 247 ops, 1000 group visits, 10000.0 states emitted; x 1.0000/s; keeps 1583680000 bytes", + "multiplier": 1, + "operator": "SummaryTumblingWindowChain", + "timing": "ingestion_time" }, - { - "consumer": 8, - "data_state": { - "primitive": "SummaryState", - "timing": "query_time" - }, - "grouping": "NotApplicable", - "intermediate_schema": { - "closed": true, - "fields": [ - { - "dtype": { - "Plain": "utf8" - }, - "name": "region", - "nullable": true, - "table": null - }, - { - "dtype": { - "ExactAggregate": [ - "Count", - "Count" - ] - }, - "name": "state", - "nullable": false, - "table": null - } - ], - "time_index": null, - "unique_keys": [ - [ - 0 - ] - ] - }, - "producer": 7, - "role": "Input", - "window": "NotApplicable" + "10": { + "cost_per_second": 5.66031365291e-8, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.00503333333333, + "detail": "merge 400000 states into 10000; x 0.0333/s; holds 1440000 bytes for 151.000 ms", + "multiplier": 1, + "operator": "SummaryMerge", + "timing": "query_time" }, - { - "consumer": 9, - "data_state": { - "primitive": "Raw", - "timing": "query_time" - }, - "grouping": "NotApplicable", - "intermediate_schema": { - "closed": true, - "fields": [ - { - "dtype": { - "Plain": "utf8" - }, - "name": "region", - "nullable": true, - "table": null - }, - { - "dtype": { - "Plain": "int64" - }, - "name": "count", - "nullable": false, - "table": null - } - ], - "time_index": null, - "unique_keys": [ - [ - 0 - ] - ] - }, - "producer": 8, - "role": "Input", - "window": "NotApplicable" + "11": { + "cost_per_second": 9.37e-9, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.000833333333333, + "detail": "finalize 10000 accumulators; x 0.0333/s", + "multiplier": 1, + "operator": "FinalizeExactAccumulator", + "timing": "query_time" }, - { - "consumer": 10, - "data_state": { - "primitive": "Raw", - "timing": "query_time" - }, - "grouping": "NotApplicable", - "intermediate_schema": { - "closed": true, - "fields": [ - { - "dtype": { - "Plain": "utf8" - }, - "name": "region", - "nullable": true, - "table": null - }, - { - "dtype": { - "Plain": "float64" - }, - "name": "sum", - "nullable": false, - "table": null - } - ], - "time_index": null, - "unique_keys": [ - [ - 0 - ] - ] - }, - "producer": 6, - "role": "Left", - "window": "NotApplicable" + "12": { + "cost_per_second": 1.03261625007e-8, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.000918333333333, + "detail": "hash aggregate 10000 rows into 10000 groups: 85 ops per row, 10000 group visits; x 0.0333/s; holds 400000 bytes for 27.550 ms", + "multiplier": 1, + "operator": "Aggregate", + "timing": "query_time" }, - { - "consumer": 10, - "data_state": { - "primitive": "Raw", - "timing": "query_time" - }, - "grouping": "NotApplicable", - "intermediate_schema": { - "closed": true, - "fields": [ - { - "dtype": { - "Plain": "utf8" - }, - "name": "region", - "nullable": true, - "table": null - }, - { - "dtype": { - "Plain": "float64" - }, - "name": "col_1", - "nullable": false, - "table": null - } - ], - "time_index": null, - "unique_keys": [ - [ - 0 - ] - ] - }, - "producer": 9, - "role": "Right", - "window": "NotApplicable" + "2": { + "cost_per_second": 1.23684019838e-7, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.011, + "detail": "time range 100s: 100 tumbling windows of states; x 1.0000/s; holds 1568 bytes for 11.000 ms", + "multiplier": 1, + "operator": "TimeRange", + "timing": "query_time" }, - { - "consumer": 11, - "data_state": { - "primitive": "Raw", - "timing": "query_time" - }, - "grouping": "NotApplicable", - "intermediate_schema": { - "closed": true, - "fields": [ - { - "dtype": { - "Plain": "utf8" - }, - "name": "region", - "nullable": true, - "table": null - }, - { - "dtype": { - "Plain": "float64" - }, - "name": "sum", - "nullable": false, - "table": null - } - ], - "time_index": null, - "unique_keys": [ - [ - 0 - ] - ] - }, - "producer": 10, - "role": "Input", - "window": "NotApplicable" + "3": { + "cost_per_second": 0.0000282916772582, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 2.511, + "detail": "merge 1000000 states into 10000; x 1.0000/s; holds 20080000 bytes for 2511.000 ms", + "multiplier": 1, + "operator": "SummaryMerge", + "timing": "query_time" + }, + "4": { + "cost_per_second": 3.9354e-7, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.035, + "detail": "estimate 10000 groups x 1; x 1.0000/s", + "multiplier": 1, + "operator": "SummaryEstimate", + "timing": "query_time" + }, + "5": { + "cost_per_second": 1.12440000552e-10, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.00001, + "detail": "operator: pass 10000 rows; x 1.0000/s; holds 48 bytes for 0.010 ms", + "multiplier": 1, + "operator": "Project", + "timing": "query_time" + }, + "6": { + "cost_per_second": 3.02475975973e-7, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.0269, + "detail": "hash aggregate 10000 rows into 10000 groups: 20 ops per row, 10000 group visits; x 1.0000/s; holds 400000 bytes for 26.900 ms", + "multiplier": 1, + "operator": "Aggregate", + "timing": "query_time" + }, + "7": { + "cost_per_second": 6.52152e-9, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.00058, + "detail": "scan 100000 samples; x 1.0000/s", + "multiplier": 1, + "operator": "Scan", + "timing": "ingestion_time" + }, + "8": { + "cost_per_second": 1.69889906735e-7, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.00907, + "detail": "build exact Rate accumulator in tumbling windows of 30000 ms for 10000 groups: 100000 rows x 70 ops, 1000 group visits, 333.3 states emitted; x 1.0000/s; keeps 59040000 bytes", + "multiplier": 1, + "operator": "SummaryTumblingWindowChain", + "timing": "ingestion_time" }, + "9": { + "cost_per_second": 1.64912002429e-9, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.000146666666667, + "detail": "time range 1200s: 40 tumbling windows of states; x 0.0333/s; holds 144 bytes for 4.400 ms", + "multiplier": 1, + "operator": "TimeRange", + "timing": "query_time" + } + }, + "source": "analytical-cost-v2 (illustrative statistics)", + "usage": { + "average_memory_bytes": 1693159272.6, + "cpu_ops_per_second": 2651381666.67, + "disk_bytes": 0, + "peak_cpu_cores": 4, + "peak_memory_bytes": 1665041856, + "scanned_bytes_per_second": 9600000.0 + } + }, + "dag": { + "edges": [ { - "consumer": 13, + "consumer": 1, "data_state": { "primitive": "Raw", "timing": "ingestion_time" @@ -6147,32 +202,28 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -6183,12 +234,12 @@ ] ] }, - "producer": 12, + "producer": 0, "role": "Input", - "window": "RequiresAlignedPanePhaseOrExactBoundaryResidual" + "window": "NotApplicable" }, { - "consumer": 14, + "consumer": 2, "data_state": { "primitive": "SummaryState", "timing": "ingestion_time" @@ -6201,52 +252,46 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { - "ExactAggregate": [ - "Rate", - "Rate" - ] + "Sketch": "DDSketch" }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 3, - 0 + 0, + 2, + 3 ] ] }, - "producer": 13, + "producer": 1, "role": "Input", "window": "NotApplicable" }, { - "consumer": 15, + "consumer": 3, "data_state": { "primitive": "SummaryState", "timing": "query_time" @@ -6259,54 +304,48 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { - "ExactAggregate": [ - "Rate", - "Rate" - ] + "Sketch": "DDSketch" }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 3, - 0 + 0, + 2, + 3 ] ] }, - "producer": 14, + "producer": 2, "role": "Input", "window": "NotApplicable" }, { - "consumer": 16, + "consumer": 4, "data_state": { - "primitive": "Raw", + "primitive": "SummaryState", "timing": "query_time" }, "grouping": "NotApplicable", @@ -6318,32 +357,28 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { - "Plain": "float64" + "Sketch": "DDSketch" }, - "name": "value", - "nullable": false, - "table": null + "name": "state", + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -6354,12 +389,12 @@ ] ] }, - "producer": 15, + "producer": 3, "role": "Input", "window": "NotApplicable" }, { - "consumer": 17, + "consumer": 5, "data_state": { "primitive": "Raw", "timing": "query_time" @@ -6373,32 +408,28 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, - "name": "value", - "nullable": false, - "table": null + "name": "quantile", + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -6409,12 +440,12 @@ ] ] }, - "producer": 16, + "producer": 4, "role": "Input", "window": "NotApplicable" }, { - "consumer": 18, + "consumer": 6, "data_state": { "primitive": "Raw", "timing": "query_time" @@ -6428,32 +459,28 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -6464,34 +491,18 @@ ] ] }, - "producer": 17, + "producer": 5, "role": "Input", "window": "NotApplicable" - } - ], - "nodes": [ + }, { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact over exact inputs" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } + "consumer": 8, + "data_state": { + "primitive": "Raw", + "timing": "ingestion_time" }, - "id": 0, - "lifetime": "None", - "output_schema": { + "grouping": "NotApplicable", + "intermediate_schema": { "closed": true, "fields": [ { @@ -6499,204 +510,160 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ 3, - 0 - ] - ] - }, - "output_state": { - "primitive": "Raw", - "timing": "ingestion_time" - }, - "payload": { - "NonASAP": { - "Scan": { - "predicates": [], - "schema": { - "closed": true, - "fields": [ - { - "dtype": { - "Plain": "timestamp" - }, - "name": "ts", - "nullable": false, - "table": null - }, - { - "dtype": { - "Plain": "float64" - }, - "name": "value", - "nullable": false, - "table": null - }, - { - "dtype": { - "Plain": "utf8" - }, - "name": "region", - "nullable": true, - "table": null - }, - { - "dtype": { - "Plain": "utf8" - }, - "name": "$promql_series_identity", - "nullable": false, - "table": null - } - ], - "time_index": 0, - "unique_keys": [ - [ - 3, - 0 - ] - ] - }, - "source": { - "TimeSeries": { - "metric": "data" - } - } - } - } - } + 0 + ] + ] + }, + "producer": 7, + "role": "Input", + "window": "NotApplicable" }, { - "guarantee": null, - "id": 1, - "lifetime": { - "AcrossEvaluations": "Memory" + "consumer": 9, + "data_state": { + "primitive": "SummaryState", + "timing": "ingestion_time" }, - "output_schema": { + "grouping": "NotApplicable", + "intermediate_schema": { "closed": true, "fields": [ { "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { - "Sketch": [ - { - "algorithm": "DDSketch", - "params": { - "DDSketch": { - "alpha": 0.05 - } - } - }, - "PerSubpopulationInstance" + "ExactAggregate": [ + "Rate", + "Rate" ] }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 3, - 0 + 0, + 2, + 3 ] ] }, - "output_state": { + "producer": 8, + "role": "Input", + "window": "NotApplicable" + }, + { + "consumer": 10, + "data_state": { "primitive": "SummaryState", - "timing": "ingestion_time" + "timing": "query_time" }, - "payload": { - "ASAP": { - "SummaryTumblingWindowChain": { - "child": 0, - "family": { - "Sketch": [ - { - "algorithm": "DDSketch", - "params": { - "DDSketch": { - "alpha": 0.05 - } - } - }, - "PerSubpopulationInstance" + "grouping": "NotApplicable", + "intermediate_schema": { + "closed": true, + "fields": [ + { + "dtype": { + "Plain": "timestamp" + }, + "name": "window_end", + "nullable": false + }, + { + "dtype": { + "ExactAggregate": [ + "Rate", + "Rate" ] }, - "filter": null, - "grouping": "PerSubpopulationInstance", - "input": { - "item": null, - "weight": { - "Column": "SampleValue" - } + "name": "state", + "nullable": false + }, + { + "dtype": { + "Plain": "utf8" }, - "reduction": "PerEntity", - "tumbling_window_count": 100, - "tumbling_window_width_ms": 1000 + "name": "region", + "nullable": true + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false } - } - } + ], + "time_index": 0, + "unique_keys": [ + [ + 0, + 2, + 3 + ] + ] + }, + "producer": 9, + "role": "Input", + "window": "NotApplicable" }, { - "guarantee": null, - "id": 2, - "lifetime": "None", - "output_schema": { + "consumer": 11, + "data_state": { + "primitive": "SummaryState", + "timing": "query_time" + }, + "grouping": "NotApplicable", + "intermediate_schema": { "closed": true, "fields": [ { @@ -6704,42 +671,31 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { - "Sketch": [ - { - "algorithm": "DDSketch", - "params": { - "DDSketch": { - "alpha": 0.05 - } - } - }, - "PerSubpopulationInstance" + "ExactAggregate": [ + "Rate", + "Rate" ] }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -6750,59 +706,18 @@ ] ] }, - "output_state": { - "primitive": "SummaryState", - "timing": "query_time" - }, - "payload": { - "ASAP": { - "SummaryMerge": { - "children": [ - 1 - ] - } - } - } + "producer": 10, + "role": "Input", + "window": "NotApplicable" }, { - "guarantee": { - "provenance": [ - { - "algorithm": "DDSketch", - "contract": "ddsketch_relative_error_alpha_v1", - "kind": "sketch_evaluation", - "params": { - "DDSketch": { - "alpha": 0.05 - } - }, - "query": "Quantile { q: 0.95 }" - } - ], - "spec": { - "accuracy_metric": "relative", - "accuracy_scope": "per_value", - "error": { - "factors": [ - { - "op": "constant", - "value": 0.05 - }, - { - "op": "unknown", - "statistic": "values_non_negative[data]" - } - ], - "op": "product" - }, - "failure_probability": { - "op": "zero" - } - } + "consumer": 12, + "data_state": { + "primitive": "Raw", + "timing": "query_time" }, - "id": 3, - "lifetime": "None", - "output_schema": { + "grouping": "NotApplicable", + "intermediate_schema": { "closed": true, "fields": [ { @@ -6810,32 +725,28 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, - "name": "quantile", - "nullable": false, - "table": null + "name": "value", + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -6846,60 +757,14 @@ ] ] }, - "output_state": { - "primitive": "Raw", - "timing": "query_time" - }, - "payload": { - "ASAP": { - "SummaryEstimate": { - "query": { - "Quantile": { - "q": 0.95 - } - }, - "summary_input": 2 - } - } - } - }, + "producer": 11, + "role": "Input", + "window": "NotApplicable" + } + ], + "nodes": [ { - "guarantee": { - "provenance": [ - { - "algorithm": "DDSketch", - "contract": "ddsketch_relative_error_alpha_v1", - "kind": "sketch_evaluation", - "params": { - "DDSketch": { - "alpha": 0.05 - } - }, - "query": "Quantile { q: 0.95 }" - } - ], - "spec": { - "accuracy_metric": "relative", - "accuracy_scope": "per_value", - "error": { - "factors": [ - { - "op": "constant", - "value": 0.05 - }, - { - "op": "unknown", - "statistic": "values_non_negative[data]" - } - ], - "op": "product" - }, - "failure_probability": { - "op": "zero" - } - } - }, - "id": 4, + "id": 0, "lifetime": "None", "output_schema": { "closed": true, @@ -6909,32 +774,28 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -6947,427 +808,314 @@ }, "output_state": { "primitive": "Raw", - "timing": "query_time" + "timing": "ingestion_time" }, "payload": { "NonASAP": { - "Project": { - "child": 3, - "cols": [ - { - "alias": "ts", - "expr": { - "Column": 0 - } - }, - { - "alias": "value", - "expr": { - "Column": 1 - } - }, - { - "alias": "region", - "expr": { - "Column": 2 - } - }, - { - "alias": "$promql_series_identity", - "expr": { - "Column": 3 + "Scan": { + "predicates": [], + "schema": { + "closed": true, + "fields": [ + { + "dtype": { + "Plain": "timestamp" + }, + "name": "ts", + "nullable": false + }, + { + "dtype": { + "Plain": "float64" + }, + "name": "value", + "nullable": false + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "region", + "nullable": true + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false } + ], + "time_index": 0, + "unique_keys": [ + [ + 3, + 0 + ] + ] + }, + "source": { + "TimeSeries": { + "metric": "data" } - ], - "qualifier": null + } } } } }, { - "guarantee": null, - "id": 5, - "lifetime": "None", + "id": 1, + "lifetime": { + "AcrossEvaluations": "Memory" + }, "output_schema": { "closed": true, "fields": [ + { + "dtype": { + "Plain": "timestamp" + }, + "name": "window_end", + "nullable": false + }, + { + "dtype": { + "Sketch": "DDSketch" + }, + "name": "state", + "nullable": false + }, { "dtype": { "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { - "ExactAggregate": [ - "Sum", - "Sum" - ] + "Plain": "utf8" }, - "name": "state", - "nullable": false, - "table": null + "name": "$promql_series_identity", + "nullable": false } ], - "time_index": null, + "time_index": 0, "unique_keys": [ [ - 0 + 0, + 2, + 3 ] ] }, "output_state": { "primitive": "SummaryState", - "timing": "query_time" + "timing": "ingestion_time" + }, + "parameters": { + "DDSketch": { + "alpha": 0.05 + } }, "payload": { "ASAP": { - "SummaryAgg": { - "child": 4, - "family": { - "ExactAggregate": [ - "Sum", - "Sum" - ] - }, + "SummaryTumblingWindowChain": { + "child": 0, "filter": null, - "grouping": "PerSubpopulationInstance", "input": { "item": null, "weight": { "Column": "SampleValue" } }, - "reduction": { - "Reduce": [ - 2 - ] - } + "reduction": "PerEntity", + "summary_type": { + "Sketch": "DDSketch" + }, + "tumbling_window_width_ms": 1000 } } + }, + "retention_ms": 100000, + "view_coverage": { + "definition_id": "0c7f962d8fde713c1de7a58752019151", + "selection": [ + { + "columns": [] + } + ] } }, { - "guarantee": { - "provenance": [ - { - "kind": "unavailable_statistic", - "statistic": "input_row_count" - }, - { - "kind": "unavailable_statistic", - "statistic": "values_non_negative[data]" - }, + "id": 2, + "lifetime": "None", + "output_schema": { + "closed": true, + "fields": [ { - "guarantee": { - "provenance": [ - { - "algorithm": "DDSketch", - "contract": "ddsketch_relative_error_alpha_v1", - "kind": "sketch_evaluation", - "params": { - "DDSketch": { - "alpha": 0.05 - } - }, - "query": "Quantile { q: 0.95 }" - } - ], - "spec": { - "accuracy_metric": "relative", - "accuracy_scope": "per_value", - "error": { - "factors": [ - { - "op": "constant", - "value": 0.05 - }, - { - "op": "unknown", - "statistic": "values_non_negative[data]" - } - ], - "op": "product" - }, - "failure_probability": { - "op": "zero" - } - } + "dtype": { + "Plain": "timestamp" }, - "input_index": 0, - "kind": "child_guarantee" - }, - { - "kind": "exact", - "reason": "exact aggregation" + "name": "window_end", + "nullable": false }, { - "kind": "composition_step", - "operator": { - "op": "exact_sum" + "dtype": { + "Sketch": "DDSketch" }, - "rule": "exact_sum_relative" - } - ], - "spec": { - "accuracy_metric": "relative", - "accuracy_scope": "per_value", - "error": { - "factors": [ - { - "op": "max", - "terms": [ - { - "factors": [ - { - "op": "constant", - "value": 0.05 - }, - { - "op": "unknown", - "statistic": "values_non_negative[data]" - } - ], - "op": "product" - } - ] - }, - { - "op": "unknown", - "statistic": "values_non_negative[data]" - } - ], - "op": "product" + "name": "state", + "nullable": false }, - "failure_probability": { - "count": { - "op": "unknown", - "statistic": "input_row_count" - }, - "inner": { - "op": "union_bound", - "terms": [ - { - "op": "zero" - } - ] - }, - "op": "scaled" - } - } - }, - "id": 6, - "lifetime": "None", - "output_schema": { - "closed": true, - "fields": [ { "dtype": { "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { - "Plain": "float64" + "Plain": "utf8" }, - "name": "sum", - "nullable": false, - "table": null + "name": "$promql_series_identity", + "nullable": false } ], - "time_index": null, + "time_index": 0, "unique_keys": [ [ - 0 + 0, + 2, + 3 ] ] }, "output_state": { - "primitive": "Raw", + "primitive": "SummaryState", "timing": "query_time" }, + "parameters": { + "DDSketch": { + "alpha": 0.05 + } + }, "payload": { - "ASAP": { - "FinalizeExactAccumulator": { - "child": 5 + "NonASAP": { + "TimeRange": { + "child": 1, + "kind": "range", + "range": { + "nanos": 0, + "secs": 100 + } } } } }, { - "guarantee": null, - "id": 7, - "lifetime": "None", + "id": 3, + "lifetime": { + "OneEvaluation": "Memory" + }, "output_schema": { "closed": true, "fields": [ { "dtype": { - "Plain": "utf8" + "Plain": "timestamp" }, - "name": "region", - "nullable": true, - "table": null + "name": "ts", + "nullable": false }, { "dtype": { - "ExactAggregate": [ - "Count", - "Count" - ] + "Sketch": "DDSketch" }, "name": "state", - "nullable": false, - "table": null - } - ], - "time_index": null, - "unique_keys": [ - [ - 0 - ] - ] - }, - "output_state": { - "primitive": "SummaryState", - "timing": "query_time" - }, - "payload": { - "ASAP": { - "SummaryAgg": { - "child": 4, - "family": { - "ExactAggregate": [ - "Count", - "Count" - ] - }, - "filter": null, - "grouping": "PerSubpopulationInstance", - "input": { - "item": null, - "weight": { - "Column": "SampleValue" - } - }, - "reduction": { - "Reduce": [ - 2 - ] - } - } - } - } - }, - { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "count of rows" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" + "nullable": false }, - "failure_probability": { - "op": "zero" - } - } - }, - "id": 8, - "lifetime": "None", - "output_schema": { - "closed": true, - "fields": [ { "dtype": { "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { - "Plain": "int64" + "Plain": "utf8" }, - "name": "count", - "nullable": false, - "table": null + "name": "$promql_series_identity", + "nullable": false } ], - "time_index": null, + "time_index": 0, "unique_keys": [ [ + 3, 0 ] ] }, "output_state": { - "primitive": "Raw", + "primitive": "SummaryState", "timing": "query_time" }, + "parameters": { + "DDSketch": { + "alpha": 0.05 + } + }, "payload": { "ASAP": { - "FinalizeExactAccumulator": { - "child": 7 + "SummaryMerge": { + "children": [ + 2 + ] } } } }, { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact over exact inputs" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, - "id": 9, + "id": 4, "lifetime": "None", "output_schema": { "closed": true, "fields": [ + { + "dtype": { + "Plain": "timestamp" + }, + "name": "ts", + "nullable": false + }, + { + "dtype": { + "Plain": "float64" + }, + "name": "quantile", + "nullable": false + }, { "dtype": { "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { - "Plain": "float64" + "Plain": "utf8" }, - "name": "col_1", - "nullable": false, - "table": null + "name": "$promql_series_identity", + "nullable": false } ], - "time_index": null, + "time_index": 0, "unique_keys": [ [ + 3, 0 ] ] @@ -7377,168 +1125,57 @@ "timing": "query_time" }, "payload": { - "NonASAP": { - "Project": { - "child": 8, - "cols": [ - { - "alias": null, - "expr": { - "Column": 0 - } - }, - { - "alias": null, - "expr": { - "Cast": { - "expr": { - "Column": 1 - }, - "to": "float64", - "try_cast": false - } - } + "ASAP": { + "SummaryEstimate": { + "query": { + "Quantile": { + "q": 0.95 } - ], - "qualifier": null + }, + "summary_input": 3 } } } }, { - "guarantee": { - "provenance": [ - { - "kind": "unavailable_statistic", - "statistic": "input_row_count" - }, - { - "kind": "unavailable_statistic", - "statistic": "values_non_negative[data]" - }, + "id": 5, + "lifetime": "None", + "output_schema": { + "closed": true, + "fields": [ { - "guarantee": { - "provenance": [ - { - "algorithm": "DDSketch", - "contract": "ddsketch_relative_error_alpha_v1", - "kind": "sketch_evaluation", - "params": { - "DDSketch": { - "alpha": 0.05 - } - }, - "query": "Quantile { q: 0.95 }" - } - ], - "spec": { - "accuracy_metric": "relative", - "accuracy_scope": "per_value", - "error": { - "factors": [ - { - "op": "constant", - "value": 0.05 - }, - { - "op": "unknown", - "statistic": "values_non_negative[data]" - } - ], - "op": "product" - }, - "failure_probability": { - "op": "zero" - } - } + "dtype": { + "Plain": "timestamp" }, - "input_index": 0, - "kind": "child_guarantee" - }, - { - "kind": "exact", - "reason": "exact aggregation" + "name": "ts", + "nullable": false }, { - "kind": "composition_step", - "operator": { - "op": "exact_sum" + "dtype": { + "Plain": "float64" }, - "rule": "exact_sum_relative" - } - ], - "spec": { - "accuracy_metric": "relative", - "accuracy_scope": "per_value", - "error": { - "factors": [ - { - "op": "max", - "terms": [ - { - "factors": [ - { - "op": "constant", - "value": 0.05 - }, - { - "op": "unknown", - "statistic": "values_non_negative[data]" - } - ], - "op": "product" - } - ] - }, - { - "op": "unknown", - "statistic": "values_non_negative[data]" - } - ], - "op": "product" + "name": "value", + "nullable": false }, - "failure_probability": { - "count": { - "op": "unknown", - "statistic": "input_row_count" - }, - "inner": { - "op": "union_bound", - "terms": [ - { - "op": "zero" - } - ] - }, - "op": "scaled" - } - } - }, - "id": 10, - "lifetime": "None", - "output_schema": { - "closed": true, - "fields": [ { "dtype": { "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { - "Plain": "float64" + "Plain": "utf8" }, - "name": "sum", - "nullable": false, - "table": null + "name": "$promql_series_identity", + "nullable": false } ], - "time_index": null, + "time_index": 0, "unique_keys": [ [ + 3, 0 ] ] @@ -7549,133 +1186,43 @@ }, "payload": { "NonASAP": { - "BinaryOp": { - "lhs": 6, - "operator": { - "checked_finite_division": false, - "checked_relative_division": false, - "kind": { - "Arithmetic": "Div" - }, - "vector_match": null - }, - "return_bool": false, - "rhs": 9 - } - } - } - }, - { - "guarantee": { - "provenance": [ - { - "kind": "unavailable_statistic", - "statistic": "input_row_count" - }, - { - "kind": "unavailable_statistic", - "statistic": "values_non_negative[data]" - }, - { - "guarantee": { - "provenance": [ - { - "algorithm": "DDSketch", - "contract": "ddsketch_relative_error_alpha_v1", - "kind": "sketch_evaluation", - "params": { - "DDSketch": { - "alpha": 0.05 - } - }, - "query": "Quantile { q: 0.95 }" + "Project": { + "child": 4, + "cols": [ + { + "alias": "ts", + "expr": { + "Column": 0 } - ], - "spec": { - "accuracy_metric": "relative", - "accuracy_scope": "per_value", - "error": { - "factors": [ - { - "op": "constant", - "value": 0.05 - }, - { - "op": "unknown", - "statistic": "values_non_negative[data]" - } - ], - "op": "product" - }, - "failure_probability": { - "op": "zero" + }, + { + "alias": "value", + "expr": { + "Column": 1 } - } - }, - "input_index": 0, - "kind": "child_guarantee" - }, - { - "kind": "exact", - "reason": "exact aggregation" - }, - { - "kind": "composition_step", - "operator": { - "op": "exact_sum" - }, - "rule": "exact_sum_relative" - } - ], - "spec": { - "accuracy_metric": "relative", - "accuracy_scope": "per_value", - "error": { - "factors": [ + }, { - "op": "max", - "terms": [ - { - "factors": [ - { - "op": "constant", - "value": 0.05 - }, - { - "op": "unknown", - "statistic": "values_non_negative[data]" - } - ], - "op": "product" - } - ] + "alias": "region", + "expr": { + "Column": 2 + } }, { - "op": "unknown", - "statistic": "values_non_negative[data]" - } - ], - "op": "product" - }, - "failure_probability": { - "count": { - "op": "unknown", - "statistic": "input_row_count" - }, - "inner": { - "op": "union_bound", - "terms": [ - { - "op": "zero" + "alias": "$promql_series_identity", + "expr": { + "Column": 3 } - ] - }, - "op": "scaled" + } + ] } } + } + }, + { + "id": 6, + "lifetime": { + "OneEvaluation": "Memory" }, - "id": 11, - "lifetime": "None", "output_schema": { "closed": true, "fields": [ @@ -7684,16 +1231,14 @@ "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "float64" }, "name": "avg", - "nullable": false, - "table": null + "nullable": false } ], "time_index": null, @@ -7709,47 +1254,30 @@ }, "payload": { "NonASAP": { - "Project": { - "child": 10, - "cols": [ - { - "alias": "region", - "expr": { - "Column": 0 - } - }, + "Aggregate": { + "child": 5, + "filters": [], + "having": null, + "measures": [ { - "alias": "avg", - "expr": { - "Column": 1 - } + "col": null, + "kind": "avg" } ], - "qualifier": null + "output_names": [ + "" + ], + "reduction": { + "Reduce": [ + 2 + ] + } } } } }, { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact over exact inputs" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, - "id": 12, + "id": 7, "lifetime": "None", "output_schema": { "closed": true, @@ -7759,32 +1287,28 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -7811,32 +1335,28 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -7857,8 +1377,7 @@ } }, { - "guarantee": null, - "id": 13, + "id": 8, "lifetime": { "AcrossEvaluations": "Memory" }, @@ -7869,9 +1388,8 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { @@ -7881,31 +1399,29 @@ ] }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 3, - 0 + 0, + 2, + 3 ] ] }, @@ -7916,15 +1432,8 @@ "payload": { "ASAP": { "SummaryTumblingWindowChain": { - "child": 12, - "family": { - "ExactAggregate": [ - "Rate", - "Rate" - ] - }, + "child": 7, "filter": null, - "grouping": "PerSubpopulationInstance", "input": { "item": null, "weight": { @@ -7932,15 +1441,28 @@ } }, "reduction": "PerEntity", - "tumbling_window_count": 40, + "summary_type": { + "ExactAggregate": [ + "Rate", + "Rate" + ] + }, "tumbling_window_width_ms": 30000 } } + }, + "retention_ms": 1200000, + "view_coverage": { + "definition_id": "96705e10b00e7f916542fec1da8dcdb6", + "selection": [ + { + "columns": [] + } + ] } }, { - "guarantee": null, - "id": 14, + "id": 9, "lifetime": "None", "output_schema": { "closed": true, @@ -7949,9 +1471,8 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { @@ -7961,31 +1482,29 @@ ] }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 3, - 0 + 0, + 2, + 3 ] ] }, @@ -7994,36 +1513,23 @@ "timing": "query_time" }, "payload": { - "ASAP": { - "SummaryMerge": { - "children": [ - 13 - ] + "NonASAP": { + "TimeRange": { + "child": 8, + "kind": "range", + "range": { + "nanos": 0, + "secs": 1200 + } } } } }, { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact accumulator over exact input" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } + "id": 10, + "lifetime": { + "OneEvaluation": "Memory" }, - "id": 15, - "lifetime": "None", "output_schema": { "closed": true, "fields": [ @@ -8032,32 +1538,31 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { - "Plain": "float64" + "ExactAggregate": [ + "Rate", + "Rate" + ] }, - "name": "value", - "nullable": false, - "table": null + "name": "state", + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -8069,126 +1574,21 @@ ] }, "output_state": { - "primitive": "Raw", + "primitive": "SummaryState", "timing": "query_time" }, "payload": { "ASAP": { - "FinalizeExactAccumulator": { - "child": 14 - } - } - } - }, - { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact over exact inputs" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, - "id": 16, - "lifetime": "None", - "output_schema": { - "closed": true, - "fields": [ - { - "dtype": { - "Plain": "timestamp" - }, - "name": "ts", - "nullable": false, - "table": null - }, - { - "dtype": { - "Plain": "float64" - }, - "name": "value", - "nullable": false, - "table": null - }, - { - "dtype": { - "Plain": "utf8" - }, - "name": "region", - "nullable": true, - "table": null - }, - { - "dtype": { - "Plain": "utf8" - }, - "name": "$promql_series_identity", - "nullable": false, - "table": null - } - ], - "time_index": 0, - "unique_keys": [ - [ - 3, - 0 - ] - ] - }, - "output_state": { - "primitive": "Raw", - "timing": "query_time" - }, - "payload": { - "NonASAP": { - "Sort": { - "child": 15, - "keys": [ - { - "ascending": false, - "expr": { - "Column": 1 - }, - "nulls_first": false - } - ], - "partition_by": [ - 2 + "SummaryMerge": { + "children": [ + 9 ] } } } }, { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact over exact inputs" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, - "id": 17, + "id": 11, "lifetime": "None", "output_schema": { "closed": true, @@ -8198,32 +1598,28 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -8239,39 +1635,18 @@ "timing": "query_time" }, "payload": { - "NonASAP": { - "Limit": { - "child": 16, - "n": 4, - "offset": 0, - "partition_by": [ - 2 - ] + "ASAP": { + "FinalizeExactAccumulator": { + "child": 10 } } } }, { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact over exact inputs" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } + "id": 12, + "lifetime": { + "OneEvaluation": "Memory" }, - "id": 18, - "lifetime": "None", "output_schema": { "closed": true, "fields": [ @@ -8280,28 +1655,30 @@ "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "float64" }, "name": "topk_4", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": null, - "unique_keys": [] + "unique_keys": [ + [ + 0, + 2 + ] + ] }, "output_state": { "primitive": "Raw", @@ -8309,44 +1686,38 @@ }, "payload": { "NonASAP": { - "Project": { - "child": 17, - "cols": [ - { - "alias": "region", - "expr": { - "Column": 2 - } - }, - { - "alias": "topk_4", - "expr": { - "Column": 1 - } - }, + "Aggregate": { + "child": 11, + "filters": [], + "having": null, + "measures": [ { - "alias": "$promql_series_identity", - "expr": { - "Column": 3 - } + "k": 4, + "kind": "top_k" } ], - "qualifier": null + "output_names": [ + "" + ], + "reduction": { + "Reduce": [ + 2 + ] + } } } } } ], "roots": [ - 11, - 18 + 6, + 12 ], "scalar_roots": [] - }, - "display_name": "Q1 Sum acc + DDSketch ×100 tumbling windows + Count acc · Q2 Rate acc ×40 tumbling windows · ingestion time: DDSketch for Q1, 100 × 1s tumbling windows, Rate acc for Q2, 40 × 30s tumbling windows" + } }, - "tumbling_window_retention_ms": { - "1": 100000, - "13": 1200000 - } + "warnings": [ + "assumed data fact: source data declares no column_magnitudes for value, so a DDSketch state over it is costed for magnitudes in [1, 1000000000]; this affects cost only, the accuracy guarantee does not depend on it", + "assumed data fact: the data arrival declares no max_out_of_order_ms, so the samples of source data_total that an ingestion-time rate, increase or irate state reads are taken to arrive in timestamp order; if this does not hold, answers may be wrong by an amount no accuracy target bounds (a late sample can be read as a counter reset), or ingestion may fail" + ] } diff --git a/crates/devtools/tests/sketch_plan_snapshots/random-mixed-seed-862-w02-q2-q7-mvp.json b/crates/devtools/tests/sketch_plan_snapshots/random-mixed-seed-862-w02-q2-q7-mvp.json index 06e307546..ac9d0db3e 100644 --- a/crates/devtools/tests/sketch_plan_snapshots/random-mixed-seed-862-w02-q2-q7-mvp.json +++ b/crates/devtools/tests/sketch_plan_snapshots/random-mixed-seed-862-w02-q2-q7-mvp.json @@ -3,6897 +3,192 @@ "materialized_views": [ { "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", + "definition_id": "0c7f962d8fde713c1de7a58752019151", "selection": [ { - "columns": [], - "relative_time": [ - { - "excluded": -1000 - }, - { - "included": 0 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 0, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 0, - "node_id": 2, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2000 - }, - { - "included": -1000 - } - ] + "columns": [] } ] }, "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] + "materialized_views": [] }, - "installed_count": 100, - "installed_position": 1, - "node_id": 5, + "node_id": 1, "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 + "retention_ms": 100000 }, { "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", + "definition_id": "b876d20377d09364dbbaba79f3786982", "selection": [ { - "columns": [], - "relative_time": [ - { - "excluded": -3000 - }, - { - "included": -2000 - } - ] + "columns": [] } ] }, "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] + "materialized_views": [] }, - "installed_count": 100, - "installed_position": 2, "node_id": 8, "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -4000 - }, - { - "included": -3000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 3, - "node_id": 11, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -5000 - }, - { - "included": -4000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 4000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 4, - "node_id": 14, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -6000 - }, - { - "included": -5000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 5000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 5, - "node_id": 17, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -7000 - }, - { - "included": -6000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 6000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 6, - "node_id": 20, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -8000 - }, - { - "included": -7000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 7000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 7, - "node_id": 23, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -9000 - }, - { - "included": -8000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 8000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 8, - "node_id": 26, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -10000 - }, - { - "included": -9000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 9000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 9, - "node_id": 29, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -11000 - }, - { - "included": -10000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 10000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 10, - "node_id": 32, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -12000 - }, - { - "included": -11000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 11000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 11, - "node_id": 35, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -13000 - }, - { - "included": -12000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 12000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] + "retention_ms": 300000 + } + ], + "queries": [ + "avg by (region) (quantile_over_time(0.75, data[100s]))", + "topk by (region) (4, increase(data_total[5m]))" + ], + "selected": { + "accepted": [ + { + "accepted_metric": "relative", + "sketches": [ + "DDSketch" + ], + "summary": "accepted relative (DDSketch)" }, - "installed_count": 100, - "installed_position": 12, - "node_id": 38, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -14000 - }, - { - "included": -13000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 13000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 13, - "node_id": 41, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -15000 - }, - { - "included": -14000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 14000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 14, - "node_id": 44, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -16000 - }, - { - "included": -15000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 15000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 15, - "node_id": 47, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -17000 - }, - { - "included": -16000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 16000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 16, - "node_id": 50, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -18000 - }, - { - "included": -17000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 17000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 17, - "node_id": 53, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -19000 - }, - { - "included": -18000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 18000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 18, - "node_id": 56, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -20000 - }, - { - "included": -19000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 19000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 19, - "node_id": 59, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -21000 - }, - { - "included": -20000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 20000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 20, - "node_id": 62, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -22000 - }, - { - "included": -21000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 21000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 21, - "node_id": 65, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -23000 - }, - { - "included": -22000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 22000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 22, - "node_id": 68, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -24000 - }, - { - "included": -23000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 23000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 23, - "node_id": 71, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -25000 - }, - { - "included": -24000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 24000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 24, - "node_id": 74, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -26000 - }, - { - "included": -25000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 25000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 25, - "node_id": 77, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -27000 - }, - { - "included": -26000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 26000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 26, - "node_id": 80, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -28000 - }, - { - "included": -27000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 27000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 27, - "node_id": 83, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -29000 - }, - { - "included": -28000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 28000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 28, - "node_id": 86, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -30000 - }, - { - "included": -29000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 29000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 29, - "node_id": 89, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -31000 - }, - { - "included": -30000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 30000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 30, - "node_id": 92, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -32000 - }, - { - "included": -31000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 31000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 31, - "node_id": 95, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -33000 - }, - { - "included": -32000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 32000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 32, - "node_id": 98, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -34000 - }, - { - "included": -33000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 33000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 33, - "node_id": 101, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -35000 - }, - { - "included": -34000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 34000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 34, - "node_id": 104, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -36000 - }, - { - "included": -35000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 35000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 35, - "node_id": 107, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -37000 - }, - { - "included": -36000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 36000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 36, - "node_id": 110, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -38000 - }, - { - "included": -37000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 37000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 37, - "node_id": 113, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -39000 - }, - { - "included": -38000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 38000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 38, - "node_id": 116, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -40000 - }, - { - "included": -39000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 39000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 39, - "node_id": 119, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -41000 - }, - { - "included": -40000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 40000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 40, - "node_id": 122, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -42000 - }, - { - "included": -41000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 41000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 41, - "node_id": 125, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -43000 - }, - { - "included": -42000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 42000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 42, - "node_id": 128, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -44000 - }, - { - "included": -43000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 43000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 43, - "node_id": 131, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -45000 - }, - { - "included": -44000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 44000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 44, - "node_id": 134, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -46000 - }, - { - "included": -45000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 45000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 45, - "node_id": 137, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -47000 - }, - { - "included": -46000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 46000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 46, - "node_id": 140, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -48000 - }, - { - "included": -47000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 47000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 47, - "node_id": 143, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -49000 - }, - { - "included": -48000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 48000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 48, - "node_id": 146, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -50000 - }, - { - "included": -49000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 49000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 49, - "node_id": 149, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -51000 - }, - { - "included": -50000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 50000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 50, - "node_id": 152, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -52000 - }, - { - "included": -51000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 51000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 51, - "node_id": 155, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -53000 - }, - { - "included": -52000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 52000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 52, - "node_id": 158, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -54000 - }, - { - "included": -53000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 53000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 53, - "node_id": 161, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -55000 - }, - { - "included": -54000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 54000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 54, - "node_id": 164, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -56000 - }, - { - "included": -55000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 55000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 55, - "node_id": 167, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -57000 - }, - { - "included": -56000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 56000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 56, - "node_id": 170, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -58000 - }, - { - "included": -57000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 57000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 57, - "node_id": 173, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -59000 - }, - { - "included": -58000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 58000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 58, - "node_id": 176, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -60000 - }, - { - "included": -59000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 59000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 59, - "node_id": 179, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -61000 - }, - { - "included": -60000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 60000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 60, - "node_id": 182, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -62000 - }, - { - "included": -61000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 61000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 61, - "node_id": 185, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -63000 - }, - { - "included": -62000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 62000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 62, - "node_id": 188, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -64000 - }, - { - "included": -63000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 63000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 63, - "node_id": 191, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -65000 - }, - { - "included": -64000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 64000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 64, - "node_id": 194, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -66000 - }, - { - "included": -65000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 65000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 65, - "node_id": 197, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -67000 - }, - { - "included": -66000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 66000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 66, - "node_id": 200, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -68000 - }, - { - "included": -67000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 67000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 67, - "node_id": 203, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -69000 - }, - { - "included": -68000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 68000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 68, - "node_id": 206, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -70000 - }, - { - "included": -69000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 69000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 69, - "node_id": 209, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -71000 - }, - { - "included": -70000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 70000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 70, - "node_id": 212, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -72000 - }, - { - "included": -71000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 71000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 71, - "node_id": 215, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -73000 - }, - { - "included": -72000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 72000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 72, - "node_id": 218, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -74000 - }, - { - "included": -73000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 73000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 73, - "node_id": 221, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -75000 - }, - { - "included": -74000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 74000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 74, - "node_id": 224, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -76000 - }, - { - "included": -75000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 75000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 75, - "node_id": 227, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -77000 - }, - { - "included": -76000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 76000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 76, - "node_id": 230, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -78000 - }, - { - "included": -77000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 77000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 77, - "node_id": 233, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -79000 - }, - { - "included": -78000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 78000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 78, - "node_id": 236, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -80000 - }, - { - "included": -79000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 79000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 79, - "node_id": 239, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -81000 - }, - { - "included": -80000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 80000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 80, - "node_id": 242, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -82000 - }, - { - "included": -81000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 81000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 81, - "node_id": 245, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -83000 - }, - { - "included": -82000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 82000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 82, - "node_id": 248, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -84000 - }, - { - "included": -83000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 83000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 83, - "node_id": 251, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -85000 - }, - { - "included": -84000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 84000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 84, - "node_id": 254, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -86000 - }, - { - "included": -85000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 85000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 85, - "node_id": 257, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -87000 - }, - { - "included": -86000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 86000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 86, - "node_id": 260, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -88000 - }, - { - "included": -87000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 87000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 87, - "node_id": 263, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -89000 - }, - { - "included": -88000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 88000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 88, - "node_id": 266, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -90000 - }, - { - "included": -89000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 89000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 89, - "node_id": 269, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -91000 - }, - { - "included": -90000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 90000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 90, - "node_id": 272, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -92000 - }, - { - "included": -91000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 91000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 91, - "node_id": 275, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -93000 - }, - { - "included": -92000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 92000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 92, - "node_id": 278, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -94000 - }, - { - "included": -93000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 93000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 93, - "node_id": 281, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -95000 - }, - { - "included": -94000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 94000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 94, - "node_id": 284, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -96000 - }, - { - "included": -95000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 95000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 95, - "node_id": 287, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -97000 - }, - { - "included": -96000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 96000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 96, - "node_id": 290, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -98000 - }, - { - "included": -97000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 97000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 97, - "node_id": 293, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -99000 - }, - { - "included": -98000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 98000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 98, - "node_id": 296, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "3cc5ca8de4166e2db2a1175162047eae", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -100000 - }, - { - "included": -99000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 99000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 1000 - } - ] - }, - "installed_count": 100, - "installed_position": 99, - "node_id": 299, - "refresh": "at_ingestion", - "retention_ms": 100000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "916dc74ffcca765ff49ae11fd1cec5e2", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -5000 - }, - { - "included": 0 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 0, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 5000 - } - ] - }, - "installed_count": 60, - "installed_position": 0, - "node_id": 312, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "916dc74ffcca765ff49ae11fd1cec5e2", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -10000 - }, - { - "included": -5000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 5000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 5000 - } - ] - }, - "installed_count": 60, - "installed_position": 1, - "node_id": 315, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "916dc74ffcca765ff49ae11fd1cec5e2", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -15000 - }, - { - "included": -10000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 10000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 5000 - } - ] - }, - "installed_count": 60, - "installed_position": 2, - "node_id": 318, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "916dc74ffcca765ff49ae11fd1cec5e2", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -20000 - }, - { - "included": -15000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 15000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 5000 - } - ] - }, - "installed_count": 60, - "installed_position": 3, - "node_id": 321, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "916dc74ffcca765ff49ae11fd1cec5e2", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -25000 - }, - { - "included": -20000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 20000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 5000 - } - ] - }, - "installed_count": 60, - "installed_position": 4, - "node_id": 324, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "916dc74ffcca765ff49ae11fd1cec5e2", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -30000 - }, - { - "included": -25000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 25000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 5000 - } - ] - }, - "installed_count": 60, - "installed_position": 5, - "node_id": 327, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "916dc74ffcca765ff49ae11fd1cec5e2", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -35000 - }, - { - "included": -30000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 30000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 5000 - } - ] - }, - "installed_count": 60, - "installed_position": 6, - "node_id": 330, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "916dc74ffcca765ff49ae11fd1cec5e2", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -40000 - }, - { - "included": -35000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 35000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 5000 - } - ] - }, - "installed_count": 60, - "installed_position": 7, - "node_id": 333, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "916dc74ffcca765ff49ae11fd1cec5e2", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -45000 - }, - { - "included": -40000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 40000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 5000 - } - ] - }, - "installed_count": 60, - "installed_position": 8, - "node_id": 336, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "916dc74ffcca765ff49ae11fd1cec5e2", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -50000 - }, - { - "included": -45000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 45000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 5000 - } - ] - }, - "installed_count": 60, - "installed_position": 9, - "node_id": 339, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "916dc74ffcca765ff49ae11fd1cec5e2", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -55000 - }, - { - "included": -50000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 50000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 5000 - } - ] - }, - "installed_count": 60, - "installed_position": 10, - "node_id": 342, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "916dc74ffcca765ff49ae11fd1cec5e2", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -60000 - }, - { - "included": -55000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 55000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 5000 - } - ] - }, - "installed_count": 60, - "installed_position": 11, - "node_id": 345, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "916dc74ffcca765ff49ae11fd1cec5e2", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -65000 - }, - { - "included": -60000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 60000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 5000 - } - ] - }, - "installed_count": 60, - "installed_position": 12, - "node_id": 348, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "916dc74ffcca765ff49ae11fd1cec5e2", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -70000 - }, - { - "included": -65000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 65000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 5000 - } - ] - }, - "installed_count": 60, - "installed_position": 13, - "node_id": 351, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "916dc74ffcca765ff49ae11fd1cec5e2", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -75000 - }, - { - "included": -70000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 70000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 5000 - } - ] - }, - "installed_count": 60, - "installed_position": 14, - "node_id": 354, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "916dc74ffcca765ff49ae11fd1cec5e2", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -80000 - }, - { - "included": -75000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 75000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 5000 - } - ] - }, - "installed_count": 60, - "installed_position": 15, - "node_id": 357, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "916dc74ffcca765ff49ae11fd1cec5e2", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -85000 - }, - { - "included": -80000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 80000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 5000 - } - ] - }, - "installed_count": 60, - "installed_position": 16, - "node_id": 360, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "916dc74ffcca765ff49ae11fd1cec5e2", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -90000 - }, - { - "included": -85000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 85000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 5000 - } - ] - }, - "installed_count": 60, - "installed_position": 17, - "node_id": 363, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "916dc74ffcca765ff49ae11fd1cec5e2", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -95000 - }, - { - "included": -90000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 90000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 5000 - } - ] - }, - "installed_count": 60, - "installed_position": 18, - "node_id": 366, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "916dc74ffcca765ff49ae11fd1cec5e2", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -100000 - }, - { - "included": -95000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 95000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 5000 - } - ] - }, - "installed_count": 60, - "installed_position": 19, - "node_id": 369, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "916dc74ffcca765ff49ae11fd1cec5e2", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -105000 - }, - { - "included": -100000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 100000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 5000 - } - ] - }, - "installed_count": 60, - "installed_position": 20, - "node_id": 372, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "916dc74ffcca765ff49ae11fd1cec5e2", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -110000 - }, - { - "included": -105000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 105000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 5000 - } - ] - }, - "installed_count": 60, - "installed_position": 21, - "node_id": 375, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "916dc74ffcca765ff49ae11fd1cec5e2", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -115000 - }, - { - "included": -110000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 110000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 5000 - } - ] - }, - "installed_count": 60, - "installed_position": 22, - "node_id": 378, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "916dc74ffcca765ff49ae11fd1cec5e2", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -120000 - }, - { - "included": -115000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 115000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 5000 - } - ] - }, - "installed_count": 60, - "installed_position": 23, - "node_id": 381, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "916dc74ffcca765ff49ae11fd1cec5e2", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -125000 - }, - { - "included": -120000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 120000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 5000 - } - ] - }, - "installed_count": 60, - "installed_position": 24, - "node_id": 384, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "916dc74ffcca765ff49ae11fd1cec5e2", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -130000 - }, - { - "included": -125000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 125000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 5000 - } - ] - }, - "installed_count": 60, - "installed_position": 25, - "node_id": 387, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "916dc74ffcca765ff49ae11fd1cec5e2", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -135000 - }, - { - "included": -130000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 130000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 5000 - } - ] - }, - "installed_count": 60, - "installed_position": 26, - "node_id": 390, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "916dc74ffcca765ff49ae11fd1cec5e2", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -140000 - }, - { - "included": -135000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 135000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 5000 - } - ] - }, - "installed_count": 60, - "installed_position": 27, - "node_id": 393, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "916dc74ffcca765ff49ae11fd1cec5e2", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -145000 - }, - { - "included": -140000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 140000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 5000 - } - ] - }, - "installed_count": 60, - "installed_position": 28, - "node_id": 396, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "916dc74ffcca765ff49ae11fd1cec5e2", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -150000 - }, - { - "included": -145000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 145000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 5000 - } - ] - }, - "installed_count": 60, - "installed_position": 29, - "node_id": 399, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "916dc74ffcca765ff49ae11fd1cec5e2", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -155000 - }, - { - "included": -150000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 150000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 5000 - } - ] - }, - "installed_count": 60, - "installed_position": 30, - "node_id": 402, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "916dc74ffcca765ff49ae11fd1cec5e2", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -160000 - }, - { - "included": -155000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 155000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 5000 - } - ] - }, - "installed_count": 60, - "installed_position": 31, - "node_id": 405, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "916dc74ffcca765ff49ae11fd1cec5e2", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -165000 - }, - { - "included": -160000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 160000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 5000 - } - ] - }, - "installed_count": 60, - "installed_position": 32, - "node_id": 408, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "916dc74ffcca765ff49ae11fd1cec5e2", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -170000 - }, - { - "included": -165000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 165000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 5000 - } - ] - }, - "installed_count": 60, - "installed_position": 33, - "node_id": 411, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "916dc74ffcca765ff49ae11fd1cec5e2", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -175000 - }, - { - "included": -170000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 170000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 5000 - } - ] - }, - "installed_count": 60, - "installed_position": 34, - "node_id": 414, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "916dc74ffcca765ff49ae11fd1cec5e2", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -180000 - }, - { - "included": -175000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 175000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 5000 - } - ] - }, - "installed_count": 60, - "installed_position": 35, - "node_id": 417, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "916dc74ffcca765ff49ae11fd1cec5e2", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -185000 - }, - { - "included": -180000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 180000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 5000 - } - ] - }, - "installed_count": 60, - "installed_position": 36, - "node_id": 420, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "916dc74ffcca765ff49ae11fd1cec5e2", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -190000 - }, - { - "included": -185000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 185000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 5000 - } - ] - }, - "installed_count": 60, - "installed_position": 37, - "node_id": 423, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "916dc74ffcca765ff49ae11fd1cec5e2", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -195000 - }, - { - "included": -190000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 190000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 5000 - } - ] - }, - "installed_count": 60, - "installed_position": 38, - "node_id": 426, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "916dc74ffcca765ff49ae11fd1cec5e2", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -200000 - }, - { - "included": -195000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 195000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 5000 - } - ] - }, - "installed_count": 60, - "installed_position": 39, - "node_id": 429, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "916dc74ffcca765ff49ae11fd1cec5e2", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -205000 - }, - { - "included": -200000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 200000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 5000 - } - ] - }, - "installed_count": 60, - "installed_position": 40, - "node_id": 432, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "916dc74ffcca765ff49ae11fd1cec5e2", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -210000 - }, - { - "included": -205000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 205000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 5000 - } - ] - }, - "installed_count": 60, - "installed_position": 41, - "node_id": 435, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "916dc74ffcca765ff49ae11fd1cec5e2", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -215000 - }, - { - "included": -210000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 210000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 5000 - } - ] - }, - "installed_count": 60, - "installed_position": 42, - "node_id": 438, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "916dc74ffcca765ff49ae11fd1cec5e2", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -220000 - }, - { - "included": -215000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 215000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 5000 - } - ] - }, - "installed_count": 60, - "installed_position": 43, - "node_id": 441, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "916dc74ffcca765ff49ae11fd1cec5e2", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -225000 - }, - { - "included": -220000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 220000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 5000 - } - ] - }, - "installed_count": 60, - "installed_position": 44, - "node_id": 444, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "916dc74ffcca765ff49ae11fd1cec5e2", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -230000 - }, - { - "included": -225000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 225000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 5000 - } - ] - }, - "installed_count": 60, - "installed_position": 45, - "node_id": 447, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "916dc74ffcca765ff49ae11fd1cec5e2", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -235000 - }, - { - "included": -230000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 230000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 5000 - } - ] - }, - "installed_count": 60, - "installed_position": 46, - "node_id": 450, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "916dc74ffcca765ff49ae11fd1cec5e2", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -240000 - }, - { - "included": -235000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 235000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 5000 - } - ] - }, - "installed_count": 60, - "installed_position": 47, - "node_id": 453, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "916dc74ffcca765ff49ae11fd1cec5e2", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -245000 - }, - { - "included": -240000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 240000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 5000 - } - ] - }, - "installed_count": 60, - "installed_position": 48, - "node_id": 456, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "916dc74ffcca765ff49ae11fd1cec5e2", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -250000 - }, - { - "included": -245000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 245000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 5000 - } - ] - }, - "installed_count": 60, - "installed_position": 49, - "node_id": 459, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "916dc74ffcca765ff49ae11fd1cec5e2", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -255000 - }, - { - "included": -250000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 250000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 5000 - } - ] - }, - "installed_count": 60, - "installed_position": 50, - "node_id": 462, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "916dc74ffcca765ff49ae11fd1cec5e2", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -260000 - }, - { - "included": -255000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 255000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 5000 - } - ] - }, - "installed_count": 60, - "installed_position": 51, - "node_id": 465, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "916dc74ffcca765ff49ae11fd1cec5e2", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -265000 - }, - { - "included": -260000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 260000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 5000 - } - ] - }, - "installed_count": 60, - "installed_position": 52, - "node_id": 468, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "916dc74ffcca765ff49ae11fd1cec5e2", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -270000 - }, - { - "included": -265000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 265000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 5000 - } - ] - }, - "installed_count": 60, - "installed_position": 53, - "node_id": 471, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "916dc74ffcca765ff49ae11fd1cec5e2", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -275000 - }, - { - "included": -270000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 270000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 5000 - } - ] - }, - "installed_count": 60, - "installed_position": 54, - "node_id": 474, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "916dc74ffcca765ff49ae11fd1cec5e2", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -280000 - }, - { - "included": -275000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 275000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 5000 - } - ] - }, - "installed_count": 60, - "installed_position": 55, - "node_id": 477, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "916dc74ffcca765ff49ae11fd1cec5e2", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -285000 - }, - { - "included": -280000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 280000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 5000 - } - ] - }, - "installed_count": 60, - "installed_position": 56, - "node_id": 480, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "916dc74ffcca765ff49ae11fd1cec5e2", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -290000 - }, - { - "included": -285000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 285000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 5000 - } - ] - }, - "installed_count": 60, - "installed_position": 57, - "node_id": 483, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "916dc74ffcca765ff49ae11fd1cec5e2", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -295000 - }, - { - "included": -290000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 290000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 5000 - } - ] - }, - "installed_count": 60, - "installed_position": 58, - "node_id": 486, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 13 - }, - { - "coverage": { - "definition_id": "916dc74ffcca765ff49ae11fd1cec5e2", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -300000 - }, - { - "included": -295000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 295000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 5000 - } - ] - }, - "installed_count": 60, - "installed_position": 59, - "node_id": 489, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 13 - } - ], - "proven_optimal": true, - "queries": [ - "avg by (region) (quantile_over_time(0.75, data[100s]))", - "topk by (region) (4, increase(data_total[5m]))" - ], - "search_status": "optimal", - "selected": { - "accepted": [ - { - "accepted_metric": "relative", - "accepted_regardless_of_guarantee": false, - "guarantee_check_rejection": null, - "sketches": [ - "DDSketch" - ], - "summary": "metric not set; accepted relative (DDSketch)", - "target_metric": null - }, - { - "accepted_metric": null, - "accepted_regardless_of_guarantee": false, - "guarantee_check_rejection": null, - "sketches": [], - "summary": "metric not set; accepted exact", - "target_metric": null - } - ], - "cost": { - "cost_per_second": 6.44592483745e-6, - "cost_unit": "cost_per_second", - "cpu_core_seconds_per_second": 0.002778, - "per_node": { - "0": { - "cost_per_second": 6.52152e-9, - "cpu_core_seconds_per_second": 0.00058, - "detail": "scan 100000 samples; x 1.0000/s", - "multiplier": 1, - "operator": "Scan", - "price_source": "analytical", - "timing": "ingestion_time" - }, - "1": { - "cost_per_second": 6.3775772514e-6, - "cpu_core_seconds_per_second": 0.0002, - "detail": "build DDSketch in 100 tumbling windows of 1000 ms for 10000 groups: 100000 rows x (1 + depth 1); x 1.0000/s; keeps 5542880000 bytes", - "multiplier": 1, - "operator": "SummaryTumblingWindowChain", - "price_source": "analytical", - "timing": "ingestion_time" - }, - "10": { - "cost_per_second": 1.12440030089e-10, - "cpu_core_seconds_per_second": 0.00001, - "detail": "operator: pass 10000 rows; x 1.0000/s; holds 24 bytes for 1.1 ms", - "multiplier": 1, - "operator": "BinaryOp", - "price_source": "analytical", - "timing": "query_time" - }, - "11": { - "cost_per_second": 1.12440030089e-10, - "cpu_core_seconds_per_second": 0.00001, - "detail": "operator: pass 10000 rows; x 1.0000/s; holds 24 bytes for 1.1 ms", - "multiplier": 1, - "operator": "Project", - "price_source": "analytical", - "timing": "query_time" - }, - "12": { - "cost_per_second": 6.52152e-9, - "cpu_core_seconds_per_second": 0.00058, - "detail": "scan 100000 samples; x 1.0000/s", - "multiplier": 1, - "operator": "Scan", - "price_source": "analytical", - "timing": "ingestion_time" - }, - "13": { - "cost_per_second": 4.15390642488e-8, - "cpu_core_seconds_per_second": 0.0002, - "detail": "build exact Increase accumulator in 60 tumbling windows of 5000 ms for 10000 groups: 100000 rows x (1 + depth 1); x 1.0000/s; keeps 34160000 bytes", - "multiplier": 1, - "operator": "SummaryTumblingWindowChain", - "price_source": "analytical", - "timing": "ingestion_time" - }, - "14": { - "cost_per_second": 1.34936244514e-9, - "cpu_core_seconds_per_second": 0.00012, - "detail": "merge 600000 states into 10000; x 0.2000/s; holds 560000 bytes for 0.6 ms", - "multiplier": 1, - "operator": "SummaryMerge", - "price_source": "analytical", - "timing": "query_time" - }, - "15": { - "cost_per_second": 2.2488e-11, - "cpu_core_seconds_per_second": 2e-6, - "detail": "finalize 10000 accumulators; x 0.2000/s", - "multiplier": 1, - "operator": "FinalizeExactAccumulator", - "price_source": "analytical", - "timing": "query_time" - }, - "16": { - "cost_per_second": 2.24880070667e-11, - "cpu_core_seconds_per_second": 2e-6, - "detail": "sort 10000 rows in 10000 partitions; x 0.2000/s; holds 48 bytes for 0.6 ms", - "multiplier": 1, - "operator": "Sort", - "price_source": "analytical", - "timing": "query_time" - }, - "17": { - "cost_per_second": 2.24880070667e-11, - "cpu_core_seconds_per_second": 2e-6, - "detail": "limit to 10000 rows; x 0.2000/s; holds 48 bytes for 0.6 ms", - "multiplier": 1, - "operator": "Limit", - "price_source": "analytical", - "timing": "query_time" - }, - "18": { - "cost_per_second": 2.24880070667e-11, - "cpu_core_seconds_per_second": 2e-6, - "detail": "operator: pass 10000 rows; x 0.2000/s; holds 48 bytes for 0.6 ms", - "multiplier": 1, - "operator": "Project", - "price_source": "analytical", - "timing": "query_time" - }, - "2": { - "cost_per_second": 1.13128030496e-8, - "cpu_core_seconds_per_second": 0.001, - "detail": "merge 1000000 states into 10000; x 1.0000/s; holds 54880000 bytes for 1.1 ms", - "multiplier": 1, - "operator": "SummaryMerge", - "price_source": "analytical", - "timing": "query_time" - }, - "3": { - "cost_per_second": 1.1244e-10, - "cpu_core_seconds_per_second": 0.00001, - "detail": "estimate 10000 groups x 1; x 1.0000/s", - "multiplier": 1, - "operator": "SummaryEstimate", - "price_source": "analytical", - "timing": "query_time" - }, - "4": { - "cost_per_second": 1.12440060178e-10, - "cpu_core_seconds_per_second": 0.00001, - "detail": "operator: pass 10000 rows; x 1.0000/s; holds 48 bytes for 1.1 ms", - "multiplier": 1, - "operator": "Project", - "price_source": "analytical", - "timing": "query_time" - }, - "5": { - "cost_per_second": 1.13142071935e-10, - "cpu_core_seconds_per_second": 0.00001, - "detail": "build exact Sum accumulator into 10000 states: 10000 rows x depth 1; x 1.0000/s; holds 560000 bytes for 1.1 ms", - "multiplier": 1, - "operator": "SummaryAgg", - "price_source": "analytical", - "timing": "query_time" - }, - "6": { - "cost_per_second": 1.1244e-10, - "cpu_core_seconds_per_second": 0.00001, - "detail": "finalize 10000 accumulators; x 1.0000/s", - "multiplier": 1, - "operator": "FinalizeExactAccumulator", - "price_source": "analytical", - "timing": "query_time" - }, - "7": { - "cost_per_second": 1.13142071935e-10, - "cpu_core_seconds_per_second": 0.00001, - "detail": "build exact Count accumulator into 10000 states: 10000 rows x depth 1; x 1.0000/s; holds 560000 bytes for 1.1 ms", - "multiplier": 1, - "operator": "SummaryAgg", - "price_source": "analytical", - "timing": "query_time" - }, - "8": { - "cost_per_second": 1.1244e-10, - "cpu_core_seconds_per_second": 0.00001, - "detail": "finalize 10000 accumulators; x 1.0000/s", - "multiplier": 1, - "operator": "FinalizeExactAccumulator", - "price_source": "analytical", - "timing": "query_time" - }, - "9": { - "cost_per_second": 1.12440030089e-10, - "cpu_core_seconds_per_second": 0.00001, - "detail": "operator: pass 10000 rows; x 1.0000/s; holds 24 bytes for 1.1 ms", - "multiplier": 1, - "operator": "Project", - "price_source": "analytical", - "timing": "query_time" - } - }, - "source": "analytical-cost-v2 (illustrative statistics)", - "usage": { - "average_memory_bytes": 5577101111.83, - "cpu_ops_per_second": 1818000.0, - "disk_bytes": 0, - "latency_per_query": [ - { - "compaction_ms": 0.0, - "query_ms": 1.09 - }, - { - "compaction_ms": 0.0, - "query_ms": 0.64 - } - ], - "peak_cpu_cores": 3, - "peak_memory_bytes": 5633600360, - "scanned_bytes_per_second": 9600000.0 - } - }, - "dag": { - "edges": [ - { - "consumer": 1, - "data_state": { - "primitive": "Raw", - "timing": "ingestion_time" - }, - "grouping": "NotApplicable", - "intermediate_schema": { - "closed": true, - "fields": [ - { - "dtype": { - "Plain": "timestamp" - }, - "name": "ts", - "nullable": false, - "table": null - }, - { - "dtype": { - "Plain": "float64" - }, - "name": "value", - "nullable": false, - "table": null - }, - { - "dtype": { - "Plain": "utf8" - }, - "name": "region", - "nullable": true, - "table": null - }, - { - "dtype": { - "Plain": "utf8" - }, - "name": "$promql_series_identity", - "nullable": false, - "table": null - } - ], - "time_index": 0, - "unique_keys": [ - [ - 3, - 0 - ] - ] - }, - "producer": 0, - "role": "Input", - "window": "RequiresAlignedPanePhaseOrExactBoundaryResidual" - }, - { - "consumer": 2, - "data_state": { - "primitive": "SummaryState", - "timing": "ingestion_time" - }, - "grouping": "NotApplicable", - "intermediate_schema": { - "closed": true, - "fields": [ - { - "dtype": { - "Plain": "timestamp" - }, - "name": "ts", - "nullable": false, - "table": null - }, - { - "dtype": { - "Sketch": [ - { - "algorithm": "DDSketch", - "params": { - "DDSketch": { - "alpha": 0.05 - } - } - }, - "PerSubpopulationInstance" - ] - }, - "name": "state", - "nullable": false, - "table": null - }, - { - "dtype": { - "Plain": "utf8" - }, - "name": "region", - "nullable": true, - "table": null - }, - { - "dtype": { - "Plain": "utf8" - }, - "name": "$promql_series_identity", - "nullable": false, - "table": null - } - ], - "time_index": 0, - "unique_keys": [ - [ - 3, - 0 - ] - ] - }, - "producer": 1, - "role": "Input", - "window": "NotApplicable" - }, - { - "consumer": 3, - "data_state": { - "primitive": "SummaryState", - "timing": "query_time" - }, - "grouping": "NotApplicable", - "intermediate_schema": { - "closed": true, - "fields": [ - { - "dtype": { - "Plain": "timestamp" - }, - "name": "ts", - "nullable": false, - "table": null - }, - { - "dtype": { - "Sketch": [ - { - "algorithm": "DDSketch", - "params": { - "DDSketch": { - "alpha": 0.05 - } - } - }, - "PerSubpopulationInstance" - ] - }, - "name": "state", - "nullable": false, - "table": null - }, - { - "dtype": { - "Plain": "utf8" - }, - "name": "region", - "nullable": true, - "table": null - }, - { - "dtype": { - "Plain": "utf8" - }, - "name": "$promql_series_identity", - "nullable": false, - "table": null - } - ], - "time_index": 0, - "unique_keys": [ - [ - 3, - 0 - ] - ] - }, - "producer": 2, - "role": "Input", - "window": "NotApplicable" - }, - { - "consumer": 4, - "data_state": { - "primitive": "Raw", - "timing": "query_time" - }, - "grouping": "NotApplicable", - "intermediate_schema": { - "closed": true, - "fields": [ - { - "dtype": { - "Plain": "timestamp" - }, - "name": "ts", - "nullable": false, - "table": null - }, - { - "dtype": { - "Plain": "float64" - }, - "name": "quantile", - "nullable": false, - "table": null - }, - { - "dtype": { - "Plain": "utf8" - }, - "name": "region", - "nullable": true, - "table": null - }, - { - "dtype": { - "Plain": "utf8" - }, - "name": "$promql_series_identity", - "nullable": false, - "table": null - } - ], - "time_index": 0, - "unique_keys": [ - [ - 3, - 0 - ] - ] - }, - "producer": 3, - "role": "Input", - "window": "NotApplicable" - }, - { - "consumer": 5, - "data_state": { - "primitive": "Raw", - "timing": "query_time" - }, - "grouping": "NotApplicable", - "intermediate_schema": { - "closed": true, - "fields": [ - { - "dtype": { - "Plain": "timestamp" - }, - "name": "ts", - "nullable": false, - "table": null - }, - { - "dtype": { - "Plain": "float64" - }, - "name": "value", - "nullable": false, - "table": null - }, - { - "dtype": { - "Plain": "utf8" - }, - "name": "region", - "nullable": true, - "table": null - }, - { - "dtype": { - "Plain": "utf8" - }, - "name": "$promql_series_identity", - "nullable": false, - "table": null - } - ], - "time_index": 0, - "unique_keys": [ - [ - 3, - 0 - ] - ] - }, - "producer": 4, - "role": "Input", - "window": "NotApplicable" - }, - { - "consumer": 6, - "data_state": { - "primitive": "SummaryState", - "timing": "query_time" - }, - "grouping": "NotApplicable", - "intermediate_schema": { - "closed": true, - "fields": [ - { - "dtype": { - "Plain": "utf8" - }, - "name": "region", - "nullable": true, - "table": null - }, - { - "dtype": { - "ExactAggregate": [ - "Sum", - "Sum" - ] - }, - "name": "state", - "nullable": false, - "table": null - } - ], - "time_index": null, - "unique_keys": [ - [ - 0 - ] - ] - }, - "producer": 5, - "role": "Input", - "window": "NotApplicable" + { + "accepted_metric": null, + "sketches": [], + "summary": "accepted exact" + } + ], + "cost": { + "cost_per_second": 0.0000323837372788, + "cpu_core_seconds_per_second": 2.70394, + "per_node": { + "0": { + "cost_per_second": 6.52152e-9, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.00058, + "detail": "scan 100000 samples; x 1.0000/s", + "multiplier": 1, + "operator": "Scan", + "timing": "ingestion_time" }, - { - "consumer": 7, - "data_state": { - "primitive": "Raw", - "timing": "query_time" - }, - "grouping": "NotApplicable", - "intermediate_schema": { - "closed": true, - "fields": [ - { - "dtype": { - "Plain": "timestamp" - }, - "name": "ts", - "nullable": false, - "table": null - }, - { - "dtype": { - "Plain": "float64" - }, - "name": "value", - "nullable": false, - "table": null - }, - { - "dtype": { - "Plain": "utf8" - }, - "name": "region", - "nullable": true, - "table": null - }, - { - "dtype": { - "Plain": "utf8" - }, - "name": "$promql_series_identity", - "nullable": false, - "table": null - } - ], - "time_index": 0, - "unique_keys": [ - [ - 3, - 0 - ] - ] - }, - "producer": 4, - "role": "Input", - "window": "NotApplicable" + "1": { + "cost_per_second": 2.39800229468e-6, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.05127, + "detail": "build DDSketch in tumbling windows of 1000 ms for 10000 groups: 100000 rows x 247 ops, 1000 group visits, 10000.0 states emitted; x 1.0000/s; keeps 1583680000 bytes", + "multiplier": 1, + "operator": "SummaryTumblingWindowChain", + "timing": "ingestion_time" }, - { - "consumer": 8, - "data_state": { - "primitive": "SummaryState", - "timing": "query_time" - }, - "grouping": "NotApplicable", - "intermediate_schema": { - "closed": true, - "fields": [ - { - "dtype": { - "Plain": "utf8" - }, - "name": "region", - "nullable": true, - "table": null - }, - { - "dtype": { - "ExactAggregate": [ - "Count", - "Count" - ] - }, - "name": "state", - "nullable": false, - "table": null - } - ], - "time_index": null, - "unique_keys": [ - [ - 0 - ] - ] - }, - "producer": 7, - "role": "Input", - "window": "NotApplicable" + "10": { + "cost_per_second": 4.97058006872e-7, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.0442, + "detail": "merge 600000 states into 10000; x 0.2000/s; holds 1440000 bytes for 221.000 ms", + "multiplier": 1, + "operator": "SummaryMerge", + "timing": "query_time" }, - { - "consumer": 9, - "data_state": { - "primitive": "Raw", - "timing": "query_time" - }, - "grouping": "NotApplicable", - "intermediate_schema": { - "closed": true, - "fields": [ - { - "dtype": { - "Plain": "utf8" - }, - "name": "region", - "nullable": true, - "table": null - }, - { - "dtype": { - "Plain": "int64" - }, - "name": "count", - "nullable": false, - "table": null - } - ], - "time_index": null, - "unique_keys": [ - [ - 0 - ] - ] - }, - "producer": 8, - "role": "Input", - "window": "NotApplicable" + "11": { + "cost_per_second": 5.622e-8, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.005, + "detail": "finalize 10000 accumulators; x 0.2000/s", + "multiplier": 1, + "operator": "FinalizeExactAccumulator", + "timing": "query_time" }, - { - "consumer": 10, - "data_state": { - "primitive": "Raw", - "timing": "query_time" - }, - "grouping": "NotApplicable", - "intermediate_schema": { - "closed": true, - "fields": [ - { - "dtype": { - "Plain": "utf8" - }, - "name": "region", - "nullable": true, - "table": null - }, - { - "dtype": { - "Plain": "float64" - }, - "name": "sum", - "nullable": false, - "table": null - } - ], - "time_index": null, - "unique_keys": [ - [ - 0 - ] - ] - }, - "producer": 6, - "role": "Left", - "window": "NotApplicable" + "12": { + "cost_per_second": 6.19569750042e-8, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.00551, + "detail": "hash aggregate 10000 rows into 10000 groups: 85 ops per row, 10000 group visits; x 0.2000/s; holds 400000 bytes for 27.550 ms", + "multiplier": 1, + "operator": "Aggregate", + "timing": "query_time" }, - { - "consumer": 10, - "data_state": { - "primitive": "Raw", - "timing": "query_time" - }, - "grouping": "NotApplicable", - "intermediate_schema": { - "closed": true, - "fields": [ - { - "dtype": { - "Plain": "utf8" - }, - "name": "region", - "nullable": true, - "table": null - }, - { - "dtype": { - "Plain": "float64" - }, - "name": "col_1", - "nullable": false, - "table": null - } - ], - "time_index": null, - "unique_keys": [ - [ - 0 - ] - ] - }, - "producer": 9, - "role": "Right", - "window": "NotApplicable" + "2": { + "cost_per_second": 1.23684019838e-7, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.011, + "detail": "time range 100s: 100 tumbling windows of states; x 1.0000/s; holds 1568 bytes for 11.000 ms", + "multiplier": 1, + "operator": "TimeRange", + "timing": "query_time" }, - { - "consumer": 11, - "data_state": { - "primitive": "Raw", - "timing": "query_time" - }, - "grouping": "NotApplicable", - "intermediate_schema": { - "closed": true, - "fields": [ - { - "dtype": { - "Plain": "utf8" - }, - "name": "region", - "nullable": true, - "table": null - }, - { - "dtype": { - "Plain": "float64" - }, - "name": "sum", - "nullable": false, - "table": null - } - ], - "time_index": null, - "unique_keys": [ - [ - 0 - ] - ] - }, - "producer": 10, - "role": "Input", - "window": "NotApplicable" + "3": { + "cost_per_second": 0.0000282916772582, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 2.511, + "detail": "merge 1000000 states into 10000; x 1.0000/s; holds 20080000 bytes for 2511.000 ms", + "multiplier": 1, + "operator": "SummaryMerge", + "timing": "query_time" + }, + "4": { + "cost_per_second": 3.9354e-7, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.035, + "detail": "estimate 10000 groups x 1; x 1.0000/s", + "multiplier": 1, + "operator": "SummaryEstimate", + "timing": "query_time" + }, + "5": { + "cost_per_second": 1.12440000552e-10, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.00001, + "detail": "operator: pass 10000 rows; x 1.0000/s; holds 48 bytes for 0.010 ms", + "multiplier": 1, + "operator": "Project", + "timing": "query_time" + }, + "6": { + "cost_per_second": 3.02475975973e-7, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.0269, + "detail": "hash aggregate 10000 rows into 10000 groups: 20 ops per row, 10000 group visits; x 1.0000/s; holds 400000 bytes for 26.900 ms", + "multiplier": 1, + "operator": "Aggregate", + "timing": "query_time" + }, + "7": { + "cost_per_second": 6.52152e-9, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.00058, + "detail": "scan 100000 samples; x 1.0000/s", + "multiplier": 1, + "operator": "Scan", + "timing": "ingestion_time" + }, + "8": { + "cost_per_second": 2.31125188068e-7, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.01157, + "detail": "build exact Increase accumulator in tumbling windows of 5000 ms for 10000 groups: 100000 rows x 70 ops, 1000 group visits, 2000.0 states emitted; x 1.0000/s; keeps 87840000 bytes", + "multiplier": 1, + "operator": "SummaryTumblingWindowChain", + "timing": "ingestion_time" }, + "9": { + "cost_per_second": 1.48420802186e-8, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.00132, + "detail": "time range 300s: 60 tumbling windows of states; x 0.2000/s; holds 144 bytes for 6.600 ms", + "multiplier": 1, + "operator": "TimeRange", + "timing": "query_time" + } + }, + "source": "analytical-cost-v2 (illustrative statistics)", + "usage": { + "average_memory_bytes": 1722017509.44, + "cpu_ops_per_second": 2702980000.0, + "disk_bytes": 0, + "peak_cpu_cores": 4, + "peak_memory_bytes": 1693841856, + "scanned_bytes_per_second": 9600000.0 + } + }, + "dag": { + "edges": [ { - "consumer": 13, + "consumer": 1, "data_state": { "primitive": "Raw", "timing": "ingestion_time" @@ -6907,32 +202,28 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -6943,12 +234,12 @@ ] ] }, - "producer": 12, + "producer": 0, "role": "Input", - "window": "RequiresAlignedPanePhaseOrExactBoundaryResidual" + "window": "NotApplicable" }, { - "consumer": 14, + "consumer": 2, "data_state": { "primitive": "SummaryState", "timing": "ingestion_time" @@ -6961,52 +252,46 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { - "ExactAggregate": [ - "Increase", - "Increase" - ] + "Sketch": "DDSketch" }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 3, - 0 + 0, + 2, + 3 ] ] }, - "producer": 13, + "producer": 1, "role": "Input", "window": "NotApplicable" }, { - "consumer": 15, + "consumer": 3, "data_state": { "primitive": "SummaryState", "timing": "query_time" @@ -7019,54 +304,48 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { - "ExactAggregate": [ - "Increase", - "Increase" - ] + "Sketch": "DDSketch" }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 3, - 0 + 0, + 2, + 3 ] ] }, - "producer": 14, + "producer": 2, "role": "Input", "window": "NotApplicable" }, { - "consumer": 16, + "consumer": 4, "data_state": { - "primitive": "Raw", + "primitive": "SummaryState", "timing": "query_time" }, "grouping": "NotApplicable", @@ -7078,32 +357,28 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { - "Plain": "float64" + "Sketch": "DDSketch" }, - "name": "value", - "nullable": false, - "table": null + "name": "state", + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -7114,12 +389,12 @@ ] ] }, - "producer": 15, + "producer": 3, "role": "Input", "window": "NotApplicable" }, { - "consumer": 17, + "consumer": 5, "data_state": { "primitive": "Raw", "timing": "query_time" @@ -7133,32 +408,28 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, - "name": "value", - "nullable": false, - "table": null + "name": "quantile", + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -7169,12 +440,12 @@ ] ] }, - "producer": 16, + "producer": 4, "role": "Input", "window": "NotApplicable" }, { - "consumer": 18, + "consumer": 6, "data_state": { "primitive": "Raw", "timing": "query_time" @@ -7188,32 +459,28 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -7224,34 +491,18 @@ ] ] }, - "producer": 17, + "producer": 5, "role": "Input", "window": "NotApplicable" - } - ], - "nodes": [ + }, { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact over exact inputs" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } + "consumer": 8, + "data_state": { + "primitive": "Raw", + "timing": "ingestion_time" }, - "id": 0, - "lifetime": "None", - "output_schema": { + "grouping": "NotApplicable", + "intermediate_schema": { "closed": true, "fields": [ { @@ -7259,204 +510,160 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ 3, - 0 - ] - ] - }, - "output_state": { - "primitive": "Raw", - "timing": "ingestion_time" - }, - "payload": { - "NonASAP": { - "Scan": { - "predicates": [], - "schema": { - "closed": true, - "fields": [ - { - "dtype": { - "Plain": "timestamp" - }, - "name": "ts", - "nullable": false, - "table": null - }, - { - "dtype": { - "Plain": "float64" - }, - "name": "value", - "nullable": false, - "table": null - }, - { - "dtype": { - "Plain": "utf8" - }, - "name": "region", - "nullable": true, - "table": null - }, - { - "dtype": { - "Plain": "utf8" - }, - "name": "$promql_series_identity", - "nullable": false, - "table": null - } - ], - "time_index": 0, - "unique_keys": [ - [ - 3, - 0 - ] - ] - }, - "source": { - "TimeSeries": { - "metric": "data" - } - } - } - } - } + 0 + ] + ] + }, + "producer": 7, + "role": "Input", + "window": "NotApplicable" }, { - "guarantee": null, - "id": 1, - "lifetime": { - "AcrossEvaluations": "Memory" + "consumer": 9, + "data_state": { + "primitive": "SummaryState", + "timing": "ingestion_time" }, - "output_schema": { + "grouping": "NotApplicable", + "intermediate_schema": { "closed": true, "fields": [ { "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { - "Sketch": [ - { - "algorithm": "DDSketch", - "params": { - "DDSketch": { - "alpha": 0.05 - } - } - }, - "PerSubpopulationInstance" + "ExactAggregate": [ + "Increase", + "Increase" ] }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 3, - 0 + 0, + 2, + 3 ] ] }, - "output_state": { + "producer": 8, + "role": "Input", + "window": "NotApplicable" + }, + { + "consumer": 10, + "data_state": { "primitive": "SummaryState", - "timing": "ingestion_time" + "timing": "query_time" }, - "payload": { - "ASAP": { - "SummaryTumblingWindowChain": { - "child": 0, - "family": { - "Sketch": [ - { - "algorithm": "DDSketch", - "params": { - "DDSketch": { - "alpha": 0.05 - } - } - }, - "PerSubpopulationInstance" + "grouping": "NotApplicable", + "intermediate_schema": { + "closed": true, + "fields": [ + { + "dtype": { + "Plain": "timestamp" + }, + "name": "window_end", + "nullable": false + }, + { + "dtype": { + "ExactAggregate": [ + "Increase", + "Increase" ] }, - "filter": null, - "grouping": "PerSubpopulationInstance", - "input": { - "item": null, - "weight": { - "Column": "SampleValue" - } + "name": "state", + "nullable": false + }, + { + "dtype": { + "Plain": "utf8" }, - "reduction": "PerEntity", - "tumbling_window_count": 100, - "tumbling_window_width_ms": 1000 + "name": "region", + "nullable": true + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false } - } - } + ], + "time_index": 0, + "unique_keys": [ + [ + 0, + 2, + 3 + ] + ] + }, + "producer": 9, + "role": "Input", + "window": "NotApplicable" }, { - "guarantee": null, - "id": 2, - "lifetime": "None", - "output_schema": { + "consumer": 11, + "data_state": { + "primitive": "SummaryState", + "timing": "query_time" + }, + "grouping": "NotApplicable", + "intermediate_schema": { "closed": true, "fields": [ { @@ -7464,42 +671,31 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { - "Sketch": [ - { - "algorithm": "DDSketch", - "params": { - "DDSketch": { - "alpha": 0.05 - } - } - }, - "PerSubpopulationInstance" + "ExactAggregate": [ + "Increase", + "Increase" ] }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -7510,59 +706,18 @@ ] ] }, - "output_state": { - "primitive": "SummaryState", - "timing": "query_time" - }, - "payload": { - "ASAP": { - "SummaryMerge": { - "children": [ - 1 - ] - } - } - } + "producer": 10, + "role": "Input", + "window": "NotApplicable" }, { - "guarantee": { - "provenance": [ - { - "algorithm": "DDSketch", - "contract": "ddsketch_relative_error_alpha_v1", - "kind": "sketch_evaluation", - "params": { - "DDSketch": { - "alpha": 0.05 - } - }, - "query": "Quantile { q: 0.75 }" - } - ], - "spec": { - "accuracy_metric": "relative", - "accuracy_scope": "per_value", - "error": { - "factors": [ - { - "op": "constant", - "value": 0.05 - }, - { - "op": "unknown", - "statistic": "values_non_negative[data]" - } - ], - "op": "product" - }, - "failure_probability": { - "op": "zero" - } - } + "consumer": 12, + "data_state": { + "primitive": "Raw", + "timing": "query_time" }, - "id": 3, - "lifetime": "None", - "output_schema": { + "grouping": "NotApplicable", + "intermediate_schema": { "closed": true, "fields": [ { @@ -7570,32 +725,28 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, - "name": "quantile", - "nullable": false, - "table": null + "name": "value", + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -7606,60 +757,14 @@ ] ] }, - "output_state": { - "primitive": "Raw", - "timing": "query_time" - }, - "payload": { - "ASAP": { - "SummaryEstimate": { - "query": { - "Quantile": { - "q": 0.75 - } - }, - "summary_input": 2 - } - } - } - }, + "producer": 11, + "role": "Input", + "window": "NotApplicable" + } + ], + "nodes": [ { - "guarantee": { - "provenance": [ - { - "algorithm": "DDSketch", - "contract": "ddsketch_relative_error_alpha_v1", - "kind": "sketch_evaluation", - "params": { - "DDSketch": { - "alpha": 0.05 - } - }, - "query": "Quantile { q: 0.75 }" - } - ], - "spec": { - "accuracy_metric": "relative", - "accuracy_scope": "per_value", - "error": { - "factors": [ - { - "op": "constant", - "value": 0.05 - }, - { - "op": "unknown", - "statistic": "values_non_negative[data]" - } - ], - "op": "product" - }, - "failure_probability": { - "op": "zero" - } - } - }, - "id": 4, + "id": 0, "lifetime": "None", "output_schema": { "closed": true, @@ -7669,32 +774,28 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -7707,427 +808,314 @@ }, "output_state": { "primitive": "Raw", - "timing": "query_time" + "timing": "ingestion_time" }, "payload": { "NonASAP": { - "Project": { - "child": 3, - "cols": [ - { - "alias": "ts", - "expr": { - "Column": 0 - } - }, - { - "alias": "value", - "expr": { - "Column": 1 - } - }, - { - "alias": "region", - "expr": { - "Column": 2 - } - }, - { - "alias": "$promql_series_identity", - "expr": { - "Column": 3 + "Scan": { + "predicates": [], + "schema": { + "closed": true, + "fields": [ + { + "dtype": { + "Plain": "timestamp" + }, + "name": "ts", + "nullable": false + }, + { + "dtype": { + "Plain": "float64" + }, + "name": "value", + "nullable": false + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "region", + "nullable": true + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false } + ], + "time_index": 0, + "unique_keys": [ + [ + 3, + 0 + ] + ] + }, + "source": { + "TimeSeries": { + "metric": "data" } - ], - "qualifier": null + } } } } }, { - "guarantee": null, - "id": 5, - "lifetime": "None", + "id": 1, + "lifetime": { + "AcrossEvaluations": "Memory" + }, "output_schema": { "closed": true, "fields": [ + { + "dtype": { + "Plain": "timestamp" + }, + "name": "window_end", + "nullable": false + }, + { + "dtype": { + "Sketch": "DDSketch" + }, + "name": "state", + "nullable": false + }, { "dtype": { "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { - "ExactAggregate": [ - "Sum", - "Sum" - ] + "Plain": "utf8" }, - "name": "state", - "nullable": false, - "table": null + "name": "$promql_series_identity", + "nullable": false } ], - "time_index": null, + "time_index": 0, "unique_keys": [ [ - 0 + 0, + 2, + 3 ] ] }, "output_state": { "primitive": "SummaryState", - "timing": "query_time" + "timing": "ingestion_time" + }, + "parameters": { + "DDSketch": { + "alpha": 0.05 + } }, "payload": { "ASAP": { - "SummaryAgg": { - "child": 4, - "family": { - "ExactAggregate": [ - "Sum", - "Sum" - ] - }, + "SummaryTumblingWindowChain": { + "child": 0, "filter": null, - "grouping": "PerSubpopulationInstance", "input": { "item": null, "weight": { "Column": "SampleValue" } }, - "reduction": { - "Reduce": [ - 2 - ] - } + "reduction": "PerEntity", + "summary_type": { + "Sketch": "DDSketch" + }, + "tumbling_window_width_ms": 1000 } } + }, + "retention_ms": 100000, + "view_coverage": { + "definition_id": "0c7f962d8fde713c1de7a58752019151", + "selection": [ + { + "columns": [] + } + ] } }, { - "guarantee": { - "provenance": [ - { - "kind": "unavailable_statistic", - "statistic": "input_row_count" - }, - { - "kind": "unavailable_statistic", - "statistic": "values_non_negative[data]" - }, + "id": 2, + "lifetime": "None", + "output_schema": { + "closed": true, + "fields": [ { - "guarantee": { - "provenance": [ - { - "algorithm": "DDSketch", - "contract": "ddsketch_relative_error_alpha_v1", - "kind": "sketch_evaluation", - "params": { - "DDSketch": { - "alpha": 0.05 - } - }, - "query": "Quantile { q: 0.75 }" - } - ], - "spec": { - "accuracy_metric": "relative", - "accuracy_scope": "per_value", - "error": { - "factors": [ - { - "op": "constant", - "value": 0.05 - }, - { - "op": "unknown", - "statistic": "values_non_negative[data]" - } - ], - "op": "product" - }, - "failure_probability": { - "op": "zero" - } - } + "dtype": { + "Plain": "timestamp" }, - "input_index": 0, - "kind": "child_guarantee" - }, - { - "kind": "exact", - "reason": "exact aggregation" + "name": "window_end", + "nullable": false }, { - "kind": "composition_step", - "operator": { - "op": "exact_sum" + "dtype": { + "Sketch": "DDSketch" }, - "rule": "exact_sum_relative" - } - ], - "spec": { - "accuracy_metric": "relative", - "accuracy_scope": "per_value", - "error": { - "factors": [ - { - "op": "max", - "terms": [ - { - "factors": [ - { - "op": "constant", - "value": 0.05 - }, - { - "op": "unknown", - "statistic": "values_non_negative[data]" - } - ], - "op": "product" - } - ] - }, - { - "op": "unknown", - "statistic": "values_non_negative[data]" - } - ], - "op": "product" + "name": "state", + "nullable": false }, - "failure_probability": { - "count": { - "op": "unknown", - "statistic": "input_row_count" - }, - "inner": { - "op": "union_bound", - "terms": [ - { - "op": "zero" - } - ] - }, - "op": "scaled" - } - } - }, - "id": 6, - "lifetime": "None", - "output_schema": { - "closed": true, - "fields": [ { "dtype": { "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { - "Plain": "float64" + "Plain": "utf8" }, - "name": "sum", - "nullable": false, - "table": null + "name": "$promql_series_identity", + "nullable": false } ], - "time_index": null, + "time_index": 0, "unique_keys": [ [ - 0 + 0, + 2, + 3 ] ] }, "output_state": { - "primitive": "Raw", + "primitive": "SummaryState", "timing": "query_time" }, + "parameters": { + "DDSketch": { + "alpha": 0.05 + } + }, "payload": { - "ASAP": { - "FinalizeExactAccumulator": { - "child": 5 + "NonASAP": { + "TimeRange": { + "child": 1, + "kind": "range", + "range": { + "nanos": 0, + "secs": 100 + } } } } }, { - "guarantee": null, - "id": 7, - "lifetime": "None", + "id": 3, + "lifetime": { + "OneEvaluation": "Memory" + }, "output_schema": { "closed": true, "fields": [ { "dtype": { - "Plain": "utf8" + "Plain": "timestamp" }, - "name": "region", - "nullable": true, - "table": null + "name": "ts", + "nullable": false }, { "dtype": { - "ExactAggregate": [ - "Count", - "Count" - ] + "Sketch": "DDSketch" }, "name": "state", - "nullable": false, - "table": null - } - ], - "time_index": null, - "unique_keys": [ - [ - 0 - ] - ] - }, - "output_state": { - "primitive": "SummaryState", - "timing": "query_time" - }, - "payload": { - "ASAP": { - "SummaryAgg": { - "child": 4, - "family": { - "ExactAggregate": [ - "Count", - "Count" - ] - }, - "filter": null, - "grouping": "PerSubpopulationInstance", - "input": { - "item": null, - "weight": { - "Column": "SampleValue" - } - }, - "reduction": { - "Reduce": [ - 2 - ] - } - } - } - } - }, - { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "count of rows" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" + "nullable": false }, - "failure_probability": { - "op": "zero" - } - } - }, - "id": 8, - "lifetime": "None", - "output_schema": { - "closed": true, - "fields": [ { "dtype": { "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { - "Plain": "int64" + "Plain": "utf8" }, - "name": "count", - "nullable": false, - "table": null + "name": "$promql_series_identity", + "nullable": false } ], - "time_index": null, + "time_index": 0, "unique_keys": [ [ + 3, 0 ] ] }, "output_state": { - "primitive": "Raw", + "primitive": "SummaryState", "timing": "query_time" }, + "parameters": { + "DDSketch": { + "alpha": 0.05 + } + }, "payload": { "ASAP": { - "FinalizeExactAccumulator": { - "child": 7 + "SummaryMerge": { + "children": [ + 2 + ] } } } }, { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact over exact inputs" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, - "id": 9, + "id": 4, "lifetime": "None", "output_schema": { "closed": true, "fields": [ + { + "dtype": { + "Plain": "timestamp" + }, + "name": "ts", + "nullable": false + }, + { + "dtype": { + "Plain": "float64" + }, + "name": "quantile", + "nullable": false + }, { "dtype": { "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { - "Plain": "float64" + "Plain": "utf8" }, - "name": "col_1", - "nullable": false, - "table": null + "name": "$promql_series_identity", + "nullable": false } ], - "time_index": null, + "time_index": 0, "unique_keys": [ [ + 3, 0 ] ] @@ -8137,168 +1125,57 @@ "timing": "query_time" }, "payload": { - "NonASAP": { - "Project": { - "child": 8, - "cols": [ - { - "alias": null, - "expr": { - "Column": 0 - } - }, - { - "alias": null, - "expr": { - "Cast": { - "expr": { - "Column": 1 - }, - "to": "float64", - "try_cast": false - } - } + "ASAP": { + "SummaryEstimate": { + "query": { + "Quantile": { + "q": 0.75 } - ], - "qualifier": null + }, + "summary_input": 3 } } } }, { - "guarantee": { - "provenance": [ - { - "kind": "unavailable_statistic", - "statistic": "input_row_count" - }, - { - "kind": "unavailable_statistic", - "statistic": "values_non_negative[data]" - }, + "id": 5, + "lifetime": "None", + "output_schema": { + "closed": true, + "fields": [ { - "guarantee": { - "provenance": [ - { - "algorithm": "DDSketch", - "contract": "ddsketch_relative_error_alpha_v1", - "kind": "sketch_evaluation", - "params": { - "DDSketch": { - "alpha": 0.05 - } - }, - "query": "Quantile { q: 0.75 }" - } - ], - "spec": { - "accuracy_metric": "relative", - "accuracy_scope": "per_value", - "error": { - "factors": [ - { - "op": "constant", - "value": 0.05 - }, - { - "op": "unknown", - "statistic": "values_non_negative[data]" - } - ], - "op": "product" - }, - "failure_probability": { - "op": "zero" - } - } + "dtype": { + "Plain": "timestamp" }, - "input_index": 0, - "kind": "child_guarantee" - }, - { - "kind": "exact", - "reason": "exact aggregation" + "name": "ts", + "nullable": false }, { - "kind": "composition_step", - "operator": { - "op": "exact_sum" + "dtype": { + "Plain": "float64" }, - "rule": "exact_sum_relative" - } - ], - "spec": { - "accuracy_metric": "relative", - "accuracy_scope": "per_value", - "error": { - "factors": [ - { - "op": "max", - "terms": [ - { - "factors": [ - { - "op": "constant", - "value": 0.05 - }, - { - "op": "unknown", - "statistic": "values_non_negative[data]" - } - ], - "op": "product" - } - ] - }, - { - "op": "unknown", - "statistic": "values_non_negative[data]" - } - ], - "op": "product" + "name": "value", + "nullable": false }, - "failure_probability": { - "count": { - "op": "unknown", - "statistic": "input_row_count" - }, - "inner": { - "op": "union_bound", - "terms": [ - { - "op": "zero" - } - ] - }, - "op": "scaled" - } - } - }, - "id": 10, - "lifetime": "None", - "output_schema": { - "closed": true, - "fields": [ { "dtype": { "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { - "Plain": "float64" + "Plain": "utf8" }, - "name": "sum", - "nullable": false, - "table": null + "name": "$promql_series_identity", + "nullable": false } ], - "time_index": null, + "time_index": 0, "unique_keys": [ [ + 3, 0 ] ] @@ -8309,133 +1186,43 @@ }, "payload": { "NonASAP": { - "BinaryOp": { - "lhs": 6, - "operator": { - "checked_finite_division": false, - "checked_relative_division": false, - "kind": { - "Arithmetic": "Div" - }, - "vector_match": null - }, - "return_bool": false, - "rhs": 9 - } - } - } - }, - { - "guarantee": { - "provenance": [ - { - "kind": "unavailable_statistic", - "statistic": "input_row_count" - }, - { - "kind": "unavailable_statistic", - "statistic": "values_non_negative[data]" - }, - { - "guarantee": { - "provenance": [ - { - "algorithm": "DDSketch", - "contract": "ddsketch_relative_error_alpha_v1", - "kind": "sketch_evaluation", - "params": { - "DDSketch": { - "alpha": 0.05 - } - }, - "query": "Quantile { q: 0.75 }" + "Project": { + "child": 4, + "cols": [ + { + "alias": "ts", + "expr": { + "Column": 0 } - ], - "spec": { - "accuracy_metric": "relative", - "accuracy_scope": "per_value", - "error": { - "factors": [ - { - "op": "constant", - "value": 0.05 - }, - { - "op": "unknown", - "statistic": "values_non_negative[data]" - } - ], - "op": "product" - }, - "failure_probability": { - "op": "zero" + }, + { + "alias": "value", + "expr": { + "Column": 1 } - } - }, - "input_index": 0, - "kind": "child_guarantee" - }, - { - "kind": "exact", - "reason": "exact aggregation" - }, - { - "kind": "composition_step", - "operator": { - "op": "exact_sum" - }, - "rule": "exact_sum_relative" - } - ], - "spec": { - "accuracy_metric": "relative", - "accuracy_scope": "per_value", - "error": { - "factors": [ + }, { - "op": "max", - "terms": [ - { - "factors": [ - { - "op": "constant", - "value": 0.05 - }, - { - "op": "unknown", - "statistic": "values_non_negative[data]" - } - ], - "op": "product" - } - ] + "alias": "region", + "expr": { + "Column": 2 + } }, { - "op": "unknown", - "statistic": "values_non_negative[data]" - } - ], - "op": "product" - }, - "failure_probability": { - "count": { - "op": "unknown", - "statistic": "input_row_count" - }, - "inner": { - "op": "union_bound", - "terms": [ - { - "op": "zero" + "alias": "$promql_series_identity", + "expr": { + "Column": 3 } - ] - }, - "op": "scaled" + } + ] } } + } + }, + { + "id": 6, + "lifetime": { + "OneEvaluation": "Memory" }, - "id": 11, - "lifetime": "None", "output_schema": { "closed": true, "fields": [ @@ -8444,16 +1231,14 @@ "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "float64" }, "name": "avg", - "nullable": false, - "table": null + "nullable": false } ], "time_index": null, @@ -8469,47 +1254,30 @@ }, "payload": { "NonASAP": { - "Project": { - "child": 10, - "cols": [ - { - "alias": "region", - "expr": { - "Column": 0 - } - }, + "Aggregate": { + "child": 5, + "filters": [], + "having": null, + "measures": [ { - "alias": "avg", - "expr": { - "Column": 1 - } + "col": null, + "kind": "avg" } ], - "qualifier": null + "output_names": [ + "" + ], + "reduction": { + "Reduce": [ + 2 + ] + } } } } }, { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact over exact inputs" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, - "id": 12, + "id": 7, "lifetime": "None", "output_schema": { "closed": true, @@ -8519,32 +1287,28 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -8571,32 +1335,28 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -8617,8 +1377,7 @@ } }, { - "guarantee": null, - "id": 13, + "id": 8, "lifetime": { "AcrossEvaluations": "Memory" }, @@ -8629,9 +1388,8 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { @@ -8641,31 +1399,29 @@ ] }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 3, - 0 + 0, + 2, + 3 ] ] }, @@ -8676,15 +1432,8 @@ "payload": { "ASAP": { "SummaryTumblingWindowChain": { - "child": 12, - "family": { - "ExactAggregate": [ - "Increase", - "Increase" - ] - }, + "child": 7, "filter": null, - "grouping": "PerSubpopulationInstance", "input": { "item": null, "weight": { @@ -8692,15 +1441,28 @@ } }, "reduction": "PerEntity", - "tumbling_window_count": 60, + "summary_type": { + "ExactAggregate": [ + "Increase", + "Increase" + ] + }, "tumbling_window_width_ms": 5000 } } + }, + "retention_ms": 300000, + "view_coverage": { + "definition_id": "b876d20377d09364dbbaba79f3786982", + "selection": [ + { + "columns": [] + } + ] } }, { - "guarantee": null, - "id": 14, + "id": 9, "lifetime": "None", "output_schema": { "closed": true, @@ -8709,9 +1471,8 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { @@ -8721,31 +1482,29 @@ ] }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 3, - 0 + 0, + 2, + 3 ] ] }, @@ -8754,36 +1513,23 @@ "timing": "query_time" }, "payload": { - "ASAP": { - "SummaryMerge": { - "children": [ - 13 - ] + "NonASAP": { + "TimeRange": { + "child": 8, + "kind": "range", + "range": { + "nanos": 0, + "secs": 300 + } } } } }, { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact accumulator over exact input" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } + "id": 10, + "lifetime": { + "OneEvaluation": "Memory" }, - "id": 15, - "lifetime": "None", "output_schema": { "closed": true, "fields": [ @@ -8792,32 +1538,31 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { - "Plain": "float64" + "ExactAggregate": [ + "Increase", + "Increase" + ] }, - "name": "value", - "nullable": false, - "table": null + "name": "state", + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -8829,126 +1574,21 @@ ] }, "output_state": { - "primitive": "Raw", + "primitive": "SummaryState", "timing": "query_time" }, "payload": { "ASAP": { - "FinalizeExactAccumulator": { - "child": 14 - } - } - } - }, - { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact over exact inputs" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, - "id": 16, - "lifetime": "None", - "output_schema": { - "closed": true, - "fields": [ - { - "dtype": { - "Plain": "timestamp" - }, - "name": "ts", - "nullable": false, - "table": null - }, - { - "dtype": { - "Plain": "float64" - }, - "name": "value", - "nullable": false, - "table": null - }, - { - "dtype": { - "Plain": "utf8" - }, - "name": "region", - "nullable": true, - "table": null - }, - { - "dtype": { - "Plain": "utf8" - }, - "name": "$promql_series_identity", - "nullable": false, - "table": null - } - ], - "time_index": 0, - "unique_keys": [ - [ - 3, - 0 - ] - ] - }, - "output_state": { - "primitive": "Raw", - "timing": "query_time" - }, - "payload": { - "NonASAP": { - "Sort": { - "child": 15, - "keys": [ - { - "ascending": false, - "expr": { - "Column": 1 - }, - "nulls_first": false - } - ], - "partition_by": [ - 2 + "SummaryMerge": { + "children": [ + 9 ] } } } }, { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact over exact inputs" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, - "id": 17, + "id": 11, "lifetime": "None", "output_schema": { "closed": true, @@ -8958,32 +1598,28 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -8999,39 +1635,18 @@ "timing": "query_time" }, "payload": { - "NonASAP": { - "Limit": { - "child": 16, - "n": 4, - "offset": 0, - "partition_by": [ - 2 - ] + "ASAP": { + "FinalizeExactAccumulator": { + "child": 10 } } } }, { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact over exact inputs" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } + "id": 12, + "lifetime": { + "OneEvaluation": "Memory" }, - "id": 18, - "lifetime": "None", "output_schema": { "closed": true, "fields": [ @@ -9040,28 +1655,30 @@ "Plain": "utf8" }, "name": "region", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "float64" }, "name": "topk_4", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": null, - "unique_keys": [] + "unique_keys": [ + [ + 0, + 2 + ] + ] }, "output_state": { "primitive": "Raw", @@ -9069,44 +1686,38 @@ }, "payload": { "NonASAP": { - "Project": { - "child": 17, - "cols": [ - { - "alias": "region", - "expr": { - "Column": 2 - } - }, - { - "alias": "topk_4", - "expr": { - "Column": 1 - } - }, + "Aggregate": { + "child": 11, + "filters": [], + "having": null, + "measures": [ { - "alias": "$promql_series_identity", - "expr": { - "Column": 3 - } + "k": 4, + "kind": "top_k" } ], - "qualifier": null + "output_names": [ + "" + ], + "reduction": { + "Reduce": [ + 2 + ] + } } } } } ], "roots": [ - 11, - 18 + 6, + 12 ], "scalar_roots": [] - }, - "display_name": "Q1 Sum acc + DDSketch ×100 tumbling windows + Count acc · Q2 Increase acc ×60 tumbling windows · ingestion time: DDSketch for Q1, 100 × 1s tumbling windows, Increase acc for Q2, 60 × 5s tumbling windows" + } }, - "tumbling_window_retention_ms": { - "1": 100000, - "13": 300000 - } + "warnings": [ + "assumed data fact: source data declares no column_magnitudes for value, so a DDSketch state over it is costed for magnitudes in [1, 1000000000]; this affects cost only, the accuracy guarantee does not depend on it", + "assumed data fact: the data arrival declares no max_out_of_order_ms, so the samples of source data_total that an ingestion-time rate, increase or irate state reads are taken to arrive in timestamp order; if this does not hold, answers may be wrong by an amount no accuracy target bounds (a late sample can be read as a counter reset), or ingestion may fail" + ] } diff --git a/crates/devtools/tests/sketch_plan_snapshots/ratio-of-quantiles-1h-mvp.json b/crates/devtools/tests/sketch_plan_snapshots/ratio-of-quantiles-1h-mvp.json index dda03f31b..08060d289 100644 --- a/crates/devtools/tests/sketch_plan_snapshots/ratio-of-quantiles-1h-mvp.json +++ b/crates/devtools/tests/sketch_plan_snapshots/ratio-of-quantiles-1h-mvp.json @@ -3,2403 +3,127 @@ "materialized_views": [ { "coverage": { - "definition_id": "ef4518647caf2874d0f473f802792403", + "definition_id": "5597d603bdf9f1a148d7c7e5344c1a97", "selection": [ { - "columns": [], - "relative_time": [ - { - "excluded": -60000 - }, - { - "included": 0 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 0, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 0, - "node_id": 2, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef4518647caf2874d0f473f802792403", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -120000 - }, - { - "included": -60000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 60000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 1, - "node_id": 5, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef4518647caf2874d0f473f802792403", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -180000 - }, - { - "included": -120000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 120000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 2, - "node_id": 8, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef4518647caf2874d0f473f802792403", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -240000 - }, - { - "included": -180000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 180000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 3, - "node_id": 11, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef4518647caf2874d0f473f802792403", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -300000 - }, - { - "included": -240000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 240000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 4, - "node_id": 14, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef4518647caf2874d0f473f802792403", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -360000 - }, - { - "included": -300000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 300000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 5, - "node_id": 17, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef4518647caf2874d0f473f802792403", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -420000 - }, - { - "included": -360000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 360000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 6, - "node_id": 20, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef4518647caf2874d0f473f802792403", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -480000 - }, - { - "included": -420000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 420000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 7, - "node_id": 23, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef4518647caf2874d0f473f802792403", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -540000 - }, - { - "included": -480000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 480000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 8, - "node_id": 26, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef4518647caf2874d0f473f802792403", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -600000 - }, - { - "included": -540000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 540000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 9, - "node_id": 29, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef4518647caf2874d0f473f802792403", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -660000 - }, - { - "included": -600000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 600000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 10, - "node_id": 32, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef4518647caf2874d0f473f802792403", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -720000 - }, - { - "included": -660000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 660000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 11, - "node_id": 35, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef4518647caf2874d0f473f802792403", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -780000 - }, - { - "included": -720000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 720000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 12, - "node_id": 38, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef4518647caf2874d0f473f802792403", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -840000 - }, - { - "included": -780000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 780000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 13, - "node_id": 41, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef4518647caf2874d0f473f802792403", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -900000 - }, - { - "included": -840000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 840000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 14, - "node_id": 44, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef4518647caf2874d0f473f802792403", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -960000 - }, - { - "included": -900000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 900000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 15, - "node_id": 47, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef4518647caf2874d0f473f802792403", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1020000 - }, - { - "included": -960000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 960000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 16, - "node_id": 50, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef4518647caf2874d0f473f802792403", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1080000 - }, - { - "included": -1020000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1020000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 17, - "node_id": 53, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef4518647caf2874d0f473f802792403", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1140000 - }, - { - "included": -1080000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1080000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 18, - "node_id": 56, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef4518647caf2874d0f473f802792403", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1200000 - }, - { - "included": -1140000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1140000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 19, - "node_id": 59, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef4518647caf2874d0f473f802792403", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1260000 - }, - { - "included": -1200000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1200000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 20, - "node_id": 62, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef4518647caf2874d0f473f802792403", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1320000 - }, - { - "included": -1260000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1260000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 21, - "node_id": 65, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef4518647caf2874d0f473f802792403", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1380000 - }, - { - "included": -1320000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1320000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 22, - "node_id": 68, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef4518647caf2874d0f473f802792403", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1440000 - }, - { - "included": -1380000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1380000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 23, - "node_id": 71, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef4518647caf2874d0f473f802792403", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1500000 - }, - { - "included": -1440000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1440000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 24, - "node_id": 74, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef4518647caf2874d0f473f802792403", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1560000 - }, - { - "included": -1500000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1500000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 25, - "node_id": 77, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef4518647caf2874d0f473f802792403", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1620000 - }, - { - "included": -1560000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1560000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 26, - "node_id": 80, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef4518647caf2874d0f473f802792403", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1680000 - }, - { - "included": -1620000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1620000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 27, - "node_id": 83, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef4518647caf2874d0f473f802792403", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1740000 - }, - { - "included": -1680000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1680000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 28, - "node_id": 86, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef4518647caf2874d0f473f802792403", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1800000 - }, - { - "included": -1740000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1740000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 29, - "node_id": 89, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef4518647caf2874d0f473f802792403", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1860000 - }, - { - "included": -1800000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1800000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 30, - "node_id": 92, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef4518647caf2874d0f473f802792403", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1920000 - }, - { - "included": -1860000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1860000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 31, - "node_id": 95, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef4518647caf2874d0f473f802792403", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1980000 - }, - { - "included": -1920000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1920000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 32, - "node_id": 98, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef4518647caf2874d0f473f802792403", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2040000 - }, - { - "included": -1980000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1980000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 33, - "node_id": 101, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef4518647caf2874d0f473f802792403", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2100000 - }, - { - "included": -2040000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2040000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 34, - "node_id": 104, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef4518647caf2874d0f473f802792403", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2160000 - }, - { - "included": -2100000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2100000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 35, - "node_id": 107, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef4518647caf2874d0f473f802792403", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2220000 - }, - { - "included": -2160000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2160000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 36, - "node_id": 110, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef4518647caf2874d0f473f802792403", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2280000 - }, - { - "included": -2220000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2220000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 37, - "node_id": 113, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef4518647caf2874d0f473f802792403", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2340000 - }, - { - "included": -2280000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2280000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 38, - "node_id": 116, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef4518647caf2874d0f473f802792403", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2400000 - }, - { - "included": -2340000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2340000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 39, - "node_id": 119, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef4518647caf2874d0f473f802792403", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2460000 - }, - { - "included": -2400000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2400000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 40, - "node_id": 122, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef4518647caf2874d0f473f802792403", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2520000 - }, - { - "included": -2460000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2460000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 41, - "node_id": 125, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef4518647caf2874d0f473f802792403", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2580000 - }, - { - "included": -2520000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2520000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 42, - "node_id": 128, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef4518647caf2874d0f473f802792403", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2640000 - }, - { - "included": -2580000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2580000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 43, - "node_id": 131, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef4518647caf2874d0f473f802792403", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2700000 - }, - { - "included": -2640000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2640000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 44, - "node_id": 134, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef4518647caf2874d0f473f802792403", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2760000 - }, - { - "included": -2700000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2700000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 45, - "node_id": 137, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef4518647caf2874d0f473f802792403", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2820000 - }, - { - "included": -2760000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2760000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 46, - "node_id": 140, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef4518647caf2874d0f473f802792403", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2880000 - }, - { - "included": -2820000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2820000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 47, - "node_id": 143, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef4518647caf2874d0f473f802792403", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2940000 - }, - { - "included": -2880000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2880000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 48, - "node_id": 146, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef4518647caf2874d0f473f802792403", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -3000000 - }, - { - "included": -2940000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2940000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 49, - "node_id": 149, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef4518647caf2874d0f473f802792403", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -3060000 - }, - { - "included": -3000000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3000000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 50, - "node_id": 152, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef4518647caf2874d0f473f802792403", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -3120000 - }, - { - "included": -3060000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3060000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 51, - "node_id": 155, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef4518647caf2874d0f473f802792403", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -3180000 - }, - { - "included": -3120000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3120000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 52, - "node_id": 158, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef4518647caf2874d0f473f802792403", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -3240000 - }, - { - "included": -3180000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3180000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 53, - "node_id": 161, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef4518647caf2874d0f473f802792403", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -3300000 - }, - { - "included": -3240000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3240000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 54, - "node_id": 164, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef4518647caf2874d0f473f802792403", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -3360000 - }, - { - "included": -3300000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3300000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 55, - "node_id": 167, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef4518647caf2874d0f473f802792403", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -3420000 - }, - { - "included": -3360000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3360000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 56, - "node_id": 170, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef4518647caf2874d0f473f802792403", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -3480000 - }, - { - "included": -3420000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3420000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 57, - "node_id": 173, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef4518647caf2874d0f473f802792403", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -3540000 - }, - { - "included": -3480000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3480000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 58, - "node_id": 176, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ef4518647caf2874d0f473f802792403", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -3600000 - }, - { - "included": -3540000 - } - ] + "columns": [] } ] }, "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3540000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] + "materialized_views": [] }, - "installed_count": 60, - "installed_position": 59, - "node_id": 179, + "node_id": 1, "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 + "retention_ms": 3600000 } ], - "proven_optimal": true, "queries": [ "quantile_over_time(0.9, data[1h]) / quantile_over_time(0.5, data[1h])" ], - "search_status": "optimal", "selected": { "accepted": [ { "accepted_metric": "relative", - "accepted_regardless_of_guarantee": false, - "guarantee_check_rejection": null, "sketches": [ "DDSketch" ], - "summary": "metric not set; accepted relative (DDSketch)", - "target_metric": null + "summary": "accepted relative (DDSketch)" } ], "cost": { - "cost_per_second": 6.4822354111e-6, - "cost_unit": "cost_per_second", - "cpu_core_seconds_per_second": 0.0124208333333, + "cost_per_second": 8.02474994718e-6, + "cpu_core_seconds_per_second": 0.483447166667, "per_node": { "0": { "cost_per_second": 9.44496e-8, + "cost_source": "analytical", "cpu_core_seconds_per_second": 0.0084, "detail": "scan 2000000 samples; x 1.0000/s", "multiplier": 1, "operator": "Scan", - "price_source": "analytical", "timing": "ingestion_time" }, "1": { - "cost_per_second": 6.38754722874e-6, - "cpu_core_seconds_per_second": 0.004, - "detail": "build DDSketch in 60 tumbling windows of 60000 ms for 10000 groups: 2000000 rows x (1 + depth 1); x 1.0000/s; keeps 5514400000 bytes", + "cost_per_second": 7.63145302553e-6, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.448586666667, + "detail": "build DDSketch in tumbling windows of 60000 ms for 10000 groups: 2000000 rows x 210 ops, 1000 group visits, 166.7 states emitted; x 1.0000/s; keeps 2249680000 bytes", "multiplier": 1, "operator": "SummaryTumblingWindowChain", - "price_source": "analytical", "timing": "ingestion_time" }, "2": { - "cost_per_second": 1.14606178698e-10, - "cpu_core_seconds_per_second": 0.00001, - "detail": "merge 600000 states into 10000; x 0.0167/s; holds 90400000 bytes for 1.2 ms", + "cost_per_second": 1.23684046661e-9, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.00011, + "detail": "time range 3600s: 60 tumbling windows of states; x 0.0167/s; holds 3688 bytes for 6.600 ms", "multiplier": 1, - "operator": "SummaryMerge", - "price_source": "analytical", + "operator": "TimeRange", "timing": "query_time" }, "3": { - "cost_per_second": 1.874e-12, - "cpu_core_seconds_per_second": 1.66666666667e-7, - "detail": "estimate 10000 groups x 1; x 0.0167/s", + "cost_per_second": 2.84486859058e-7, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.0251833333333, + "detail": "merge 600000 states into 10000; x 0.0167/s; holds 45760000 bytes for 1511.000 ms", "multiplier": 1, - "operator": "SummaryEstimate", - "price_source": "analytical", + "operator": "SummaryMerge", "timing": "query_time" }, "4": { - "cost_per_second": 1.87400076679e-12, - "cpu_core_seconds_per_second": 1.66666666667e-7, - "detail": "operator: pass 10000 rows; x 0.0167/s; holds 32 bytes for 1.2 ms", + "cost_per_second": 6.559e-9, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.000583333333333, + "detail": "estimate 10000 groups x 1; x 0.0167/s", "multiplier": 1, - "operator": "Project", - "price_source": "analytical", + "operator": "SummaryEstimate", "timing": "query_time" }, "5": { - "cost_per_second": 1.14606178698e-10, - "cpu_core_seconds_per_second": 0.00001, - "detail": "merge 600000 states into 10000; x 0.0167/s; holds 90400000 bytes for 1.2 ms", + "cost_per_second": 1.87406134311e-12, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 1.66666666667e-7, + "detail": "operator: pass 10000 rows; x 0.0167/s; holds 320000 bytes for 0.010 ms", "multiplier": 1, - "operator": "SummaryMerge", - "price_source": "analytical", + "operator": "Project", "timing": "query_time" }, "6": { - "cost_per_second": 1.874e-12, - "cpu_core_seconds_per_second": 1.66666666667e-7, + "cost_per_second": 6.559e-9, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.000583333333333, "detail": "estimate 10000 groups x 1; x 0.0167/s", "multiplier": 1, "operator": "SummaryEstimate", - "price_source": "analytical", "timing": "query_time" }, "7": { - "cost_per_second": 1.87400076679e-12, + "cost_per_second": 1.87406134311e-12, + "cost_source": "analytical", "cpu_core_seconds_per_second": 1.66666666667e-7, - "detail": "operator: pass 10000 rows; x 0.0167/s; holds 32 bytes for 1.2 ms", + "detail": "operator: pass 10000 rows; x 0.0167/s; holds 320000 bytes for 0.010 ms", "multiplier": 1, "operator": "Project", - "price_source": "analytical", "timing": "query_time" }, "8": { - "cost_per_second": 1.87400076679e-12, + "cost_per_second": 1.87400000613e-12, + "cost_source": "analytical", "cpu_core_seconds_per_second": 1.66666666667e-7, - "detail": "operator: pass 10000 rows; x 0.0167/s; holds 32 bytes for 1.2 ms", + "detail": "operator: pass 10000 rows; x 0.0167/s; holds 32 bytes for 0.010 ms", "multiplier": 1, "operator": "BinaryOp", - "price_source": "analytical", "timing": "query_time" } }, "source": "analytical-cost-v2 (illustrative statistics)", "usage": { - "average_memory_bytes": 5514403766.67, - "cpu_ops_per_second": 6020833.33333, + "average_memory_bytes": 2250832389.85, + "cpu_ops_per_second": 477047166.667, "disk_bytes": 0, - "latency_per_query": [ - { - "compaction_ms": 0.0, - "query_ms": 1.25 - } - ], "peak_cpu_cores": 2, - "peak_memory_bytes": 5695200128, + "peak_memory_bytes": 2296083752, "scanned_bytes_per_second": 64000000.0 } }, @@ -2420,24 +144,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -2450,7 +171,7 @@ }, "producer": 0, "role": "Input", - "window": "RequiresAlignedPanePhaseOrExactBoundaryResidual" + "window": "NotApplicable" }, { "consumer": 2, @@ -2466,42 +187,29 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { - "Sketch": [ - { - "algorithm": "DDSketch", - "params": { - "DDSketch": { - "alpha": 0.0240999270515 - } - } - }, - "PerSubpopulationInstance" - ] + "Sketch": "DDSketch" }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 2, - 0 + 0, + 2 ] ] }, @@ -2523,42 +231,29 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { - "Sketch": [ - { - "algorithm": "DDSketch", - "params": { - "DDSketch": { - "alpha": 0.0240999270515 - } - } - }, - "PerSubpopulationInstance" - ] + "Sketch": "DDSketch" }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 2, - 0 + 0, + 2 ] ] }, @@ -2569,7 +264,7 @@ { "consumer": 4, "data_state": { - "primitive": "Raw", + "primitive": "SummaryState", "timing": "query_time" }, "grouping": "NotApplicable", @@ -2581,24 +276,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { - "Plain": "float64" + "Sketch": "DDSketch" }, - "name": "quantile", - "nullable": false, - "table": null + "name": "state", + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -2616,8 +308,8 @@ { "consumer": 5, "data_state": { - "primitive": "SummaryState", - "timing": "ingestion_time" + "primitive": "Raw", + "timing": "query_time" }, "grouping": "NotApplicable", "intermediate_schema": { @@ -2628,34 +320,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { - "Sketch": [ - { - "algorithm": "DDSketch", - "params": { - "DDSketch": { - "alpha": 0.0240999270515 - } - } - }, - "PerSubpopulationInstance" - ] + "Plain": "float64" }, - "name": "state", - "nullable": false, - "table": null + "name": "quantile", + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -2666,7 +345,7 @@ ] ] }, - "producer": 1, + "producer": 4, "role": "Input", "window": "NotApplicable" }, @@ -2685,34 +364,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { - "Sketch": [ - { - "algorithm": "DDSketch", - "params": { - "DDSketch": { - "alpha": 0.0240999270515 - } - } - }, - "PerSubpopulationInstance" - ] + "Sketch": "DDSketch" }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -2723,7 +389,7 @@ ] ] }, - "producer": 5, + "producer": 3, "role": "Input", "window": "NotApplicable" }, @@ -2742,24 +408,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "quantile", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -2789,24 +452,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -2817,7 +477,7 @@ ] ] }, - "producer": 4, + "producer": 7, "role": "Left", "window": "NotApplicable" }, @@ -2836,24 +496,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -2864,31 +521,13 @@ ] ] }, - "producer": 7, + "producer": 5, "role": "Right", "window": "NotApplicable" } ], "nodes": [ { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact over exact inputs" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 0, "lifetime": "None", "output_schema": { @@ -2899,24 +538,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -2943,24 +579,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -2981,7 +614,6 @@ } }, { - "guarantee": null, "id": 1, "lifetime": { "AcrossEvaluations": "Memory" @@ -2993,42 +625,29 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { - "Sketch": [ - { - "algorithm": "DDSketch", - "params": { - "DDSketch": { - "alpha": 0.0240999270515 - } - } - }, - "PerSubpopulationInstance" - ] + "Sketch": "DDSketch" }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 2, - 0 + 0, + 2 ] ] }, @@ -3036,25 +655,16 @@ "primitive": "SummaryState", "timing": "ingestion_time" }, + "parameters": { + "DDSketch": { + "alpha": 0.02 + } + }, "payload": { "ASAP": { "SummaryTumblingWindowChain": { "child": 0, - "family": { - "Sketch": [ - { - "algorithm": "DDSketch", - "params": { - "DDSketch": { - "alpha": 0.0240999270515 - } - } - }, - "PerSubpopulationInstance" - ] - }, "filter": null, - "grouping": "PerSubpopulationInstance", "input": { "item": null, "weight": { @@ -3062,14 +672,24 @@ } }, "reduction": "PerEntity", - "tumbling_window_count": 60, + "summary_type": { + "Sketch": "DDSketch" + }, "tumbling_window_width_ms": 60000 } } + }, + "retention_ms": 3600000, + "view_coverage": { + "definition_id": "5597d603bdf9f1a148d7c7e5344c1a97", + "selection": [ + { + "columns": [] + } + ] } }, { - "guarantee": null, "id": 2, "lifetime": "None", "output_schema": { @@ -3079,42 +699,29 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { - "Sketch": [ - { - "algorithm": "DDSketch", - "params": { - "DDSketch": { - "alpha": 0.0240999270515 - } - } - }, - "PerSubpopulationInstance" - ] + "Sketch": "DDSketch" }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 2, - 0 + 0, + 2 ] ] }, @@ -3122,54 +729,29 @@ "primitive": "SummaryState", "timing": "query_time" }, + "parameters": { + "DDSketch": { + "alpha": 0.02 + } + }, "payload": { - "ASAP": { - "SummaryMerge": { - "children": [ - 1 - ] + "NonASAP": { + "TimeRange": { + "child": 1, + "kind": "range", + "range": { + "nanos": 0, + "secs": 3600 + } } } } }, { - "guarantee": { - "provenance": [ - { - "algorithm": "DDSketch", - "contract": "ddsketch_relative_error_alpha_v1", - "kind": "sketch_evaluation", - "params": { - "DDSketch": { - "alpha": 0.0240999270515 - } - }, - "query": "Quantile { q: 0.9 }" - } - ], - "spec": { - "accuracy_metric": "relative", - "accuracy_scope": "per_value", - "error": { - "factors": [ - { - "op": "constant", - "value": 0.0240999270515 - }, - { - "op": "unknown", - "statistic": "values_non_negative[data]" - } - ], - "op": "product" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 3, - "lifetime": "None", + "lifetime": { + "OneEvaluation": "Memory" + }, "output_schema": { "closed": true, "fields": [ @@ -3178,24 +760,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { - "Plain": "float64" + "Sketch": "DDSketch" }, - "name": "quantile", - "nullable": false, - "table": null + "name": "state", + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -3207,58 +786,25 @@ ] }, "output_state": { - "primitive": "Raw", + "primitive": "SummaryState", "timing": "query_time" }, + "parameters": { + "DDSketch": { + "alpha": 0.02 + } + }, "payload": { "ASAP": { - "SummaryEstimate": { - "query": { - "Quantile": { - "q": 0.9 - } - }, - "summary_input": 2 + "SummaryMerge": { + "children": [ + 2 + ] } } } }, { - "guarantee": { - "provenance": [ - { - "algorithm": "DDSketch", - "contract": "ddsketch_relative_error_alpha_v1", - "kind": "sketch_evaluation", - "params": { - "DDSketch": { - "alpha": 0.0240999270515 - } - }, - "query": "Quantile { q: 0.9 }" - } - ], - "spec": { - "accuracy_metric": "relative", - "accuracy_scope": "per_value", - "error": { - "factors": [ - { - "op": "constant", - "value": 0.0240999270515 - }, - { - "op": "unknown", - "statistic": "values_non_negative[data]" - } - ], - "op": "product" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 4, "lifetime": "None", "output_schema": { @@ -3269,24 +815,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, - "name": "value", - "nullable": false, - "table": null + "name": "quantile", + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -3302,38 +845,23 @@ "timing": "query_time" }, "payload": { - "NonASAP": { - "Project": { - "child": 3, - "cols": [ - { - "alias": "ts", - "expr": { - "Column": 0 - } - }, - { - "alias": "value", - "expr": { - "Column": 1 - } - }, - { - "alias": "$promql_series_identity", - "expr": { - "Column": 2 - } + "ASAP": { + "SummaryEstimate": { + "query": { + "Quantile": { + "q": 0.5 } - ], - "qualifier": null + }, + "summary_input": 3 } } } }, { - "guarantee": null, "id": 5, - "lifetime": "None", + "lifetime": { + "OneEvaluation": "Memory" + }, "output_schema": { "closed": true, "fields": [ @@ -3342,34 +870,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { - "Sketch": [ - { - "algorithm": "DDSketch", - "params": { - "DDSketch": { - "alpha": 0.0240999270515 - } - } - }, - "PerSubpopulationInstance" - ] + "Plain": "float64" }, - "name": "state", - "nullable": false, - "table": null + "name": "value", + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -3381,55 +896,38 @@ ] }, "output_state": { - "primitive": "SummaryState", + "primitive": "Raw", "timing": "query_time" }, "payload": { - "ASAP": { - "SummaryMerge": { - "children": [ - 1 - ] - } - } - } - }, - { - "guarantee": { - "provenance": [ - { - "algorithm": "DDSketch", - "contract": "ddsketch_relative_error_alpha_v1", - "kind": "sketch_evaluation", - "params": { - "DDSketch": { - "alpha": 0.0240999270515 - } - }, - "query": "Quantile { q: 0.5 }" - } - ], - "spec": { - "accuracy_metric": "relative", - "accuracy_scope": "per_value", - "error": { - "factors": [ + "NonASAP": { + "Project": { + "child": 4, + "cols": [ + { + "alias": "ts", + "expr": { + "Column": 0 + } + }, { - "op": "constant", - "value": 0.0240999270515 + "alias": "value", + "expr": { + "Column": 1 + } }, { - "op": "unknown", - "statistic": "values_non_negative[data]" + "alias": "$promql_series_identity", + "expr": { + "Column": 2 + } } - ], - "op": "product" - }, - "failure_probability": { - "op": "zero" + ] } } - }, + } + }, + { "id": 6, "lifetime": "None", "output_schema": { @@ -3440,24 +938,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "quantile", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -3477,52 +972,19 @@ "SummaryEstimate": { "query": { "Quantile": { - "q": 0.5 + "q": 0.9 } }, - "summary_input": 5 + "summary_input": 3 } } } }, { - "guarantee": { - "provenance": [ - { - "algorithm": "DDSketch", - "contract": "ddsketch_relative_error_alpha_v1", - "kind": "sketch_evaluation", - "params": { - "DDSketch": { - "alpha": 0.0240999270515 - } - }, - "query": "Quantile { q: 0.5 }" - } - ], - "spec": { - "accuracy_metric": "relative", - "accuracy_scope": "per_value", - "error": { - "factors": [ - { - "op": "constant", - "value": 0.0240999270515 - }, - { - "op": "unknown", - "statistic": "values_non_negative[data]" - } - ], - "op": "product" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 7, - "lifetime": "None", + "lifetime": { + "OneEvaluation": "Memory" + }, "output_schema": { "closed": true, "fields": [ @@ -3531,24 +993,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -3586,14 +1045,12 @@ "Column": 2 } } - ], - "qualifier": null + ] } } } }, { - "guarantee": null, "id": 8, "lifetime": "None", "output_schema": { @@ -3604,24 +1061,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -3639,7 +1093,7 @@ "payload": { "NonASAP": { "BinaryOp": { - "lhs": 4, + "lhs": 7, "operator": { "checked_finite_division": false, "checked_relative_division": false, @@ -3649,7 +1103,7 @@ "vector_match": null }, "return_bool": false, - "rhs": 7 + "rhs": 5 } } } @@ -3659,10 +1113,9 @@ 8 ], "scalar_roots": [] - }, - "display_name": "Q1 DDSketch ×60 tumbling windows · ingestion time: DDSketch for Q1, 60 × 1m tumbling windows" + } }, - "tumbling_window_retention_ms": { - "1": 3600000 - } + "warnings": [ + "assumed data fact: source data declares no column_magnitudes for value, so a DDSketch state over it is costed for magnitudes in [1, 1000000000]; this affects cost only, the accuracy guarantee does not depend on it" + ] } diff --git a/crates/devtools/tests/sketch_plan_snapshots/shared-quantiles-1h-mvp.json b/crates/devtools/tests/sketch_plan_snapshots/shared-quantiles-1h-mvp.json index ee5c5b6b1..d53947023 100644 --- a/crates/devtools/tests/sketch_plan_snapshots/shared-quantiles-1h-mvp.json +++ b/crates/devtools/tests/sketch_plan_snapshots/shared-quantiles-1h-mvp.json @@ -3,2409 +3,126 @@ "materialized_views": [ { "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", + "definition_id": "5597d603bdf9f1a148d7c7e5344c1a97", "selection": [ { - "columns": [], - "relative_time": [ - { - "excluded": -60000 - }, - { - "included": 0 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 0, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 0, - "node_id": 2, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -120000 - }, - { - "included": -60000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 60000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 1, - "node_id": 5, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -180000 - }, - { - "included": -120000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 120000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 2, - "node_id": 8, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -240000 - }, - { - "included": -180000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 180000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 3, - "node_id": 11, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -300000 - }, - { - "included": -240000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 240000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 4, - "node_id": 14, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -360000 - }, - { - "included": -300000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 300000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 5, - "node_id": 17, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -420000 - }, - { - "included": -360000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 360000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 6, - "node_id": 20, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -480000 - }, - { - "included": -420000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 420000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 7, - "node_id": 23, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -540000 - }, - { - "included": -480000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 480000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 8, - "node_id": 26, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -600000 - }, - { - "included": -540000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 540000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 9, - "node_id": 29, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -660000 - }, - { - "included": -600000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 600000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 10, - "node_id": 32, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -720000 - }, - { - "included": -660000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 660000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 11, - "node_id": 35, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -780000 - }, - { - "included": -720000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 720000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 12, - "node_id": 38, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -840000 - }, - { - "included": -780000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 780000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 13, - "node_id": 41, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -900000 - }, - { - "included": -840000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 840000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 14, - "node_id": 44, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -960000 - }, - { - "included": -900000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 900000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 15, - "node_id": 47, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1020000 - }, - { - "included": -960000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 960000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 16, - "node_id": 50, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1080000 - }, - { - "included": -1020000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1020000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 17, - "node_id": 53, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1140000 - }, - { - "included": -1080000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1080000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 18, - "node_id": 56, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1200000 - }, - { - "included": -1140000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1140000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 19, - "node_id": 59, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1260000 - }, - { - "included": -1200000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1200000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 20, - "node_id": 62, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1320000 - }, - { - "included": -1260000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1260000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 21, - "node_id": 65, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1380000 - }, - { - "included": -1320000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1320000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 22, - "node_id": 68, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1440000 - }, - { - "included": -1380000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1380000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 23, - "node_id": 71, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1500000 - }, - { - "included": -1440000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1440000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 24, - "node_id": 74, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1560000 - }, - { - "included": -1500000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1500000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 25, - "node_id": 77, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1620000 - }, - { - "included": -1560000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1560000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 26, - "node_id": 80, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1680000 - }, - { - "included": -1620000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1620000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 27, - "node_id": 83, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1740000 - }, - { - "included": -1680000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1680000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 28, - "node_id": 86, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1800000 - }, - { - "included": -1740000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1740000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 29, - "node_id": 89, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1860000 - }, - { - "included": -1800000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1800000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 30, - "node_id": 92, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1920000 - }, - { - "included": -1860000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1860000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 31, - "node_id": 95, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -1980000 - }, - { - "included": -1920000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1920000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 32, - "node_id": 98, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2040000 - }, - { - "included": -1980000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 1980000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 33, - "node_id": 101, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2100000 - }, - { - "included": -2040000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2040000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 34, - "node_id": 104, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2160000 - }, - { - "included": -2100000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2100000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 35, - "node_id": 107, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2220000 - }, - { - "included": -2160000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2160000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 36, - "node_id": 110, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2280000 - }, - { - "included": -2220000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2220000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 37, - "node_id": 113, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2340000 - }, - { - "included": -2280000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2280000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 38, - "node_id": 116, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2400000 - }, - { - "included": -2340000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2340000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 39, - "node_id": 119, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2460000 - }, - { - "included": -2400000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2400000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 40, - "node_id": 122, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2520000 - }, - { - "included": -2460000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2460000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 41, - "node_id": 125, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2580000 - }, - { - "included": -2520000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2520000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 42, - "node_id": 128, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2640000 - }, - { - "included": -2580000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2580000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 43, - "node_id": 131, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2700000 - }, - { - "included": -2640000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2640000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 44, - "node_id": 134, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2760000 - }, - { - "included": -2700000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2700000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 45, - "node_id": 137, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2820000 - }, - { - "included": -2760000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2760000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 46, - "node_id": 140, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2880000 - }, - { - "included": -2820000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2820000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 47, - "node_id": 143, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -2940000 - }, - { - "included": -2880000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2880000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 48, - "node_id": 146, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -3000000 - }, - { - "included": -2940000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 2940000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 49, - "node_id": 149, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -3060000 - }, - { - "included": -3000000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3000000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 50, - "node_id": 152, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -3120000 - }, - { - "included": -3060000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3060000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 51, - "node_id": 155, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -3180000 - }, - { - "included": -3120000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3120000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 52, - "node_id": 158, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -3240000 - }, - { - "included": -3180000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3180000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 53, - "node_id": 161, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -3300000 - }, - { - "included": -3240000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3240000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 54, - "node_id": 164, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -3360000 - }, - { - "included": -3300000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3300000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 55, - "node_id": 167, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -3420000 - }, - { - "included": -3360000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3360000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 56, - "node_id": 170, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -3480000 - }, - { - "included": -3420000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3420000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 57, - "node_id": 173, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -3540000 - }, - { - "included": -3480000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3480000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] - }, - "installed_count": 60, - "installed_position": 58, - "node_id": 176, - "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "ebb7a8bd672401a580c2477ccdc5b96d", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -3600000 - }, - { - "included": -3540000 - } - ] + "columns": [] } ] }, "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 3540000, - "source": { - "TimeSeries": { - "metric": "data" - } - }, - "width_ms": 60000 - } - ] + "materialized_views": [] }, - "installed_count": 60, - "installed_position": 59, - "node_id": 179, + "node_id": 1, "refresh": "at_ingestion", - "retention_ms": 3600000, - "stage2_node_id": 1 + "retention_ms": 3600000 } ], - "proven_optimal": true, "queries": [ "quantile_over_time(0.5, data[1h])", "quantile_over_time(0.9, data[1h])" ], - "search_status": "optimal", "selected": { "accepted": [ { "accepted_metric": "relative", - "accepted_regardless_of_guarantee": false, - "guarantee_check_rejection": null, "sketches": [ "DDSketch" ], - "summary": "metric not set; accepted relative (DDSketch)", - "target_metric": null + "summary": "accepted relative (DDSketch)" }, { "accepted_metric": "relative", - "accepted_regardless_of_guarantee": false, - "guarantee_check_rejection": null, "sketches": [ "DDSketch" ], - "summary": "metric not set; accepted relative (DDSketch)", - "target_metric": null + "summary": "accepted relative (DDSketch)" } ], "cost": { - "cost_per_second": 3.99010517691e-6, - "cost_unit": "cost_per_second", - "cpu_core_seconds_per_second": 0.0124206666667, + "cost_per_second": 6.99139378635e-6, + "cpu_core_seconds_per_second": 0.483447, "per_node": { "0": { "cost_per_second": 9.44496e-8, + "cost_source": "analytical", "cpu_core_seconds_per_second": 0.0084, "detail": "scan 2000000 samples; x 1.0000/s", "multiplier": 1, "operator": "Scan", - "price_source": "analytical", "timing": "ingestion_time" }, "1": { - "cost_per_second": 3.89542189639e-6, - "cpu_core_seconds_per_second": 0.004, - "detail": "build DDSketch in 60 tumbling windows of 60000 ms for 10000 groups: 2000000 rows x (1 + depth 1); x 1.0000/s; keeps 3347680000 bytes", + "cost_per_second": 6.59868036528e-6, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.448586666667, + "detail": "build DDSketch in tumbling windows of 60000 ms for 10000 groups: 2000000 rows x 210 ops, 1000 group visits, 166.7 states emitted; x 1.0000/s; keeps 1351760000 bytes", "multiplier": 1, "operator": "SummaryTumblingWindowChain", - "price_source": "analytical", "timing": "ingestion_time" }, "2": { - "cost_per_second": 1.13092261327e-10, - "cpu_core_seconds_per_second": 0.00001, - "detail": "merge 600000 states into 10000; x 0.0167/s; holds 54880000 bytes for 0.6 ms", + "cost_per_second": 1.23684028037e-9, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.00011, + "detail": "time range 3600s: 60 tumbling windows of states; x 0.0167/s; holds 2216 bytes for 6.600 ms", "multiplier": 1, - "operator": "SummaryMerge", - "price_source": "analytical", + "operator": "TimeRange", "timing": "query_time" }, "3": { - "cost_per_second": 1.874e-12, - "cpu_core_seconds_per_second": 1.66666666667e-7, - "detail": "estimate 10000 groups x 1; x 0.0167/s", + "cost_per_second": 2.83905232793e-7, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.0251833333333, + "detail": "merge 600000 states into 10000; x 0.0167/s; holds 25680000 bytes for 1511.000 ms", "multiplier": 1, - "operator": "SummaryEstimate", - "price_source": "analytical", + "operator": "SummaryMerge", "timing": "query_time" }, "4": { - "cost_per_second": 1.87400038033e-12, - "cpu_core_seconds_per_second": 1.66666666667e-7, - "detail": "operator: pass 10000 rows; x 0.0167/s; holds 32 bytes for 0.6 ms", + "cost_per_second": 6.559e-9, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.000583333333333, + "detail": "estimate 10000 groups x 1; x 0.0167/s", "multiplier": 1, - "operator": "Project", - "price_source": "analytical", + "operator": "SummaryEstimate", "timing": "query_time" }, "5": { - "cost_per_second": 1.13092261327e-10, - "cpu_core_seconds_per_second": 0.00001, - "detail": "merge 600000 states into 10000; x 0.0167/s; holds 54880000 bytes for 0.6 ms", + "cost_per_second": 1.87400000613e-12, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 1.66666666667e-7, + "detail": "operator: pass 10000 rows; x 0.0167/s; holds 32 bytes for 0.010 ms", "multiplier": 1, - "operator": "SummaryMerge", - "price_source": "analytical", + "operator": "Project", "timing": "query_time" }, "6": { - "cost_per_second": 1.874e-12, - "cpu_core_seconds_per_second": 1.66666666667e-7, + "cost_per_second": 6.559e-9, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.000583333333333, "detail": "estimate 10000 groups x 1; x 0.0167/s", "multiplier": 1, "operator": "SummaryEstimate", - "price_source": "analytical", "timing": "query_time" }, "7": { - "cost_per_second": 1.87400038033e-12, + "cost_per_second": 1.87400000613e-12, + "cost_source": "analytical", "cpu_core_seconds_per_second": 1.66666666667e-7, - "detail": "operator: pass 10000 rows; x 0.0167/s; holds 32 bytes for 0.6 ms", + "detail": "operator: pass 10000 rows; x 0.0167/s; holds 32 bytes for 0.010 ms", "multiplier": 1, "operator": "Project", - "price_source": "analytical", "timing": "query_time" } }, "source": "analytical-cost-v2 (illustrative statistics)", "usage": { - "average_memory_bytes": 3347681134.19, - "cpu_ops_per_second": 6020666.66667, + "average_memory_bytes": 1352406708.24, + "cpu_ops_per_second": 477047000.0, "disk_bytes": 0, - "latency_per_query": [ - { - "compaction_ms": 0.0, - "query_ms": 0.62 - }, - { - "compaction_ms": 0.0, - "query_ms": 0.62 - } - ], "peak_cpu_cores": 3, - "peak_memory_bytes": 3457440096, + "peak_memory_bytes": 1377442312, "scanned_bytes_per_second": 64000000.0 } }, @@ -2426,24 +143,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -2456,7 +170,7 @@ }, "producer": 0, "role": "Input", - "window": "RequiresAlignedPanePhaseOrExactBoundaryResidual" + "window": "NotApplicable" }, { "consumer": 2, @@ -2472,42 +186,29 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { - "Sketch": [ - { - "algorithm": "DDSketch", - "params": { - "DDSketch": { - "alpha": 0.05 - } - } - }, - "PerSubpopulationInstance" - ] + "Sketch": "DDSketch" }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 2, - 0 + 0, + 2 ] ] }, @@ -2529,42 +230,29 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { - "Sketch": [ - { - "algorithm": "DDSketch", - "params": { - "DDSketch": { - "alpha": 0.05 - } - } - }, - "PerSubpopulationInstance" - ] + "Sketch": "DDSketch" }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 2, - 0 + 0, + 2 ] ] }, @@ -2575,7 +263,7 @@ { "consumer": 4, "data_state": { - "primitive": "Raw", + "primitive": "SummaryState", "timing": "query_time" }, "grouping": "NotApplicable", @@ -2587,24 +275,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { - "Plain": "float64" + "Sketch": "DDSketch" }, - "name": "quantile", - "nullable": false, - "table": null + "name": "state", + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -2622,8 +307,8 @@ { "consumer": 5, "data_state": { - "primitive": "SummaryState", - "timing": "ingestion_time" + "primitive": "Raw", + "timing": "query_time" }, "grouping": "NotApplicable", "intermediate_schema": { @@ -2634,34 +319,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { - "Sketch": [ - { - "algorithm": "DDSketch", - "params": { - "DDSketch": { - "alpha": 0.05 - } - } - }, - "PerSubpopulationInstance" - ] + "Plain": "float64" }, - "name": "state", - "nullable": false, - "table": null + "name": "quantile", + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -2672,7 +344,7 @@ ] ] }, - "producer": 1, + "producer": 4, "role": "Input", "window": "NotApplicable" }, @@ -2691,34 +363,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { - "Sketch": [ - { - "algorithm": "DDSketch", - "params": { - "DDSketch": { - "alpha": 0.05 - } - } - }, - "PerSubpopulationInstance" - ] + "Sketch": "DDSketch" }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -2729,7 +388,7 @@ ] ] }, - "producer": 5, + "producer": 3, "role": "Input", "window": "NotApplicable" }, @@ -2748,24 +407,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "quantile", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -2783,24 +439,6 @@ ], "nodes": [ { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact over exact inputs" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 0, "lifetime": "None", "output_schema": { @@ -2811,24 +449,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -2855,24 +490,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -2893,7 +525,6 @@ } }, { - "guarantee": null, "id": 1, "lifetime": { "AcrossEvaluations": "Memory" @@ -2905,42 +536,29 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { - "Sketch": [ - { - "algorithm": "DDSketch", - "params": { - "DDSketch": { - "alpha": 0.05 - } - } - }, - "PerSubpopulationInstance" - ] + "Sketch": "DDSketch" }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 2, - 0 + 0, + 2 ] ] }, @@ -2948,25 +566,16 @@ "primitive": "SummaryState", "timing": "ingestion_time" }, + "parameters": { + "DDSketch": { + "alpha": 0.05 + } + }, "payload": { "ASAP": { "SummaryTumblingWindowChain": { "child": 0, - "family": { - "Sketch": [ - { - "algorithm": "DDSketch", - "params": { - "DDSketch": { - "alpha": 0.05 - } - } - }, - "PerSubpopulationInstance" - ] - }, "filter": null, - "grouping": "PerSubpopulationInstance", "input": { "item": null, "weight": { @@ -2974,14 +583,24 @@ } }, "reduction": "PerEntity", - "tumbling_window_count": 60, + "summary_type": { + "Sketch": "DDSketch" + }, "tumbling_window_width_ms": 60000 } } + }, + "retention_ms": 3600000, + "view_coverage": { + "definition_id": "5597d603bdf9f1a148d7c7e5344c1a97", + "selection": [ + { + "columns": [] + } + ] } }, { - "guarantee": null, "id": 2, "lifetime": "None", "output_schema": { @@ -2991,42 +610,29 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { - "Sketch": [ - { - "algorithm": "DDSketch", - "params": { - "DDSketch": { - "alpha": 0.05 - } - } - }, - "PerSubpopulationInstance" - ] + "Sketch": "DDSketch" }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 2, - 0 + 0, + 2 ] ] }, @@ -3034,54 +640,29 @@ "primitive": "SummaryState", "timing": "query_time" }, + "parameters": { + "DDSketch": { + "alpha": 0.05 + } + }, "payload": { - "ASAP": { - "SummaryMerge": { - "children": [ - 1 - ] + "NonASAP": { + "TimeRange": { + "child": 1, + "kind": "range", + "range": { + "nanos": 0, + "secs": 3600 + } } } } }, { - "guarantee": { - "provenance": [ - { - "algorithm": "DDSketch", - "contract": "ddsketch_relative_error_alpha_v1", - "kind": "sketch_evaluation", - "params": { - "DDSketch": { - "alpha": 0.05 - } - }, - "query": "Quantile { q: 0.5 }" - } - ], - "spec": { - "accuracy_metric": "relative", - "accuracy_scope": "per_value", - "error": { - "factors": [ - { - "op": "constant", - "value": 0.05 - }, - { - "op": "unknown", - "statistic": "values_non_negative[data]" - } - ], - "op": "product" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 3, - "lifetime": "None", + "lifetime": { + "OneEvaluation": "Memory" + }, "output_schema": { "closed": true, "fields": [ @@ -3090,24 +671,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { - "Plain": "float64" + "Sketch": "DDSketch" }, - "name": "quantile", - "nullable": false, - "table": null + "name": "state", + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -3119,58 +697,25 @@ ] }, "output_state": { - "primitive": "Raw", + "primitive": "SummaryState", "timing": "query_time" }, + "parameters": { + "DDSketch": { + "alpha": 0.05 + } + }, "payload": { "ASAP": { - "SummaryEstimate": { - "query": { - "Quantile": { - "q": 0.5 - } - }, - "summary_input": 2 + "SummaryMerge": { + "children": [ + 2 + ] } } } }, { - "guarantee": { - "provenance": [ - { - "algorithm": "DDSketch", - "contract": "ddsketch_relative_error_alpha_v1", - "kind": "sketch_evaluation", - "params": { - "DDSketch": { - "alpha": 0.05 - } - }, - "query": "Quantile { q: 0.5 }" - } - ], - "spec": { - "accuracy_metric": "relative", - "accuracy_scope": "per_value", - "error": { - "factors": [ - { - "op": "constant", - "value": 0.05 - }, - { - "op": "unknown", - "statistic": "values_non_negative[data]" - } - ], - "op": "product" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 4, "lifetime": "None", "output_schema": { @@ -3181,24 +726,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, - "name": "value", - "nullable": false, - "table": null + "name": "quantile", + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -3214,36 +756,19 @@ "timing": "query_time" }, "payload": { - "NonASAP": { - "Project": { - "child": 3, - "cols": [ - { - "alias": "ts", - "expr": { - "Column": 0 - } - }, - { - "alias": "value", - "expr": { - "Column": 1 - } - }, - { - "alias": "$promql_series_identity", - "expr": { - "Column": 2 - } + "ASAP": { + "SummaryEstimate": { + "query": { + "Quantile": { + "q": 0.9 } - ], - "qualifier": null + }, + "summary_input": 3 } } } }, { - "guarantee": null, "id": 5, "lifetime": "None", "output_schema": { @@ -3254,34 +779,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { - "Sketch": [ - { - "algorithm": "DDSketch", - "params": { - "DDSketch": { - "alpha": 0.05 - } - } - }, - "PerSubpopulationInstance" - ] + "Plain": "float64" }, - "name": "state", - "nullable": false, - "table": null + "name": "value", + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -3293,55 +805,38 @@ ] }, "output_state": { - "primitive": "SummaryState", + "primitive": "Raw", "timing": "query_time" }, "payload": { - "ASAP": { - "SummaryMerge": { - "children": [ - 1 - ] - } - } - } - }, - { - "guarantee": { - "provenance": [ - { - "algorithm": "DDSketch", - "contract": "ddsketch_relative_error_alpha_v1", - "kind": "sketch_evaluation", - "params": { - "DDSketch": { - "alpha": 0.05 - } - }, - "query": "Quantile { q: 0.9 }" - } - ], - "spec": { - "accuracy_metric": "relative", - "accuracy_scope": "per_value", - "error": { - "factors": [ + "NonASAP": { + "Project": { + "child": 4, + "cols": [ + { + "alias": "ts", + "expr": { + "Column": 0 + } + }, { - "op": "constant", - "value": 0.05 + "alias": "value", + "expr": { + "Column": 1 + } }, { - "op": "unknown", - "statistic": "values_non_negative[data]" + "alias": "$promql_series_identity", + "expr": { + "Column": 2 + } } - ], - "op": "product" - }, - "failure_probability": { - "op": "zero" + ] } } - }, + } + }, + { "id": 6, "lifetime": "None", "output_schema": { @@ -3352,24 +847,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "quantile", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -3389,50 +881,15 @@ "SummaryEstimate": { "query": { "Quantile": { - "q": 0.9 + "q": 0.5 } }, - "summary_input": 5 + "summary_input": 3 } } } }, { - "guarantee": { - "provenance": [ - { - "algorithm": "DDSketch", - "contract": "ddsketch_relative_error_alpha_v1", - "kind": "sketch_evaluation", - "params": { - "DDSketch": { - "alpha": 0.05 - } - }, - "query": "Quantile { q: 0.9 }" - } - ], - "spec": { - "accuracy_metric": "relative", - "accuracy_scope": "per_value", - "error": { - "factors": [ - { - "op": "constant", - "value": 0.05 - }, - { - "op": "unknown", - "statistic": "values_non_negative[data]" - } - ], - "op": "product" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 7, "lifetime": "None", "output_schema": { @@ -3443,24 +900,21 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -3498,22 +952,20 @@ "Column": 2 } } - ], - "qualifier": null + ] } } } } ], "roots": [ - 4, - 7 + 7, + 5 ], "scalar_roots": [] - }, - "display_name": "Q1 DDSketch ×60 tumbling windows · Q2 DDSketch ×60 tumbling windows · shared DDSketch ×60 tumbling windows (Q1, Q2) · shared Scan (Q1, Q2) · ingestion time: DDSketch for Q1, Q2, 60 × 1m tumbling windows" + } }, - "tumbling_window_retention_ms": { - "1": 3600000 - } + "warnings": [ + "assumed data fact: source data declares no column_magnitudes for value, so a DDSketch state over it is costed for magnitudes in [1, 1000000000]; this affects cost only, the accuracy guarantee does not depend on it" + ] } diff --git a/crates/devtools/tests/sketch_plan_snapshots/topk-rate-guarantees-ignored-mvp.json b/crates/devtools/tests/sketch_plan_snapshots/topk-rate-guarantees-ignored-mvp.json index 12caf51a6..249e5523d 100644 --- a/crates/devtools/tests/sketch_plan_snapshots/topk-rate-guarantees-ignored-mvp.json +++ b/crates/devtools/tests/sketch_plan_snapshots/topk-rate-guarantees-ignored-mvp.json @@ -3,1245 +3,118 @@ "materialized_views": [ { "coverage": { - "definition_id": "9dfffca4bd58dda229c67ca3b6042d4f", + "definition_id": "12f38a39e56799163522b8353c998877", "selection": [ { - "columns": [], - "relative_time": [ - { - "excluded": -10000 - }, - { - "included": 0 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 0, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 0, - "node_id": 2, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "9dfffca4bd58dda229c67ca3b6042d4f", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -20000 - }, - { - "included": -10000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 10000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 1, - "node_id": 5, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "9dfffca4bd58dda229c67ca3b6042d4f", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -30000 - }, - { - "included": -20000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 20000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 2, - "node_id": 8, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "9dfffca4bd58dda229c67ca3b6042d4f", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -40000 - }, - { - "included": -30000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 30000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 3, - "node_id": 11, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "9dfffca4bd58dda229c67ca3b6042d4f", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -50000 - }, - { - "included": -40000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 40000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 4, - "node_id": 14, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "9dfffca4bd58dda229c67ca3b6042d4f", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -60000 - }, - { - "included": -50000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 50000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 5, - "node_id": 17, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "9dfffca4bd58dda229c67ca3b6042d4f", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -70000 - }, - { - "included": -60000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 60000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 6, - "node_id": 20, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "9dfffca4bd58dda229c67ca3b6042d4f", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -80000 - }, - { - "included": -70000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 70000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 7, - "node_id": 23, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "9dfffca4bd58dda229c67ca3b6042d4f", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -90000 - }, - { - "included": -80000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 80000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 8, - "node_id": 26, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "9dfffca4bd58dda229c67ca3b6042d4f", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -100000 - }, - { - "included": -90000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 90000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 9, - "node_id": 29, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "9dfffca4bd58dda229c67ca3b6042d4f", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -110000 - }, - { - "included": -100000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 100000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 10, - "node_id": 32, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "9dfffca4bd58dda229c67ca3b6042d4f", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -120000 - }, - { - "included": -110000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 110000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 11, - "node_id": 35, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "9dfffca4bd58dda229c67ca3b6042d4f", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -130000 - }, - { - "included": -120000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 120000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 12, - "node_id": 38, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "9dfffca4bd58dda229c67ca3b6042d4f", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -140000 - }, - { - "included": -130000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 130000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 13, - "node_id": 41, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "9dfffca4bd58dda229c67ca3b6042d4f", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -150000 - }, - { - "included": -140000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 140000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 14, - "node_id": 44, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "9dfffca4bd58dda229c67ca3b6042d4f", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -160000 - }, - { - "included": -150000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 150000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 15, - "node_id": 47, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "9dfffca4bd58dda229c67ca3b6042d4f", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -170000 - }, - { - "included": -160000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 160000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 16, - "node_id": 50, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "9dfffca4bd58dda229c67ca3b6042d4f", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -180000 - }, - { - "included": -170000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 170000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 17, - "node_id": 53, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "9dfffca4bd58dda229c67ca3b6042d4f", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -190000 - }, - { - "included": -180000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 180000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 18, - "node_id": 56, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "9dfffca4bd58dda229c67ca3b6042d4f", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -200000 - }, - { - "included": -190000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 190000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 19, - "node_id": 59, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "9dfffca4bd58dda229c67ca3b6042d4f", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -210000 - }, - { - "included": -200000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 200000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 20, - "node_id": 62, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "9dfffca4bd58dda229c67ca3b6042d4f", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -220000 - }, - { - "included": -210000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 210000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 21, - "node_id": 65, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "9dfffca4bd58dda229c67ca3b6042d4f", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -230000 - }, - { - "included": -220000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 220000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 22, - "node_id": 68, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "9dfffca4bd58dda229c67ca3b6042d4f", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -240000 - }, - { - "included": -230000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 230000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 23, - "node_id": 71, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "9dfffca4bd58dda229c67ca3b6042d4f", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -250000 - }, - { - "included": -240000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 240000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 24, - "node_id": 74, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "9dfffca4bd58dda229c67ca3b6042d4f", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -260000 - }, - { - "included": -250000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 250000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 25, - "node_id": 77, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "9dfffca4bd58dda229c67ca3b6042d4f", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -270000 - }, - { - "included": -260000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 260000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 26, - "node_id": 80, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "9dfffca4bd58dda229c67ca3b6042d4f", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -280000 - }, - { - "included": -270000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 270000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 27, - "node_id": 83, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "9dfffca4bd58dda229c67ca3b6042d4f", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -290000 - }, - { - "included": -280000 - } - ] - } - ] - }, - "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 280000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 10000 - } - ] - }, - "installed_count": 30, - "installed_position": 28, - "node_id": 86, - "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 - }, - { - "coverage": { - "definition_id": "9dfffca4bd58dda229c67ca3b6042d4f", - "selection": [ - { - "columns": [], - "relative_time": [ - { - "excluded": -300000 - }, - { - "included": -290000 - } - ] + "columns": [] } ] }, "inputs": { - "materialized_views": [], - "raw_data_windows": [ - { - "end_offset_ms": 290000, - "source": { - "TimeSeries": { - "metric": "data_total" - } - }, - "width_ms": 10000 - } - ] + "materialized_views": [] }, - "installed_count": 30, - "installed_position": 29, - "node_id": 89, + "node_id": 1, "refresh": "at_ingestion", - "retention_ms": 300000, - "stage2_node_id": 1 + "retention_ms": 300000 } ], - "proven_optimal": true, "queries": [ "topk(5, sum by (job) (rate(data_total[5m])))" ], - "search_status": "optimal", "selected": { "accepted": [ { "accepted_metric": null, - "accepted_regardless_of_guarantee": true, - "guarantee_check_rejection": "no_accuracy_guarantee", "sketches": [ "CmsWithHeap" ], - "summary": "metric not set; accepted by the deployment's accuracy model regardless of guarantee, ignoring a failed guarantee check (NoAccuracyGuarantee) (CmsWithHeap)", - "target_metric": null + "summary": "no accuracy target (CmsWithHeap)" } ], "cost": { - "cost_per_second": 2.91198328572e-8, - "cost_unit": "cost_per_second", - "cpu_core_seconds_per_second": 0.000814001, + "cost_per_second": 3.34907479468e-7, + "cpu_core_seconds_per_second": 0.0252173675, "per_node": { "0": { "cost_per_second": 6.52152e-9, + "cost_source": "analytical", "cpu_core_seconds_per_second": 0.00058, "detail": "scan 100000 samples; x 1.0000/s", "multiplier": 1, "operator": "Scan", - "price_source": "analytical", "timing": "ingestion_time" }, "1": { - "cost_per_second": 2.22159834707e-8, - "cpu_core_seconds_per_second": 0.0002, - "detail": "build exact Rate accumulator in 30 tumbling windows of 10000 ms for 10000 groups: 100000 rows x (1 + depth 1); x 1.0000/s; keeps 17360000 bytes", + "cost_per_second": 1.64571266068e-7, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.01007, + "detail": "build exact Rate accumulator in tumbling windows of 10000 ms for 10000 groups: 100000 rows x 70 ops, 1000 group visits, 1000.0 states emitted; x 1.0000/s; keeps 44640000 bytes", "multiplier": 1, "operator": "SummaryTumblingWindowChain", - "price_source": "analytical", "timing": "ingestion_time" }, "2": { - "cost_per_second": 3.37341900136e-10, - "cpu_core_seconds_per_second": 0.00003, - "detail": "merge 300000 states into 10000; x 0.1000/s; holds 560000 bytes for 0.3 ms", + "cost_per_second": 3.71052005466e-9, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.00033, + "detail": "time range 300s: 30 tumbling windows of states; x 0.1000/s; holds 144 bytes for 3.300 ms", "multiplier": 1, - "operator": "SummaryMerge", - "price_source": "analytical", + "operator": "TimeRange", "timing": "query_time" }, "3": { - "cost_per_second": 1.1244e-11, - "cpu_core_seconds_per_second": 1e-6, + "cost_per_second": 1.30449612663e-7, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.0116, + "detail": "merge 300000 states into 10000; x 0.1000/s; holds 1440000 bytes for 116.000 ms", + "multiplier": 1, + "operator": "SummaryMerge", + "timing": "query_time" + }, + "4": { + "cost_per_second": 2.811e-8, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.0025, "detail": "finalize 10000 accumulators; x 0.1000/s", "multiplier": 1, "operator": "FinalizeExactAccumulator", - "price_source": "analytical", "timing": "query_time" }, - "4": { - "cost_per_second": 3.37322413708e-11, - "cpu_core_seconds_per_second": 3e-6, - "detail": "build CmsWithHeap into 1 states: 10000 rows x depth 3; x 0.1000/s; holds 6172 bytes for 0.3 ms", + "5": { + "cost_per_second": 1.54061966081e-9, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 0.000137017, + "detail": "build CmsWithHeap into 1 states: 10000 rows x 137 ops, 1 group visits; x 0.1000/s; holds 3254 bytes for 1.370 ms", "multiplier": 1, "operator": "SummaryAgg", - "price_source": "analytical", "timing": "query_time" }, - "5": { - "cost_per_second": 5.622e-15, - "cpu_core_seconds_per_second": 5e-10, - "detail": "estimate 1 groups x 5; x 0.1000/s", + "6": { + "cost_per_second": 3.93540004831e-12, + "cost_source": "analytical", + "cpu_core_seconds_per_second": 3.5e-7, + "detail": "estimate 1 groups x 5; x 0.1000/s; holds 120 bytes for 0.003 ms", "multiplier": 1, "operator": "SummaryEstimate", - "price_source": "analytical", "timing": "query_time" }, - "6": { - "cost_per_second": 5.62293857724e-15, + "7": { + "cost_per_second": 5.6220000138e-15, + "cost_source": "analytical", "cpu_core_seconds_per_second": 5e-10, - "detail": "operator: pass 5 rows; x 0.1000/s; holds 24 bytes for 0.3 ms", + "detail": "operator: pass 5 rows; x 0.1000/s; holds 24 bytes for 0.000 ms", "multiplier": 1, "operator": "Project", - "price_source": "analytical", "timing": "query_time" } }, "source": "analytical-cost-v2 (illustrative statistics)", "usage": { - "average_memory_bytes": 17360019.2512, - "cpu_ops_per_second": 334001.0, + "average_memory_bytes": 44656704.4934, + "cpu_ops_per_second": 24737367.5, "disk_bytes": 0, - "latency_per_query": [ - { - "compaction_ms": 0.0, - "query_ms": 0.34001 - } - ], "peak_cpu_cores": 2, - "peak_memory_bytes": 17926244, + "peak_memory_bytes": 46083590, "scanned_bytes_per_second": 4800000.0 } }, @@ -1262,32 +135,28 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -1300,7 +169,7 @@ }, "producer": 0, "role": "Input", - "window": "RequiresAlignedPanePhaseOrExactBoundaryResidual" + "window": "NotApplicable" }, { "consumer": 2, @@ -1316,9 +185,8 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { @@ -1328,31 +196,29 @@ ] }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 3, - 0 + 0, + 2, + 3 ] ] }, @@ -1367,6 +233,61 @@ "timing": "query_time" }, "grouping": "NotApplicable", + "intermediate_schema": { + "closed": true, + "fields": [ + { + "dtype": { + "Plain": "timestamp" + }, + "name": "window_end", + "nullable": false + }, + { + "dtype": { + "ExactAggregate": [ + "Rate", + "Rate" + ] + }, + "name": "state", + "nullable": false + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "job", + "nullable": true + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false + } + ], + "time_index": 0, + "unique_keys": [ + [ + 0, + 2, + 3 + ] + ] + }, + "producer": 2, + "role": "Input", + "window": "NotApplicable" + }, + { + "consumer": 4, + "data_state": { + "primitive": "SummaryState", + "timing": "query_time" + }, + "grouping": "NotApplicable", "intermediate_schema": { "closed": true, "fields": [ @@ -1375,8 +296,7 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { @@ -1386,24 +306,21 @@ ] }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -1414,12 +331,12 @@ ] ] }, - "producer": 2, + "producer": 3, "role": "Input", "window": "NotApplicable" }, { - "consumer": 4, + "consumer": 5, "data_state": { "primitive": "Raw", "timing": "query_time" @@ -1433,32 +350,28 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -1469,12 +382,12 @@ ] ] }, - "producer": 3, + "producer": 4, "role": "Input", "window": "NotApplicable" }, { - "consumer": 5, + "consumer": 6, "data_state": { "primitive": "SummaryState", "timing": "query_time" @@ -1485,34 +398,21 @@ "fields": [ { "dtype": { - "Sketch": [ - { - "algorithm": "CmsWithHeap", - "params": { - "CmsWithHeap": { - "depth": 2, - "heap_size": 20, - "width": 55 - } - } - }, - "PerSubpopulationInstance" - ] + "Sketch": "CmsWithHeap" }, "name": "state", - "nullable": false, - "table": null + "nullable": false } ], "time_index": null, "unique_keys": [] }, - "producer": 4, + "producer": 5, "role": "Input", "window": "NotApplicable" }, { - "consumer": 6, + "consumer": 7, "data_state": { "primitive": "Raw", "timing": "query_time" @@ -1526,16 +426,14 @@ "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "float64" }, "name": "topk", - "nullable": false, - "table": null + "nullable": false } ], "time_index": null, @@ -1545,31 +443,13 @@ ] ] }, - "producer": 5, + "producer": 6, "role": "Input", "window": "NotApplicable" } ], "nodes": [ { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact over exact inputs" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, "id": 0, "lifetime": "None", "output_schema": { @@ -1580,32 +460,28 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -1632,32 +508,28 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -1678,7 +550,6 @@ } }, { - "guarantee": null, "id": 1, "lifetime": { "AcrossEvaluations": "Memory" @@ -1690,9 +561,8 @@ "dtype": { "Plain": "timestamp" }, - "name": "ts", - "nullable": false, - "table": null + "name": "window_end", + "nullable": false }, { "dtype": { @@ -1702,31 +572,29 @@ ] }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, "unique_keys": [ [ - 3, - 0 + 0, + 2, + 3 ] ] }, @@ -1738,14 +606,7 @@ "ASAP": { "SummaryTumblingWindowChain": { "child": 0, - "family": { - "ExactAggregate": [ - "Rate", - "Rate" - ] - }, "filter": null, - "grouping": "PerSubpopulationInstance", "input": { "item": null, "weight": { @@ -1753,16 +614,95 @@ } }, "reduction": "PerEntity", - "tumbling_window_count": 30, + "summary_type": { + "ExactAggregate": [ + "Rate", + "Rate" + ] + }, "tumbling_window_width_ms": 10000 } } + }, + "retention_ms": 300000, + "view_coverage": { + "definition_id": "12f38a39e56799163522b8353c998877", + "selection": [ + { + "columns": [] + } + ] } }, { - "guarantee": null, "id": 2, "lifetime": "None", + "output_schema": { + "closed": true, + "fields": [ + { + "dtype": { + "Plain": "timestamp" + }, + "name": "window_end", + "nullable": false + }, + { + "dtype": { + "ExactAggregate": [ + "Rate", + "Rate" + ] + }, + "name": "state", + "nullable": false + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "job", + "nullable": true + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false + } + ], + "time_index": 0, + "unique_keys": [ + [ + 0, + 2, + 3 + ] + ] + }, + "output_state": { + "primitive": "SummaryState", + "timing": "query_time" + }, + "payload": { + "NonASAP": { + "TimeRange": { + "child": 1, + "kind": "range", + "range": { + "nanos": 0, + "secs": 300 + } + } + } + } + }, + { + "id": 3, + "lifetime": { + "OneEvaluation": "Memory" + }, "output_schema": { "closed": true, "fields": [ @@ -1771,8 +711,7 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { @@ -1782,24 +721,21 @@ ] }, "name": "state", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -1818,32 +754,14 @@ "ASAP": { "SummaryMerge": { "children": [ - 1 + 2 ] } } } }, { - "guarantee": { - "provenance": [ - { - "kind": "exact", - "reason": "exact accumulator over exact input" - } - ], - "spec": { - "accuracy_metric": "absolute", - "accuracy_scope": "per_value", - "error": { - "op": "zero" - }, - "failure_probability": { - "op": "zero" - } - } - }, - "id": 3, + "id": 4, "lifetime": "None", "output_schema": { "closed": true, @@ -1853,32 +771,28 @@ "Plain": "timestamp" }, "name": "ts", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "float64" }, "name": "value", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "utf8" }, "name": "$promql_series_identity", - "nullable": false, - "table": null + "nullable": false } ], "time_index": 0, @@ -1896,37 +810,25 @@ "payload": { "ASAP": { "FinalizeExactAccumulator": { - "child": 2 + "child": 3 } } } }, { - "guarantee": null, - "id": 4, - "lifetime": "None", + "id": 5, + "lifetime": { + "OneEvaluation": "Memory" + }, "output_schema": { "closed": true, "fields": [ { "dtype": { - "Sketch": [ - { - "algorithm": "CmsWithHeap", - "params": { - "CmsWithHeap": { - "depth": 2, - "heap_size": 20, - "width": 55 - } - } - }, - "PerSubpopulationInstance" - ] + "Sketch": "CmsWithHeap" }, "name": "state", - "nullable": false, - "table": null + "nullable": false } ], "time_index": null, @@ -1936,27 +838,18 @@ "primitive": "SummaryState", "timing": "query_time" }, + "parameters": { + "CmsWithHeap": { + "depth": 5, + "heap_size": 10, + "width": 28 + } + }, "payload": { "ASAP": { "SummaryAgg": { - "child": 3, - "family": { - "Sketch": [ - { - "algorithm": "CmsWithHeap", - "params": { - "CmsWithHeap": { - "depth": 2, - "heap_size": 20, - "width": 55 - } - } - }, - "PerSubpopulationInstance" - ] - }, + "child": 4, "filter": null, - "grouping": "PerSubpopulationInstance", "input": { "item": { "Column": { @@ -1969,15 +862,19 @@ }, "reduction": { "Reduce": [] + }, + "summary_type": { + "Sketch": "CmsWithHeap" } } } } }, { - "guarantee": null, - "id": 5, - "lifetime": "None", + "id": 6, + "lifetime": { + "OneEvaluation": "Memory" + }, "output_schema": { "closed": true, "fields": [ @@ -1986,16 +883,14 @@ "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true }, { "dtype": { "Plain": "float64" }, "name": "topk", - "nullable": false, - "table": null + "nullable": false } ], "time_index": null, @@ -2017,14 +912,13 @@ "k": 5 } }, - "summary_input": 4 + "summary_input": 5 } } } }, { - "guarantee": null, - "id": 6, + "id": 7, "lifetime": "None", "output_schema": { "closed": true, @@ -2034,16 +928,14 @@ "Plain": "float64" }, "name": "topk_5", - "nullable": false, - "table": null + "nullable": false }, { "dtype": { "Plain": "utf8" }, "name": "job", - "nullable": true, - "table": null + "nullable": true } ], "time_index": null, @@ -2060,7 +952,7 @@ "payload": { "NonASAP": { "Project": { - "child": 5, + "child": 6, "cols": [ { "alias": "topk_5", @@ -2074,21 +966,19 @@ "Column": 0 } } - ], - "qualifier": null + ] } } } } ], "roots": [ - 6 + 7 ], "scalar_roots": [] - }, - "display_name": "Q1 CMS+heap + Rate acc ×30 tumbling windows · ingestion time: Rate acc for Q1, 30 × 10s tumbling windows" + } }, - "tumbling_window_retention_ms": { - "1": 300000 - } + "warnings": [ + "assumed data fact: the data arrival declares no max_out_of_order_ms, so the samples of source data_total that an ingestion-time rate, increase or irate state reads are taken to arrive in timestamp order; if this does not hold, answers may be wrong by an amount no accuracy target bounds (a late sample can be read as a counter reset), or ingestion may fail" + ] } diff --git a/crates/devtools/tests/viewer_contract.rs b/crates/devtools/tests/viewer_contract.rs index 440e4db60..ad54cfd77 100644 --- a/crates/devtools/tests/viewer_contract.rs +++ b/crates/devtools/tests/viewer_contract.rs @@ -316,13 +316,10 @@ fn wire_name(value: T) -> String { } /// The viewer knows every `MaterializedViewRefresh` -/// (`materialized_views[].refresh`) and every Stage 3 `RejectionCause`, and -/// splits the causes into failed checks and losses as -/// `RejectionCause::failed_a_check` does. +/// (`materialized_views[].refresh`). #[test] -fn viewer_knows_every_view_refresh_and_rejection_cause() { +fn viewer_knows_every_view_refresh() { use asap_planner::MaterializedViewRefresh as R; - use asap_stage3_plan_selection::RejectionCause as C; let refreshes = [R::AtIngestion, R::OnFirstRead]; for refresh in refreshes { // A new variant fails to compile here until it is listed above. @@ -337,48 +334,4 @@ fn viewer_knows_every_view_refresh_and_rejection_cause() { .map(wire_name) .collect::>() ); - - let causes = [ - C::AccuracyTargetMissed, - C::NoAccuracyGuarantee, - C::LatencyBoundExceeded, - C::LatencyOverRepeatInterval, - C::OverClusterCapacity, - C::Infeasible, - C::CostNotRepresentable, - C::Costlier, - C::Slower, - C::MoreMemory, - C::TiedLostOnEnumerationOrder, - ]; - for cause in causes { - match cause { - C::AccuracyTargetMissed - | C::NoAccuracyGuarantee - | C::LatencyBoundExceeded - | C::LatencyOverRepeatInterval - | C::OverClusterCapacity - | C::Infeasible - | C::CostNotRepresentable - | C::Costlier - | C::Slower - | C::MoreMemory - | C::TiedLostOnEnumerationOrder => {} - } - } - let failed: BTreeSet = causes - .into_iter() - .filter(|c| c.failed_a_check()) - .map(wire_name) - .collect(); - let lost: BTreeSet = causes - .into_iter() - .filter(|c| !c.failed_a_check()) - .map(wire_name) - .collect(); - assert_eq!(stages_js_strings("const FAILED_CHECK_CAUSES = ["), failed); - assert_eq!( - stages_js_strings("const REJECTION_CAUSES = FAILED_CHECK_CAUSES.concat(["), - lost - ); } diff --git a/crates/stage1-pass2-reuse-candidates/src/lib.rs b/crates/stage1-pass2-reuse-candidates/src/lib.rs index 7331be4b0..6e1703380 100644 --- a/crates/stage1-pass2-reuse-candidates/src/lib.rs +++ b/crates/stage1-pass2-reuse-candidates/src/lib.rs @@ -65,7 +65,8 @@ pub enum Pass2RuleError { /// dead-candidate elimination again over the memo with them (a merge /// whose θ serves no consumer); /// 6. the logical edges between the surviving candidates, with -/// $\alpha^{\log}$ from `evidence` or the theoretical guarantee, and the +/// $\alpha^{\log}$ from `evidence` or the theoretical guarantee, every +/// failing edge with its reason (eliminated candidates' included), and the /// groups each query may read only exactly /// (`ErrorMetricsOfMemo::exact_only_groups_of_query`). /// @@ -118,15 +119,24 @@ pub fn generate_logical_asap_memo_candidates( ); let pruned = eliminate_dead_candidates(&memo, &candidate_edges); let dead = candidates_of(&pruned); + let logical_edge = |edge: &edges::CandidateEdge| LogicalEdge { + producer_candidate_id: edge.producer, + consumer_candidate_id: edge.consumer, + input_index_of_consumer: edge.input, + satisfies_logical_requirement: edge.satisfies.is_ok(), + }; + // Before the dead endpoints are dropped: why each failing edge fails. + let failing_logical_edges = candidate_edges + .iter() + .filter_map(|edge| { + let reason = edge.satisfies.as_ref().err()?; + Some((logical_edge(edge), reason.clone())) + }) + .collect(); let edges = candidate_edges - .into_iter() + .iter() .filter(|edge| !dead.contains(&edge.producer) && !dead.contains(&edge.consumer)) - .map(|edge| LogicalEdge { - producer_candidate_id: edge.producer, - consumer_candidate_id: edge.consumer, - input_index_of_consumer: edge.input, - satisfies_logical_requirement: edge.satisfies.is_ok(), - }) + .map(logical_edge) .collect(); let exact_only_groups_of_query = error_metrics.exact_only_groups_of_query( &memo, @@ -137,6 +147,7 @@ pub fn generate_logical_asap_memo_candidates( Ok(LogicalASAPMemoCandidates { memo, edges, + failing_logical_edges, pruned, evaluation_grid_of_group, exact_only_groups_of_query, diff --git a/crates/stage2-physical-optimization/src/test_fixtures.rs b/crates/stage2-physical-optimization/src/test_fixtures.rs index 8457b0bae..feb34a87a 100644 --- a/crates/stage2-physical-optimization/src/test_fixtures.rs +++ b/crates/stage2-physical-optimization/src/test_fixtures.rs @@ -95,6 +95,7 @@ pub fn logical( exact_only_groups_of_query: vec![BTreeSet::new(); memo.queries().len()], memo, edges, + failing_logical_edges: vec![], pruned: vec![], evaluation_grid_of_group, } diff --git a/crates/stage3-plan-selection/src/cost/tests.rs b/crates/stage3-plan-selection/src/cost/tests.rs index c64f88ac9..15dd5564e 100644 --- a/crates/stage3-plan-selection/src/cost/tests.rs +++ b/crates/stage3-plan-selection/src/cost/tests.rs @@ -148,6 +148,7 @@ fn no_execution_reading_a_producer_costs_less_than_nothing() { LogicalASAPMemoCandidates { memo, edges: vec![], + failing_logical_edges: vec![], pruned: vec![], evaluation_grid_of_group: vec![None; groups], exact_only_groups_of_query: vec![], diff --git a/crates/stage3-plan-selection/src/milp/model/tests.rs b/crates/stage3-plan-selection/src/milp/model/tests.rs index 3bfb13968..59dfdc616 100644 --- a/crates/stage3-plan-selection/src/milp/model/tests.rs +++ b/crates/stage3-plan-selection/src/milp/model/tests.rs @@ -106,6 +106,7 @@ fn physical( LogicalASAPMemoCandidates { memo, edges: vec![], + failing_logical_edges: vec![], pruned: vec![], evaluation_grid_of_group: vec![None; groups], exact_only_groups_of_query: vec![], diff --git a/crates/types/src/plan/physical_asap_dag.rs b/crates/types/src/plan/physical_asap_dag.rs index b263241c5..9b421daa0 100644 --- a/crates/types/src/plan/physical_asap_dag.rs +++ b/crates/types/src/plan/physical_asap_dag.rs @@ -302,26 +302,42 @@ pub enum PhysicalASAPDAGValidationError { } impl PhysicalASAPDAG { - /// The plan's query roots as IR, in entry order: each operator root's - /// node, or the standalone scalar query keyed by its entry. Shared nodes - /// stay shared. For tests that inspect a plan as operator trees. - #[cfg(any(test, feature = "test-support"))] - pub fn query_roots(&self) -> Result, PhysicalASAPDAGValidationError> { + /// Each query's root in node ids, in workload order: query `i` is the + /// scalar root keyed `i` when there is one, and otherwise the next entry + /// of `roots`. Example: `roots = [n4]`, `scalar_roots = [(0, n2 / n3)]` + /// gives `[Scalar(n2 / n3), Operator(n4)]`. + pub fn root_nodes_per_query(&self) -> Vec> { use crate::ir::QueryRoot; - let nodes = self.operator_nodes()?; let mut operator_roots = self.roots.iter(); - let count = self.roots.len() + self.scalar_roots.len(); - Ok((0..count) + (0..self.roots.len() + self.scalar_roots.len()) .map( - |entry| match self.scalar_roots.iter().find(|(index, _)| *index == entry) { - Some((_, expr)) => { - QueryRoot::Scalar(expr.map_operator_refs(&mut |id| Rc::clone(&nodes[id]))) - } - None => QueryRoot::Operator(Rc::clone( - &nodes[operator_roots.next().expect("one operator root per entry")], - )), + |query| match self.scalar_roots.iter().find(|(index, _)| *index == query) { + Some((_, expr)) => QueryRoot::Scalar(expr.clone()), + None => QueryRoot::Operator( + *operator_roots.next().expect("one operator root per query"), + ), }, ) + .collect() + } + + /// The plan's query roots as IR, in workload order + /// ([`Self::root_nodes_per_query`] with each node id replaced by its + /// operator tree). Shared nodes stay shared. For tests that inspect a + /// plan as operator trees. + #[cfg(any(test, feature = "test-support"))] + pub fn query_roots(&self) -> Result, PhysicalASAPDAGValidationError> { + use crate::ir::QueryRoot; + let nodes = self.operator_nodes()?; + Ok(self + .root_nodes_per_query() + .into_iter() + .map(|root| match root { + QueryRoot::Operator(id) => QueryRoot::Operator(Rc::clone(&nodes[&id])), + QueryRoot::Scalar(expr) => { + QueryRoot::Scalar(expr.map_operator_refs(&mut |id| Rc::clone(&nodes[id]))) + } + }) .collect()) } diff --git a/crates/types/src/stages/logical/candidate/mod.rs b/crates/types/src/stages/logical/candidate/mod.rs index a9e646f9a..be0a15d5f 100644 --- a/crates/types/src/stages/logical/candidate/mod.rs +++ b/crates/types/src/stages/logical/candidate/mod.rs @@ -87,8 +87,10 @@ pub fn listed_build_parameters(build: &Operator) -> Vec, + /// Every $(u, c, i)$ with $\alpha^{\log} = 0$, eliminated candidates' + /// edges included, with the rule that failed, as Pass 2 decided it (the + /// chain grid, then values per evaluation, then the metric and accuracy + /// rules), sorted by consumer, then input, then producer. E.g. a KLL + /// readout G5[0] of a query with δ over its whole result and $n_q$ + /// unknown: `(G4[3] → G5[0], input 0, ValuesPerEvaluationUnknown { + /// consumer: G5[0], query: 0 })`. A possible edge not listed holds. + pub failing_logical_edges: Vec<(LogicalEdge, RootPruneReason)>, /// The candidates dead-candidate elimination removed: no logical edge /// names them, and Stage 2 does not expand them. pub pruned: Vec, diff --git a/crates/types/src/stages/physical/memo_candidates/tests.rs b/crates/types/src/stages/physical/memo_candidates/tests.rs index 14a3ee2fa..622211238 100644 --- a/crates/types/src/stages/physical/memo_candidates/tests.rs +++ b/crates/types/src/stages/physical/memo_candidates/tests.rs @@ -129,6 +129,7 @@ fn candidates( LogicalASAPMemoCandidates { memo, edges: vec![], + failing_logical_edges: vec![], pruned: vec![], evaluation_grid_of_group, exact_only_groups_of_query: vec![], @@ -687,6 +688,7 @@ fn new_indexes_only_admissible_edges() { LogicalASAPMemoCandidates { memo: memo(), edges: vec![], + failing_logical_edges: vec![], pruned: vec![], evaluation_grid_of_group: vec![None; 5], exact_only_groups_of_query: vec![], @@ -777,6 +779,7 @@ fn a_stored_chain_and_an_ingestion_time_result_are_stored_on_their_refresh_windo LogicalASAPMemoCandidates { memo: memo.clone(), edges: vec![], + failing_logical_edges: vec![], pruned: vec![], evaluation_grid_of_group: vec![grid_60s; group_count], exact_only_groups_of_query: vec![], diff --git a/crates/types/src/stages/pruning.rs b/crates/types/src/stages/pruning.rs index e56309f88..003a8d45c 100644 --- a/crates/types/src/stages/pruning.rs +++ b/crates/types/src/stages/pruning.rs @@ -34,8 +34,10 @@ pub struct PruneCause { } /// Why the edge at the end of a [`PruneCause`]'s path fails, or why the -/// candidate has no edge at all. +/// candidate has no edge at all. JSON: the snake_case variant name, e.g. +/// `{"below_required_accuracy": {...}}` or `"no_consumer"`. #[derive(Debug, Clone, serde::Serialize)] +#[serde(rename_all = "snake_case")] pub enum RootPruneReason { /// The producer's guarantee, at the required failure probability, /// exceeds the consumer input's required error. @@ -110,8 +112,10 @@ pub struct PrunedCandidate { /// Why a stage removed a candidate: first Stage 2's (the physical candidate /// cannot be built, run or read by a query output), then Stage 3's, before -/// its selection MILP. +/// its selection MILP. JSON: the snake_case variant name, e.g. +/// `"raw_data_not_retained"`, or `{"dominated_by": 3}`. #[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, serde::Serialize)] +#[serde(rename_all = "snake_case")] pub enum PruneReason { /// Stage 2: it runs at ingestion time, and the source has no ingestion /// pipeline (its data is at rest or its arrival unknown). diff --git a/docs/user_guide_docs/asap-planner.md b/docs/user_guide_docs/asap-planner.md index f8c293b0f..c45148cb4 100644 --- a/docs/user_guide_docs/asap-planner.md +++ b/docs/user_guide_docs/asap-planner.md @@ -34,19 +34,20 @@ minutes in a debug build. ```text asap_planner (--workload )... --run-until stage0|stage1_pass1|stage1_pass2|stage2|stage3 - [--format text|json] [--out ] [--sql-catalog-table ]... + [--format text|json] [--printing-full-candidates] [--out ] [--sql-catalog-table ]... [--cost-inputs ] [--deployment-capabilities ] [--ignore-accuracy-guarantees] ``` | Option | Meaning | | --- | --- | -| `--workload ` | A `PlanningWorkload` JSON file: the queries, their requirements and recurrence, and the data-source facts. Repeat it to plan several files together, including PromQL and SQL files; sources they share must have the same facts. Required. | +| `--workload ` | A `PlanningWorkload` JSON file: the queries, their requirements and recurrence, and the data-source facts. Repeat it to plan several files together, including PromQL, MetricsQL and SQL files; sources they share must have the same facts. Required. | | `--run-until ` | The last stage to run. Every stage up to and including it is printed. Required. | -| `--format text\|json` | `text` (the default) for reading; `json` for an `asap-stage-pipeline/v1` document. | +| `--format text\|json` | `text` (the default) for reading; `json` for an `asap-stage-pipeline/v2` document. | +| `--printing-full-candidates` | Text only: print every candidate, edge and pruned candidate. By default the candidates of a memo group that differ only in θ are folded into one line (their indices, each θ parameter's values, and how many Pass 2 pruned, by cause), the candidates Pass 2 kept are printed in full with their edges, and the pruned lists of Pass 2 and Stage 2 are counts per cause. | | `--out ` | Write to this file instead of standard output. | | `--sql-catalog-table ` | One SQL table a SQL workload reads, as `{"name", "columns": [{"name", "type", "nullable"}], "time_index"}`. Repeat it per table. | -| `--cost-inputs ` | The cost model and its prices (`CostInputs` JSON). The files in `tools/dag-viewer/cost-inputs/` are ready to use. Without it, the built-in usage-over-time model is used. Stage 3 prices only usage-over-time models (`aws-fargate-us-east-1.json` is one); a provisioned- or cluster-capacity model ends Stage 3 with an `UnsupportedCostModel` error. | +| `--cost-inputs ` | The cost model and its costs (`CostInputs` JSON). The files in `tools/dag-viewer/cost-inputs/` are ready to use. Without it, the built-in usage-over-time model is used. Stage 3 costs only usage-over-time models (`aws-fargate-us-east-1.json` is one); a provisioned- or cluster-capacity model ends Stage 3 with an `UnsupportedCostModel` error. | | `--deployment-capabilities ` | What the deployment can do, e.g. `{"raw_data_retained": true, "query_time_results_retained": false}`. Set it to match your deployment. | | `--ignore-accuracy-guarantees` | Accept candidates without an accuracy guarantee. By default each query's accuracy requirement is enforced. | @@ -61,10 +62,10 @@ under the stage that failed. | `--run-until` | Prints, after the stages before it | | --- | --- | | `stage0` | The frontends' query DAGs and their memo groups. | -| `stage1_pass1` | Stage 1 Pass 1: each computation's candidates, and the candidates pruned with their reason. | -| `stage1_pass2` | Stage 1 Pass 2: the memo after the reuse rules and chain groups (a chain group for every legal tumbling-window width), with the logical edges and the pruned candidates, which Stage 2 implements. | -| `stage2` | Stage 2: the physical candidates and the ones pruned. | -| `stage3` | Stage 3: the selected plan, its cost per node, each query's accepted accuracy and the search status. | +| `stage1_pass1` | Stage 1 Pass 1: each computation's candidates. | +| `stage1_pass2` | Stage 1 Pass 2: the memo after its rules (with a chain group for every legal tumbling-window width), each candidate's logical edges with $\alpha^{\log}$ and why an edge fails, and the candidates Pass 2 eliminated with their causes. | +| `stage2` | Stage 2: the physical candidates, their physical edges with $\alpha^{\mathrm{phys}}$ and why an edge fails, and the candidates pruned with their reasons. | +| `stage3` | Stage 3: the selected plan, its cost per node, its materialized views (node, refresh, retention), each query's accepted accuracy, the candidates Stage 3 pruned and the selection MILP's size. | Every stage sees every legal tumbling-window width at once, so the selected plan is the best over all of them. @@ -73,7 +74,7 @@ plan is the best over all of them. The example below plans the checked-in [range-heavy workload](../../tools/dag-viewer/memo-examples/promql/rangeheavy-t8-t5-1h.json) -with raw data retained and AWS Fargate prices: +with raw data retained and AWS Fargate costs: ```sh echo '{"raw_data_retained": true, "query_time_results_retained": false}' > capabilities.json @@ -105,16 +106,19 @@ With `--format json`, open the document in the cargo run --release -p asap-devtools --bin asap_planner -- \ --workload tools/dag-viewer/workloads/planner-layering/example1.json \ --run-until stage3 --format json --out plan.json -python3 -c 'import json; print(json.dumps(json.load(open("plan.json"))["stage3_plan_selection"]["search_report"], indent=2))' +python3 -c 'import json; print(json.dumps(json.load(open("plan.json"))["stage3_plan_selection"]["cost"], indent=2))' ``` Sections of stages that were not run are `null`. In -`stage3_plan_selection`, check `selected_dag`, `cost`, `search_report` and -any `error`: a written document alone does not mean planning succeeded. -A selected plan is proven optimal only when `search_report.status` is -`optimal`. Costs are model predictions, not measured execution times or a +`stage3_plan_selection`, check `plan`, `cost` and any `error`: a written +document alone does not mean planning succeeded. Costs are model predictions, not measured execution times or a cloud bill. +The JSON document numbers memo groups and candidates canonically (depth +first from the query roots, in query order), while `--format text` prints +the planner's own ids, so one group can be `G2` in the JSON and `G5` in the +text. + ## Troubleshooting - **`asap_planner: command not found`:** use `cargo run ... -- ...`, or the diff --git a/tools/dag-viewer/README.md b/tools/dag-viewer/README.md index f713f4a0f..3db1fb906 100644 --- a/tools/dag-viewer/README.md +++ b/tools/dag-viewer/README.md @@ -1,18 +1,21 @@ # ASAPPlanner Stage Viewer A browser view of what the planner does with one workload, read from an -`asap-stage-pipeline/v1` document written by the `asap_planner` devtool +`asap-stage-pipeline/v2` document written by the `asap_planner` devtool (`asap_planner --workload --run-until stage3 --format json`). The document shows: - **Stage 0**, the query-language frontends' DAG, with one root per query; -- **Stage 1**, the logical candidates of the memo and the selected plan's - logical DAG (`L1`); -- **Stage 2**, the selected plan as an executable physical DAG (`P1`), with - each node's timing and lifetime, and the physical candidates Stage 2 - pruned; -- **Stage 3**, the selected plan's cost, the accuracy it accepted per query, - its materialized views and the search report. +- **Stage 1**, the memo's logical candidates after Pass 1 and Pass 2, the + logical edges between them ($\alpha^{\log}$ and why it is 0), and the + candidates Pass 2 pruned with their root causes; +- **Stage 2**, every physical candidate with its physical edges + ($\alpha^{\mathrm{phys}}$ and why it is 0), and the physical candidates + Stage 2 pruned with their reasons; +- **Stage 3**, the selected plan (a physical DAG) with each node's timing, + lifetime and cost, the accuracy it accepted per query, its materialized + views, the candidates it pruned and the size and times of its MILP, or why + no plan was selected. To run it, see [RUNNING.md](RUNNING.md). @@ -30,53 +33,62 @@ To run it, see [RUNNING.md](RUNNING.md). section. - **Query workload** with each query's accuracy, latency, recurrence, predictability and range evaluation, and a line that counts the - candidates at each stage. + candidates at each stage: the Pass 2 memo's logical candidates and how + many Pass 2 pruned, the physical candidates and how many Stage 2 and + Stage 3 pruned, and whether a plan was selected. - **Data workload** beside it: the arrival and one row per declared source with exactly the facts it declares (series count, metric type, non-negative values, sample interval, stream norms, distinct bound, distinct values per label, distinct values per label set) and the queries whose text names it, plus the deployment capabilities the run planned with. -- **Stage 3 · plans by cost**: the selected plan, then the Stage 2 pruned - candidates and the Stage 1 candidates that did not compose, each with its - reason. The header counts the candidates, names the search and its report - (status, cost lower bound, MILP size, times), says whether accuracy - guarantees were enforced or ignored, shows the run's warnings, and says - why Stage 1 or plan selection failed when it did. -- **Three lanes**, stacked top to bottom so each is as wide as the page: - Stage 0, the Stage 1 candidate, and its Stage 2 plan with each node's - timing (also its fill: green at ingestion time, orange at query time) and - Stage 3 cost (`× n` when a node that is not materialized is computed once - per reader). Each lane is the whole workload's DAG, so a node shared by - several queries appears once and says `shared ×n`. Summary builds print - their configuration (for example `CmsWithHeap · depth 7 · heap 100 · - width 272`) and whether there is one per group, one per series, or one - shared instance. A standalone scalar query (`time()`, `scalar(v)`, - `1 + 2`) is a dashed `Scalar` node over the nodes it reads. Query roots - have a thick border. A `SummaryTumblingWindowChain` is one node (`6 - tumbling windows × 10 s`); clicking it draws its tumbling windows, newest - first, each with its window (`window t−20 s … t−10 s`), and clicking a - tumbling window folds them back. +- **Stage 3 · plan selection**: one row, the selected plan with its cost + per second and the accuracy Stage 3 accepted per query, or why no plan + was selected. The header says why Stage 1 or Stage 2 failed when it did, + shows the selection's warnings, says whether accuracy guarantees were + enforced or ignored, and names the search with its MILP's size and times. +- **Two lanes**, stacked top to bottom so each is as wide as the page: + Stage 0 and Stage 3's plan, with each plan node's timing (also its fill: + green at ingestion time, orange at query time) and Stage 3 cost (`× n` + when a node that is not materialized is computed once per consumer). + Each lane is the whole workload's DAG, so a node shared by several queries + appears once and says `shared ×n`. Summary builds print their summary type + (`summary: Kll`, `summary: exact Sum`) and grouping; a + `SummaryTumblingWindowChain` prints its tumbling-window width. A + standalone scalar query (`time()`, `scalar(v)`, `1 + 2`) is a dashed + `Scalar` node over the nodes it reads. Query roots have a thick border. - **Highlighting**: the **highlight query** menu dims, in every lane, what that query's root does not reach. Clicking a node dims everything but what it reads and what reads it; clicking the background goes back to the query's highlight. -- **Materialized views** under the Stage 2 lane: the selected plan's - materialized views, one row per Stage 2 node: its coverage definition, - operator and summary parameters, tumbling windows (`6 × 10 s` for a - tumbling-window chain), how it is refreshed, its retention, the Stage 3 - cost and the queries that read it. A chain's row lists its installed - tumbling windows under **installed**. Clicking a row selects its node in - the Stage 2 lane. +- **Materialized views** under the plan lane: one row per view: its plan + node, coverage definition, operator and summary type, how it is + refreshed, what its refresh reads (the views it reads, else raw data), + its retention, the node's Stage 3 cost and the queries that read it. A plan that + is not installable says why instead. Clicking a row selects its node in + the plan lane. - **Memo**: the memo after each planner step, one tab per step (`Stage 0`, - `Stage 1, Pass 1`, local window composition, `Stage 1, Pass 2`), opening - on the last. Each memo group is a box `G ·