Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
67 commits
Select commit Hold shift + click to select a range
8fe46ae
docs(concepts): memo groups, properties, requirements and the propert…
zzylol Nov 5, 2026
8d59b7d
docs(concepts): memo group as expression, input requirements and cand…
zzylol Nov 5, 2026
32bb7cc
docs(concepts): align memo group, candidate, property and requirement…
zzylol Nov 5, 2026
67dd612
docs(concepts): data structures and interfaces per stage, from the #1…
zzylol Nov 5, 2026
31ec62b
docs(concepts): keep the per-stage interfaces as section 3, drop the …
zzylol Nov 5, 2026
a99e9ff
docs(concepts): sections 1-2 in the formulation's terms; section 4 ru…
zzylol Nov 5, 2026
3fb4039
docs(concepts): error propagation function per consumer; validity and…
zzylol Nov 5, 2026
730be23
docs(concepts): ErrorPropagation approved
zzylol Nov 5, 2026
d6aea8a
docs(concepts): λ is a propagation factor read from the query expression
zzylol Nov 5, 2026
8b9c74d
docs(concepts): keep the candidate's origin, listed as not in the for…
zzylol Nov 5, 2026
a1c1ecd
docs(concepts): consumer and query/context instead of reader; Physica…
zzylol Nov 5, 2026
3e8faca
docs(concepts): section 5, printing every logical DAG a memo group re…
zzylol Nov 5, 2026
86d7ace
docs(concepts): section 5 size bound; reaches covers self-reads
zzylol Nov 5, 2026
7bfb7f4
docs(concepts): logical properties and requirement mirror the physica…
zzylol Nov 5, 2026
1554891
docs(concepts): drop fields other things fix; LogicalProperties {para…
zzylol Nov 5, 2026
0952692
docs(concepts): parameter_list approved
zzylol Nov 5, 2026
4e7c752
docs(concepts): context_times -> Context::evaluation_schedule
zzylol Nov 5, 2026
2990b05
docs(concepts): constraints, not rows (constraint (K), *_constraints …
zzylol Nov 6, 2026
294066a
Update memo-groups-properties-requirements.md
zzylol Nov 6, 2026
b33c0bb
docs(concepts): fix review issues: one Memo type, decision tree resto…
zzylol Nov 6, 2026
bc00292
docs(concepts): delete both consumer-count policies; no constraint (J)
zzylol Nov 6, 2026
019fdb8
docs(concepts): (K) encoding approved; worst-case counting affects bo…
zzylol Nov 6, 2026
7e86e17
docs(concepts): running example follows the metric rule (B reads a DD…
zzylol Nov 6, 2026
428aad9
docs(concepts): retention need = time horizon of the served queries; …
zzylol Nov 6, 2026
39d25f5
docs(concepts): CandidateForMemoGroupId.index -> index_in_group
zzylol Nov 6, 2026
949b3e1
docs(concepts): define schedule σ; a grid window is readable once it …
zzylol Nov 6, 2026
12ec9fb
docs(concepts): drop the physical schedule σ; p^phys = (τ, ℓ), grids …
zzylol Nov 6, 2026
1342661
docs(concepts): theta listed, floors free; heap 1-2-5 steps; one eval…
zzylol Nov 6, 2026
9f56b58
docs(concepts): rename the interfaces as agreed with the reviewer
zzylol Nov 6, 2026
9b82b79
docs(concepts): note the empty parameter lists of unsized families (#…
zzylol Nov 6, 2026
05bc1ea
docs(concepts): execution contexts reuse the workload's recurrence; o…
zzylol Nov 6, 2026
2f984cc
docs(concepts): Memo methods return Result; CandidateError
zzylol Nov 6, 2026
836885c
docs(concepts): a memo group is its expression; private Memo groups
zzylol Nov 6, 2026
7a87ff7
docs(concepts): GroupProperties derived from input groups; SummaryDef…
zzylol Nov 6, 2026
4c44e01
docs(concepts): SummaryStateDefinition, OperatorInput and MemoGroupAs…
zzylol Nov 6, 2026
e364e86
docs(concepts): unlifted_operators; point to the derivation examples
zzylol Nov 6, 2026
f9d1790
docs(concepts): unlifted_operators name the groups their subqueries read
zzylol Nov 6, 2026
9c810de
docs(concepts): column lineage replaces table qualifiers, with memo-g…
zzylol Nov 6, 2026
b461462
docs(concepts): input_group is optional when Scan predicates lift; de…
zzylol Nov 6, 2026
1baa2a1
docs(concepts): SourceColumn is one struct shared by lineage and summ…
zzylol Nov 7, 2026
25b9d8e
docs(concepts): chain groups, general rollup rule, dead-candidate eli…
zzylol Nov 7, 2026
8066235
docs(concepts): no executor-support check, ingestion-time counter and…
zzylol Nov 7, 2026
5d08902
docs(concepts): Stage 2 entry returns no Result; current crate path
zzylol Nov 7, 2026
1e7bbb5
docs(concepts): IntervalConstraintLimit replaces TimeBound, with exam…
zzylol Nov 7, 2026
37650ff
docs(concepts): ranges are (t − r, t]; @ end()/@ start() lifted only …
zzylol Nov 8, 2026
f6b4e70
docs(concepts): Hydra is a rollup strategy, with a worked example
zzylol Nov 8, 2026
97062af
docs(concepts): execution contexts explained with an example; grid co…
zzylol Nov 8, 2026
86919ae
docs(concepts): when the candidate graph can have a cycle; t only on …
zzylol Nov 8, 2026
9b0cee0
docs(concepts): apply the 2026-11-08 decisions across the memo model
zzylol Nov 8, 2026
271ca7d
docs(concepts): δ levels 1e-2..1e-5 (CMS depths 5/7/10/12), result-se…
zzylol Nov 8, 2026
a33e4cc
docs(concepts): guarantees at a failure probability, AccuracyEvidence…
zzylol Nov 9, 2026
344f321
docs(concepts): Count Sketch depths, delivers_on over Stage 2's candi…
zzylol Nov 9, 2026
ae2551e
docs(concepts): declared column signs in the memo, value-sign rows fo…
zzylol Nov 9, 2026
b8c7056
docs(concepts): lineage over chains, window selections, @ end()/@ sta…
zzylol Nov 9, 2026
e4f8787
docs(concepts): inputs of an ingestion-time materialized candidate ru…
zzylol Nov 9, 2026
950b9b3
docs(concepts): RawDataNotRetained also rejects a Scan materialized a…
zzylol Nov 9, 2026
7f9e985
docs(concepts): a candidate's own cost reads the logical size over th…
zzylol Nov 9, 2026
fcb8cd0
docs(concepts): merges carry θ, metric rule cuts edges, held memory p…
zzylol Nov 9, 2026
5d5d458
docs(concepts): scan read r_e and constraint (S) in the objective; HL…
zzylol Nov 9, 2026
ae231ff
docs(concepts): a query-time Scan execution reads the most its contex…
zzylol Nov 9, 2026
bae8a8d
docs(concepts): Pass 2 δ levels continue in powers of ten below the p…
zzylol Nov 9, 2026
3d6ea13
docs(concepts): match the memo interfaces as implemented
zzylol Nov 10, 2026
1ecd13d
docs(concepts): SketchParams and parameter_list paths in the summary_…
zzylol Nov 10, 2026
7fb4202
docs(concepts): lineage key sentence without PR numbers
zzylol Nov 10, 2026
192359b
docs(concepts): no PR or issue numbers in the concepts docs
zzylol Nov 10, 2026
7eeff73
docs(concepts): coverage in the IR, the plan's content id by expansio…
zzylol Nov 10, 2026
15d61cf
docs(concepts): AcrossEvaluations for a time-independent group withou…
zzylol Nov 12, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/design_docs/concepts/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,4 @@ These pages introduce the vocabulary and representations used throughout the rep
- [Mapping concepts](asap-aware-mapping.md)
- [Pre-ASAP IR](pre-asap-ir.md)
- [Post-ASAP IR](post-asap-ir.md)
- [Memo groups, properties and requirements](memo-groups-properties-requirements.md)
21 changes: 13 additions & 8 deletions docs/design_docs/concepts/accuracy-models.md
Original file line number Diff line number Diff line change
Expand Up @@ -290,14 +290,19 @@ A deployment declares `DataWorkload.sources[source].max_distinct_per_evaluation`
for the source the distinct count reads. It asserts an enforced maximum
distinct population per evaluation, including all merged tumbling windows; the classic
estimator and independent uniform bucket hashing are the model's
assumptions. Stage 1 sizes the precision from the query or allocated local
target and leaves the failure probability symbolic, as
`classic_hll_failure_probability[p=P,eps=ε](max_distinct[source])`. Pass 1
then raises the precision when the source's declared bound needs it (for its
share of the failure probability), and Stage 3 binds the failure probability
from that bound in the normal selection checks. A bound outside the
contract's range gives a failure probability of 1; a missing bound leaves it
unknown, so the target is not met.
assumptions. The memo holds the scanned sources' facts, and
`Memo::max_distinct_per_evaluation(group)` sums the bounds of the sources
below the HLL state. The theoretical guarantee of precision p for the distinct
count at the readout's required failure probability δ is the least relative
error ε whose classic failure probability over that bound is at most δ; the
readout's edge (α^log, Pass 2) holds when that ε meets its requirement. A
bound outside the contract's range certifies no error, and a missing bound
none either: the edge fails with `BelowRequiredAccuracy`. The readout's
metric (relative) does not depend on the bound.

Example: `COUNT(DISTINCT user_id)` within relative 0.05 at δ = 0.01. With
`max_distinct_per_evaluation` 9,636, precision 18 meets it; with 1,000,000
(outside the contract) or no bound, no HLL precision does.

The current model supports maxima from 1 to 131,072 (2^17) and precisions from 4 to 18,
and certifies only configurations that remain in the linear-counting branch.
Expand Down
Loading