Repository navigation
datagen: labelled time-series API for library use; drop the ERP export - #249
Merged
Merged
Conversation
Add aqpbm_datagen::TimeSeriesDescription: series_num distinct label sets drawn from per-label cardinalities with optional Zipf skew, sampled every sample_interval_ms over duration_ms, values from an existing DataDistribution, optional counter mode, one seed. samples() yields (labels, timestamp_ms, value) time-major. The crate's default features need only serde, rand and thiserror, so another workspace can depend on it by git rev. Delete approxbench erp (erp.rs, erp_cmd.rs, CLI wiring), its methodology doc, its run/analysis scripts and the doc references: nothing consumes it.
…weighted label sets - Series values mix the distribution's own seed: stream_seed(seed ^ value_distribution.seed(), i). A distribution seed of 0 keeps the previous output. - samples() yields Result items: a counter past 2^53 yields one error naming the series and timestamp, then the iterator ends, instead of stopping silently. - Label skew is documented as the skew of the draws. When the product of the cardinalities is at most 4 x series_num, label sets come from weighted sampling without replacement (keys u^(1/w)), which always succeeds and is 4-18x faster near the product. - GeneratedTimeSeries holds its description; one shared bad() helper. - Renames: value -> value_distribution, LabelSpec -> SeriesLabelSpec. - TableDescription::from_path, serde_json and serde_norway move behind a `files` feature (implied by `import`); dependency lists and crate description corrected. - atomic_costs: reduce_scores inlined into reduce_one.
zzylol
force-pushed
the
feat/datagen-library-drop-erp
branch
from
November 16, 2026 20:39
cf8f867 to
6236d43
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
aqpbm-datagenonly generates typed tables, andapproxbench erpis an export with no consumer.TimeSeriesDescriptiontoaqpbm-datagen(per-label cardinalities with optional Zipf skew, sample interval, duration, an existingDataDistributionper series, counter mode, one seed), putTableDescription::from_pathbehind a newfilesfeature, and delete the ERP export with its docs and scripts.aqpbm-datagen = { git = "https://github.com/ProjectASAP/sketch-bench", rev = "cf8f86735e6f4c59d9827e3ae5d5bd28e8cd3ee1" }builds from an outside workspace with only serde, rand, rand_distr, rand_xoshiro and thiserror (no serde_json, serde_norway, sketch libraries, polars or CLI). Example: 50 series overhost(100 values) xendpoint(8 values, Zipf 1.1), every 10 s for 60 s, gives 300 samples; 10^4 series x 360 samples stream at 7.2 M samples/s in 6.9 MB.Review guide
main. Not stacked.aqpbm-datagen/src/timeseries.rs(new API, the two label-set algorithms, and its tests).aqpbm-cli(module, subcommand,mainmatch arm);reduce_scoresinlined intoatomic_costs::reduce_one(same checks, same order); the sharederror::badhelper; thefilesfeature and theconfigs/datagentests reading YAML through a test helper; doc and comment edits.Behavior: before → after
Before: an outside crate wanting labelled series had to build a
TableDescriptionper label, dedupe label sets, lay out timestamps and run a counter itself, and takingaqpbm-datagenalso pulled in serde_json and serde_norway. After:Why not reuse
aqpbm-usecase::data: it already turns a use-caseSpecinto labelled series, but it depends onaqpbm-datagenwith theimportfeature, so taking it pulls in polars, csv, chrono, flate2 and anyhow; its spec (metrics, groups, phases,{i}templates) is also richer than a library caller needs. The two generators differ in three conventions:aqpbm-usecase::dataTimeSeriesDescriptionstart_ms + interval(offsets1..=duration/interval)start_ms(k * interval < duration)start_msnames the first sample directly and the grid is the half-open window[start_ms, start_ms + duration_ms); a caller wanting the usecase grid setsstart_ms + interval.seed ^ value_distribution.seedseed ^ stream * 0x9E3779B97F4A7C15splitmix64(seed ^ splitmix64(stream))i * φ/ stream 0, and switching either side changes recorded outputs.Follow-up #250 makes usecase's synthetic path draw from these conventions.
Interfaces
aqpbm-datagen/src/timeseries.rs(new; re-exported fromlib.rs)aqpbm-datagen/src/table.rs,aqpbm-datagen/Cargo.tomlimpl TableDescription { /// Load a description from a `.yaml`/`.yml` (serde_norway) or otherwise JSON file. + /// Behind the `files` feature. + #[cfg(feature = "files")] pub fn from_path(path: &Path) -> Result<Self, DataGenError>; } [features] schema = ["dep:schemars"] +# Load a description from a YAML or JSON file (`TableDescription::from_path`). +files = ["dep:serde_json", "dep:serde_norway"] -import = ["dep:anyhow", "dep:chrono", "dep:csv", "dep:flate2", "dep:polars"] +import = ["files", "dep:anyhow", "dep:chrono", "dep:csv", "dep:flate2", "dep:polars"] [dependencies] -serde_json.workspace = true -serde_norway.workspace = true +serde_json = { workspace = true, optional = true } +serde_norway = { workspace = true, optional = true }Every in-repo caller of
from_path(aqpbm-cli) already enablesimport, which now impliesfiles.aqpbm-cli/src/cli.rs,aqpbm-cli/src/lib.rs,aqpbm-cli/src/atomic_costs.rspub enum Cmd { Sketchbench(SketchbenchArgs), AtomicCosts(atomic_costs_cmd::AtomicCostsArgs), - /// Preserve distribution-conditioned error and resource measurements as ERP v1. - Erp(erp_cmd::ErpArgs), Flatten(flatten_cmd::FlattenArgs), } -pub mod erp; -pub mod erp_cmd; pub fn reduce_one(record: &MergedRecord, accuracy_metric: &str) -> Result<AtomicCostEntry, SkipReason>; -/// [`reduce_one`] without naming the row's metric (`accuracy_metric` is empty), for a consumer that keeps every score (ERP). -pub(crate) fn reduce_scores(record: &MergedRecord) -> Result<AtomicCostEntry, SkipReason>;Removed because nothing reads ERP output: a grep for
erp,erp_v1,erp_cmdanderp_shapeacross the repo found only the deleted files and doc mentions.reduce_scoresexisted only for ERP; its body now sits inreduce_one.Usage (the rustdoc example in
timeseries.rs, run as a doctest):Algorithm changes
Label sets
BadParamafter 10^6 draws (322 found).TimeSeriesDescription::generate):series_num(WEIGHTED_SAMPLING_SETS_PER_SERIES),weighted_label_sets: give every set(i, j, ...)the weightw = (i+1)^-s_1 (j+1)^-s_2 ...and the keyu^(1/w),uuniform; keep theseries_numlargest keys (select, then sort), largest first. Keys are compared asln(-ln u) - ln wso tiny weights do not underflow.drawn_label_sets: each label draws its index from Zipf(cardinality, skewness) on one seeded stream; keep each new set untilseries_numexist, giving up withBadParamafter 1000 xseries_numdraws.far_below_the_label_set_count_the_series_keep_the_draw_skew,near_the_label_set_count_heavy_sets_are_taken_first,series_num_equal_to_the_label_set_count_always_succeeds.series_num; drawn path unchanged, O(series_num x labels) expected. Measured (release, one core, labels 1000 x 100 x 5): 4×10^5 series 2.1 s → 0.44 s, 4.99×10^5 series 9.7 s → 0.53 s.BadParamafter 2×10^6 draws, 0.9 s), with a message that names the fix.Values
Series
i's values come from its own streamsplitmix64(s ^ splitmix64(i))withs = seed ^ value_distribution.seed, so series are independent and replays are deterministic. Counter mode reusesrule::MonotonicAcc. O(1) per sample, 7.2 M samples/s.Decisions and open questions
Needs reviewer decision: none.
Decided:
stream_seed(seed ^ value_distribution.seed(), i). A seed of 0 keeps the output of the first revision. Rejected: ignoring it silently (a validated field doing nothing), refusing a nonzero one.samples()yieldsResultitems; a counter past 2^53 yields oneErrnaming the series and timestamp, then the iterator ends. Rejected: refusing up front, which can only bound bounded distributions (Uniform, Zipf); Pareto and Normal, the usual counter increments, would have to be refused outright or still stop silently.series_num.value→value_distribution,LabelSpec→SeriesLabelSpec(the crate root already uses "label" for a column name);series_numkept to matchrow_num/column_num.{name}_{index}(for examplehost_17); the index is the Zipf rank, so_0is the most frequent value. Rejected: randomStringOptsstrings, which add nothing for series labels.aqpbm-usecase::dataon purpose (table above); aqpbm-usecase: draw synthetic series from aqpbm-datagen's TimeSeriesDescription conventions #250 follows up.from_pathmoves behind afilesfeature so a library caller does not take serde_json and serde_norway.Verification
cargo test -p aqpbm-datagen(default features)timeseries.rscargo test -p aqpbm-datagen --all-featuresimporttestscargo test -p aqpbm-clicargo clippy --workspace --all-targets --all-features -- -D warnings-p aqpbm-datagen -p aqpbm-cliwith default features, and-p aqpbm-datagen --no-default-featurescargo fmt --all --checkcf8f86735e6f4c59d9827e3ae5d5bd28e8cd3ee1cargo tree: 22 crates (40 in the first revision): serde, rand, rand_distr, rand_xoshiro, thiserror and their dependencies; no serde_json, serde_norway, polars, sketch library or clapNew tests (written failing first where the behavior changed):
the_value_distributions_seed_is_mixed_into_the_values_only: seriesi's first value is streamiunderseed ^ value seed; a value seed of 0 matchesseedalone, 3 changes every value and no label set.a_counter_past_two_to_the_53_ends_with_an_error: Pareto(α=0.3) counter, 10 series x 100,000 samples, ends with oneErrnaming the series and 2^53 as its last item (the first revision returned 41,249 samples and no error).series_num_equal_to_the_label_set_count_always_succeeds: 1000 of 50 x 20 sets at skewness 3 (the first revision failed with 322 found).far_below_the_label_set_count_the_series_keep_the_draw_skew: 200 of 5000 sets,code_0on about 57% of the series (Zipf 1.5 over 5 values).near_the_label_set_count_heavy_sets_are_taken_first: 100 of 200 sets via the weighted path, heavycode_0sets mostly taken, lightcode_4sets mostly left.the_same_description_generates_the_same_samples_and_the_seed_changes_them,every_series_is_distinct_and_sampled_on_the_interval,a_counter_is_the_running_sum_of_the_plain_draws,bad_descriptions_are_refused_by_name,no_labels_means_one_series,a_description_round_trips_through_yaml, and the doctest.Outside-consumer checks (separate workspace, release):
size_hint.Oksamples, then oneErr("the counter of series 2 passed 2^53 at timestamp 46340 ms ...") as the last item.Resultitems.Review status
Round 1 (quality and functional reviews of the first revision, 74ef8d0) found nothing blocking. Addressed here:
aqpbm-usecase::data, conventions differreduce_scoresleft over from ERPreduce_one, "(ERP)" doc goneGeneratedTimeSeriescopied five description fieldsResultitems; test with Pareto(α=0.3)bad()pub(crate) fn badinerror.rsMonotonicAccprimed withpush(0.0)series_numlib.rsand docsfrom_pathfilesfeaturevalue_distribution,SeriesLabelSpecsamples()comment about the first counter draw{name}_{index}Follow-ups
aqpbm-usecase's synthetic path draws fromTimeSeriesDescriptionconventions (grid and stream seed in one place).aqpbm-planevalis left unchanged here; its executor driver will be replaced later.aqpbm-cli/src/erp.rs,aqpbm-cli/src/erp_cmd.rs,docs/erp-v1-workloads.md,scripts/sketch_runs/run_erp_v1_figure1.sh,scripts/sketch_runs/analyze_erp_v1_figure1.py, the ERP section ofdocs/aqpbm-cli.md, and the ERP-provenance text inconfigs/workloads/README.mdandburst.rs.