Problem
No proposer currently generates new, unique, non-memorized values for a genuine unique-ID (or other unique) column, so the evaluator's recommender has no sound candidate to recommend for that column type.
Every generator in the ChoiceProposer family (ZipfChoiceProposer, UniformChoiceProposer, WeightedChoiceProposer — proposers/choice.py) samples with replacement from at most MAXIMUM_CHOICES (500) distinct observed values, via dist_gen.choice_direct/zipf_choice_direct/weighted_choice in providers.py, which call random.choice/random.choices under the hood. Once the generated row count exceeds the number of distinct sampled values, duplicates are guaranteed — which would violate a real PRIMARY KEY/UNIQUE constraint on a genuine ID column.
The only other fallback, dist_gen.constant (providers.py:403), is worse: it returns the exact same value every time, so it duplicates as early as the second generated row.
In short: for a column that must contain unique values, every currently-registered proposer (see everything_factory() in proposers/init.py) will eventually produce a constraint violation once enough rows are generated. The evaluator's recommendation for such a column is therefore never actually usable at scale — it's picking "least-bad among broken options," not a correct one.
Proposed solution
Add a new UniqueSequenceProposer (+ matching ProposerFactory) that is guaranteed not to repeat values, registered alongside the existing factories in everything_factory() (proposers/init.py:43-60).
Acceptance criteria
- New proposer(s) never produce a duplicate value across a full data-generation run for a column previously flagged as unique (verify against a dense serial PK column, e.g. Pagila
actor_id).
- Registered in
everything_factory() so it appears as a candidate in propose/propose all.
- Wins the evaluator's ranking for genuinely unique columns (see
datafaker/evaluators/) without tripping weak_recommendation_warning (in proposal_ranking.py).
- Existing uniqueness/resample penalty logic in the evaluator (currently penalizing
ChoiceProposer and numeric-column candidates for exactly this failure mode) should no longer need to penalize this new proposer, since it isn't a resampler.
Context
Found while building an evaluator/recommender module (datafaker/evaluators/) that scores and ranks candidate generators for a column. The evaluator currently applies a uniqueness/resample penalty (1 - real_uniqueness × copy_fraction) to flag exactly this problem, but a penalty on bad options isn't a substitute for having a correct one to recommend.
Problem
No proposer currently generates new, unique, non-memorized values for a genuine unique-ID (or other unique) column, so the evaluator's recommender has no sound candidate to recommend for that column type.
Every generator in the
ChoiceProposerfamily (ZipfChoiceProposer,UniformChoiceProposer,WeightedChoiceProposer— proposers/choice.py) samples with replacement from at mostMAXIMUM_CHOICES(500) distinct observed values, viadist_gen.choice_direct/zipf_choice_direct/weighted_choicein providers.py, which callrandom.choice/random.choicesunder the hood. Once the generated row count exceeds the number of distinct sampled values, duplicates are guaranteed — which would violate a realPRIMARY KEY/UNIQUEconstraint on a genuine ID column.The only other fallback,
dist_gen.constant(providers.py:403), is worse: it returns the exact same value every time, so it duplicates as early as the second generated row.In short: for a column that must contain unique values, every currently-registered proposer (see
everything_factory()in proposers/init.py) will eventually produce a constraint violation once enough rows are generated. The evaluator's recommendation for such a column is therefore never actually usable at scale — it's picking "least-bad among broken options," not a correct one.Proposed solution
Add a new
UniqueSequenceProposer(+ matchingProposerFactory) that is guaranteed not to repeat values, registered alongside the existing factories ineverything_factory()(proposers/init.py:43-60).Acceptance criteria
actor_id).everything_factory()so it appears as a candidate inpropose/propose all.datafaker/evaluators/) without trippingweak_recommendation_warning(in proposal_ranking.py).ChoiceProposerand numeric-column candidates for exactly this failure mode) should no longer need to penalize this new proposer, since it isn't a resampler.Context
Found while building an evaluator/recommender module (
datafaker/evaluators/) that scores and ranks candidate generators for a column. The evaluator currently applies a uniqueness/resample penalty (1 - real_uniqueness × copy_fraction) to flag exactly this problem, but a penalty on bad options isn't a substitute for having a correct one to recommend.