You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Indexer: a benchmark over a generated 30,000-entity collection #96
A repeatable benchmark for indexing at production scale, built before the new writers so that tuning has numbers to work from.
A generator for a 30,000-entity collection in two shapes:
one crate holding all of it
the same collection split across thousands of crates (a top collection, sub-collection crates, object crates)
Text files of 50 KB, 1 MB and 5 MB among its files.
A command that indexes the generated collection into a fresh database and search index and reports wall-clock time and the indexing process's peak memory for each shape.
The first run measures today's indexer as the baseline. Later tickets run it against the new indexer.
The command is run by hand, not by pnpm test or CI.
The targets it will check, from the spec: the largest collection fully indexed in under 10 minutes with the worker under 1 GB. The aggregates read check is added by the aggregates ticket.
Acceptance criteria
One command generates both shapes deterministically (same input → same crates).
One command indexes each shape and prints time and peak memory.
The baseline numbers for today's indexer are recorded in a comment on this issue.
The README or the command's help says how to run it.
Activity