Skip to content

Indexer: a benchmark over a generated 30,000-entity collection #96

Open

Description

@johnf

Parent

Spec: indexer redesign

What to build

A repeatable benchmark for indexing at production scale, built before the new writers so that tuning has numbers to work from.

  • A generator for a 30,000-entity collection in two shapes:
    • one crate holding all of it
    • the same collection split across thousands of crates (a top collection, sub-collection crates, object crates)
  • Text files of 50 KB, 1 MB and 5 MB among its files.
  • A command that indexes the generated collection into a fresh database and search index and reports wall-clock time and the indexing process's peak memory for each shape.
  • The first run measures today's indexer as the baseline. Later tickets run it against the new indexer.
  • The command is run by hand, not by pnpm test or CI.

The targets it will check, from the spec: the largest collection fully indexed in under 10 minutes with the worker under 1 GB. The aggregates read check is added by the aggregates ticket.

Acceptance criteria

  • One command generates both shapes deterministically (same input → same crates).
  • One command indexes each shape and prints time and peak memory.
  • The baseline numbers for today's indexer are recorded in a comment on this issue.
  • The README or the command's help says how to run it.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    ready-for-agentFully specified, ready for an AFK agent

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions