Skip to content

Indexer: precomputed Oni aggregates #105

Open

Description

@johnf

Parent

Spec: indexer redesign

What to build

Oni's Aggregates, precomputed by the indexer and summed per request.

  • Rule 13: derivation produces, for each entity with something of this crate's at or below it (including ancestors in parent crates), what this crate adds:

    • the collections, objects, subCollections and files counts below it
    • the language, communicationMode and mediaType values at or below it, using the same labels as the search facets

    Counts follow the indexer's classification. Files are counted whether held or not, and the entity itself isn't counted. People, organisations and software get zero counts and empty lists.

  • Table: an aggregates table in our own models, one row per (crate, entity). An index job writes only its own crate's rows, as the last batch. A delete removes them without reading the repository.

  • Read: the Oni entity transformer reads aggregates for a page with one flat query, summing counts and combining lists. This replaces the recursive memberOf query and its per-request batching.

  • Child crates aren't reindexed when a parent changes.

  • Benchmark: the benchmark gains the aggregates read check: a page of 100 entities with aggregates is at most 50 ms slower than without. If it isn't, add an index on the aggregates table.

Acceptance criteria

  • Derivation tests for rule 13 cover counts, lists, files not held, and people with empty aggregates.
  • A collection split across two child crates reads as the sum of both, with languages combined (the spec's "Sydney Speaks" example).
  • Deleting one child crate removes only its share.
  • Oni's entity and list endpoints return the same counts and lists as before for the test crates, with no recursive query left.
  • The benchmark reports the aggregates read, and it meets the 50 ms target (the number is recorded in a comment here).
  • Indexer coverage stays at 100%. pnpm test and pnpm lint pass.

Metadata

Metadata

Assignees

No one assigned

    Labels

    ready-for-agentFully specified, ready for an AFK agent

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions