Skip to content

Indexer: shared entities kept per crate, and clashes fail the job #104

Open

Description

@johnf

Parent

Spec: indexer redesign

What to build

Shared entities and clashes, as ADR 0006 records. This fixes Deleting a crate's database index removes entities other crates share.

  • Crate links:
    • A crate link table in our own models: one row per (crate, entity) pair, indexed both ways. For people, organisations and software it holds that crate's derived copy.
    • It is the only record of which crates contain an entity. meta.crateId is dropped, and per-crate counts and deletes use the links.
    • A crate that mentions a person without the LDaC profile gets no link.
  • Winner: a shared entity's row and search document are wholly one crate's copy: a crate whose metadata licence is open first, then the lowest crate id. Copies are never merged.
  • Delete and reindex:
    • Deleting a crate removes its links, then the entities and documents with no link left.
    • A shared entity whose winning crate goes is rebuilt from the next crate's stored copy, without reading the repository.
    • Reindexing a crate that no longer contains an entity drops its link the same way.
  • Clashes: a collection, object or file another crate already holds, or an id described as a different kind of entity, fails the second crate's job, with an error naming both crates and the entity. The holder keeps the row.
  • The migration may empty the old Entity and File rows: there's no upgrade path for existing data.

Check the findings of Indexer: check production crates for profiled stubs of their parent collection before starting.

Acceptance criteria

  • Deleting one of two crates that describe the same person leaves the person searchable and served.
  • Deleting the winning crate makes the row and document the next crate's copy.
  • Indexing the crates in either order gives the same rows.
  • An open-licence copy wins over a restricted one with a lower crate id.
  • A second crate claiming a collection fails its job naming both crates; the first crate's row is untouched.
  • meta.crateId is gone, and the admin page counts come from the links.
  • Indexer coverage stays at 100%. pnpm test and pnpm lint pass.

Metadata

Metadata

Assignees

No one assigned

    Labels

    ready-for-agentFully specified, ready for an AFK agent

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions