Skip to content

Cutting production over to the new indexer #86

Closed

Description

@johnf

Question

When the redesigned indexer ships, how does production move to it? Is a full delete-and-rebuild of the database index, search index, aggregates and index state acceptable (and how long may the catalogue be empty or partial while it runs), or do existing rows and documents need migrating in place? What order do the migration, the worker's first start and the rebuild happen in, and what is the way back if it goes wrong?

Activity

  1. self-assigned this
    on Nov 17, 2026
  2. johnf commented on Nov 17, 2026

    @johnf
    CollaboratorAuthor

    Resolution

    There is no cutover of existing data. The redesigned indexer goes onto a brand new VM with nothing on it, and traffic switches to that VM once it is ready. ldacapi is not in production yet, so no user sees the rebuild.

    Rebuild, not migrate. The database index, search index, aggregates and index state are built from the repository. Nothing converts old rows, old search documents or old meta.crateId values.

    No upgrade path in the code. The implementation may assume empty index tables: its migration can drop or empty the old Entity and File rows. Developers and any test server on the old schema run pnpm db:deploy, delete the search index and index again. The release notes say "rebuild both indexes after upgrading" and nothing more.

    Order on the new VM.

    1. Migrate the database (pnpm db:deploy).
    2. Put the crates in the repository.
    3. Start the API and the worker.
    4. Ask for every crate to be indexed (CLI or admin pages).
    5. Switch traffic once the admin crates page shows every crate as indexed and none failed.

    This becomes the README's install section. "Ready" is read off the admin pages (see #71); no readiness command or health gate is built for it. The API may be up during the rebuild because nobody is pointed at it yet.

    The way back. Don't switch traffic, or switch it back. No database dump, down-migrations or rollback procedure are needed for this.

    Rebuilds after go-live: not designed here. Once there are real users, a release that changes what the indexer derives needs every crate reindexed. The default is "reindex all" through the queue with the API up; a release that changes the search index's mapping takes a planned maintenance window. A no-gap rebuild (second search index and a switch) is out of scope for this map; the queue design does not block adding it later. The spec notes this as a known limit.

    For other tickets. This ticket sets no outage target for "Indexing performance at scale" (#88): the first full index runs before anyone uses the VM.

    No glossary change and no ADR: a fresh install is easy to reverse and unsurprising.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions