You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Where index state lives and what the admin pages read #71
Where does index state live, and what do the admin pages read? Decide:
What state exists per crate (queued / indexing / deleting / indexed / failed, last indexed version, errors) and where it's stored (e.g. Postgres in prisma/models).
How a queue survives a restart and works across replicas (if indexing is queued at all).
How the admin pages get state and counts without scanning the entity table by meta.crateId.
How a partial failure (search documents removed, database delete failed) is recorded and recovered.
Input from Where indexing runs and what triggers it (#70, resolved): indexing runs in a separate worker draining one durable queue that the API and CLI add to. This ticket now also owns: where that queue is stored; recording the OCFL version each crate was indexed at (incremental indexing depends on it); keeping failed jobs and their errors; a crate having at most one running and one waiting job; returning a dead worker's job to the queue (up to three times); and the admin pages showing when no worker is running.
Passed on from "How shared entities are stored and deleted" (#72): that ticket adds a crate↔entity link table in our own models, keyed by crate id, as the only record of which crates contain an entity (meta.crateId is dropped, and per-crate entity counts come from the links). Open for this ticket: whether crates get their own table that the links point to. A shared entity (person, organisation, software) counts in each crate that contains it.
Decided by grilling. Recorded as ADR 0007 and glossary terms (Index state, Indexed version) in #84.
Where it lives
Everything is in our own Postgres tables (prisma/models): index jobs (the queue), one record per crate, and workers.
Rejected: pg-boss (its own schema outside our migrations, its own job states to map onto ours); Redis with BullMQ (a new service); keeping every finished job as the history; counting rows by meta.crateId.
The queue
A table of index jobs, claimed with FOR UPDATE SKIP LOCKED. One running and one waiting job per crate is enforced by partial unique indexes.
A job that succeeds is deleted; the record holds its result. A failed job is kept with its error until the next job for that crate is asked for, which replaces it. So a crate has at most one failed, one running and one waiting job, and the table doesn't grow.
Who asked for what stays in the audit log, not in job history.
The crate record
One per crate known to be in the repository: name, indexed version, when it was indexed, and the number of entities and files the job wrote.
The indexed version is the last thing written and the first thing cleared. Index and reindex: clear the version, write rows, then documents, then set version, time and counts. Delete: clear the version, remove documents, then rows.
Deleting a crate's indexes keeps its record, as not indexed. A record is removed only when a job finds its crate is gone from the repository. "Delete everything" clears every version and keeps the records.
Incremental indexing skips a crate only when its indexed version equals its head version; it never skips a crate with no indexed version.
Partial failure
A failure part-way (for example documents removed, database delete failed) leaves a record with no indexed version and a failed job holding the error. The crate shows as failed.
Recovery is asking again: index and delete are both safe to repeat. There is no separate partial state and no repair code.
Workers
Each worker, and each CLI run while it drains the queue, writes a heartbeat every 30 seconds. A running job names its worker.
A worker silent for 5 minutes is treated as dead: its job goes back to waiting with its attempt count raised, and is marked failed on the third return. Any worker or CLI run makes this check when it looks for work. (60 seconds was rejected: parsing a large crate blocks Node's one thread, so a healthy worker can miss heartbeats.)
A worker shutting down cleanly removes itself and releases its job at once.
A job checks it still owns its claim before writing its result, so a worker wrongly judged dead drops its work.
Admin pages
They read only Postgres, never the repository. A crate is listed once something has queued a job for it (Deposit, Import, "Index all"). A crate another tool put in the repository is unlisted until someone runs "Index all"; a repository indexed in place shows an empty list until then.
One index state per crate, not one per index: not indexed, queued, indexing, deleting, indexed (with version, time and counts), failed (with the error). Derived: a running or waiting job wins, then a failed job, then the record. This replaces the note in Where indexing runs and what triggers it #70 that the pages could show the two indexes separately.
No "out of date" state, as that would mean reading the repository on each page load.
The top of the page counts crates per index state, from one grouped query. No grand total of entities, since shared entities would be counted twice.
"No worker running" is shown whenever no heartbeat is fresh, whether or not jobs are waiting.
Input from "Oni aggregates: precompute or compute per request" (#73, resolved): aggregates live in a new table of our own, one row per (crate, entity) pair, written and deleted only by that crate's index job. It is one more thing an index job writes and a crate delete removes.
Activity
johnf commentedon Nov 17, 2026
Input from Where indexing runs and what triggers it (#70, resolved): indexing runs in a separate worker draining one durable queue that the API and CLI add to. This ticket now also owns: where that queue is stored; recording the OCFL version each crate was indexed at (incremental indexing depends on it); keeping failed jobs and their errors; a crate having at most one running and one waiting job; returning a dead worker's job to the queue (up to three times); and the admin pages showing when no worker is running.
johnf commentedon Nov 17, 2026
Passed on from "How shared entities are stored and deleted" (#72): that ticket adds a crate↔entity link table in our own models, keyed by crate id, as the only record of which crates contain an entity (
meta.crateIdis dropped, and per-crate entity counts come from the links). Open for this ticket: whether crates get their own table that the links point to. A shared entity (person, organisation, software) counts in each crate that contains it.johnf commentedon Nov 17, 2026
Resolution
Decided by grilling. Recorded as ADR 0007 and glossary terms (Index state, Indexed version) in #84.
Where it lives
prisma/models): index jobs (the queue), one record per crate, and workers.meta.crateId.The queue
FOR UPDATE SKIP LOCKED. One running and one waiting job per crate is enforced by partial unique indexes.The crate record
Partial failure
Workers
Admin pages
Handed on
johnf commentedon Nov 17, 2026
Input from "Oni aggregates: precompute or compute per request" (#73, resolved): aggregates live in a new table of our own, one row per (crate, entity) pair, written and deleted only by that crate's index job. It is one more thing an index job writes and a crate delete removes.
docs: record that index state lives in Postgres and the admin pages r…