You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
How are entities that appear in more than one crate written and deleted? Decide between (or beyond):
Upsert plus a crate↔entity join table in our own models (prisma/models).
Reference counting.
roCrateIds on the entity, as in the spec's deposit extension (see the arocapi research ticket).
Also: which crate's copy wins when two crates describe the same @id differently; how deleting one crate leaves other crates' entities intact; and how lookups by crate avoid scanning the entity table.
Input from "Where indexing runs and what triggers it" (#70, resolved): the unit of work is one index job per crate, incremental by default (a crate whose head version is already indexed is skipped), and jobs for different crates can run at the same time on different workers. Open for this ticket: two crates sharing an entity may be indexed or deleted concurrently, and a changed parent crate can leave its children's rootCollection stale, since nothing in #70 reindexes the children.
Decided by grilling with the maintainer. Fixes the design behind #35.
1. Crate links
A crate↔entity link table in our own models (prisma/models), one row per (crate, entity) pair, indexed both ways.
It is the only record of which crates contain an entity. meta.crateId is dropped from the entity row; counts, deletes and lookups by crate use the links.
Rejected: a reference count (can't say which crates, drifts when a run dies, no lookup by crate); a roCrateIds array in meta (no index, so every lookup by crate is a scan). If the deposit API needs roCrateIds, it is read from the links.
2. Collections, objects and files belong to one crate
The same collection, object or file in two crates is an error in the data, not a shared entity.
The crate that already holds it keeps the row. The second crate's copy of that entity is refused; the rest of that crate is indexed. Nothing is overwritten or merged.
The clash is reported once per crate as an error naming both crates, on every run until the data is fixed.
The same id described as different kinds by two crates (a person in one, an object in another) is the same clash.
Which crate "holds" it depends on indexing order. Accepted, since it is an error either way.
Rejected: refusing the whole second crate; picking a winner by rule (hides a data bug).
3. People, organisations and software are shared entities
Each link for one of these keeps that crate's own derived copy.
The row and the search document are wholly one crate's copy: name, description, licences and virtual crate together, never mixed.
The winner: a crate whose metadata licence is open first, then the lowest crate id. A full rebuild gives the same rows whatever order crates are indexed in.
Rejected: last indexed wins (the repository walk is unordered, so licences could change between rebuilds); merging copies; lowest crate id alone (a restricted copy could win while an open one sits in the index).
4. Root collection
Person, Organisation and SoftwareApplication rows always have a null rootCollection, shared or not.
Read from arocapi 5.0.0-beta.10 and crate-works/oni @ b3d2591: null breaks nothing. Oni's Person page is unchanged; Organisation and SoftwareApplication pages lose the collection breadcrumb and "Member of" card; these entities leave the Collection facet.
Listing a person's objects on their page is a later extension, not part of this design.
5. Deleting and reindexing
Deleting a crate removes its links, then removes the entities (and search documents) left with no link.
A shared entity whose winning crate goes is rebuilt from the next crate's stored copy, with no read of the repository.
Reindexing a crate that no longer contains an entity drops that link the same way.
A shared entity counts in each crate that contains it.
A crate that mentions a person without the LDaC profile gets no link.
Changes to earlier decisions
Two lines of "What the indexer derives from a crate" (#69) change:
Point 2, "rootCollection is unchanged": now null for Person, Organisation and SoftwareApplication.
Point 5, "meta (crateId, rocrate, accessControl) stays as today": crateId is dropped.
Whether one crate's database writes happen in a single transaction: to "Where indexing runs and what triggers it" (Where indexing runs and what triggers it #70), or implementation.
For implementation
Production crates weren't checked for member crates that carry a profiled stub of their parent collection. Under point 2 such a stub is refused and reported. The test crates don't do this.
ADR 0006 and the glossary term "Shared entity": #83 (draft).
Correction to "Passed on": "Where indexing runs and what triggers it" (#70) was already closed when this resolved, so whether one crate's database writes happen in a single transaction is left to implementation.
Input from Where index state lives and what the admin pages read (#71, resolved): each crate has a record holding its indexed version and the number of entities and files its index job wrote. Those counts are per crate, and the admin pages show no grand total of entities, so they don't depend on how a shared entity is counted. A crate's record is removed only when a job finds the crate is gone from the repository; deleting its indexes keeps the record as not indexed.
Point 2 changed. When a crate describes a collection, object or file that another crate already holds, the holder still keeps the row, but the second crate's index job now fails with an error naming both crates and the entity. The rest of that crate is no longer indexed around the clash. The crate has no indexed version and shows as failed on the admin pages until its data is fixed.
Why: refusing one entity and indexing the rest made a clash the only data error that didn't fail its job. That conflicted with "Progress and failure reporting for index runs" (#87), and left the clash visible only in the log. Decided by the maintainer while writing Spec: indexer redesign. ADR 0006 is updated in #93.
Activity
johnf commentedon Nov 17, 2026
Input from "Where indexing runs and what triggers it" (#70, resolved): the unit of work is one index job per crate, incremental by default (a crate whose head version is already indexed is skipped), and jobs for different crates can run at the same time on different workers. Open for this ticket: two crates sharing an entity may be indexed or deleted concurrently, and a changed parent crate can leave its children's
rootCollectionstale, since nothing in #70 reindexes the children.johnf commentedon Nov 17, 2026
Resolution
Decided by grilling with the maintainer. Fixes the design behind #35.
1. Crate links
prisma/models), one row per (crate, entity) pair, indexed both ways.meta.crateIdis dropped from the entity row; counts, deletes and lookups by crate use the links.roCrateIdsarray inmeta(no index, so every lookup by crate is a scan). If the deposit API needsroCrateIds, it is read from the links.2. Collections, objects and files belong to one crate
3. People, organisations and software are shared entities
4. Root collection
rootCollection, shared or not.crate-works/oni@ b3d2591: null breaks nothing. Oni's Person page is unchanged; Organisation and SoftwareApplication pages lose the collection breadcrumb and "Member of" card; these entities leave the Collection facet.5. Deleting and reindexing
Changes to earlier decisions
Two lines of "What the indexer derives from a crate" (#69) change:
rootCollectionis unchanged": now null for Person, Organisation and SoftwareApplication.meta(crateId,rocrate,accessControl) stays as today":crateIdis dropped.Passed on
For implementation
ADR 0006 and the glossary term "Shared entity": #83 (draft).
johnf commentedon Nov 17, 2026
Correction to "Passed on": "Where indexing runs and what triggers it" (#70) was already closed when this resolved, so whether one crate's database writes happen in a single transaction is left to implementation.
johnf commentedon Nov 17, 2026
Input from Where index state lives and what the admin pages read (#71, resolved): each crate has a record holding its indexed version and the number of entities and files its index job wrote. Those counts are per crate, and the admin pages show no grand total of entities, so they don't depend on how a shared entity is counted. A crate's record is removed only when a job finds the crate is gone from the repository; deleting its indexes keeps the record as not indexed.
johnf commentedon Nov 17, 2026
Point 2 changed. When a crate describes a collection, object or file that another crate already holds, the holder still keeps the row, but the second crate's index job now fails with an error naming both crates and the entity. The rest of that crate is no longer indexed around the clash. The crate has no indexed version and shows as failed on the admin pages until its data is fixed.
Why: refusing one entity and indexing the rest made a clash the only data error that didn't fail its job. That conflicted with "Progress and failure reporting for index runs" (#87), and left the clash visible only in the log. Decided by the maintainer while writing Spec: indexer redesign. ADR 0006 is updated in #93.