You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
What @ocfl/ocfl-fs 1.0 beta offers for change detection and reading #67
What does @ocfl/ocfl-fs 1.0 beta (1.0.0-beta.59 installed) offer that changes how and when ldacapi indexes? Specifically:
Detecting change: reading an object's head version or inventory digest cheaply, so the indexer can tell whether a crate changed since it was last indexed.
Listing objects in a storage root efficiently (streaming, pagination) at production scale.
Reading ro-crate-metadata.json and file metadata (size, digest, path) without reading content, including for metadata-only crates that list files they don't contain.
Writing: what Import and Deposit need (creating objects and versions) versus what indexing in place needs (read-only), and any locking or concurrency guarantees between a writer and an indexer.
Anything new in the 1.0 API (vs what src/ocfl.ts and src/ocfl-repository.ts use today) worth adopting.
Sources: node_modules/@ocfl/ocfl-fs, the ocfl-js repository, the OCFL 1.1 spec.
Findings, with sources for each claim: docs/research/ocfl-fs-1-0-beta.md on research/ocfl-fs-1-0-beta. Sources: the installed 1.0.0-beta.59 source (node_modules/@ocfl/ocfl-fs/src/), ocfl-js main 6dc9df2 (guides, ADRs 0005 to 0007, commits since beta.59), and the OCFL 1.1 spec.
Detecting change
No persistent change token is public. An opened object gives head.number and head.createdAt. The inventory digest is computed on every open but kept internal (_digest, src/object-on-disk.ts).
object.isCurrent() costs one stat (device, inode, size, mtime of the root inventory). It only works against an object held in memory, since the identity it compares is private, and it is "a cheap check, not a guarantee" (src/ocfl-object.ts).
Reading anything about an object, even its head number, means root.open(id), which reads, parses, hashes and checks the whole root inventory against its sidecar. root.has(id) is the only cheaper read, and it gives existence only.
On disk, OCFL 1.1 requires an inventory.json.<alg> sidecar beside every inventory, written last (§3.6, E058 to E062). The library doesn't give out object root paths (ADR 0006), but StorageRoot.layout is public and the id-to-path mapping is deterministic (§4.3, E083). During an ocfl-fs commit the sidecar briefly lags the root inventory.
There is no watch, event or "changed since" API.
Listing
root.objects() is an async iterator over an opendir walk. It handles one object at a time, opening each one fully, in no order, and has no pagination, cursor, filter or resume. Damaged objects are yielded as { error, objectRoot, id? } and the walk carries on (src/list.ts).
Today src/ocfl.ts reads each inventory at least twice per run: once in idsMatching, once in indexObject.
Reading
ro-crate-metadata.json can only be read through open() plus file.text().
file.path, file.digest, createdAt and modifiedAt come from the inventory with no I/O, and size() is one stat (src/ocfl-file.ts).
A file a crate lists but the object doesn't hold has no OcflFile at all. Today that is logged as an error in src/indexer/database.ts, which then falls back to contentSize.
Writing and concurrency
Indexing in place uses only read calls, and none of them write.
Import and Deposit use create, update and a Draft: write, import, clear, and so on.
draft.import keeps files it doesn't replace, so scripts/crates-import.ts (no clear()) carries removed files into the next version.
Readers take no lock and never wait. Each write has one atomic commit point: the rename of the root inventory. An opened object is a snapshot.
Writers lock each object and refuse a second writer (UPDATE_IN_PROGRESS); they never queue or expire locks (ADR 0005).
During a remove, an object can open and then have files missing.
Unreleased on ocfl-js main, breaking for ldacapi
42550f8: a missing or empty storage root throws STORAGE_ROOT_NOT_FOUND instead of INVALID_STORAGE_ROOT/E069, which isMissing() in src/ocfl-repository.ts tests for.
Activity
johnf commentedon Nov 17, 2026
Answer
Findings, with sources for each claim:
docs/research/ocfl-fs-1-0-beta.mdonresearch/ocfl-fs-1-0-beta. Sources: the installed 1.0.0-beta.59 source (node_modules/@ocfl/ocfl-fs/src/), ocfl-jsmain6dc9df2 (guides, ADRs 0005 to 0007, commits since beta.59), and the OCFL 1.1 spec.Detecting change
head.numberandhead.createdAt. The inventory digest is computed on every open but kept internal (_digest,src/object-on-disk.ts).object.isCurrent()costs onestat(device, inode, size, mtime of the root inventory). It only works against an object held in memory, since the identity it compares is private, and it is "a cheap check, not a guarantee" (src/ocfl-object.ts).root.open(id), which reads, parses, hashes and checks the whole root inventory against its sidecar.root.has(id)is the only cheaper read, and it gives existence only.inventory.json.<alg>sidecar beside every inventory, written last (§3.6, E058 to E062). The library doesn't give out object root paths (ADR 0006), butStorageRoot.layoutis public and the id-to-path mapping is deterministic (§4.3, E083). During an ocfl-fs commit the sidecar briefly lags the root inventory.Listing
root.objects()is an async iterator over anopendirwalk. It handles one object at a time, opening each one fully, in no order, and has no pagination, cursor, filter or resume. Damaged objects are yielded as{ error, objectRoot, id? }and the walk carries on (src/list.ts).src/ocfl.tsreads each inventory at least twice per run: once inidsMatching, once inindexObject.Reading
ro-crate-metadata.jsoncan only be read throughopen()plusfile.text().file.path,file.digest,createdAtandmodifiedAtcome from the inventory with no I/O, andsize()is onestat(src/ocfl-file.ts).OcflFileat all. Today that is logged as an error insrc/indexer/database.ts, which then falls back tocontentSize.Writing and concurrency
create,updateand aDraft:write,import,clear, and so on.draft.importkeeps files it doesn't replace, soscripts/crates-import.ts(noclear()) carries removed files into the next version.UPDATE_IN_PROGRESS); they never queue or expire locks (ADR 0005).remove, an object can open and then have files missing.Unreleased on ocfl-js
main, breaking for ldacapiSTORAGE_ROOT_NOT_FOUNDinstead ofINVALID_STORAGE_ROOT/E069, whichisMissing()insrc/ocfl-repository.tstests for.draft.import(source, { target, filter }).INVALID_OBJECT, notENOENT.