Skip to content

Layout of src/ after the redesign #77

Closed

Description

@johnf

Question

What does src/ look like after the redesign (e.g. api/, oidc/, admin/, storage/, indexer/)? Where do the OCFL repository, the indexer, index state and LDaCA constants (indexType, openLicenses) live, and what replaces src/repository.ts and src/ocfl.ts's module-level singletons?

Activity

  1. self-assigned this
    on Nov 17, 2026
  2. johnf commented on Nov 17, 2026

    @johnf
    CollaboratorAuthor

    Resolution

    Decided by grilling. No ADR (a folder layout is cheap to change) and no glossary changes.

    Folder shape

    src/
      index.ts            API entry point
      worker.ts           worker entry point (new)
      app.ts  configuration.ts  logger.ts  prisma.ts
      admin/              admin pages
      oidc/               sign-in provider
      access/             auth.ts, licences.ts ("is this licence open"), access-control.ts (the label type)
      oni/                the Oni entity transformer and the aggregates read query
      repository/         open/create the OCFL repository, the content storage backend,
                          the file storage meta builder, the id rule
      index-queue/        add index/reindex/delete jobs, read index state
      indexer/            "work the queue"
    scripts/              CLI: ocfl-init, crates-import, index, generate-keys
    test/                 helpers, and tests that go from a crate to an HTTP response
    
    • No api/ folder: app.ts with access/ and oni/ already is the API.
    • Rejected: a catch-all shared/ folder for the functions both sides need; access/ and repository/ say what they hold.

    Inside the two new folders

    src/index-queue/
      index-queue.ts      the whole interface: add index/reindex/delete jobs, read index state
    src/indexer/
      indexer.ts          the whole interface: "work the queue" (until empty, or forever)
      jobs.ts             claim, heartbeat, retry
      crate.ts            read a crate's metadata and file list from the OCFL repository
      derive.ts           pure: crate metadata in, rows/documents/aggregates out
      rows.ts  documents.ts  aggregates.ts     the three writers
    prisma/models/
      index.prisma        job queue, crate record, worker heartbeat, crate↔entity links
      aggregates.prisma   the aggregates table
    
    • Each folder's interface is one file named after the folder; no index.ts barrels. Outside callers import only index-queue/index-queue.ts and indexer/indexer.ts.
    • Held firm: the two interface files, and that derive.ts does no I/O and imports neither configuration.ts nor prisma.ts. The other file names inside indexer/ are a starting point for the implementer.
    • src/utils.ts goes: its one function is used only by the indexer and moves into derive.ts.
    • The COLLECTION/OBJECT type IRIs stay exported from oni/; the indexer imports them.

    What replaces repository.ts and ocfl.ts

    • Both are deleted, along with the Repository interface and its dynamic import. It had one implementation and bundled four jobs the redesign has split apart (listing crates, index state, indexing, file storage).
    • Each entry point opens what it needs and passes it in:
      • src/index.ts opens the OCFL repository, builds the content storage backend and the index queue, and calls buildApp({ search, storage, indexQueue }). The admin pages receive only the index queue.
      • src/worker.ts and scripts/index.ts call the indexer's one operation with the Prisma client, OpenSearch client and OCFL repository as arguments.
    • No module-level mutable state in repository/, index-queue/ or indexer/. The in-memory state cache goes; index state is in Postgres.
    • configuration.ts, logger.ts and prisma.ts stay as import-anywhere modules.
    • Rejected: keeping a slimmer Repository interface for a future non-OCFL store, until a second store exists.

    LDaCA constants

    • openLicenses becomes a required environment setting, parsed in configuration.ts and read only by access/licences.ts. No default in code: the app refuses to start without it, rather than treating every licence as closed. The four Creative Commons URLs move to .env.example and the deployment config.
    • indexType becomes a constant inside indexer/derive.ts, not configuration.

    Tests

    • A test of one module sits beside it.
    • A test that puts a crate in and checks what the HTTP API returns lives in test/. That moves search.test.ts, root-collection.test.ts, damaged-crate.test.ts, collection-cycle.test.ts, search-total.test.ts, licence.test.ts, oni.test.ts and app.test.ts out of src/.
    • Rejected: exempting every *.test.ts from the lint rule (an admin test could then reach into the indexer).
    • What each test covers is left to the map's test strategy question.

    The lint rule's paths

    • Biome noRestrictedImports: src/indexer/** may be imported only from src/indexer/**, src/worker.ts, scripts/** and test/**.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions