[imnasnainaec]

Part of #424; closes #432.

Summary

This adds a conformance suite at the repository seam: the tests every AnalysisRepository implementation must pass. It runs in CI against DraftRepository.

 AnalysisRepository (the seam)
+  describeAnalysisRepositoryConformance(name, createStore)
+    createStore() -> ConformanceStore
+      open(sourceProjectId)
+      storeNewerBuildDraft(sourceProjectId)
+      damageBook(sourceProjectId, bookCode, 'corrupt' | 'lost')
+      damageDraft(sourceProjectId)
+      failCleanup(sourceProjectId) -> restore
+      holdsLeftovers(sourceProjectId)
   DraftRepository
+    draftRepository.conformance.test.ts   # over map-backed storage

src/__tests__/analysis-repository-conformance.ts            # the suite, backend-agnostic
src/__tests__/services/draftRepository.conformance.test.ts  # runs it against DraftRepository
src/__tests__/services/draftRepository.test.ts              # drops cases the suite now covers
src/__tests__/test-helpers.ts                               # memoryStorage, silentLogger
src/types/analysis-repository.ts                            # docs state what the suite checks
AGENTS.md                                                   # where seam and implementation tests go

What the suite checks

• Loading (exportAll)
  • A draft never saved loads as an empty one for its source project.
  • Values are JSON-plain. Absent optional fields are omitted, never null.
  • The caller may change a loaded draft, nested values included, without changing what is stored.
  • A book whose stored analysis is corrupt or lost is named, and the rest of the draft loads.
  • An unusable draft loads as an empty one.
• Loading for a copy (exportForCopy)
  • A saved draft exports whole, naming no book. A draft never saved exports as an empty one.
  • A corrupt or lost book is named, and the rest exports.
  • An unusable draft is refused rather than exported empty.
• Saving and loading back (replaceAll)
  • Every record kind and every optional field survives, in the same repository and in a newly opened one.
  • A payload linked from two books stays one record, and an edit to it saves.
  • A save is stamped with this build's model version.
  • Changing a draft after saving it, nested values included, doesn't change what is stored, and saving it again stores the change.
  • A save resolves once the draft lands, even if cleanup after it fails. A later save, even from a newly opened repository, finishes that cleanup.
• Replacing
  • A save replaces the stored draft whole, whether or not this repository loaded it first. Records and metadata the saved draft lacks are dropped.
  • After another repository replaces the draft, a load sees that draft, and a save from the reloaded repository is stored in full.
  • An emptied analysis saves as empty, and records a later save brings back are restored.
• Books that could not be read, after a load that named them
  • A save keeps such a book as stored, whatever the draft carries for it, and resolves with the books whose analysis it left out.
  • With discardUnreadable, a save writes or wipes the book like any other.
• Marking clean (markClean)
  • The stored draft is recorded as clean, its content as it stands, unreadable books included.
  • A draft never saved, or already clean, is left alone.
  • An unusable draft is refused.
• Isolation: drafts of different source projects stay apart.
• Newer builds: a newer build's draft is neither read, copied, marked clean nor overwritten.

Ground rules

• Record lists are compared in id order. The seam promises no list order. Arrays inside a record, such as a phrase's tokens or a token's morphemes, must keep theirs.
• Interface docs match the suite. AnalysisRepository states the empty-draft, ordering, stamping and caller-ownership rules, so the suite tests only what the interface promises.
• Only the seam is in the suite. Behavior particular to DraftRepository stays in its own tests: the shard journal, rewriting only changed shards, the bytes a kept shard holds, the error each refusal throws. DraftRepository tests the suite now covers are dropped. AGENTS.md says where each kind of test goes.
• Storage is damaged only through the store. The suite never reads or writes storage itself, so the hooks describe damage in seam terms (a book, the draft) rather than in shards and keys.

Evidence

• Unit suite: passes at 100% coverage, along with tsc --noEmit, ESLint and Prettier.

• The suite can fail. Each mutation of DraftRepository was run, then reverted:

  mutation                                       conformance tests
  none                                           39 passed
  drop the model-version stamp on save           1 failed
  skip the newer-build check before a save       1 failed
  lose one token link per book on load           24 failed
  keep stored books the saved draft lacks        7 failed
  fail a save when cleanup fails                 1 failed
  never retry a failed cleanup                   1 failed
  ignore discardUnreadable                       2 failed
  always discard unreadable books                2 failed
  resolve with no book left out                  1 failed
  export an unusable draft for copy as empty     2 failed
  markClean drops the stored books               2 failed
  markClean skips the newer-build check          1 failed
  load a lost book without naming it             2 failed
  

• No unexplained warnings. The runner fails a test if DraftRepository logs a warning, unless that test damaged storage or failed cleanup. The fixtures are valid drafts it accepts without complaint.

Deviations from the issue and the plan

• Today's calls only. The suite covers exportAll, exportForCopy, replaceAll and markClean, the calls the interface has.
  • apply, undo and the index lookups join it as 3b ([3b] Add apply with inverse capture and per-session undo #441) and 3c ([3c] Add derived indexes with a rebuild-and-diff command #442) build them. 3b's done-when already names conformance tests.
  • 4c ([4c] Delete the WebView write path #455) removes exportAll and replaceAll. Most of today's tests get rewritten against the reads that replace them. The suite itself stays.
• "In-memory implementation" is DraftRepository over map-backed storage, in the same sense as Route getDraft and saveDraft through an async repository (#431) #466.
• Shared test helpers moved. memoryStorage and silentLogger moved to test-helpers.ts. The new runner needs both, and silentLogger already had two copies.

Decisions with a strong alternative

Reviewer push-back is welcome.

• Order-insensitive comparison
  • Alternative: require list order to survive a round trip.
  • Why not: the plan's contract rules promise no order, and 1a made picks follow a rule, not list order. DraftRepository already regroups records by book on load.
• An exported suite function
  • Alternative: one test file with describe.each over implementations.
  • Why not: a later backend's runner may live elsewhere, such as an adapter over Build a host C# program in the extension to prepare for Harmony integration #462's C# program.
• Store hooks to plant a newer build's draft and damage storage
  • Alternative: leave the newer-build guard, unreadable books and failed cleanup to each implementation's own tests.
  • Why not: the plan makes the guard part of the seam ("keeps projectStorage's guards"), and Move Save and Save As into the host #475 made unreadable books and settling once content lands part of the interface. The suite checks each through the seam: a load still refuses the draft or still names the book, and holdsLeftovers says whether a later save finished cleanup.
• open is async
  • Alternative: a synchronous open, matching DraftRepository's constructor.
  • Why not: a backend in another process may need to wait before it can serve a repository.
• Not added: a run over Node file storage (perf/storage.ts)
  • Alternative: a second runner, since the repository also runs in Node.
  • Why not: the issue asks only for the in-memory implementation. A second runner is a few lines if it's wanted.
• One writer at a time, documented rather than tested
  • Alternative: pin interleaved saves from two repositories in the suite, and make DraftRepository safe under them.
  • Why not: production keeps one repository per source project, and the fix needs save invalidation that DraftRepository doesn't have. AnalysisRepository instead forbids overlapping calls for one draft, whether on one repository or several. After one repository saves a draft, any other that had already loaded or saved it must load it again before its next save.
• Unreadable books checked only after a load
  • Alternative: also check a save from a repository that never loaded the draft.
  • Why not: DraftRepository learns which books are unreadable only by loading, so such a save writes over a corrupt book, or wipes it if the draft carries nothing for it. replaceAll's doc says the book is kept unless options discards it, without that condition. Either the doc or DraftRepository should change; that belongs in a follow-up, not a test PR.

Comments on other issues

• 3a ([3a] Add host sessions #440): comment
• 3b ([3b] Add apply with inverse capture and per-session undo #441): comment
• 3c ([3c] Add derived indexes with a rebuild-and-diff command #442): comment
• 4c ([4c] Delete the WebView write path #455): comment
• 1d (Move Save and Save As into the host #475): comment on #475, comment here
• Build a host C# program in the extension to prepare for Harmony integration #462: comment

Merge Danger

Door: two-way

Only tests and docs change, plus one shared-helper move. A revert removes the suite and restores the dropped DraftRepository tests.

Blast Radius: tests

Nothing ships differently. A later change to DraftRepository or the seam must keep the suite passing, and a new backend must implement the store hooks to run it.

🤖 Generated with Claude Code

---

This change is

Summary by CodeRabbit

• Documentation
  • Clarified storage expectations, including how project data is handled, when saves are complete, and how unreadable analysis is managed.
  • Added guidance for ensuring analysis storage implementations meet the documented requirements.
• Tests
  • Expanded repository behavior checks for saving, loading, copying, data isolation, and recovery from damaged or unavailable records.
  • Consolidated shared test utilities to support consistent coverage.

[coderabbitai]

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

• Last reviewed commit: 84509d9
• Latest commit, not reviewed: 1d44ede

Use the following commands to manage reviews:

• @coderabbitai resume to resume automatic reviews.
• @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

• ▶️ Resume reviews
• 🔍 Trigger review

📝 Walkthrough

The change documents AnalysisRepository behavior, adds a reusable repository conformance suite, and runs it against DraftRepository. Shared test helpers provide in-memory storage and a silent logger.

Changes: Analysis repository conformance

Layer / File(s)	Summary
Document repository behavior src/types/analysis-repository.ts	The documentation defines JSON-plain values, mutation isolation, record ordering, call overlap, empty results, and replacement behavior for unreadable data and cleanup.
Add reusable conformance tests src/__tests__/analysis-repository-conformance.ts, src/__tests__/test-helpers.ts	The suite tests exports, replacement, persistence, mutation isolation, model versions, unreadable data, cleanup, and markClean. Shared helpers provide in-memory storage and a silent logger.
Run the suite against DraftRepository src/__tests__/services/draftRepository.conformance.test.ts, src/__tests__/services/draftRepository.test.ts, src/__tests__/utils/analysis-violations.test.ts, AGENTS.md	The DraftRepository test setup runs the conformance suite. Existing tests reuse shared helpers, and project storage guidance describes the suite and identifies ProjectSummaries as PAPI-free.

Priority: ➖ Normal

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Other

Possibly related issues

• sillsdev/interlinearizer-extension#441: The suite covers linked payload persistence and draft replacement, but does not add the requested apply and undo coverage.

Suggested reviewers: alex-rawlings-yyc

Merge Risk: 🔵 Low · up to 84509

Clarify the cleanup guarantee so repository implementations and callers have an accurate contract.

Pre-merge checks: 4 · 1

Failed checks: 1 warning

Check name	Status	Explanation	Resolution
Out of Scope Changes check		The conformance suite, its DraftRepository runner, shared test helpers, seam documentation, and related test cleanup support [#432]. The PR also changes AGENTS.md to describe ProjectSummaries as…	Remove the unrelated ProjectSummaries documentation change from AGENTS.md, or provide evidence that it is required for [#432].

Passed checks: 4 passed

Check name	Status	Explanation
Description Check		Check skipped - CodeRabbit’s high-level summary is enabled.
Title check		The title clearly and concisely describes the main change: adding a repository conformance suite.
Linked Issues check		[#432] requires a permanent repository conformance suite that runs in CI against the in-memory implementation. The PR adds describeAnalysisRepositoryConformance and runs it for DraftRepository ove…
Docstring Coverage		Docstring coverage is 85.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 20 functions across 6 files.

Full details: Linked Issues check

Explanation

[#432] requires a permanent repository conformance suite that runs in CI against the in-memory implementation. The PR adds describeAnalysisRepositoryConformance and runs it for DraftRepository over map-backed storage in draftRepository.conformance.test.ts. The suite covers the repository seam described in the PR evidence. Shared helpers, seam documentation, and removal of duplicate implementation tests support this objective.

Full details: Out of Scope Changes check

Explanation

The conformance suite, its DraftRepository runner, shared test helpers, seam documentation, and related test cleanup support [#432]. The PR also changes AGENTS.md to describe ProjectSummaries as PAPI-free. The supplied change summary does not connect that documentation change to the conformance-suite objective.

Resolution

Remove the unrelated ProjectSummaries documentation change from AGENTS.md, or provide evidence that it is required for [#432].

✨ Finishing Touches

📝 Generate docstrings

• Commit to this branch
• Create a new PR

🧪 Generate unit tests (beta)

• Commit to this branch
• Create a new PR

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[alex-rawlings-yyc]

These comments were generated by Claude Opus 5.5 (Claude Code) and reviewed by me before posting.

[imnasnainaec]

⛏️ #475 changes the repository interface this suite defines. Cases to add:

• replaceAll resolves with the books it left out. A book whose shard its load couldn't read keeps that shard as stored, whatever the draft carries for it, unless discardUnreadable. This changes 1b, where an edit overwrote the shard.
• replaceAll resolves once the draft has landed, even if deleting an orphaned shard or updating the journal fails, and the next save retries that.
• exportAll: resolves with the draft and the books it couldn't read.
• exportForCopy: names missing and unusable shards; rejects an unreadable envelope.
• markClean: rewrites only the envelope; does nothing for a missing or already-clean draft; refuses a draft from a newer build or one that fails validation.

[alex-rawlings-yyc]

These comments were generated by Claude Opus 5.5 (Claude Code) and reviewed by me before posting.

[alex-rawlings-yyc]

These comments were generated by Claude Opus 5.5 (Claude Code) and reviewed by me before posting.

The Evidence table's "none 38 passed" is stale: splitting the in-place-change test made it 39.

[alex-rawlings-yyc]

These comments were generated by Claude Opus 5.5 (Claude Code) and reviewed by me before posting.

[alex-rawlings-yyc]

@alex-rawlings-yyc reviewed 6 files and all commit messages, and resolved 21 discussions.
Reviewable status: complete! all files reviewed, all discussions resolved (waiting on imnasnainaec).