fix(components): fail closed on a full or dead repo IndexedDB - #438
Open
lodystage[bot] wants to merge 5 commits into
Open
fix(components): fail closed on a full or dead repo IndexedDB#438lodystage[bot] wants to merge 5 commits into
lodystage[bot] wants to merge 5 commits into
Conversation
When the disk or the origin's storage quota ran out, IndexedDB rejected writes with QuotaExceededError and then killed the connection, so every later db.transaction() threw InvalidStateError. Session creation broke on the READ path — openPersistedDoc for a new room already runs a readwrite transaction — and each retry pasted the raw Chromium DOMException into a fresh toast. Freeing disk space did not help: the dying connection only goes away with the process. Classify the failure and latch one one-way breaker under the repo, then explain it once in a blocking recovery screen. - lib/storage-crisis.ts classifies (quota / unavailable) by walking the cause chain, and latches the first failure for the page lifetime. - providers/crisis-aware-storage-adapter.ts wraps the adaptor passed to LoroRepo.create. Once latched, every method rejects with StorageCrisisError without touching IndexedDB — reads included, since an undefined read would look like "no such document" and invite a write that overwrites durable history. close() stays delegated: it opens no transaction and runtime dispose needs it. - StorageCrisisDialog (mounted above RuntimeProvider, since loadMeta runs before anything renders) dismisses the stale toasts, states that a restart is required, and keeps the raw engine text behind a toggle. - app.restartApp() / app.quitApp() relaunch the process. A renderer reload is not enough; Chromium binds the dying backing store to it. Classification sits in Lody rather than in a loro-repo patch because StorageAdapter is a public interface and every method is async, so a bare synchronous throw surfaces as a rejection the wrapper already sees. Verified: 3208 component tests (27 new), 79 Electron tests, typecheck, lint, i18n, and the public/platform/code-collab boundary guards. Refs #417 Model: claude-opus-5[1m] Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Merging main brought the document maintenance process (#431), which trims every AGENTS.md under an 8192-byte gate. The storage-crisis entry landed as a 17-line block and pushed packages/components/AGENTS.md to 8898, the only error `pnpm run docs check` reported. Route it per .agents/README.md#where-content-goes rather than trimming words: the binding constraint stays as one compressed bullet in the nearest AGENTS.md, the cross-module explanation moves to .agents/docs/components-storage-crisis.md, and the decision with its rejected alternatives becomes a proposed bug-fix note in both languages. The note records what a future maintainer could plausibly get wrong: an in-memory repo fallback copied from the resilient cursor store, reads answering undefined instead of rejecting, auto-reopen on InvalidStateError, and patching loro-repo's dist. It also records that loro-repo does not lose data when a save fails — persistDocUpdate rolls its version pointer back and MetaPersister advances only after save resolves — and names the message-regex fallback as the known limit that upstream work would remove. `pnpm run docs check`: no errors. packages/components/AGENTS.md is 7905 bytes, 287 under the gate; it was already past the 7000-byte warning on main at 7489. Model: claude-opus-5[1m] Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The upstream session verified all three findings against loro-repo source and landed loro-dev/loro-repo#129 (a RepoStorageError carrying code: 'quota' | 'unavailable' | 'unknown', plus loadDoc no longer needing readwrite) and issue #130 for the missing retry trigger. Name both instead of "upstream work was opened". Two corrections to the note's own claims: - The verification counts predated the main merge. They are now 3216 tests across 438 files and 91 Electron tests, attributed to the merged branch at 90c6eaf, and the limits section admits that toasts raised after the screen mounts are not dismissed. - The brief this repo sent upstream asked for loadDoc's transaction to be split in two. That was wrong: the readwrite is deliberate and commented upstream, because splitting the read from delete(docId) drops updates appended in between. The note records the actual fix, which generalizes the existing compare-then-write helper. Also record why `code` is deliberately not read yet: the contract is open for review upstream and could be renamed, and a field that never matches would be dead code behind working heuristics. Verified that #129's rename to RepoStorageError is safe here — nothing in Lody compares an error name to 'Error', and classifyStorageFailure walks `cause` regardless. Adds the Lody PR link the note rules ask for, and states that the issue stays open for Phase 2. `pnpm run docs check`: no errors. Model: claude-opus-5[1m] Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The note said the failed-save behavior is "a UX and correctness-of-failure problem, not a durability bug". True but imprecise: nothing re-triggers a flush after a failure, so the re-queued document waits for the next doc event and the meta Flock for its next subscription callback. No data is lost, but the last change before a transient failure stays unpersisted until the user edits again. Filed as loro-dev/loro-repo#130. Also strengthen the rejected auto-reopen alternative with evidence found afterwards: ensureDb caches the promise from a FAILED open and never clears it, unlike the close() and versionchange paths, so the adaptor is already dead for the page lifetime once opening fails (loro-dev/loro-repo#131). An app-level reopen would have been built on a layer that cannot reopen itself. Both are upstream and out of scope here; they are recorded because one corrects a claim this note makes and the other supports a decision it defends. `pnpm run docs check`: no errors. Model: claude-opus-5[1m] Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Related issue
Refs #417
Phase 1 only (classify + fail-closed breaker + blocking recovery screen). The Issue stays open for Phase 2, the filesystem-only Manage storage panel.
Problem / pressure
When a user's disk filled, the renderer's CRDT replica (
lody-loro-repo-db-<workspaceId>) hitQuotaExceededError, and Chromium then tore down the backing store while keeping theIDBDatabaseobject alive. Every laterdb.transaction()threwInvalidStateError: ... The database connection is closing.Two things made this worse than a failed operation:
IndexedDBStorageAdaptor.loadDocopens its transactionreadwritebecause it may consolidate queued updates. So opening a brand-new session room — which writes nothing and returnsundefined— still took a readwrite transaction and threw. Session creation failed before any write was attempted, so a guard in the composer would have covered one caller out of many; archive, send, and workspace-catalog writes hit the same dead connection right after.location.reload()— the store is bound to the renderer PROCESS. So the toast kept firing after the user had already fixed the underlying problem, each retry pasting the raw DOMException into a fresh one.Summary
Classify the failure under the repo and latch a one-way breaker, then explain it once instead of repeating it.
lib/storage-crisis.ts— classifiesquota/unavailableby walking thecausechain, and latches the FIRST failure for the page lifetime so the recovery screen keeps naming the original cause. Anything unclassified keeps its existing behavior.providers/crisis-aware-storage-adapter.ts— wraps the adaptor passed toLoroRepo.create. Re-throws asStorageCrisisError, so rawIDBDatabasetext cannot reach a toast even on the first failure. Only forwards the optionalStorageAdaptermethods the inner adaptor actually implements.components/storage-crisis-dialog.tsx— mounted in__root.tsxaboveRuntimeProvider, becauseloadMetaruns before anything renders. Dismisses the stale toasts, states that a restart is required, keeps the engine text behind a toggle.app.restartApp()/app.quitApp()—app.relaunch()+app.quit(). A renderer reload is not a restart here.--z-storage-crisis: 110, above--z-toast.Classification sits in Lody rather than in
patches/loro-repo.patch:StorageAdapteris a public interface and every method isasync, so a bare synchronousdb.transaction()throw already surfaces as a rejection the wrapper sees. Patching the publisheddist/(two builds) would need re-applying on every version bump for no user-visible gain. Upstream work was opened againstloro-dev/loro-repofor the two things a wrapper cannot fix: a stablecodeon storage errors, andloadDocnot requiringreadwrite.Rationale is routed per
.agents/README.md#where-content-goes— a compressed rule inpackages/components/AGENTS.md, the explanation in.agents/docs/components-storage-crisis.md, the decision and rejected alternatives in a proposed bug-fix note (EN + ZH).Before / after
会话创建失败withFailed to execute 'transaction' on 'IDBDatabase': The database connection is closing.undefinedfrom a dead storeundefinedreads as "no such document" and invites a write over durable historyrestartApp()relaunches the process; browser shells fall back to reloadTest plan
Run on the merged branch (main brought 12 commits, including #431):
vitest runin@lody/components: 438 files / 3216 tests pass, 27 new acrossstorage-crisis,crisis-aware-storage-adapter, andstorage-crisis-dialog. Covers classification by name and by message,cause-chain walking, a cyclic chain terminating, fail-closed reads, optional methods not being advertised when absent,close()still delegating during a crisis, and the dialog's restart/quit/reload branches.@lody/electron: typecheck clean, 91 tests pass.pnpm test:scripts: 27 pass.tsgo --noEmitclean for@lody/components.pnpm lint(0 errors),check-i18n(en + zh_CN complete),check:public-boundary,check:platform-boundaries,check:code-collab-imports.pnpm run docs check: no errors.packages/components/AGENTS.mdis 7905 bytes — under the 8192 gate, but see the gap below.Not done: the disk-full condition is reproduced from fixtures, not a real exhausted volume; no manual end-to-end run on a full disk. Phase 2 (Manage storage panel) and Phase 3 (upstream PR) are out of scope.
Environment note for anyone re-running locally: with
NODE_ENV=productionexported in your shell, React 19's production build omitsactand every.tsxtest fails withact is not a function— including ones untouched here. UseNODE_ENV=test.Context handoff
Instructions for reviewing agents
providers/crisis-aware-storage-adapter.ts(does the wrapper preserveStorageAdaptersemantics loro-repo relies on?) andlib/storage-crisis.tsclassifyStorageFailure(are the match rules narrow enough not to latch on a transient error?).undefined; the latch being one-way with no retry;close()being the one method still delegated during a crisis; wrapping in Lody instead of patchingloro-repo.packages/components/AGENTS.mdsits 287 bytes under the gate — already past the 7000-byte warning on main at 7489, and I did not restructure other contributors' entries to buy room.Authoring context
patches/loro-repo.patchedit. Keep the sibling stream-cursor store's fail-open behavior untouched.restartApp()/quitApp()end the process; the existingbefore-quithandler still drains the embedded CLI. The screen deliberately offers no cache-clear button —deleteDatabase()blocks while the runtime holds a connection and is the wrong tool for a full disk.InvalidStateError(on a full disk the reopen fails too, and a self-healing storage layer makes failing closed harder); no in-memory repo fallback.codework would settle.