Repository navigation
[2e] Read the whole text in the host with a form-in-text index #439
Copy link
Copy link
Closed
Milestone
Description
Part of Phase 2 (#425) of the data layer plan.
Whole-text read in BookService: reads every book of the source text, keeps a form-in-text index, and reports progress and failed books through getTextReadStatus; addressed by source project, and from 3k by session. Not called by the WebView yet.
Done when: on the #407 datasets the index finds the same catalog forms not in the text as main's whole-text read, a book that fails to read is reported and turns off "not in the text" marking, and a Scripture edit updates that book's entries.
Depends on: 2a (#435), 2d (#438).
From the plan
API
| Call | Kind | Request | Response | Replaces today | Issue |
|---|---|---|---|---|---|
getTextReadStatus |
query | source project (2e to 4c), or session (from 3k) | Books read, books in the text, books that failed to read | The read status TextReadingStatus shows |
2e, 3k |
Cross-book answers
| Answer | Today | Planned | Freshness | Issue |
|---|---|---|---|---|
| Forms in the text | The concordance's whole-text read in the WebView, run when a catalog filter or the concordance needs it | 2e's form-in-text index, read at session open; getCatalogRows filters by it |
At session open, and per book on a Scripture edit | 2e, 5a |
| Text read status | TextReadingStatus over the WebView's read |
getTextReadStatus |
Refetched when named | 2e, 5a |
Performance checkpoint
| After | What to check | Baseline (large-loadable) |
|---|---|---|
| 2e | Whole-text read time and host heap with every book read; decides whether the host reads every book at session open or, as on main, only on demand | Read every book 22.9 / 23.5 s |
Risks
| Risk | Effect | Mitigation |
|---|---|---|
| Reading every book at session open costs too much | Slow session open, or a large host heap | The read runs in the background and the catalog shows its progress; the checkpoint after 2e decides, and the fallback is reading on demand, as on main, with the host re-anchoring each book as getTokenizedBook serves it |
Decisions
| Question | Decision | From |
|---|---|---|
| Whole-text read | Carried: the host reads every book of the source text at session open, re-anchors each linked book as it arrives and builds a form-in-text index, so Stale covers the whole draft once the read finishes. The checkpoint after 2e decides whether that cost is affordable; if not, the read stays on demand, as on main | — |
Considered and rejected
- Moving the text-only answers into Phase 2 (the concordance's occurrences and the form-in-text check, joined with the WebView's analysis), so 4c would remove the text bridge. It adds WebView wiring to Phase 2 and splits the concordance across two issues. It's the fallback if the 2e checkpoint or 4a's latency gate fails because the text is read twice from 4a to 5d.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels
Activity
alex-rawlings-yyc commentedon Oct 16, 2026
BookServicekeeps only each project's most recently requested book (2a, #460). Reading the whole text throughgetTokenizedBookpushes the open book out of that cache and stops watching its text. That's intended for 2a, but 2e will need its own watches rather than relying on the cache's.imnasnainaec commentedon Oct 19, 2026
2d (#468) watches each project's open book (the last one outlined) separately from
BookService's one-book cache, and reports its edits throughbookService.onOpenBookTextChanged. That leaves for 2e:stop()stall from Fetch and tokenize books in a host BookService (#435) #460). Onlydeactivateawaits it.Keeping the open book in the cache beside the latest book fixes all three: one subscription per book, no eviction. The read still needs its own watches to track edits to every book it keeps.
onOpenBookTextChangedis small enough to fold into them.A patch against #468's head (65f5ee6) that keeps the open book in the cache, with its tests:
Patch
Keep the open book in BookService's cache beside the latest
Read the whole text in the host with a form-in-text index (#439)
Keep the open book in BookService's cache beside the latest
Read the whole text in the host with a form-in-text index (#439)
Keep the open book in BookService's cache beside the latest
Read the whole text in the host with a form-in-text index (#439)
Keep the open book in BookService's cache beside the latest
Read the whole text in the host with a form-in-text index (#439)