ADFA-4853: Wizard content pre-selection + post-install provisioning#260
Merged
Conversation
ADR-4853 records the wizard pre-selection + post-install provisioning design. Adds
the offline data layer it needs: a generator (tools/gen-books-catalog-asset.py) that
trims the dashboard catalog.db to a gzipped JSONL asset (id,title,author,lang,url),
the bundled asset itself (1100 books, ~46 KB), BooksCatalogAsset (reads the asset and
emits the same row shape as the live REST search), and BooksWishlist (the persisted
order {id,title,url}, app-private so it survives the install). UI wiring and the
post-install drain come next.
Bumps cover cell margins (6dp to 8dp) so cards don't read as one block, and makes cover-color assignment neighbor-aware: seeded from the title hash for variety, then nudged so a card never shares a color with its left neighbor or the card above it (2-column grid). Color still carries no meaning; this only reduces same-color runs.
The 8dp cell margin was set on the cell's own params but discarded when the cell was added to the row with fresh weight params, so cards touched. Move a small gutter (5dp) onto the row LayoutParams actually used, giving an even ~10dp gap between columns and rows — a light separation, less than the Library cards.
Reuses the Get More Books screen for pre-install selection. In wizard mode the source is the bundled offline catalog (BooksCatalogAsset), there is no live server, and Add writes the picks to BooksWishlist (drained after install) instead of the download service; 'in your books' becomes 'Added'. Educational/My books chips and the downloads link are hidden offline. SetupLibraryActivity.openBooksWizard() launches it. Not yet wired into the wizard flow (content step + install queue come next).
After choosing the edition, Step 1 now opens the Get More hub in wizard mode instead of jumping straight to the old content picker. In wizard mode the hub gates by the chosen tier's plan (not a live probe): Books only for Full, Wikipedia and Maps always, Courses hidden. Books opens the offline catalog + wishlist; Wikipedia/Maps are placeholders for now. A wizard-only Continue proceeds to the existing install step. Transitional: the system install still runs via Step 2; the unified install queue that drains the wishlists replaces it in the next slice.
Wikipedia/ZIM in the wizard reuses the same offline browse (assets/kiwix_catalog.csv); the only change is the terminal: in wizard mode ZIM Confirm banks the cart into ZimWishlist and returns to the hub instead of downloading live (button reads 'Add to your setup'). Maps is identical pre/post-install (it stops everything and runs the proot), so the wizard Maps card reuses the existing Maps flow rather than a special mode. Books + ZIM selections are now persisted orders awaiting the post-install drain.
Reuses the same Get More hub (no duplicate): in wizard mode the header now reads 'Set up your library' so it reads as the wizard's content step. The Continue button moved from code into the layout, styled like the other primary buttons (rounded teal), shown only in wizard mode — fixes the flat, off-style button.
The 'Set up your library' step now carries the same header as the old Step 2: the System -> Content spine and the Used / System / Your picks / Free storage bar, shown only in wizard mode (the conventional Get More hub is unchanged). System uses the tier's OS size; Your picks sums the ZIM wishlist bytes plus a small per-book estimate, and refreshes on resume as selections change.
Logs the asset load (row count) and any read failure under tags K2Go-BooksAsset / K2Go-Books, and shows a distinct 'Couldn't read the offline catalog' message in wizard mode instead of the network one, so a failure to read books_catalog.jsonl.gz is diagnosable pre-install.
…APK) The bundled books_catalog.jsonl.gz was never packaged into the APK (FileNotFoundException at runtime): a .gz double-extension binary matched '* text=auto eol=lf' in .gitattributes and got treated as EOL-normalized text, unlike the single-extension kiwix_catalog.csv that packages fine. Switch to an uncompressed, single-extension books_catalog.jsonl (207 KB, 1100 books) and read it without gzip — same approach as the Kiwix CSV. Generator updated to emit .jsonl.
Adds BooksProvisioner: once the system is installed and the server is up, it hands the banked Books wishlist to the existing BooksDownloadService (the live, foreground, per-item-retry engine) and clears the wishlist — no new download machinery. The home triggers it one-shot when it sees the server alive with a pending order, so wizard book picks provision themselves seamlessly after install. ZIM live drain + the setup progress screen come next.
ZimProvisioner mirrors BooksProvisioner: once installed and the server is up, it resolves each banked 'project|lang|flavour' selector against the offline catalog (KiwixCatalog / kiwix_catalog.csv) into a real ZIM file + label + size and hands them to the existing ZimDownloadService (live REST, foreground, per-item retry), then clears the wishlist. The home drains Books and ZIM one-shot when it sees the server alive.
The 'Set up your library' hub and the old Step 2 content picker were both in the flow (hub Continue -> Step 2), showing two redundant content screens. Continue now triggers the system install itself (InstallService ACTION_START, companion=false: OS/tier only, maps ships in the image) and opens LibraryActivity, the same as the old 'Download library' button. Content picked in the hub is banked as wishlists and drains live once the server is up, so Step 2 is out of the wizard path (still reachable only via the hidden A/B flip).
Adds K2Go-Provision logging to the home trigger and both provisioners (wishlist counts, drain hand-off, ZIM catalog load/resolve) so we can see whether the drain fires post-install and why content wasn't provisioned in a fresh install.
ZimDownloadService now logs, per ZIM, the filename it sends to the REST engine and whether the job finished or failed (with the server's error), so we can see why a handed-off ZIM ends up not installed (empty filename vs server rejection like 'no ZIMs requested').
Mirror the ZIM per-job logging on BooksDownloadService: log the id/title/url sent to /api/books/download, a missing job id, POST failures, per-job done, and server error phases, so we can see why handed-off books don't provision.
Root cause of nothing installing: the drain fired as soon as /home answered (nginx up), but dash-node (the REST engine on :4000) was still warming up, so every /api/*/download POST returned 502 Bad Gateway and the jobs failed with no retry. Now the home probes an /api endpoint first and only drains once it answers non-5xx; the wishlist is untouched until then and the ~3s poll retries, so provisioning waits for the engine to be ready.
SetupProgressActivity no longer re-draws the checklist. It now shows one summary row per active stream (title + 'X of N' / Done / 'k failed' + chevron); tapping a row opens that stream's detail, whose per-item checklist is drawn by the shared ProvisioningChecklist (same renderer as the Get More cards) with back to the index. Removes the third copy of the checklist; the wizard and Get More converge on one renderer.
Remove Step2OptionAFragment (collapsible picker), Step2OptionBFragment (5-step spine), AbFlip, and their layouts, plus SetupLibraryActivity.goToStep2/step2Fragment/flipAbTest/ optionB. These were unreachable from the current flow (Step 1 -> hub -> Continue -> install) and only accessible via the hidden A/B flip, so they only added confusion.
Books and ZIM now retry a failed item up to 3 times (4s apart) before marking it FAILED, covering nginx 502 while the engine warms up and flaky Gutenberg/mirror fetches. Only after 3 attempts does an item show as failed (manual Retry still available). Keeps the initial install hands-off — it advances on its own and only stops on real errors.
Tapping a Finishing-setup index row now hosts the actual ZimPreparingFragment / BooksDownloadsFragment (in a new 'from index' mode: they only observe, hiding their own Finish/Run-in-background), so the user sees the real live progress (bar, %, checklist, per-item retry) — not a re-drawn copy. In a detail the primary action is Back (to the index); Finish is secondary with a confirm dialog. Index keeps Finish (single exit) + Run in background. Row affordance uses a Material chevron; adds ic_chevron_right / ic_arrow_left. Get More flow unchanged (fromIndex defaults false).
…roundable) Note two pending items in the ADR so they surface when we review open work: (1) a background jobs monitor screen (live monitor + failed-item history) that the Finishing setup Finish note will point to; (2) Maps is not backgroundable — Maps changes (outside FQR) stop the server, so it must run to completion in the foreground, and the Maps card must explain the shipped minimal maps, FQR vs full rebuild, and that a quality reinstall is an advanced decision.
…ct, real buttons) The screen now uses an XML layout so its buttons are real MaterialButtons matching the installer (fixing the off-style Back/Finish). A status dot (like Library) shows 'Starting services...' until the REST engine answers; only then does the drain start, so no spinner spins before the system can download. Each stream row shows from the start: waiting dot -> active spinner -> green check / amber alert, tapping opens the real card. On full success it auto-redirects to the library after a 3s countdown (Cancel keeps you here and reveals Finish); on failure it shows Finish plus a note pointing to the future background jobs monitor. Run in background leaves; the Library keeps provisioning.
… flash) On install SUCCESS, if the wizard banked content, LibraryActivity opens SetupProgressActivity directly (over the library) instead of briefly landing on the home. The home no longer auto-opens the screen; it keeps the readiness-gated drain only as a background fallback so 'Run in background' still provisions. The screen owns the visible starting/drain flow.
The dot line now carries only the short, changeable phrase — 'Starting services' (amber)
that flips to 'Adding your content' (green) — so only one easy-to-read line changes with
state. The explanation ('It will redirect you automatically when finished. You can keep
using the app by running the installation in the background.') moves to its own line below,
separated by a gap, and stays fixed.
Returning from a real detail card reclaimed the service listener with an async commit(), but the fragment's onDestroyView (which nulls the listener) ran afterwards and clobbered it — so a job finishing while back on the index left the spinner stuck until the card was reopened. Use commitNow() so the fragment tears down (and nulls) before the index reclaims the listener; the index now reacts to completion (and auto-redirect) as expected.
… cards Adds SetupProgressActivity: a post-install screen with one card per content stream being provisioned, in a fixed order (Wikipedia/ZIM, then Books; Maps later). Each card shows a per-item checklist driven live by ZimDownloadService/BooksDownloadService (round check / teal dot / amber + Retry), and a section only appears if that stream has a session — so the user sees exactly what they picked being added. Finish clears the sessions; Run in background leaves them going. The home opens it once the drain starts.
…f truth) The per-item checklist (round check / colored dot / label + optional sub-label / inline Retry) was hand-copied in ZimPreparingFragment and BooksDownloadsFragment. Extract it to a single ProvisioningChecklist.render() and repoint both — identical rendering, now maintained in one place. ZIM passes its 'failed' suffix label; Books passes its state sub-label. No behavior change (regression check on device: both detail cards look/work the same). Prepares the wizard index to link to these cards instead of re-drawing them.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Lets the first-run wizard pre-select content offline (before anything is installed) and provisions it automatically once the system is up — the "leave a food order; cook it when the kitchen is ready" model. Builds on ADFA-4850 (Get More → Books) and reuses its download engine. Design: controller/docs/ADR-4853.
Wizard (pre-install)
Post-install provisioning (live)
Dashboard REST (static/dashboard): adds /api/books/languages for the language filter.
Deploy / verification