diff --git a/CHANGELOG.md b/CHANGELOG.md index db9cbd3f6..39e48c1e6 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -21,6 +21,18 @@ released versions carry their date on the heading. ### Features +- 2026-10-08: **The Import form says when a program the import needs is + missing, with Try again.** A WhatsApp import needs wtsexporter, and + **Convert** and **Compress** need ffmpeg and ffprobe. When one hasn't been + downloaded or its download failed, the Import form names it and gives the + reason, with **Try again**, which downloads it at once, and a link to + Troubleshooting. A WhatsApp import can't start without wtsexporter. An + import with **Convert** or **Compress** still starts, as before. An import + started while its program is still downloading waits for it, and its + progress line says so, such as "Waiting for the wtsexporter download: 12 MB + of 30 MB (40%)". When that download fails, the run fails with the reason + and names the troubleshooting section (#1053). + - 2026-10-08: **The desktop app downloads ffmpeg, ffprobe and wtsexporter by itself.** Each time it starts, it downloads in the background whatever is missing from its Tools Directory, with no button and nothing to wait diff --git a/docs/src/content/docs/docs/user/features/messages/attachments-and-media.md b/docs/src/content/docs/docs/user/features/messages/attachments-and-media.md index dc61c4943..518f81530 100644 --- a/docs/src/content/docs/docs/user/features/messages/attachments-and-media.md +++ b/docs/src/content/docs/docs/user/features/messages/attachments-and-media.md @@ -82,10 +82,12 @@ These commands install both on `PATH`: The desktop app finds the programs on `PATH`, else in its Tools Directory, and takes both from the same one. [**Settings → System**](/docs/user/features/settings/system/#media) shows each under **Media**: `Found` with its path, its download's progress, why the download failed, `not found`, or `not used` with the reason. -An Import Run looks for ffmpeg at the Staging Review, not when the run starts, because Staging copies the original files and needs neither program. -When ffmpeg or ffprobe can't be used, the review names it, such as `Media can't use ffprobe.`, and the **Convert media** or **Compress media** button is disabled. -The desktop app downloads both programs into the Tools Directory when it starts, so a program missing at the review is usually still downloading. -When **Settings → System** says the download failed, the package manager commands above install ffmpeg and ffprobe instead. +With **Convert** or **Compress** chosen, the Import form says when ffmpeg or ffprobe can't be used, and why, but the run still starts, because Staging copies the original files and needs neither program. +At the Staging Review, the review names the program, such as `Media can't use ffprobe.`, and the **Convert media** or **Compress media** button is disabled. +**Try again**, on the form and at the review, downloads the program into the Tools Directory at once. +A program still downloading doesn't stop the review. +Media waits for the download, and its progress line says so. +When the download fails, the package manager commands above install ffmpeg and ffprobe instead. The run keeps waiting at the review until the program is there. ## The size limit diff --git a/docs/src/content/docs/docs/user/features/owner/troubleshooting.md b/docs/src/content/docs/docs/user/features/owner/troubleshooting.md index 697a41ab3..0bf51cd11 100644 --- a/docs/src/content/docs/docs/user/features/owner/troubleshooting.md +++ b/docs/src/content/docs/docs/user/features/owner/troubleshooting.md @@ -98,6 +98,11 @@ Import continues once the field is empty. A WhatsApp import runs a separate program, `wtsexporter`, which the desktop app downloads when it starts. [**Settings → System**](/docs/user/features/settings/system/#media) shows whether it arrived, or why the download failed, and [WhatsApp](/docs/user/import-sources/whatsapp/#wtsexporter) says where it comes from. +The Import form names the program and the reason, and its **Try again** downloads it at once instead of at the next start. + +The app has no `wtsexporter` download for some computers, such as Linux on ARM. +There the Import form says so and offers no **Try again**, because nothing can be downloaded. +A `wtsexporter` put in the Tools Directory by hand is used instead. ### A WhatsApp import doesn't stop when cancelled @@ -112,6 +117,7 @@ Two of the **Attachments** choices on the Import form run `ffmpeg` and `ffprobe` In the desktop app, **Settings → System** shows the Tools Directory under **Media**. The app looks on the system `PATH`, then in the Tools Directory, and takes both programs from the same one. The lines there report each program as found, with its path, not found, or not used, with the reason. +When a download failed, **Try again** on the Import form, or at the Staging Review, downloads the program at once instead of at the next start. ## Convert diff --git a/docs/src/content/docs/docs/user/features/settings/system.md b/docs/src/content/docs/docs/user/features/settings/system.md index 29109b50f..8df124dad 100644 --- a/docs/src/content/docs/docs/user/features/settings/system.md +++ b/docs/src/content/docs/docs/user/features/settings/system.md @@ -66,13 +66,16 @@ One line per program reports the result: - A cross with ` not found` and where to put it. A missing ffmpeg or ffprobe goes beside the other one, or both go in the Tools Directory, because the two are used only from one place. A missing wtsexporter goes in the Tools Directory. +- A cross with ` not found, and the app has no download of it for this computer` and where to put a copy by hand. + The app has no wtsexporter download for Linux on ARM. - A cross with ` not used` and the reason. For ffmpeg and ffprobe, the reason is that they were found in two places, one only on `PATH` and the other only in the Tools Directory. For wtsexporter, the reason is that it has no permission to run. - A download arrow with `Downloading ` and how much has arrived, such as `12 MB of 29 MB (41%)`. The line updates each second until the download ends. -- A cross with ` download failed` and the reason: no connection to the download's server, the status the server answered, a checksum that didn't match, a file that couldn't be written to the Tools Directory, or a program in the Tools Directory that doesn't run on this computer. - A failed download is tried again the next time the app starts. +- A cross with ` download failed` and the reason: no connection to the download's server, the status the server answered, a checksum that didn't match, a file that couldn't be written to the Tools Directory, a download that was interrupted or couldn't be started, or a program in the Tools Directory that doesn't run on this computer. + A failed download is tried again the next time the app starts, or at once with **Try again** on the Import form. + The line ends with a link to the program's section in [Troubleshooting](/docs/user/features/owner/troubleshooting/#import). A program that doesn't run is not downloaded again, because the download would be the same file. For ffmpeg and ffprobe, the reason says to install them with a package manager, because the app uses the copy on `PATH`. While an older copy is in place, the line shows that copy as found instead, because the old copy is still the one used. diff --git a/docs/src/content/docs/docs/user/import-sources/whatsapp.md b/docs/src/content/docs/docs/user/import-sources/whatsapp.md index 6fc82f57e..783b8b901 100644 --- a/docs/src/content/docs/docs/user/import-sources/whatsapp.md +++ b/docs/src/content/docs/docs/user/import-sources/whatsapp.md @@ -16,6 +16,8 @@ Two things make it hard: The desktop app runs `wtsexporter`, from Message Crate's fork of the open-source [WhatsApp Chat Exporter](https://github.com/KnugiHK/WhatsApp-Chat-Exporter), [messagecrate/WhatsApp-Chat-Exporter](https://github.com/messagecrate/WhatsApp-Chat-Exporter), because only the fork records the ids a quoted reply is linked by. Each time it starts, the desktop app downloads the pinned release into its Tools Directory when it isn't there, in the background, and [**Settings → System**](/docs/user/features/settings/system/#media) shows the program or how its download stands. The app runs `wtsexporter` only from the Tools Directory, never from `PATH`. +A WhatsApp import can't start without it, and the Import form names the reason and offers **Try again**. +An import started while `wtsexporter` is still downloading waits for the download, and its progress line says so. A program in the Tools Directory whose checksum is the pinned program's is kept, whoever put it there, and anything else under its name is replaced by the pinned program. ## WhatsApp on Android diff --git a/src-tauri/src/commands/events.rs b/src-tauri/src/commands/events.rs index ce7be2d89..aa65ce68b 100644 --- a/src-tauri/src/commands/events.rs +++ b/src-tauri/src/commands/events.rs @@ -8,6 +8,8 @@ use message_crate_core::{IssueSink, ProgressEvent, ProgressSink, RunIssue, Write use serde::Serialize; use tauri::{AppHandle, Emitter}; +use crate::tool_downloads::Program; + /// One log line for the UI's log panel. Payload: `String`. pub const LOG: &str = "extract:log"; /// Progress-bar numbers. Payload: [`ExtractProgressEvent`]. @@ -92,6 +94,12 @@ pub struct ExtractProgressEvent { /// Extra step status the UI shows. On `setup` it is the step's label. #[serde(skip_serializing_if = "Option::is_none")] pub status: Option, + /// The program the run waits for while it downloads (#1053). The event + /// then has no counts, `bytes_done` is how much has arrived and + /// `bytes_total` the file's size when the server said, and the window + /// says "Waiting for the download" with them. + #[serde(skip_serializing_if = "Option::is_none")] + pub waiting: Option, } impl ExtractProgressEvent { @@ -104,6 +112,7 @@ impl ExtractProgressEvent { bytes_done: None, bytes_total: None, status: None, + waiting: None, } } } diff --git a/src-tauri/src/commands/extract.rs b/src-tauri/src/commands/extract.rs index 2c45fc60f..03a7cb8d7 100644 --- a/src-tauri/src/commands/extract.rs +++ b/src-tauri/src/commands/extract.rs @@ -40,6 +40,7 @@ use super::paths::{logs_dir, scratch_dir}; use crate::app_directories::RunLog; use crate::run_directories::RunDirectories; use crate::state::{AppState, JobName}; +use crate::tool_downloads::{Program, ToolDownloads}; /// Ask this process to stop the job that is running. Does nothing when no /// job runs. @@ -160,6 +161,7 @@ pub struct ExtractArgs { pub fn extract( state: tauri::State<'_, Arc>>, directories: tauri::State<'_, RunDirectories>, + downloads: tauri::State<'_, ToolDownloads>, app: tauri::AppHandle, args: ExtractArgs, ) -> Result<(), String> { @@ -201,7 +203,9 @@ pub fn extract( let job = start_job(&state, JobName::Staging)?; let app_handle = app.clone(); - config.cancel = Some(job.cancel_flag()); + let cancel = job.cancel_flag(); + config.cancel = Some(cancel.clone()); + let downloads = downloads.inner().clone(); // Two channels, two jobs: log lines are for the person reading the log // panel, progress events are for the bar. Nothing reads counts out of // the prose. @@ -217,6 +221,16 @@ pub fn extract( config.issues = Some(events::run_issue_sink(&app_handle, &run_log)); spawn_job(app, job, move || { + // A program still downloading is waited for, and the wait is the + // Staging row's progress line (#1053). + super::tools::wait_for_downloads( + &app_handle, + &downloads, + programs_run_by(&config.source), + &cancel, + "setup", + ) + .inspect_err(|error| run_log.error(error))?; let run_result = run_staging(&config, &output_dir, &media_settings) .inspect_err(|error| run_log.error(error))?; let payload = finished_payload(&run_result); @@ -494,6 +508,17 @@ fn build_exporter_config( .map_err(|errors| errors.join("; ")) } +/// The programs the Tools Directory holds that `source`'s exporter runs: a +/// WhatsApp import runs wtsexporter. Staging copies attachments as they are +/// whatever the person chose, so it never runs ffmpeg; the Media stage does +/// (`transcode_staging`). +fn programs_run_by(source: &SourceConfig) -> &'static [Program] { + match source { + SourceConfig::Whatsapp(_) => &[Program::Wtsexporter], + _ => &[], + } +} + /// Call the exporter that matches `config.source`. /// /// # Errors diff --git a/src-tauri/src/commands/extract/tests.rs b/src-tauri/src/commands/extract/tests.rs index 6eafd8dcb..55a2db6fc 100644 --- a/src-tauri/src/commands/extract/tests.rs +++ b/src-tauri/src/commands/extract/tests.rs @@ -742,3 +742,43 @@ fn a_staged_jsonl_attachment_that_is_not_utf8_does_not_fail_the_count() { assert_eq!(counts, (1, 1)); } + +/// A WhatsApp import waits for wtsexporter's download, and no other source +/// waits for a program: Staging runs ffmpeg for none of them (#1053). +#[test] +fn only_a_whatsapp_import_waits_for_a_download() { + use message_crate_core::{ + AppleConfig, FormatConfig, GoSmsProConfig, ImazingConfig, OpenExtractConfig, + SmsBackupPlusConfig, SmsBackupRestoreConfig, WhatsappConfig, + }; + let none: &[Program] = &[]; + let sources = [ + SourceConfig::GoSmsPro(GoSmsProConfig { + owner_phones: Vec::new(), + }), + SourceConfig::SmsBackupRestore(SmsBackupRestoreConfig { + owner_phones: Vec::new(), + }), + SourceConfig::SmsBackupPlus(SmsBackupPlusConfig { + owner_phones: Vec::new(), + owner_emails: Vec::new(), + phone_country: None, + verbose: false, + include_summary: false, + }), + SourceConfig::OpenExtract(OpenExtractConfig::default()), + SourceConfig::Imazing(ImazingConfig::default()), + SourceConfig::Apple(AppleConfig::default()), + SourceConfig::Format(FormatConfig::default()), + ]; + for source in &sources { + assert_eq!(programs_run_by(source), none, "{source:?}"); + } + for platform in [WhatsappPlatform::Android, WhatsappPlatform::Ios] { + let source = SourceConfig::Whatsapp(WhatsappConfig { + platform: Some(platform), + ..WhatsappConfig::default() + }); + assert_eq!(programs_run_by(&source), [Program::Wtsexporter]); + } +} diff --git a/src-tauri/src/commands/staging.rs b/src-tauri/src/commands/staging.rs index 3b16bd1f0..c635b79d7 100644 --- a/src-tauri/src/commands/staging.rs +++ b/src-tauri/src/commands/staging.rs @@ -38,6 +38,7 @@ use super::jobs::{spawn_job, start_job}; use crate::app_directories::RunLog; use crate::run_directories::{self, RunDirectories, StagingRoot}; use crate::state::{AppState, JobName}; +use crate::tool_downloads::{Program, ToolDownloads}; /// The directory `summarize_staging`, `transcode_staging`, `delete_run_dir` /// and the Import Run record commands act on. @@ -158,6 +159,7 @@ pub async fn summarize_staging( bytes_done: None, bytes_total: None, status: None, + waiting: None, }, ); }) @@ -241,6 +243,7 @@ fn transcode_summary(report: &TranscodeReport) -> String { pub fn transcode_staging( state: tauri::State<'_, Arc>>, directories: tauri::State<'_, RunDirectories>, + downloads: tauri::State<'_, ToolDownloads>, app: tauri::AppHandle, args: RunDirArgs, ) -> Result<(), String> { @@ -255,8 +258,19 @@ pub fn transcode_staging( let app_handle = app.clone(); let issues = events::run_issue_sink(&app, &run_log); + let downloads = downloads.inner().clone(); spawn_job(app, job, move || { if media_stage_converts { + // ffmpeg or ffprobe still downloading is waited for, and the + // wait is the Media row's progress line (#1053). + super::tools::wait_for_downloads( + &app_handle, + &downloads, + &[Program::Ffmpeg, Program::Ffprobe], + &cancel, + "media", + ) + .inspect_err(|error| run_log.error(error))?; events::log_to_run( &app_handle, &run_log, @@ -284,6 +298,7 @@ pub fn transcode_staging( bytes_done: None, bytes_total: None, status: None, + waiting: None, }, ); }, diff --git a/src-tauri/src/commands/tools.rs b/src-tauri/src/commands/tools.rs index c0a187ab3..df2978266 100644 --- a/src-tauri/src/commands/tools.rs +++ b/src-tauri/src/commands/tools.rs @@ -8,8 +8,13 @@ //! where a program is run from. use std::path::PathBuf; +use std::sync::atomic::Ordering; -use crate::tool_downloads::{DownloadState, Program, ToolDownloads}; +use message_crate_core::{CancelFlag, Cancelled}; +use tauri::AppHandle; + +use super::events::{self, ExtractProgressEvent}; +use crate::tool_downloads::{self, DownloadState, Program, ToolDownloads, WaitError}; /// Where one program is, as Settings shows it. /// @@ -25,6 +30,11 @@ pub enum ToolStatus { }, /// The program is on neither place it is looked for. Missing, + /// The program is on neither place it is looked for, and the app has no + /// download of it for this computer (wtsexporter on Linux on ARM), so + /// Try again can't bring it. A copy put in the Tools Directory by hand + /// is used. + Unavailable, /// The program was found but is not used, for `reason`. Unusable { /// Why it is not used. @@ -38,7 +48,8 @@ pub enum ToolStatus { total: Option, }, /// The program's download failed, for `reason`, and the program is not - /// found. It is tried again the next time the app starts. + /// found. It is tried again the next time the app starts, or with Try + /// again on the Import form. DownloadFailed { /// Why the download failed. reason: String, @@ -56,6 +67,15 @@ impl ToolStatus { } } + /// This status, with a missing program the app has no download for + /// (`pinned` is false) told apart as unavailable. + fn with_pin(self, pinned: bool) -> Self { + match self { + Self::Missing if !pinned => Self::Unavailable, + status => status, + } + } + /// This status as `download` changes it. A download in progress is /// shown whatever was found, because the file is about to change. A /// failed one is shown only when the program is missing: a program found @@ -80,6 +100,12 @@ impl ToolStatus { pub struct ToolsStatus { /// The Tools Directory, `None` when the app has none (no home directory). pub tools_dir: Option, + /// Whether a check of the Tools Directory runs in this process now, the + /// start-up check or Try again. A program it has not looked at yet shows + /// as missing, so the window keeps asking while this is true. Another + /// app's check on the same Tools Directory is not counted: Try again + /// reports it as [`RetryResult::AlreadyRunning`]. + pub checking: bool, /// ffmpeg, from `PATH` or the Tools Directory. pub ffmpeg: ToolStatus, /// ffprobe, from `PATH` or the Tools Directory. @@ -92,11 +118,156 @@ pub struct ToolsStatus { /// and say how each one's download stands. #[tauri::command] pub fn tools_status(downloads: tauri::State<'_, ToolDownloads>) -> ToolsStatus { - tools_status_with(&downloads) + tools_status_with(&downloads, &pinned_programs()) +} + +/// The programs the app has a download of for this computer. +fn pinned_programs() -> Vec { + tool_downloads::pinned_for(std::env::consts::OS, std::env::consts::ARCH) + .into_iter() + .map(|pin| pin.program) + .collect() +} + +/// What Try again did, as [`retry_tool_downloads`] tells the window. +#[derive(Debug, Clone, Copy, PartialEq, Eq, serde::Serialize)] +#[serde(rename_all = "camelCase")] +pub enum RetryResult { + /// A check started in this process, and `checking` is true until it ends. + Started, + /// Another check holds the lock on the Tools Directory: the start-up + /// check, an earlier Try again, or another app's check. Nothing started. + /// `checking` is true only when that check runs in this process, so the + /// window can't tell from the status when another app's check ends. + AlreadyRunning, + /// The app has no Tools Directory (no home directory), which the status + /// shows as `toolsDir: null`. + NoToolsDirectory, + /// The check's thread could not be started, and each program it would + /// have downloaded shows as a failed download with that reason. + CouldNotStart, +} + +/// Check the Tools Directory again and download what is missing, as **Try +/// again** on the Import form asks (#1053). Returns at once: the check runs +/// on a thread of its own, and the window follows it through +/// [`tools_status`]. +#[tauri::command] +pub fn retry_tool_downloads(downloads: tauri::State<'_, ToolDownloads>) -> RetryResult { + let Some(dir) = media::tools_dir() else { + return RetryResult::NoToolsDirectory; + }; + let status = tools_status_with(&downloads, &pinned_programs()); + let not_found: Vec = [ + (Program::Ffmpeg, &status.ffmpeg), + (Program::Ffprobe, &status.ffprobe), + (Program::Wtsexporter, &status.wtsexporter), + ] + .into_iter() + .filter(|(_, status)| !matches!(status, ToolStatus::Found { .. })) + .map(|(program, _)| program) + .collect(); + let pinned = tool_downloads::pinned_for(std::env::consts::OS, std::env::consts::ARCH); + match tool_downloads::retry( + dir, + tool_downloads::GITHUB.to_string(), + pinned, + &downloads, + ¬_found, + ) { + // Nothing joins the check's thread: the window follows the check + // through `tools_status`. + tool_downloads::Retry::Started(_thread) => RetryResult::Started, + tool_downloads::Retry::AlreadyRunning => RetryResult::AlreadyRunning, + tool_downloads::Retry::CouldNotStart => RetryResult::CouldNotStart, + } } -/// [`tools_status`], with the downloads in `downloads`. -fn tools_status_with(downloads: &ToolDownloads) -> ToolsStatus { +/// How far a download waited for moves before the window hears of it again, +/// so a download heard of at every 64 KiB read doesn't send an event each. +const WAITING_STEP_BYTES: u64 = 256 * 1024; + +/// Wait while any of `programs` is downloading, before an import runs it +/// (#1053). The wait goes to the window as `extract:progress` events on +/// `step` ([`waiting_event`]): one when it starts, and one each time the +/// program waited for changes or its download moves by +/// [`WAITING_STEP_BYTES`]. The window writes the progress line from them, +/// in the byte format the Import form uses. +/// +/// # Errors +/// +/// Returns [`Cancelled`] when the run is cancelled while it waits, and the +/// failed download's reason when a download waited for fails. +pub(crate) fn wait_for_downloads( + app: &AppHandle, + downloads: &ToolDownloads, + programs: &[Program], + cancel: &CancelFlag, + step: &str, +) -> anyhow::Result<()> { + let mut last: Option<(Program, u64, Option)> = None; + downloads + .wait_for( + programs, + &|| cancel.load(Ordering::Relaxed), + &mut |program, received, total| { + let now = (program, received, total); + if !worth_sending(last, now) { + return; + } + events::emit( + app, + events::PROGRESS, + waiting_event(step, program, received, total), + ); + last = Some(now); + }, + ) + .map_err(|err| match err { + WaitError::Cancelled => anyhow::Error::new(Cancelled), + failed @ WaitError::Failed { .. } => anyhow::Error::new(failed), + }) +} + +/// Whether the wait at `now` is news to the window, which last heard `last`. +fn worth_sending( + last: Option<(Program, u64, Option)>, + now: (Program, u64, Option), +) -> bool { + let Some((program, received, total)) = last else { + return true; + }; + program != now.0 || total != now.2 || now.1.abs_diff(received) >= WAITING_STEP_BYTES +} + +/// The `extract:progress` event of a run on `step` waiting for `program`'s +/// download, `received` bytes in of `total`: marked by `waiting`, with no +/// counts and the bytes in `bytes_done` and `bytes_total`. +fn waiting_event( + step: &str, + program: Program, + received: u64, + total: Option, +) -> ExtractProgressEvent { + ExtractProgressEvent { + step: step.to_string(), + done: 0, + total: 0, + bytes_done: Some(received), + bytes_total: total, + status: None, + waiting: Some(program), + } +} + +/// [`tools_status`], with the downloads in `downloads` and a download for +/// this computer of each of `pinned`. +fn tools_status_with(downloads: &ToolDownloads, pinned: &[Program]) -> ToolsStatus { + let status = |program: Program, found: ToolStatus| { + found + .with_pin(pinned.contains(&program)) + .with_download(downloads.get(program)) + }; let (ffmpeg, ffprobe) = match media::ffmpeg_tools() { Ok(tools) => (ToolStatus::of(tools.ffmpeg), ToolStatus::of(tools.ffprobe)), // Found in two places: neither is used, and both say why. @@ -115,9 +286,10 @@ fn tools_status_with(downloads: &ToolDownloads) -> ToolsStatus { }; ToolsStatus { tools_dir: media::tools_dir().map(|dir| dir.display().to_string()), - ffmpeg: ffmpeg.with_download(downloads.get(Program::Ffmpeg)), - ffprobe: ffprobe.with_download(downloads.get(Program::Ffprobe)), - wtsexporter: wtsexporter.with_download(downloads.get(Program::Wtsexporter)), + checking: downloads.checking(), + ffmpeg: status(Program::Ffmpeg, ffmpeg), + ffprobe: status(Program::Ffprobe, ffprobe), + wtsexporter: status(Program::Wtsexporter, wtsexporter), } } @@ -125,11 +297,69 @@ fn tools_status_with(downloads: &ToolDownloads) -> ToolsStatus { mod tests { use super::*; + /// A run waiting for a download is sent marked by the program, with its + /// bytes and no counts: the shape `progressDetail` in + /// `web/src/screens/import/useImportJob.ts` reads, whose tests feed the + /// same JSON. + #[test] + fn a_run_waiting_for_a_download_is_sent_as_the_program_and_its_bytes() { + assert_eq!( + serde_json::to_value(waiting_event( + "setup", + Program::Wtsexporter, + 12 * 1024 * 1024, + Some(30 * 1024 * 1024) + )) + .unwrap(), + serde_json::json!({ + "step": "setup", + "done": 0, + "total": 0, + "bytes_done": 12_582_912, + "bytes_total": 31_457_280, + "waiting": "wtsexporter", + }) + ); + assert_eq!( + serde_json::to_value(waiting_event("media", Program::Ffmpeg, 0, None)).unwrap(), + serde_json::json!({ + "step": "media", + "done": 0, + "total": 0, + "bytes_done": 0, + "waiting": "ffmpeg", + }) + ); + } + + /// The window hears of a wait when it starts, when the program or the + /// size changes, and when the download moves by a step, not at every + /// read. + #[test] + fn a_wait_is_sent_when_it_moves_by_a_step() { + let first = (Program::Ffmpeg, 0, Some(10_000_000)); + assert!(worth_sending(None, first)); + assert!(!worth_sending( + Some(first), + (Program::Ffmpeg, 64 * 1024, Some(10_000_000)) + )); + assert!(worth_sending( + Some(first), + (Program::Ffmpeg, WAITING_STEP_BYTES, Some(10_000_000)) + )); + assert!(worth_sending( + Some(first), + (Program::Ffprobe, 0, Some(10_000_000)) + )); + assert!(worth_sending(Some(first), (Program::Ffmpeg, 0, None))); + } + /// The window reads `state` and, for a found program, `path`. #[test] fn a_status_is_sent_as_its_state_and_path() { let status = ToolsStatus { tools_dir: Some("/home/sam/message-crate/tools".into()), + checking: true, ffmpeg: ToolStatus::of(Some(PathBuf::from("/usr/bin/ffmpeg"))), ffprobe: ToolStatus::of(Some(PathBuf::from("/usr/bin/ffprobe"))), wtsexporter: ToolStatus::of(None), @@ -138,6 +368,7 @@ mod tests { serde_json::to_value(&status).unwrap(), serde_json::json!({ "toolsDir": "/home/sam/message-crate/tools", + "checking": true, "ffmpeg": { "state": "found", "path": "/usr/bin/ffmpeg" }, "ffprobe": { "state": "found", "path": "/usr/bin/ffprobe" }, "wtsexporter": { "state": "missing" }, @@ -170,12 +401,23 @@ mod tests { let name = whatsapp_exporter::wtsexporter_file_name(); media::set_tools_dir(Some(tools.path().to_path_buf())); let downloads = ToolDownloads::default(); - let missing = tools_status_with(&downloads).wtsexporter; + let pinned = [Program::Wtsexporter]; + let missing = tools_status_with(&downloads, &pinned).wtsexporter; + let unavailable = tools_status_with(&downloads, &[]).wtsexporter; media::testutil::write_with_mode(&tools.path().join(name), 0o755); - let found = tools_status_with(&downloads).wtsexporter; + let found = tools_status_with(&downloads, &pinned).wtsexporter; + let found_by_hand = tools_status_with(&downloads, &[]).wtsexporter; media::set_tools_dir(previous); assert_eq!(missing, ToolStatus::Missing); + // No download for this computer: told apart, so the form offers no + // Try again; a copy put there by hand is found all the same. + assert_eq!(unavailable, ToolStatus::Unavailable); + assert_eq!( + serde_json::to_value(&unavailable).unwrap(), + serde_json::json!({ "state": "unavailable" }) + ); + assert_eq!(found_by_hand, found); assert_eq!( found, ToolStatus::Found { diff --git a/src-tauri/src/commands/upload.rs b/src-tauri/src/commands/upload.rs index ccfe5e5cf..b3dc017e1 100644 --- a/src-tauri/src/commands/upload.rs +++ b/src-tauri/src/commands/upload.rs @@ -29,6 +29,7 @@ fn finished_upload_events( bytes_done: None, bytes_total: None, status: None, + waiting: None, }; let summary = serde_json::json!({ "summary": format!( @@ -214,6 +215,7 @@ fn forward_upload_event(app: &tauri::AppHandle, event: ProgressEvent) { bytes_done: None, bytes_total: None, status: None, + waiting: None, }, ); } diff --git a/src-tauri/src/main.rs b/src-tauri/src/main.rs index ce5690ac0..dad8a6449 100644 --- a/src-tauri/src/main.rs +++ b/src-tauri/src/main.rs @@ -60,7 +60,7 @@ fn main() { // nothing waits for it (`tool_downloads`). if let Some(home) = dirs::home_dir() { let tools = app_directories::use_tools_dir_in(&home); - tool_downloads::start(tools, app.state::().inner().clone()); + tool_downloads::start(tools, app.state::().inner()); } let app_data_dir = app.path().app_data_dir()?; let record = app_data_dir.join(run_directories::RECORD_FILE); @@ -108,6 +108,7 @@ fn main() { commands::staging::read_import_run_record, commands::staging::save_import_run_record, commands::tools::tools_status, + commands::tools::retry_tool_downloads, ]); builder diff --git a/src-tauri/src/tool_downloads.rs b/src-tauri/src/tool_downloads.rs index dacd6fb1f..ab9d926e6 100644 --- a/src-tauri/src/tool_downloads.rs +++ b/src-tauri/src/tool_downloads.rs @@ -2,8 +2,11 @@ //! (#1053, `docs/adr/0019`). //! //! At every start the app checks the Tools Directory in the background and -//! downloads what is missing or older than the release pinned here. Nothing -//! waits for it, and a failed download is tried again at the next start. +//! downloads what is missing or older than the release pinned here. Login +//! and browsing don't wait for it, and a failed download is tried again at +//! the next start, or when **Try again** on the Import form asks +//! ([`retry`]). An import that needs a program still downloading waits for +//! it ([`ToolDownloads::wait_for`]). //! //! - **ffmpeg and ffprobe** are not downloaded when both are on `PATH`, //! because a person who installed ffmpeg chose it. @@ -44,7 +47,9 @@ use std::fmt; use std::fs::File; use std::io::{self, Read, Seek, Write}; use std::path::{Path, PathBuf}; -use std::sync::{Arc, Mutex}; +use std::sync::atomic::{AtomicUsize, Ordering}; +use std::sync::{Arc, Condvar, Mutex, PoisonError}; +use std::thread::JoinHandle; use std::time::Duration; use sha2::{Digest, Sha256}; @@ -368,6 +373,11 @@ pub enum DownloadError { CouldNotWrite(String), /// The program passed its checksum and is in place, and does not run. DoesNotRun(Program), + /// The check's thread could not be started, so nothing was downloaded. + CouldNotStart(String), + /// The check stopped partway through the download, such as on a panic + /// in its thread, so the download never ended. + Interrupted, } impl fmt::Display for DownloadError { @@ -399,11 +409,12 @@ impl fmt::Display for DownloadError { Install it with your package manager instead; the app uses the copy on PATH.", program = program.name() ), - Self::DoesNotRun(Program::Wtsexporter) => write!( - f, - "wtsexporter in the Tools Directory doesn't run on this computer. \ - See \"Import can't find wtsexporter\" in Troubleshooting at messagecrate.app." - ), + // The run's error and Settings add the troubleshooting section. + Self::DoesNotRun(Program::Wtsexporter) => { + f.write_str("wtsexporter in the Tools Directory doesn't run on this computer.") + } + Self::CouldNotStart(why) => write!(f, "The download could not be started: {why}."), + Self::Interrupted => f.write_str("The download was interrupted."), } } } @@ -552,34 +563,216 @@ pub enum DownloadState { }, } -/// The download state of each program, shared by the start-up check and -/// `tools_status`. A program with no entry is not downloading and has not -/// failed since the app started. +/// The download state of each program, shared by the checks, `tools_status` +/// and an import waiting for a program. A program with no entry is not +/// downloading and has not failed since the app started. #[derive(Debug, Clone, Default)] -pub struct ToolDownloads(Arc>>); +pub struct ToolDownloads(Arc); + +/// The states, the signal a waiting import hears when one changes, and how +/// many checks run in this process. +#[derive(Debug, Default)] +struct Shared { + states: Mutex>, + changed: Condvar, + checks: AtomicUsize, +} + +/// One check in this process, from the moment it is started until it ends, +/// owned by the thread that runs it. +/// +/// While one exists [`ToolDownloads::checking`] says so, and the window +/// keeps asking for the status, because a program the check has not yet +/// looked at shows as missing. When it is dropped, at the end of the check +/// or on a panic in its thread, every program it marked that is still +/// downloading becomes failed with [`DownloadError::Interrupted`], so an +/// import waiting for one wakes and the Import form offers Try again. +#[derive(Debug)] +struct CheckRun { + downloads: ToolDownloads, + marked: Vec, +} + +impl CheckRun { + /// Show `program` as downloading, with nothing received yet. + fn mark_downloading(&mut self, program: Program) { + self.downloads.set( + program, + DownloadState::Downloading { + received: 0, + total: None, + }, + ); + if !self.marked.contains(&program) { + self.marked.push(program); + } + } +} + +impl Drop for CheckRun { + fn drop(&mut self) { + { + let mut states = self.downloads.states(); + for program in &self.marked { + if matches!(states.get(program), Some(DownloadState::Downloading { .. })) { + states.insert( + *program, + DownloadState::Failed { + reason: DownloadError::Interrupted.to_string(), + }, + ); + } + } + } + self.downloads.0.checks.fetch_sub(1, Ordering::SeqCst); + self.downloads.0.changed.notify_all(); + } +} + +/// How often an import waiting for a download looks at its cancel flag when +/// no state changes. +const WAIT_TICK: Duration = Duration::from_millis(250); impl ToolDownloads { + fn states(&self) -> std::sync::MutexGuard<'_, HashMap> { + self.0.states.lock().unwrap_or_else(PoisonError::into_inner) + } + + /// Whether a check runs in this process now, or has been started and not + /// yet taken the lock. + pub fn checking(&self) -> bool { + self.0.checks.load(Ordering::SeqCst) > 0 + } + + /// Start a check: [`Self::checking`] holds until the returned run is + /// dropped. + fn begin_check(&self) -> CheckRun { + self.0.checks.fetch_add(1, Ordering::SeqCst); + CheckRun { + downloads: self.clone(), + marked: Vec::new(), + } + } + /// Where `program`'s download stands, if anywhere. pub fn get(&self, program: Program) -> Option { - self.0 - .lock() - .expect("tool downloads lock") - .get(&program) - .cloned() + self.states().get(&program).cloned() } fn set(&self, program: Program, state: DownloadState) { - self.0 - .lock() - .expect("tool downloads lock") - .insert(program, state); + self.states().insert(program, state); + self.0.changed.notify_all(); } fn clear(&self, program: Program) { - self.0.lock().expect("tool downloads lock").remove(&program); + self.states().remove(&program); + self.0.changed.notify_all(); + } + + /// Wait while any of `programs` is downloading, so an import that needs + /// a program still downloading goes on once it has arrived (#1053). + /// + /// `waiting` hears the program waited for and its progress each time it + /// changes, for the import's progress line. `cancelled` is asked at + /// least every [`WAIT_TICK`], so a Cancel ends the wait. + /// + /// Returns at once when none of `programs` is downloading: the import + /// then finds the program or reports it missing as it always did. + /// + /// # Errors + /// + /// [`WaitError::Failed`] when a download waited for ended failed, with + /// its reason; [`WaitError::Cancelled`] when `cancelled` said so. + pub fn wait_for( + &self, + programs: &[Program], + cancelled: &dyn Fn() -> bool, + waiting: &mut dyn FnMut(Program, u64, Option), + ) -> Result<(), WaitError> { + let mut waited = Vec::new(); + let mut states = self.states(); + loop { + let downloading = programs + .iter() + .find_map(|&program| match states.get(&program) { + Some(DownloadState::Downloading { received, total }) => { + Some((program, *received, *total)) + } + _ => None, + }); + let Some((program, received, total)) = downloading else { + break; + }; + if !waited.contains(&program) { + waited.push(program); + } + drop(states); + if cancelled() { + return Err(WaitError::Cancelled); + } + waiting(program, received, total); + states = self.states(); + // Woken by any change, or after a tick to look at the cancel flag. + states = self + .0 + .changed + .wait_timeout(states, WAIT_TICK) + .unwrap_or_else(PoisonError::into_inner) + .0; + } + for program in waited { + if let Some(DownloadState::Failed { reason }) = states.get(&program) { + return Err(WaitError::Failed { + program, + reason: reason.clone(), + }); + } + } + Ok(()) } } +/// Why an import's wait for a download ended without the program. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum WaitError { + /// The download waited for failed, for `reason`. + Failed { + /// The program whose download failed. + program: Program, + /// Why, as [`DownloadError`] says it. + reason: String, + }, + /// The import was cancelled while it waited. + Cancelled, +} + +impl fmt::Display for WaitError { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + match self { + Self::Failed { program, reason } => write!( + f, + "The {} download failed. {reason} Try again on the Import form downloads it again. {}", + program.name(), + troubleshooting(*program) + ), + Self::Cancelled => f.write_str("cancelled"), + } + } +} + +impl std::error::Error for WaitError {} + +/// The sentence that sends a person to the user guide's troubleshooting +/// section for `program`, which says how to install it or where to put a +/// copy by hand. The Import form and Settings link the same section. +pub fn troubleshooting(program: Program) -> String { + let section = match program { + Program::Ffmpeg | Program::Ffprobe => "ffmpeg or ffprobe not found", + Program::Wtsexporter => "Import can't find wtsexporter", + }; + format!("See \"{section}\" in Troubleshooting at messagecrate.app.") +} + /// The state of `program` in place that does not run. fn does_not_run(program: Program) -> DownloadState { DownloadState::Failed { @@ -606,27 +799,138 @@ fn lock_check(dir: &Path) -> io::Result> { /// Check `dir` against `pinned` and download, one after another, what is /// missing or not the pinned release, from `base`. ffmpeg and ffprobe are /// passed over when both are on `PATH`. Each download's progress and -/// failure go to `downloads`, and each file written goes into the record. +/// failure go to the downloads `run` belongs to, and each file written goes +/// into the record. /// /// Another app's check running on `dir` already does the work, so this one /// returns at once. When the lock can't be taken because the Tools Directory /// can't be written, the check goes on and its downloads say why. -pub fn download_missing(dir: &Path, base: &str, pinned: &[Pinned], downloads: &ToolDownloads) { - let _lock = match lock_check(dir) { +fn download_missing(dir: &Path, base: &str, pinned: &[Pinned], mut run: CheckRun) { + let lock = match lock_check(dir) { Ok(Some(lock)) => Some(lock), Ok(None) => return, Err(_) => None, }; + check(dir, base, pinned, &mut run, lock); +} + +/// What [`retry`] did. +#[derive(Debug)] +pub enum Retry { + /// A check started, on the thread this carries. + Started(JoinHandle<()>), + /// Another check holds the lock on the Tools Directory: the start-up + /// check, an earlier Try again, or another app's check. Nothing started, + /// and the check already running brings the programs. + AlreadyRunning, + /// The check's thread could not be started: each program in `not_found` + /// that has a pin fails with [`DownloadError::CouldNotStart`]. + CouldNotStart, +} + +/// Start the check of `dir` again on a thread of its own, as **Try again** +/// on the Import form asks (#1053), and return without waiting for it. +/// +/// `not_found` are the programs the window says are not found. They show as +/// downloading before this returns, so the window, which asks again only +/// while a download runs, keeps asking until the check has decided each one. +/// +/// Returns [`Retry::Started`] with the check's thread, or says why no check +/// started: [`Retry::AlreadyRunning`] or [`Retry::CouldNotStart`]. +pub fn retry( + dir: PathBuf, + base: String, + pinned: Vec, + downloads: &ToolDownloads, + not_found: &[Program], +) -> Retry { + retry_with(dir, base, pinned, downloads, not_found, |work| { + std::thread::Builder::new() + .name("tool-downloads-retry".into()) + .spawn(work) + }) +} + +/// The work [`retry`] hands to a new thread. +type Work = Box; + +/// [`retry`], starting the check's thread with `spawn`, so a test can make +/// the start fail. +fn retry_with( + dir: PathBuf, + base: String, + pinned: Vec, + downloads: &ToolDownloads, + not_found: &[Program], + spawn: impl FnOnce(Work) -> io::Result>, +) -> Retry { + let lock = match lock_check(&dir) { + Ok(Some(lock)) => Some(lock), + Ok(None) => return Retry::AlreadyRunning, + Err(_) => None, + }; + let mut run = downloads.begin_check(); + let marked: Vec = pinned + .iter() + .map(|pin| pin.program) + .filter(|program| not_found.contains(program)) + .collect(); + for &program in &marked { + run.mark_downloading(program); + } + // The run goes to the thread only once the thread has started. Were it + // moved into the closure, a failed start would drop it there, and its + // drop would mark these programs interrupted, and wake a waiting import + // with that reason, before the cause is known. + let (hand_over, handed) = std::sync::mpsc::sync_channel::(1); + let spawned = spawn(Box::new(move || { + if let Ok(mut run) = handed.recv() { + check(&dir, &base, &pinned, &mut run, lock); + } + })); + match spawned { + Ok(handle) => { + // The thread waits on `handed` until this arrives, so the send + // can't fail. + let _ = hand_over.send(run); + Retry::Started(handle) + } + // Each program fails with the thread's error first, so the run's + // drop, which fails only the ones still downloading, leaves them. + Err(err) => { + for program in marked { + downloads.set( + program, + DownloadState::Failed { + reason: DownloadError::CouldNotStart(err.to_string()).to_string(), + }, + ); + } + drop(run); + Retry::CouldNotStart + } + } +} + +/// The check [`download_missing`] and [`retry`] run as `run`, holding +/// `_lock`, or no lock when the Tools Directory can't be written. Every +/// pinned program ends with its state decided: none when it is in place and +/// runs, failed with the reason otherwise. +fn check(dir: &Path, base: &str, pinned: &[Pinned], run: &mut CheckRun, _lock: Option) { + // A clone of the handle (one `Arc`), because `run` is borrowed mutably + // below to mark the programs wanted as downloading. + let downloads = &run.downloads.clone(); delete_leftovers(dir); let ffmpeg_on_path = media::ffmpeg_on_path(); let mut manifest = read_manifest(dir); let mut wanted = Vec::new(); for pin in pinned { if ffmpeg_on_path && matches!(pin.program, Program::Ffmpeg | Program::Ffprobe) { + downloads.clear(pin.program); continue; } match need(pin, dir, &manifest) { - Need::Keep => {} + Need::Keep => downloads.clear(pin.program), Need::Adopt => { let path = dir.join(pin.program.file_name()); let _ = make_executable(&path); @@ -634,7 +938,9 @@ pub fn download_missing(dir: &Path, base: &str, pinned: &[Pinned], downloads: &T manifest.insert(pin.program, Written::of(pin, stamp)); let _ = write_manifest(dir, &manifest); } - if !runs(pin.program, dir) { + if runs(pin.program, dir) { + downloads.clear(pin.program); + } else { downloads.set(pin.program, does_not_run(pin.program)); } } @@ -649,13 +955,7 @@ pub fn download_missing(dir: &Path, base: &str, pinned: &[Pinned], downloads: &T // first request, so Settings opened while a connection is made, or // between two downloads, keeps asking until the last one ends. for pin in &wanted { - downloads.set( - pin.program, - DownloadState::Downloading { - received: 0, - total: None, - }, - ); + run.mark_downloading(pin.program); } let fail_all = |err: DownloadError| { for pin in &wanted { @@ -688,17 +988,12 @@ pub fn download_missing(dir: &Path, base: &str, pinned: &[Pinned], downloads: &T downloads.set(program, does_not_run(program)); } if let Err(err) = write_manifest(dir, &manifest) { - // The program is in place and works; without the record - // the next start takes it as someone else's. - downloads.set( - program, - DownloadState::Failed { - reason: DownloadError::CouldNotWrite(format!( - "{} could not be written: {err}", - MANIFEST_FILE - )) - .to_string(), - }, + // The program is in place and the lookup finds it, so + // its download did not fail. Without the record the + // next start reads the file again and adopts it. + eprintln!( + "The {} download is in place, and {MANIFEST_FILE} could not be written: {err}", + program.name() ); } } @@ -714,11 +1009,15 @@ pub fn download_missing(dir: &Path, base: &str, pinned: &[Pinned], downloads: &T /// Start the check of the Tools Directory `dir` on a thread of its own, for /// this computer's platform, so login and browsing don't wait for it. -pub fn start(dir: PathBuf, downloads: ToolDownloads) { +/// +/// The check counts as running from before the thread starts, so a window +/// that asks for the status at once keeps asking until the check ends. +pub fn start(dir: PathBuf, downloads: &ToolDownloads) { let pinned = pinned_for(std::env::consts::OS, std::env::consts::ARCH); + let run = downloads.begin_check(); let _ = std::thread::Builder::new() .name("tool-downloads".into()) - .spawn(move || download_missing(&dir, GITHUB, &pinned, &downloads)); + .spawn(move || download_missing(&dir, GITHUB, &pinned, run)); } #[cfg(test)] diff --git a/src-tauri/src/tool_downloads/tests.rs b/src-tauri/src/tool_downloads/tests.rs index c1b2fd227..16e6c0300 100644 --- a/src-tauri/src/tool_downloads/tests.rs +++ b/src-tauri/src/tool_downloads/tests.rs @@ -5,6 +5,16 @@ use std::ffi::OsString; use httpmock::prelude::*; /// The SHA-256 of `bytes`, in lowercase hex. +impl Retry { + /// The check's thread, when one started. + fn started(self) -> Option> { + match self { + Self::Started(handle) => Some(handle), + Self::AlreadyRunning | Self::CouldNotStart => None, + } + } +} + fn sha256_hex(bytes: &[u8]) -> String { hex::encode(Sha256::digest(bytes)) } @@ -77,6 +87,11 @@ fn no_ffmpeg_on_path(dir: &Path) -> media::testutil::ToolsHidden { media::testutil::locate_tools(OsString::new(), dir.to_path_buf()) } +/// [`download_missing`] as one check of `downloads`, on this thread. +fn check_now(dir: &Path, base: &str, pinned: &[Pinned], downloads: &ToolDownloads) { + download_missing(dir, base, pinned, downloads.begin_check()); +} + /// The names in `dir` other than `keep` and the lock file, to see that no /// temporary file was left behind. fn other_files(dir: &Path, keep: &[&str]) -> Vec { @@ -107,7 +122,7 @@ fn a_download_whose_checksum_does_not_match_is_refused() { serve(&server, &pinned, b"a file changed on the way"); let downloads = ToolDownloads::default(); - download_missing( + check_now( tools.path(), &server.base_url(), std::slice::from_ref(&pinned), @@ -151,7 +166,7 @@ fn an_older_program_is_replaced_only_after_a_good_download() { // 404 Not Found: the old file stays. let server = MockServer::start(); - download_missing( + check_now( tools.path(), &server.base_url(), std::slice::from_ref(&new), @@ -165,7 +180,7 @@ fn an_older_program_is_replaced_only_after_a_good_download() { // A file that does not match: the old file stays. let mut wrong = serve(&server, &new, b"not the new release"); - download_missing( + check_now( tools.path(), &server.base_url(), std::slice::from_ref(&new), @@ -177,7 +192,7 @@ fn an_older_program_is_replaced_only_after_a_good_download() { // A good download replaces it, and the record names the new release. serve(&server, &new, &new_bytes); - download_missing( + check_now( tools.path(), &server.base_url(), std::slice::from_ref(&new), @@ -209,7 +224,7 @@ fn a_gzipped_program_is_unpacked_and_made_executable() { let server = MockServer::start(); serve(&server, &pinned, &published); - download_missing( + check_now( tools.path(), &server.base_url(), &[pinned], @@ -238,7 +253,7 @@ fn a_gzipped_program_that_is_not_the_pinned_one_is_refused() { serve(&server, &pinned, &published); let downloads = ToolDownloads::default(); - download_missing( + check_now( tools.path(), &server.base_url(), std::slice::from_ref(&pinned), @@ -285,7 +300,7 @@ fn ffmpeg_on_path_is_not_downloaded() { let ffprobe_asked = serve(&server, &ffprobe, &ffprobe_gz); let wts_asked = serve(&server, &wts, &wts_bytes); - download_missing( + check_now( tools.path(), &server.base_url(), &[ffmpeg, ffprobe, wts], @@ -316,7 +331,7 @@ fn ffmpeg_on_path_without_ffprobe_is_downloaded() { serve(&server, &ffmpeg, &ffmpeg_gz); serve(&server, &ffprobe, &ffprobe_gz); - download_missing( + check_now( tools.path(), &server.base_url(), &[ffmpeg, ffprobe], @@ -352,7 +367,7 @@ fn the_pinned_program_the_app_wrote_is_kept() { let server = MockServer::start(); let asked = serve(&server, &pinned, &published); - download_missing( + check_now( tools.path(), &server.base_url(), &[pinned], @@ -389,7 +404,7 @@ fn a_file_put_over_the_apps_own_is_replaced() { let server = MockServer::start(); let asked = serve(&server, &pinned, &published); - download_missing( + check_now( tools.path(), &server.base_url(), std::slice::from_ref(&pinned), @@ -420,7 +435,7 @@ fn ffmpeg_put_there_by_hand_is_replaced_after_a_good_download() { let downloads = ToolDownloads::default(); // No network: what is there stays. - download_missing( + check_now( tools.path(), "http://127.0.0.1:9", &[ffmpeg.clone(), ffprobe.clone()], @@ -433,7 +448,7 @@ fn ffmpeg_put_there_by_hand_is_replaced_after_a_good_download() { let server = MockServer::start(); serve(&server, &ffmpeg, &ffmpeg_gz); serve(&server, &ffprobe, &ffprobe_gz); - download_missing( + check_now( tools.path(), &server.base_url(), &[ffmpeg, ffprobe], @@ -460,7 +475,7 @@ fn a_pinned_program_that_does_not_run_is_not_downloaded_again() { for _start in 0..2 { let downloads = ToolDownloads::default(); - download_missing( + check_now( tools.path(), &server.base_url(), std::slice::from_ref(&pinned), @@ -496,7 +511,7 @@ fn a_pinned_program_put_there_by_hand_that_does_not_run_is_not_downloaded() { let asked = serve(&server, &pinned, &published); let downloads = ToolDownloads::default(); - download_missing( + check_now( tools.path(), &server.base_url(), std::slice::from_ref(&pinned), @@ -527,7 +542,7 @@ fn ffmpeg_with_no_record_is_adopted_when_it_is_the_pinned_program() { let asked = serve(&server, &pinned, &published); let downloads = ToolDownloads::default(); - download_missing( + check_now( tools.path(), &server.base_url(), std::slice::from_ref(&pinned), @@ -570,7 +585,7 @@ fn a_changed_stamp_on_the_pinned_program_is_recorded_again() { let server = MockServer::start(); let asked = serve(&server, &pinned, &published); - download_missing( + check_now( tools.path(), &server.base_url(), std::slice::from_ref(&pinned), @@ -599,7 +614,7 @@ fn a_program_put_there_by_hand_is_kept_only_when_it_is_the_pinned_file() { let mut asked = serve(&server, &pinned, &published); std::fs::write(&target, b"pinned").unwrap(); - download_missing( + check_now( tools.path(), &server.base_url(), std::slice::from_ref(&pinned), @@ -614,7 +629,7 @@ fn a_program_put_there_by_hand_is_kept_only_when_it_is_the_pinned_file() { std::fs::write(&target, b"a pipx shim, or an older release").unwrap(); let asked = serve(&server, &pinned, &published); - download_missing( + check_now( tools.path(), &server.base_url(), &[pinned], @@ -640,7 +655,7 @@ fn an_interrupted_downloads_leftovers_are_deleted() { std::fs::write(tools.path().join(name), b"left").unwrap(); } - download_missing( + check_now( tools.path(), "http://127.0.0.1:9", &[], @@ -680,7 +695,7 @@ fn every_program_wanted_shows_as_downloading_before_its_request() { let mut seen_waiting = false; std::thread::scope(|scope| { - let check = scope.spawn(|| download_missing(&dir, &base, &[ffmpeg, wts], &downloads)); + let check = scope.spawn(|| check_now(&dir, &base, &[ffmpeg, wts], &downloads)); while !check.is_finished() { if downloads.get(Program::Wtsexporter) == Some(DownloadState::Downloading { @@ -725,15 +740,13 @@ fn a_second_check_at_the_same_time_does_nothing() { let dir = tools.path().to_path_buf(); std::thread::scope(|scope| { - let check = - scope.spawn(|| download_missing(&dir, &base, std::slice::from_ref(&pinned), &first)); + let check = scope.spawn(|| check_now(&dir, &base, std::slice::from_ref(&pinned), &first)); // The first check holds the lock once it shows a download. while first.get(Program::Wtsexporter).is_none() { assert!(!check.is_finished(), "the first check ended at once"); std::thread::sleep(Duration::from_millis(1)); } - let other = - scope.spawn(|| download_missing(&dir, &base, std::slice::from_ref(&pinned), &second)); + let other = scope.spawn(|| check_now(&dir, &base, std::slice::from_ref(&pinned), &second)); other.join().unwrap(); assert!( !check.is_finished(), @@ -759,7 +772,7 @@ fn no_network_is_a_failed_download_that_says_so() { let downloads = ToolDownloads::default(); // Port 9 on this computer, where nothing listens. - download_missing(tools.path(), "http://127.0.0.1:9", &[pinned], &downloads); + check_now(tools.path(), "http://127.0.0.1:9", &[pinned], &downloads); let Some(DownloadState::Failed { reason }) = downloads.get(Program::Wtsexporter) else { panic!("the download did not fail"); @@ -804,3 +817,407 @@ fn every_platform_is_pinned() { } } } + +/// An import started while its program downloads waits, hears the +/// download's progress, and goes on once the program has arrived. +#[test] +fn an_import_waits_for_a_download_and_goes_on_when_it_arrives() { + let downloads = ToolDownloads::default(); + downloads.set( + Program::Wtsexporter, + DownloadState::Downloading { + received: 0, + total: None, + }, + ); + let mut heard = Vec::new(); + + std::thread::scope(|scope| { + scope.spawn(|| { + std::thread::sleep(Duration::from_millis(50)); + downloads.set( + Program::Wtsexporter, + DownloadState::Downloading { + received: 12_000_000, + total: Some(30_000_000), + }, + ); + std::thread::sleep(Duration::from_millis(50)); + downloads.clear(Program::Wtsexporter); + }); + let waited = downloads.wait_for( + &[Program::Wtsexporter], + &|| false, + &mut |program, received, total| { + heard.push((program, received, total)); + }, + ); + assert_eq!(waited, Ok(())); + }); + + assert_eq!(heard.first(), Some(&(Program::Wtsexporter, 0, None))); + assert!( + heard.contains(&(Program::Wtsexporter, 12_000_000, Some(30_000_000))), + "the wait did not hear the progress: {heard:?}" + ); +} + +/// A download that fails while an import waits for it fails the import, +/// with the download's reason. +#[test] +fn an_import_waiting_for_a_download_that_fails_fails_with_its_reason() { + let downloads = ToolDownloads::default(); + downloads.set( + Program::Ffprobe, + DownloadState::Downloading { + received: 10, + total: Some(100), + }, + ); + + let waited = std::thread::scope(|scope| { + scope.spawn(|| { + std::thread::sleep(Duration::from_millis(50)); + downloads.set( + Program::Ffprobe, + DownloadState::Failed { + reason: "The download's server answered 404 Not Found.".into(), + }, + ); + }); + downloads.wait_for( + &[Program::Ffmpeg, Program::Ffprobe], + &|| false, + &mut |_, _, _| {}, + ) + }); + + let err = waited.unwrap_err(); + assert_eq!( + err, + WaitError::Failed { + program: Program::Ffprobe, + reason: "The download's server answered 404 Not Found.".into(), + } + ); + // The run's error sends the person to the troubleshooting section. + assert_eq!( + err.to_string(), + "The ffprobe download failed. The download's server answered 404 Not Found. \ + Try again on the Import form downloads it again. \ + See \"ffmpeg or ffprobe not found\" in Troubleshooting at messagecrate.app." + ); +} + +/// A program not downloading is not waited for, and a failure from before +/// the import started is the lookup's to report, not the wait's. +#[test] +fn an_import_does_not_wait_for_a_program_not_downloading() { + let downloads = ToolDownloads::default(); + downloads.set( + Program::Wtsexporter, + DownloadState::Failed { + reason: "no network".into(), + }, + ); + let mut heard = 0; + + let waited = downloads.wait_for(&[Program::Wtsexporter], &|| false, &mut |_, _, _| { + heard += 1 + }); + + assert_eq!(waited, Ok(())); + assert_eq!(heard, 0); +} + +/// A Cancel ends the wait, though the download goes on. +#[test] +fn a_cancel_ends_the_wait() { + let downloads = ToolDownloads::default(); + downloads.set( + Program::Wtsexporter, + DownloadState::Downloading { + received: 0, + total: None, + }, + ); + let cancel = std::sync::atomic::AtomicBool::new(false); + + let waited = std::thread::scope(|scope| { + scope.spawn(|| { + std::thread::sleep(Duration::from_millis(50)); + cancel.store(true, std::sync::atomic::Ordering::Relaxed); + }); + downloads.wait_for( + &[Program::Wtsexporter], + &|| cancel.load(std::sync::atomic::Ordering::Relaxed), + &mut |_, _, _| {}, + ) + }); + + assert_eq!(waited, Err(WaitError::Cancelled)); +} + +/// Try again runs the check once on a thread of its own and returns before +/// the download ends, with the program already shown as downloading so the +/// window keeps asking. +#[test] +fn try_again_runs_the_check_once_and_returns_at_once() { + let tools = tempfile::tempdir().unwrap(); + let _tools = no_ffmpeg_on_path(tools.path()); + let (pinned, published) = pin(Program::Wtsexporter, "r1", b"wtsexporter", false); + let server = MockServer::start(); + let path = format!( + "/{}/releases/download/{}/{}", + pinned.repo, pinned.release, pinned.asset + ); + let asked = server.mock(|when, then| { + when.method(GET).path(path); + then.status(200) + .delay(Duration::from_millis(500)) + .body(published); + }); + let downloads = ToolDownloads::default(); + downloads.set( + Program::Wtsexporter, + DownloadState::Failed { + reason: "no network".into(), + }, + ); + + let started = std::time::Instant::now(); + let check = retry( + tools.path().to_path_buf(), + server.base_url(), + vec![pinned], + &downloads, + &[Program::Wtsexporter], + ) + .started() + .expect("a check started"); + assert!( + started.elapsed() < Duration::from_millis(400), + "Try again waited for the download" + ); + assert!(matches!( + downloads.get(Program::Wtsexporter), + Some(DownloadState::Downloading { .. }) + )); + check.join().unwrap(); + + asked.assert_calls(1); + assert_eq!(downloads.get(Program::Wtsexporter), None); + assert_eq!( + std::fs::read(tools.path().join(Program::Wtsexporter.file_name())).unwrap(), + b"wtsexporter" + ); +} + +/// Try again while a check holds the lock starts nothing: no second +/// download, and the state the running check keeps is left alone. +#[test] +fn try_again_while_a_check_runs_starts_nothing() { + let tools = tempfile::tempdir().unwrap(); + let _tools = no_ffmpeg_on_path(tools.path()); + let (pinned, published) = pin(Program::Wtsexporter, "r1", b"wtsexporter", false); + let server = MockServer::start(); + let asked = serve(&server, &pinned, &published); + let downloads = ToolDownloads::default(); + let running = DownloadState::Downloading { + received: 5, + total: Some(10), + }; + downloads.set(Program::Wtsexporter, running.clone()); + let _held = lock_check(tools.path()).unwrap().expect("the lock is free"); + + let check = retry( + tools.path().to_path_buf(), + server.base_url(), + vec![pinned], + &downloads, + &[Program::Wtsexporter], + ); + + assert!( + matches!(check, Retry::AlreadyRunning), + "a second check started" + ); + asked.assert_calls(0); + assert_eq!(downloads.get(Program::Wtsexporter), Some(running)); +} + +/// A program the check finds in place loses the failure an earlier check +/// left, so Try again after a file was put there by hand clears it. +#[test] +fn a_program_found_in_place_clears_an_earlier_failure() { + let tools = tempfile::tempdir().unwrap(); + let _tools = no_ffmpeg_on_path(tools.path()); + let (pinned, published) = pin(Program::Wtsexporter, "r1", b"wtsexporter", false); + std::fs::write( + tools.path().join(Program::Wtsexporter.file_name()), + &published, + ) + .unwrap(); + let downloads = ToolDownloads::default(); + downloads.set( + Program::Wtsexporter, + DownloadState::Failed { + reason: "no network".into(), + }, + ); + + // No server: the file there is the pinned one, so nothing is asked for. + check_now( + tools.path(), + "http://127.0.0.1:9", + std::slice::from_ref(&pinned), + &downloads, + ); + + assert_eq!(downloads.get(Program::Wtsexporter), None); +} + +/// A check counts as running from Try again until it ends, so the window, +/// which shows a program the check has not looked at yet as missing, keeps +/// asking for the status meanwhile. +#[test] +fn a_check_counts_as_running_until_it_ends() { + let tools = tempfile::tempdir().unwrap(); + let _tools = no_ffmpeg_on_path(tools.path()); + let (pinned, published) = pin(Program::Wtsexporter, "r1", b"wtsexporter", false); + let server = MockServer::start(); + let path = format!( + "/{}/releases/download/{}/{}", + pinned.repo, pinned.release, pinned.asset + ); + server.mock(|when, then| { + when.method(GET).path(path); + then.status(200) + .delay(Duration::from_millis(200)) + .body(published); + }); + let downloads = ToolDownloads::default(); + assert!(!downloads.checking()); + + let check = retry( + tools.path().to_path_buf(), + server.base_url(), + vec![pinned], + &downloads, + &[], + ) + .started() + .expect("a check started"); + assert!(downloads.checking(), "Try again's check is not running"); + check.join().unwrap(); + + assert!(!downloads.checking(), "the check still runs after it ended"); +} + +/// A check whose thread panics partway through a download leaves no program +/// downloading: the program it marked fails as interrupted, and an import +/// waiting for it wakes with that reason instead of waiting until Cancel. +#[test] +fn a_check_that_panics_mid_download_fails_the_download() { + let downloads = ToolDownloads::default(); + let mut run = downloads.begin_check(); + run.mark_downloading(Program::Wtsexporter); + + let (panicked, waited) = std::thread::scope(|scope| { + let check = scope.spawn(move || { + let _run = run; + std::thread::sleep(Duration::from_millis(50)); + panic!("the fake download panics"); + }); + let waited = downloads.wait_for(&[Program::Wtsexporter], &|| false, &mut |_, _, _| {}); + (check.join().is_err(), waited) + }); + + assert!(panicked); + assert_eq!( + waited, + Err(WaitError::Failed { + program: Program::Wtsexporter, + reason: "The download was interrupted.".into(), + }) + ); + assert!(!downloads.checking()); +} + +/// A download that arrives and runs is not a failed download when its +/// record can't be written: the program is found, and an import waiting for +/// it goes on. +#[test] +fn a_record_that_cannot_be_written_does_not_fail_a_good_download() { + let tools = tempfile::tempdir().unwrap(); + let _tools = no_ffmpeg_on_path(tools.path()); + // A directory where the record goes: renaming the record over it fails. + std::fs::create_dir(tools.path().join(MANIFEST_FILE)).unwrap(); + let (pinned, published) = pin(Program::Wtsexporter, "r1", b"wtsexporter", false); + let server = MockServer::start(); + let path = format!( + "/{}/releases/download/{}/{}", + pinned.repo, pinned.release, pinned.asset + ); + server.mock(|when, then| { + when.method(GET).path(path); + then.status(200) + .delay(Duration::from_millis(100)) + .body(published); + }); + let downloads = ToolDownloads::default(); + + let check = retry( + tools.path().to_path_buf(), + server.base_url(), + vec![pinned], + &downloads, + &[Program::Wtsexporter], + ) + .started() + .expect("a check started"); + let waited = downloads.wait_for(&[Program::Wtsexporter], &|| false, &mut |_, _, _| {}); + check.join().unwrap(); + + assert_eq!(waited, Ok(())); + assert_eq!(downloads.get(Program::Wtsexporter), None); + assert!(runs(Program::Wtsexporter, tools.path())); +} + +/// A Try again whose thread can't start fails each program it marked with +/// that reason, and no program is ever shown as interrupted on the way, so +/// an import waiting for one wakes with the real cause. +#[test] +fn a_try_again_whose_thread_cannot_start_fails_with_that_reason() { + let tools = tempfile::tempdir().unwrap(); + let (pinned, _) = pin(Program::Wtsexporter, "r1", b"wtsexporter", false); + let downloads = ToolDownloads::default(); + let mut seen = None; + + let retried = retry_with( + tools.path().to_path_buf(), + "http://127.0.0.1:9".into(), + vec![pinned], + &downloads, + &[Program::Wtsexporter], + |work| { + // A failed start drops the work without running it. + drop(work); + seen = downloads.get(Program::Wtsexporter); + Err(io::Error::other("no threads left")) + }, + ); + + assert!(matches!(retried, Retry::CouldNotStart)); + assert!( + matches!(seen, Some(DownloadState::Downloading { .. })), + "the program was {seen:?} before the start failed" + ); + assert_eq!( + downloads.get(Program::Wtsexporter), + Some(DownloadState::Failed { + reason: DownloadError::CouldNotStart("no threads left".into()).to_string(), + }) + ); + assert!(!downloads.checking()); +} diff --git a/web/src/lib/tauri.ts b/web/src/lib/tauri.ts index bea9e02e4..3b26e982b 100644 --- a/web/src/lib/tauri.ts +++ b/web/src/lib/tauri.ts @@ -485,6 +485,9 @@ export async function invokeFormat(config: { }); } +/** A program the desktop app keeps in its Tools Directory, as `tools_status` names it. */ +export type ToolName = "ffmpeg" | "ffprobe" | "wtsexporter"; + /** * Where one program the desktop app runs is. Tagged by `state`, one tag per * state, so a state can be added beside the others without changing them. @@ -492,10 +495,12 @@ export async function invokeFormat(config: { export type ToolStatus = | { state: "found"; path: string } | { state: "missing" } + /** Missing, and the app has no download of it for this computer: only a copy put in the Tools Directory by hand is used. */ + | { state: "unavailable" } | { state: "unusable"; reason: string } /** Downloading into the Tools Directory: bytes so far, and the total when the server said. */ | { state: "downloading"; received: number; total: number | null } - /** The download failed and the program is not found. Tried again at the next start. */ + /** The download failed and the program is not found. Tried again at the next start, or with Try again. */ | { state: "downloadFailed"; reason: string }; /** @@ -505,6 +510,13 @@ export type ToolStatus = */ export interface ToolsStatus { toolsDir: string | null; + /** + * A check of the Tools Directory runs in this process now, the start-up + * check or Try again. A program it has not looked at yet shows as missing + * until it does. Another app's check on the same Tools Directory is not + * counted: Try again reports it as `alreadyRunning`. + */ + checking: boolean; ffmpeg: ToolStatus; ffprobe: ToolStatus; wtsexporter: ToolStatus; @@ -515,6 +527,25 @@ export async function invokeToolsStatus(): Promise { return invoke("tools_status"); } +/** + * What Try again did. `started`: a check started in this process. + * `alreadyRunning`: another check holds the lock on the Tools Directory (the + * start-up check, an earlier Try again, or another app's check), and nothing + * started. `noToolsDirectory`: the app has none (`toolsDir` is null). + * `couldNotStart`: the check's thread could not be started, which each + * program it would have downloaded shows as a failed download with that + * reason. + */ +export type RetryResult = "started" | "alreadyRunning" | "noToolsDirectory" | "couldNotStart"; + +/** + * Check the Tools Directory again and download what is missing, as Try again + * on the Import form asks. Returns at once, with what it did. + */ +export async function invokeRetryToolDownloads(): Promise { + return invoke("retry_tool_downloads"); +} + /** Whether any of ffmpeg, ffprobe and wtsexporter is downloading now. */ export function toolsDownloading(status: ToolsStatus): boolean { return [status.ffmpeg, status.ffprobe, status.wtsexporter].some( @@ -525,9 +556,17 @@ export function toolsDownloading(status: ToolsStatus): boolean { /** The programs Convert and Compress run. */ export type MediaToolName = "ffmpeg" | "ffprobe"; -/** Which of ffmpeg and ffprobe, which Convert and Compress run, cannot be used. */ +/** + * Which of ffmpeg and ffprobe, which Convert and Compress run, cannot be used. + * One still downloading can: the Media stage waits for it. + */ export function ffmpegMissing(status: ToolsStatus): MediaToolName[] { - return (["ffmpeg", "ffprobe"] as const).filter((name) => status[name].state !== "found"); + return (["ffmpeg", "ffprobe"] as const).filter((name) => !toolUsable(status[name])); +} + +/** Whether an import can run a program: it is found, or downloading, which the import waits for. */ +export function toolUsable(status: ToolStatus): boolean { + return status.state === "found" || status.state === "downloading"; } export interface HomeDirInfo { diff --git a/web/src/lib/toolStatusCopy.ts b/web/src/lib/toolStatusCopy.ts new file mode 100644 index 000000000..616a65f00 --- /dev/null +++ b/web/src/lib/toolStatusCopy.ts @@ -0,0 +1,88 @@ +import { formatBytes } from "./attachmentProgressCopy"; +import type { ToolName, ToolStatus } from "./tauri"; + +/** + * A download's progress: "12 MB of 29 MB (41%)", or "12 MB so far" with no + * total. The Import form, Settings and the run's progress line all say it + * this way, so one download reads the same in each. + */ +function downloadProgress(received: number, total: number | null): string { + if (total == null || total <= 0) return `${formatBytes(received)} so far`; + const percent = Math.min(100, Math.floor((received / total) * 100)); + return `${formatBytes(received)} of ${formatBytes(total)} (${percent}%)`; +} + +/** + * The line for a program that is not found, the same in Settings and on the + * Import form, one case per `state` the desktop process sends. It is split + * around the program's name so Settings can set the name as code: + * `${before}${name}${after}` is the sentence. + */ +export function toolStatusLine(status: Exclude): { + before: string; + after: string; +} { + switch (status.state) { + case "missing": + return { before: "", after: " not found." }; + case "unavailable": + return { + before: "", + after: " not found, and the app has no download of it for this computer.", + }; + case "unusable": + return { before: "", after: ` not used. ${status.reason}` }; + case "downloading": + return { + before: "Downloading ", + after: `: ${downloadProgress(status.received, status.total)}.`, + }; + case "downloadFailed": + return { before: "", after: ` download failed. ${status.reason}` }; + } +} + +/** + * The progress line of a run waiting for `name`'s download (#1053): + * "Waiting for the wtsexporter download: 12 MB of 30 MB (40%)", or no + * numbers before the first byte arrives. + */ +export function waitingLine(name: ToolName, received: number, total: number | null): string { + const lead = `Waiting for the ${name} download`; + if (received <= 0 && (total == null || total <= 0)) return lead; + return `${lead}: ${downloadProgress(received, total)}`; +} + +const TROUBLESHOOTING = "https://messagecrate.app/docs/user/features/owner/troubleshooting/"; + +/** The user guide's troubleshooting section for `name`: its heading and its address. */ +export function troubleshootingSection(name: ToolName): { heading: string; url: string } { + return name === "wtsexporter" + ? { + heading: "Import can't find wtsexporter", + url: `${TROUBLESHOOTING}#import-cant-find-wtsexporter`, + } + : { + heading: "ffmpeg or ffprobe not found", + url: `${TROUBLESHOOTING}#ffmpeg-or-ffprobe-not-found`, + }; +} + +/** + * The sentence that sends a person to `name`'s troubleshooting section, word + * for word as a run's failed-download error ends (`troubleshooting` in + * src-tauri/src/tool_downloads.rs). Split around the section's heading, so + * Settings can set the heading as a link to `section.url`: + * `${before}${section.heading}${after}` is the sentence. + */ +export function troubleshootingSentence(name: ToolName): { + before: string; + section: { heading: string; url: string }; + after: string; +} { + return { + before: 'See "', + section: troubleshootingSection(name), + after: '" in Troubleshooting at messagecrate.app.', + }; +} diff --git a/web/src/lib/types.ts b/web/src/lib/types.ts index 852cae0f4..19f582448 100644 --- a/web/src/lib/types.ts +++ b/web/src/lib/types.ts @@ -1,4 +1,5 @@ import type { components } from "./serverApi.types"; +import type { ToolName } from "./tauri"; type Schema = components["schemas"]; @@ -103,6 +104,11 @@ export interface ImportProgressEvent { bytes_done?: number; bytes_total?: number; status?: string; + /** + * The program the run waits for while it downloads (#1053). The event then + * has no counts, and `bytes_done` and `bytes_total` are the download's. + */ + waiting?: ToolName; } /** diff --git a/web/src/lib/useToolsStatus.ts b/web/src/lib/useToolsStatus.ts new file mode 100644 index 000000000..b7cab0941 --- /dev/null +++ b/web/src/lib/useToolsStatus.ts @@ -0,0 +1,52 @@ +import { useMutation, useQuery, useQueryClient } from "@tanstack/react-query"; +import { keys } from "./queryKeys"; +import { invokeRetryToolDownloads, invokeToolsStatus, toolsDownloading } from "./tauri"; +import { isTauri } from "./tauri-check"; + +/** How often the window asks the desktop process again while a check or a download runs. */ +const DOWNLOAD_POLL_MS = 1000; + +/** + * How often the window asks again after Try again found another app's check + * running, which `checking` does not count, and for how long: a download + * that arrives shows as found, and a failed one is never heard of here. + */ +const OTHER_CHECK_POLL_MS = 2000; +export const OTHER_CHECK_POLL_FOR_MS = 2 * 60 * 1000; + +/** + * Where ffmpeg, ffprobe and wtsexporter are and how their downloads stand, + * asked again each second while a check or a download runs: a program the + * check has not looked at yet shows as missing until it does. Until + * `otherCheckUntil` (a time in milliseconds) it is also asked every 2 s, for + * another app's check that `checking` does not count. Settings and the Import + * screen read the same entry. It belongs to this computer, not to an account, + * so this is a plain `useQuery`; the browser has no desktop process to ask. + */ +export function useToolsStatus({ otherCheckUntil = 0 }: { otherCheckUntil?: number } = {}) { + return useQuery({ + queryKey: keys.desktopToolsStatus.all, + queryFn: invokeToolsStatus, + enabled: isTauri(), + retry: false, + refetchInterval: (query) => { + const data = query.state.data; + if (data && (data.checking || toolsDownloading(data))) return DOWNLOAD_POLL_MS; + return Date.now() < otherCheckUntil ? OTHER_CHECK_POLL_MS : false; + }, + }); +} + +/** + * Try again: the desktop process checks the Tools Directory and downloads + * what is missing, and the status is asked for at once. The programs it + * downloads show as downloading by then, so the status keeps being asked for + * until they arrive or fail. + */ +export function useRetryToolDownloads() { + const client = useQueryClient(); + return useMutation({ + mutationFn: invokeRetryToolDownloads, + onSettled: () => client.invalidateQueries({ queryKey: keys.desktopToolsStatus.all }), + }); +} diff --git a/web/src/screens/import/ImportFormFields.test.tsx b/web/src/screens/import/ImportFormFields.test.tsx index 961e60dec..e616624c7 100644 --- a/web/src/screens/import/ImportFormFields.test.tsx +++ b/web/src/screens/import/ImportFormFields.test.tsx @@ -1,10 +1,11 @@ /** @vitest-environment jsdom */ -import { cleanup, render, screen } from "@testing-library/react"; +import { cleanup, screen, waitFor } from "@testing-library/react"; import { afterEach, describe, expect, it, vi } from "vitest"; import { holdDesktopJob } from "../../lib/desktopJob"; import { EXPORT_SOURCES } from "../../lib/exportSources"; import { IMESSAGE_METHODS, IMESSAGE_SOURCE_ID } from "../../lib/imessageImport"; +import type { ToolStatus, ToolsStatus } from "../../lib/tauri"; import { emptyWhatsappPathStats, WHATSAPP_ERR_CRYPT_KEY, @@ -13,6 +14,7 @@ import { WHATSAPP_METHODS, WHATSAPP_SOURCE_ID, } from "../../lib/whatsappImport"; +import { renderWithProviders as render } from "../../test/providers"; import { setupUser } from "../../test/user"; import ImportFormFields, { type ImportFormFieldsProps } from "./ImportFormFields"; @@ -20,8 +22,28 @@ vi.mock("@tauri-apps/plugin-dialog", () => ({ open: vi.fn(), })); +/** The desktop process the form asks where wtsexporter, ffmpeg and ffprobe are. */ +const desktop = vi.hoisted(() => ({ + isTauri: false, + toolsStatus: vi.fn(), + retry: vi.fn(), +})); + +vi.mock("../../lib/tauri-check", () => ({ + isTauri: () => desktop.isTauri, +})); + +vi.mock("../../lib/tauri", async (importOriginal) => ({ + ...(await importOriginal()), + invokeToolsStatus: () => desktop.toolsStatus(), + invokeRetryToolDownloads: () => desktop.retry(), +})); + afterEach(() => { cleanup(); + desktop.isTauri = false; + desktop.toolsStatus.mockReset(); + desktop.retry.mockReset(); }); const presentFile = { exists: true, isFile: true, isDirectory: false }; @@ -774,3 +796,280 @@ describe("ImportFormFields required marks", () => { expect(marked).toEqual(needed.sort()); }); }); + +describe("ImportFormFields programs the import needs", () => { + const found = (path: string): ToolStatus => ({ state: "found", path }); + const status = (over: Partial): ToolsStatus => ({ + toolsDir: "/home/sam/message-crate/tools", + checking: false, + ffmpeg: found("/usr/bin/ffmpeg"), + ffprobe: found("/usr/bin/ffprobe"), + wtsexporter: found("/home/sam/message-crate/tools/wtsexporter"), + ...over, + }); + const readyWhatsapp: Partial = { + source: "whatsapp-android", + backupPath: "/tmp/wa", + whatsappOwnerPhone: "+15555550100", + whatsappStats: { + backup: presentDir, + contactsDb: null, + media: null, + db: null, + hasMsgstoreDb: true, + cryptName: null, + backupEncrypted: null, + }, + }; + + it("blocks a WhatsApp import whose wtsexporter download failed, with the reason and Try again", async () => { + desktop.isTauri = true; + desktop.toolsStatus.mockResolvedValue( + status({ + wtsexporter: { + state: "downloadFailed", + reason: "No connection to the download's server: dns error.", + }, + }), + ); + renderForm(readyWhatsapp); + + expect( + await screen.findByText( + "wtsexporter download failed. No connection to the download's server: dns error. A WhatsApp import can't start without wtsexporter.", + ), + ).toBeTruthy(); + expect(screen.getByRole("button", { name: "Try again" })).toBeTruthy(); + expect(screen.getByRole("link", { name: "Troubleshooting" }).getAttribute("href")).toBe( + "https://messagecrate.app/docs/user/features/owner/troubleshooting/#import-cant-find-wtsexporter", + ); + expect(importButton()).toBeDisabled(); + }); + + it("blocks a WhatsApp import while wtsexporter is not found", async () => { + desktop.isTauri = true; + desktop.toolsStatus.mockResolvedValue(status({ wtsexporter: { state: "missing" } })); + renderForm(readyWhatsapp); + + expect( + await screen.findByText( + "wtsexporter not found. A WhatsApp import can't start without wtsexporter.", + ), + ).toBeTruthy(); + expect(importButton()).toBeDisabled(); + }); + + it("blocks a WhatsApp import on a computer the app has no wtsexporter download for, without Try again", async () => { + desktop.isTauri = true; + desktop.toolsStatus.mockResolvedValue(status({ wtsexporter: { state: "unavailable" } })); + renderForm(readyWhatsapp); + + expect( + await screen.findByText( + "wtsexporter not found, and the app has no download of it for this computer. A WhatsApp import can't start without wtsexporter.", + ), + ).toBeTruthy(); + expect(screen.queryByRole("button", { name: "Try again" })).toBeNull(); + expect(screen.getByRole("link", { name: "Troubleshooting" }).getAttribute("href")).toBe( + "https://messagecrate.app/docs/user/features/owner/troubleshooting/#import-cant-find-wtsexporter", + ); + expect(importButton()).toBeDisabled(); + }); + + it("blocks a WhatsApp import while wtsexporter can't be used", async () => { + desktop.isTauri = true; + desktop.toolsStatus.mockResolvedValue( + status({ wtsexporter: { state: "unusable", reason: "It is not executable." } }), + ); + renderForm(readyWhatsapp); + + expect(await screen.findByText(/wtsexporter not used\. It is not executable\./)).toBeTruthy(); + expect(importButton()).toBeDisabled(); + }); + + it("lets a WhatsApp import start while wtsexporter downloads, saying it waits", async () => { + desktop.isTauri = true; + desktop.toolsStatus.mockResolvedValue( + status({ + wtsexporter: { state: "downloading", received: 12 * 1024 * 1024, total: 30 * 1024 * 1024 }, + }), + ); + renderForm(readyWhatsapp); + + expect( + await screen.findByText( + "Downloading wtsexporter: 12 MB of 30 MB (40%). The import waits for wtsexporter.", + ), + ).toBeTruthy(); + expect(screen.queryByRole("button", { name: "Try again" })).toBeNull(); + expect(importButton()).toBeEnabled(); + }); + + it("keeps asking while the start-up check runs, and unblocks once wtsexporter arrives", async () => { + desktop.isTauri = true; + // Read before the check has looked at wtsexporter: missing, and checking. + desktop.toolsStatus + .mockResolvedValueOnce(status({ checking: true, wtsexporter: { state: "missing" } })) + .mockResolvedValue(status({})); + renderForm(readyWhatsapp); + + expect(await screen.findByText(/wtsexporter not found/)).toBeTruthy(); + expect(importButton()).toBeDisabled(); + await waitFor(() => expect(importButton()).toBeEnabled(), { timeout: 3000 }); + }); + + it("lets a WhatsApp import start once wtsexporter is found, and says nothing", async () => { + desktop.isTauri = true; + desktop.toolsStatus.mockResolvedValue(status({})); + renderForm(readyWhatsapp); + + await waitFor(() => expect(desktop.toolsStatus).toHaveBeenCalled()); + expect(screen.queryByRole("button", { name: "Try again" })).toBeNull(); + expect(importButton()).toBeEnabled(); + }); + + it("names the program Media waits for while it downloads", async () => { + desktop.isTauri = true; + desktop.toolsStatus.mockResolvedValue( + status({ + ffmpeg: { state: "downloading", received: 12 * 1024 * 1024, total: 30 * 1024 * 1024 }, + }), + ); + renderForm({ ...readyWhatsapp, attachmentMedia: "convert" }); + + expect( + await screen.findByText("Downloading ffmpeg: 12 MB of 30 MB (40%). Media waits for ffmpeg."), + ).toBeTruthy(); + expect(screen.queryByRole("button", { name: "Try again" })).toBeNull(); + }); + + it("says Convert needs ffmpeg without blocking the import", async () => { + desktop.isTauri = true; + desktop.toolsStatus.mockResolvedValue( + status({ ffmpeg: { state: "missing" }, ffprobe: { state: "missing" } }), + ); + renderForm({ ...readyWhatsapp, attachmentMedia: "convert" }); + + expect( + await screen.findByText( + "ffmpeg not found. ffprobe not found. Convert and Compress need ffmpeg and ffprobe. Copy imports attachments as they are.", + ), + ).toBeTruthy(); + expect(screen.getByRole("link", { name: "Troubleshooting" }).getAttribute("href")).toBe( + "https://messagecrate.app/docs/user/features/owner/troubleshooting/#ffmpeg-or-ffprobe-not-found", + ); + expect(importButton()).toBeEnabled(); + }); + + it("Try again while a check runs says so, and follows that check until it ends", async () => { + const user = setupUser(); + desktop.isTauri = true; + desktop.toolsStatus.mockResolvedValue( + status({ wtsexporter: { state: "downloadFailed", reason: "No connection." } }), + ); + desktop.retry.mockImplementation(async () => { + // The start-up check still runs: nothing new starts, and it brings + // wtsexporter a moment later. + desktop.toolsStatus + .mockResolvedValueOnce( + status({ + checking: true, + wtsexporter: { state: "downloadFailed", reason: "No connection." }, + }), + ) + .mockResolvedValue(status({})); + return "alreadyRunning"; + }); + renderForm(readyWhatsapp); + + await user.click(await screen.findByRole("button", { name: "Try again" })); + + expect( + await screen.findByText( + "A check of the Tools Directory is already running. This form shows its result when it ends.", + ), + ).toBeTruthy(); + await waitFor(() => expect(importButton()).toBeEnabled(), { timeout: 3000 }); + }); + + it("the already-running line goes when this process's check ends with the program still failed", async () => { + const user = setupUser(); + desktop.isTauri = true; + const failed = status({ wtsexporter: { state: "downloadFailed", reason: "No connection." } }); + desktop.toolsStatus.mockResolvedValue(failed); + desktop.retry.mockImplementation(async () => { + // The start-up check still runs, then ends without wtsexporter. Two + // answers say so, since a poll may land before the click resolves. + const running = status({ + checking: true, + wtsexporter: { state: "downloadFailed", reason: "No connection." }, + }); + desktop.toolsStatus + .mockResolvedValueOnce(running) + .mockResolvedValueOnce(running) + .mockResolvedValue(failed); + return "alreadyRunning"; + }); + renderForm(readyWhatsapp); + + await user.click(await screen.findByRole("button", { name: "Try again" })); + + const line = + "A check of the Tools Directory is already running. This form shows its result when it ends."; + expect(await screen.findByText(line)).toBeTruthy(); + await waitFor(() => expect(screen.queryByText(line)).toBeNull(), { timeout: 4000 }); + expect(importButton()).toBeDisabled(); + }, 10_000); + + it("Try again while another app's check runs says so, and asks until the program arrives", async () => { + const user = setupUser(); + desktop.isTauri = true; + const failed = status({ wtsexporter: { state: "downloadFailed", reason: "No connection." } }); + desktop.toolsStatus.mockResolvedValue(failed); + desktop.retry.mockImplementation(async () => { + // Another app holds the Tools Directory: this process counts no check, + // and that app's download lands a moment later. + desktop.toolsStatus.mockResolvedValueOnce(failed).mockResolvedValue(status({})); + return "alreadyRunning"; + }); + renderForm(readyWhatsapp); + + await user.click(await screen.findByRole("button", { name: "Try again" })); + + expect( + await screen.findByText( + "A check of the Tools Directory is already running. This form shows its result when it ends.", + ), + ).toBeTruthy(); + await waitFor(() => expect(importButton()).toBeEnabled(), { timeout: 5000 }); + }, 10_000); + + it("Try again starts the download and follows it until the program arrives", async () => { + const user = setupUser(); + desktop.isTauri = true; + desktop.toolsStatus.mockResolvedValue( + status({ wtsexporter: { state: "downloadFailed", reason: "No connection." } }), + ); + desktop.retry.mockImplementation(async () => { + // The desktop process shows the program as downloading before it answers. + desktop.toolsStatus + .mockResolvedValueOnce( + status({ wtsexporter: { state: "downloading", received: 0, total: null } }), + ) + .mockResolvedValue(status({})); + return "started"; + }); + renderForm(readyWhatsapp); + + await user.click(await screen.findByRole("button", { name: "Try again" })); + + expect(desktop.retry).toHaveBeenCalledTimes(1); + expect(await screen.findByText(/Downloading wtsexporter/)).toBeTruthy(); + // Asked again while it downloads, until it is found. + await waitFor(() => expect(screen.queryByText(/Downloading wtsexporter/)).toBeNull(), { + timeout: 3000, + }); + expect(desktop.toolsStatus.mock.calls.length).toBeGreaterThanOrEqual(3); + expect(importButton()).toBeEnabled(); + }); +}); diff --git a/web/src/screens/import/ImportFormFields.tsx b/web/src/screens/import/ImportFormFields.tsx index 0f74b8462..cbe2c5378 100644 --- a/web/src/screens/import/ImportFormFields.tsx +++ b/web/src/screens/import/ImportFormFields.tsx @@ -32,8 +32,10 @@ import { showsAttachmentOptions } from "../../lib/importSource"; import type { PhoneCountryChoice } from "../../lib/phoneCountries"; import { ownerPhonesNeedMismatchAck } from "../../lib/phoneTokens"; import { parseSelectKey } from "../../lib/selectKey"; +import { toolUsable } from "../../lib/tauri"; import type { AttachmentMediaMode } from "../../lib/types"; import { accentLink } from "../../lib/uiStyles"; +import { useToolsStatus } from "../../lib/useToolsStatus"; import { isWhatsappMethod, WHATSAPP_METHODS, @@ -58,6 +60,8 @@ import { StackedField, sectionGap, } from "./ImportFormUi"; +import { MissingProgramNotice } from "./MissingProgramNotice"; +import { mediaJobVerb } from "./reviewForecast"; export type ImportFormFieldsProps = { source: string; @@ -386,8 +390,18 @@ export default function ImportFormFields(props: ImportFormFieldsProps) { const runningJob = useDesktopJob(); const blockedBy = runningJob !== null && runningJob !== "Import Run" ? runningJob : null; + // A WhatsApp import runs wtsexporter and can't start without it; one still + // downloading is waited for. Convert and Compress run ffmpeg and ffprobe + // after Staging, so a missing one is said here and blocks nothing yet. + const tools = useToolsStatus().data ?? null; + const wtsexporterBlocked = + whatsappMethod !== null && tools !== null && !toolUsable(tools.wtsexporter); + const mediaRunsFfmpeg = + showsAttachmentOptions(props.source) && mediaJobVerb(props.attachmentMedia) !== null; + const canImport = blockedBy === null && + !wtsexporterBlocked && (imessageReadiness ? imessageReadiness.enabled && !props.running : whatsappReadiness @@ -860,6 +874,13 @@ export default function ImportFormFields(props: ImportFormFieldsProps) {

) : null} + {tools && whatsappMethod !== null ? ( + + ) : null} + {tools && mediaRunsFfmpeg ? ( + + ) : null} +
+ ) : null} + + Troubleshooting + + {showAlreadyRunning ? ( + + A check of the Tools Directory is already running. This form shows its result when it + ends. + + ) : null} + {retry.isError ? ( + + The download didn't start.{" "} + {retry.error instanceof Error ? retry.error.message : String(retry.error)} + + ) : null} +
+ ); +} + +/** + * Says which of `programs` the chosen import needs and can't use, why (the + * download failed, with its reason, or it hasn't been downloaded), and offers + * Try again. A program still downloading is shown with its progress: the + * import waits for it. Nothing is shown when every program is found. + */ +export function MissingProgramNotice({ + programs, + status, + need, +}: { + programs: readonly ToolName[]; + status: ToolsStatus; + need: ProgramNeed; +}) { + const lines = programs.flatMap((name) => { + const program = status[name]; + if (program.state === "found") return []; + const { before, after } = toolStatusLine(program); + return [`${before}${name}${after}`]; + }); + if (lines.length === 0) return null; + const unusable = programs.filter((name) => !toolUsable(status[name])); + const blocked = unusable.length > 0; + const downloading = programs.filter((name) => status[name].state === "downloading"); + // A program the app has no download of for this computer can't be brought + // by Try again: the troubleshooting section says where a copy goes. + const retryable = unusable.some((name) => status[name].state !== "unavailable"); + return ( +
+

+ {lines.join(" ")} {consequence(need, blocked, downloading)} +

+ {blocked ? : null} +
+ ); +} diff --git a/web/src/screens/import/useImportJob.test.tsx b/web/src/screens/import/useImportJob.test.tsx index 908c1b0b2..c4a9d6d0a 100644 --- a/web/src/screens/import/useImportJob.test.tsx +++ b/web/src/screens/import/useImportJob.test.tsx @@ -262,6 +262,7 @@ const MIB = 1024 * 1024; function okProbe(): ToolsStatus { return { toolsDir: "/home/demo/message-crate/tools", + checking: false, ffmpeg: { state: "found", path: "/usr/bin/ffmpeg" }, ffprobe: { state: "found", path: "/usr/bin/ffprobe" }, wtsexporter: { state: "missing" }, @@ -811,6 +812,49 @@ describe("useImportJob wiring", () => { expect(result.current.phase).toBe("staging_review"); }); + it("says on the Staging row that a WhatsApp import waits for the wtsexporter download (#1053)", async () => { + let release: () => void = () => {}; + const held = new Promise((resolve) => { + release = resolve; + }); + runMock.mockReset(); + runMock.mockImplementationOnce( + async ( + fn: () => Promise, + _onLog?: (line: string) => void, + onProgress?: (event: ImportProgressEvent) => void, + ) => { + await fn(); + // The JSON `waiting_event` sends, as + // `a_run_waiting_for_a_download_is_sent_as_the_program_and_its_bytes` + // in src-tauri/src/commands/tools.rs asserts it. + onProgress?.({ + step: "setup", + done: 0, + total: 0, + bytes_done: 12_582_912, + bytes_total: 31_457_280, + waiting: "wtsexporter", + }); + await held; + return EXTRACT_RESULT; + }, + ); + const { result } = renderHook(() => useImportJob()); + let started: Promise = Promise.resolve(); + act(() => { + started = result.current.startImport(form({ attachmentMedia: "copy" })); + }); + await waitFor(() => + expect(result.current.steps[0]?.detail).toBe( + "Waiting for the wtsexporter download: 12 MB of 30 MB (40%)", + ), + ); + expect(result.current.steps[0]?.status).toBe("active"); + release(); + await act(() => started); + }); + it("keeps a line per stage on the Staging row while they run together", async () => { // Reading messages, copying attachments, and writing conversation files // report at the same time. With one shared line, each event replaced the @@ -1437,6 +1481,51 @@ describe("useImportJob wiring", () => { }); }); + it("says on the Media row that Media waits for the ffmpeg download (#1053)", async () => { + let release: () => void = () => {}; + const held = new Promise((resolve) => { + release = resolve; + }); + runMock.mockImplementationOnce( + async ( + fn: () => Promise, + _onLog?: (line: string) => void, + onProgress?: (event: ImportProgressEvent) => void, + ) => { + await fn(); + // The JSON `waiting_event` sends before the first byte arrives, as + // `a_run_waiting_for_a_download_is_sent_as_the_program_and_its_bytes` + // in src-tauri/src/commands/tools.rs asserts it, then one with bytes. + onProgress?.({ step: "media", done: 0, total: 0, bytes_done: 0, waiting: "ffmpeg" }); + onProgress?.({ + step: "media", + done: 0, + total: 0, + bytes_done: 12_582_912, + bytes_total: 31_457_280, + waiting: "ffmpeg", + }); + await held; + return { summary: "Transcode finished.", transcode: undefined }; + }, + ); + const { result } = renderHook(() => useImportJob()); + await act(() => result.current.startImport(form({ attachmentMedia: "convert" }))); + let approved: Promise = Promise.resolve(); + act(() => { + approved = result.current.approve(); + }); + + await waitFor(() => + expect(result.current.steps[1]?.detail).toBe( + "Waiting for the ffmpeg download: 12 MB of 30 MB (40%)", + ), + ); + expect(result.current.steps[1]?.status).toBe("active"); + release(); + await act(() => approved); + }); + it("writes an issue that arrives during Media into the directory before Media ends (#1479)", async () => { const mediaIssue: ImportIssueEvent = { kind: "skip", diff --git a/web/src/screens/import/useImportJob.ts b/web/src/screens/import/useImportJob.ts index afa50db61..3b59a42da 100644 --- a/web/src/screens/import/useImportJob.ts +++ b/web/src/screens/import/useImportJob.ts @@ -55,6 +55,7 @@ import { type UploadFinishedReport, } from "../../lib/tauri"; import { isTauri } from "../../lib/tauri-check"; +import { waitingLine } from "../../lib/toolStatusCopy"; import type { AttachmentMediaMode, ConversationStatus, @@ -769,6 +770,10 @@ function rowDetail(event: ImportProgressEvent): string { /** The detail line for one progress event. */ function progressDetail(event: ImportProgressEvent): string { + // A run waiting for a download, on the Setup or the Media row, says only that. + if (event.waiting) { + return waitingLine(event.waiting, event.bytes_done ?? 0, event.bytes_total ?? null); + } if (event.step === "setup") return setupDetail(event); if (event.step === "attachments") { const last = scratch.lastAttachmentProgress; diff --git a/web/src/screens/settings/SystemSection.test.tsx b/web/src/screens/settings/SystemSection.test.tsx index 01b141987..d7770dc56 100644 --- a/web/src/screens/settings/SystemSection.test.tsx +++ b/web/src/screens/settings/SystemSection.test.tsx @@ -68,6 +68,7 @@ beforeEach(() => { toolsStatus.mockReset(); toolsStatus.mockResolvedValue({ toolsDir: "/home/demo/message-crate/tools", + checking: false, ffmpeg: { state: "found", path: "/usr/bin/ffmpeg" }, ffprobe: { state: "found", path: "/usr/bin/ffprobe" }, wtsexporter: { state: "found", path: "/home/demo/message-crate/tools/wtsexporter" }, @@ -302,6 +303,7 @@ describe("SystemSection", () => { it("says which program is missing", async () => { toolsStatus.mockResolvedValue({ toolsDir: "/home/demo/message-crate/tools", + checking: false, ffmpeg: { state: "missing" }, ffprobe: { state: "found", path: "/usr/bin/ffprobe" }, wtsexporter: { state: "missing" }, @@ -325,6 +327,7 @@ describe("SystemSection", () => { it("says to put ffmpeg and ffprobe in the Tools Directory when neither is found", async () => { toolsStatus.mockResolvedValue({ toolsDir: "/home/demo/message-crate/tools", + checking: false, ffmpeg: { state: "missing" }, ffprobe: { state: "missing" }, wtsexporter: { state: "missing" }, @@ -347,6 +350,7 @@ describe("SystemSection", () => { "ffmpeg is on PATH at /usr/bin/ffmpeg and ffprobe is in the Tools Directory at /home/demo/message-crate/tools/ffprobe."; toolsStatus.mockResolvedValue({ toolsDir: "/home/demo/message-crate/tools", + checking: false, ffmpeg: { state: "unusable", reason }, ffprobe: { state: "unusable", reason }, wtsexporter: { state: "missing" }, @@ -364,25 +368,57 @@ describe("SystemSection", () => { expect(media?.querySelector("input")).toBeNull(); }); + it("asks again while a check runs, so a program it hasn't reached yet shows once it arrives", async () => { + toolsStatus.mockResolvedValueOnce({ + toolsDir: "/home/demo/message-crate/tools", + checking: true, + ffmpeg: { state: "found", path: "/usr/bin/ffmpeg" }, + ffprobe: { state: "found", path: "/usr/bin/ffprobe" }, + wtsexporter: { state: "missing" }, + }); + render(); + expect(await screen.findByLabelText(/wtsexporter not found/)).toBeTruthy(); + // The next answer, a second later, has wtsexporter in place. + expect(await screen.findByLabelText(/Found wtsexporter/i, {}, { timeout: 3000 })).toBeTruthy(); + }); + it("shows a download's progress, and asks again until it ends", async () => { toolsStatus.mockResolvedValueOnce({ toolsDir: "/home/demo/message-crate/tools", + checking: false, ffmpeg: { state: "downloading", received: 12 * 1024 * 1024, total: 29 * 1024 * 1024 }, ffprobe: { state: "downloading", received: 0, total: null }, wtsexporter: { state: "found", path: "/home/demo/message-crate/tools/wtsexporter" }, }); render(); - expect(await screen.findByLabelText("Downloading ffmpeg - 12 MB of 29 MB (41%)")).toBeTruthy(); - expect(screen.getByLabelText("Downloading ffprobe - 0 B so far")).toBeTruthy(); + expect(await screen.findByLabelText("Downloading ffmpeg: 12 MB of 29 MB (41%).")).toBeTruthy(); + expect(screen.getByLabelText("Downloading ffprobe: 0 B so far.")).toBeTruthy(); // The next answer, a second later, has both in place. expect(await screen.findByLabelText(/Found ffmpeg/i, {}, { timeout: 3000 })).toBeTruthy(); expect(screen.getByLabelText(/Found ffprobe/i)).toBeTruthy(); }); + it("says a program the app has no download for goes in the Tools Directory by hand", async () => { + toolsStatus.mockResolvedValue({ + toolsDir: "/home/demo/message-crate/tools", + checking: false, + ffmpeg: { state: "found", path: "/usr/bin/ffmpeg" }, + ffprobe: { state: "found", path: "/usr/bin/ffprobe" }, + wtsexporter: { state: "unavailable" }, + }); + render(); + expect( + await screen.findByLabelText( + "wtsexporter not found, and the app has no download of it for this computer. Put a copy in the Tools Directory, /home/demo/message-crate/tools.", + ), + ).toBeTruthy(); + }); + it("says why a download failed", async () => { const reason = "No connection to the download's server: error sending request."; toolsStatus.mockResolvedValue({ toolsDir: "/home/demo/message-crate/tools", + checking: false, ffmpeg: { state: "found", path: "/usr/bin/ffmpeg" }, ffprobe: { state: "found", path: "/usr/bin/ffprobe" }, wtsexporter: { state: "downloadFailed", reason }, @@ -390,9 +426,14 @@ describe("SystemSection", () => { render(); expect( await screen.findByLabelText( - `wtsexporter download failed. ${reason} It is tried again the next time the app starts.`, + `wtsexporter download failed. ${reason} It is tried again the next time the app starts, or with Try again on the Import form. See "Import can't find wtsexporter" in Troubleshooting at messagecrate.app.`, ), ).toBeTruthy(); + expect( + screen.getByRole("link", { name: "Import can't find wtsexporter" }).getAttribute("href"), + ).toBe( + "https://messagecrate.app/docs/user/features/owner/troubleshooting/#import-cant-find-wtsexporter", + ); }); it("says so when the desktop process cannot be asked", async () => { diff --git a/web/src/screens/settings/SystemSection.tsx b/web/src/screens/settings/SystemSection.tsx index 1510aedda..ff87d6449 100644 --- a/web/src/screens/settings/SystemSection.tsx +++ b/web/src/screens/settings/SystemSection.tsx @@ -1,12 +1,10 @@ -import { useQuery } from "@tanstack/react-query"; -import { useEffect, useId, useRef, useState } from "react"; +import { type ReactNode, useEffect, useId, useRef, useState } from "react"; import Checkbox from "../../components/Checkbox"; import { CheckIcon, DownloadIcon, XIcon } from "../../components/icons"; import OpenPathButton from "../../components/OpenPathButton"; import PathPicker from "../../components/PathPicker"; import PlainButton from "../../components/PlainButton"; import { getBaseUrl } from "../../lib/api"; -import { formatBytes } from "../../lib/attachmentProgressCopy"; import { APP_BUILD } from "../../lib/build"; import { getOpenToNetwork, @@ -16,19 +14,20 @@ import { setLocalServerOpenToNetwork, setOpenToNetwork, } from "../../lib/localServer"; -import { keys } from "../../lib/queryKeys"; import { getRememberImporterPaths, setRememberImporterPaths } from "../../lib/system-settings"; import { invokeExportDirectory, invokeSetStagingRoot, invokeStagingRoot, - invokeToolsStatus, + type ToolName, type ToolStatus, type ToolsStatus, - toolsDownloading, } from "../../lib/tauri"; import { isTauri } from "../../lib/tauri-check"; import { readerLicenseUrl, readerSourceUrl } from "../../lib/thirdPartySoftware"; +import { toolStatusLine, troubleshootingSentence } from "../../lib/toolStatusCopy"; +import { accentLink } from "../../lib/uiStyles"; +import { useToolsStatus } from "../../lib/useToolsStatus"; const sectionHeading = "m-0 mb-2 text-[12px] font-semibold uppercase tracking-[0.05em] text-muted"; @@ -56,25 +55,10 @@ function stagingHelpExample(stagingDir: string, defaultDir: string): string { return `${trimmed}/${EXAMPLE_STAGING}`; } -type ToolName = "ffmpeg" | "ffprobe" | "wtsexporter"; - -/** A download's progress: "12 MB of 29 MB (41%)", or "12 MB so far" with no total. */ -function downloadProgress(received: number, total: number | null): string { - if (total == null || total <= 0) return `${formatBytes(received)} so far`; - const percent = Math.min(100, Math.floor((received / total) * 100)); - return `${formatBytes(received)} of ${formatBytes(total)} (${percent}%)`; -} - /** Said once on every failed download's line, whatever the reason: each is retried. */ -const DOWNLOAD_RETRIED = "It is tried again the next time the app starts."; +const DOWNLOAD_RETRIED = + "It is tried again the next time the app starts, or with Try again on the Import form."; -/** How often Settings asks the desktop process again while a download runs. */ -const DOWNLOAD_POLL_MS = 1000; - -/** - * One program's status line: the path it was found at, that it is missing, - * or why it is not used. One case per `state` the desktop process sends. - */ /** * Where to put a program that is missing, as the words before the Tools * Directory's path (`dir`), or the whole sentence when `dir` is null. @@ -99,6 +83,69 @@ function missingFix( return toolsDir ? { text: `Put it and ${other} in the Tools Directory`, dir: toolsDir } : null; } +/** + * What Settings says after a program's status line: where a missing one + * goes, and for a failed download that it is tried again and the + * troubleshooting section. As text for the row's label, and as the row shows it. + */ +function statusExtra( + name: ToolName, + status: ToolStatus, + partner: ToolStatus | null, + toolsDir: string | null, +): { label: string; node: ReactNode } | null { + const fixWithDir = (fix: { text: string; dir: string | null } | null) => + fix == null + ? null + : { + label: fix.dir ? `${fix.text}, ${fix.dir}.` : `${fix.text}.`, + node: ( + <> + {fix.text} + {fix.dir ? ( + <> + , {fix.dir} + + ) : null} + . + + ), + }; + switch (status.state) { + // An app opened from the Dock on macOS doesn't see the PATH a shell + // sets, so a program installed with Homebrew is missing here until it + // is linked into the Tools Directory. + case "missing": + return fixWithDir(missingFix(name, partner, toolsDir)); + // No download for this computer: only a copy put there by hand is used. + case "unavailable": + return fixWithDir( + toolsDir ? { text: "Put a copy in the Tools Directory", dir: toolsDir } : null, + ); + case "downloadFailed": { + const { before, section, after } = troubleshootingSentence(name); + return { + label: `${DOWNLOAD_RETRIED} ${before}${section.heading}${after}`, + node: ( + <> + {DOWNLOAD_RETRIED} {before} + + {section.heading} + + {after} + + ), + }; + } + default: + return null; + } +} + +/** + * One program's status line: the path it was found at, or the line the + * Import form shows too (`toolStatusLine`), with what Settings adds after it. + */ function ToolStatusRow({ name, status, @@ -111,86 +158,39 @@ function ToolStatusRow({ partner: ToolStatus | null; toolsDir: string | null; }) { - switch (status.state) { - case "found": { - const label = `Found ${name} - ${status.path}`; - return ( -
  • - - - Found {name} - {" - "} - {status.path} - -
  • - ); - } - case "missing": { - // An app opened from the Dock on macOS doesn't see the PATH a shell - // sets, so a program installed with Homebrew is missing here until it - // is linked into the Tools Directory. - const fix = missingFix(name, partner, toolsDir); - const fixText = fix == null ? "" : fix.dir ? ` ${fix.text}, ${fix.dir}.` : ` ${fix.text}.`; - const label = `${name} not found.${fixText}`; - return ( -
  • - - - {name} not found. - {fix ? ( - <> - {" "} - {fix.text} - {fix.dir ? ( - <> - , {fix.dir} - - ) : null} - . - - ) : null} - -
  • - ); - } - case "unusable": { - const label = `${name} not used. ${status.reason}`; - return ( -
  • - - - {name} not used. {status.reason} - -
  • - ); - } - case "downloading": { - const progress = downloadProgress(status.received, status.total); - const label = `Downloading ${name} - ${progress}`; - return ( -
  • - - - Downloading {name} - {" - "} - {progress} - -
  • - ); - } - case "downloadFailed": { - const label = `${name} download failed. ${status.reason} ${DOWNLOAD_RETRIED}`; - return ( -
  • - - - {name} download failed. {status.reason}{" "} - {DOWNLOAD_RETRIED} - -
  • - ); - } + if (status.state === "found") { + return ( +
  • + + + Found {name} + {" - "} + {status.path} + +
  • + ); } + const line = toolStatusLine(status); + const extra = statusExtra(name, status, partner, toolsDir); + const label = `${line.before}${name}${line.after}${extra ? ` ${extra.label}` : ""}`; + return ( +
  • + {status.state === "downloading" ? ( + + ) : ( + + )} + + {line.before} + {name} + {line.after} + {extra ? <> {extra.node} : null} + +
  • + ); } /** @@ -421,16 +421,8 @@ export function SystemSection() { /** The Staging Directory the desktop process holds now. */ const [stagingError, setStagingError] = useState(null); const [rememberPaths, setRememberPaths] = useState(false); - // Where the programs are belongs to this computer, not to an account, so - // this is a plain `useQuery`, asked again each second while a download runs. - const toolsQuery = useQuery({ - queryKey: keys.desktopToolsStatus.all, - queryFn: invokeToolsStatus, - enabled: isTauri(), - retry: false, - refetchInterval: (query) => - query.state.data && toolsDownloading(query.state.data) ? DOWNLOAD_POLL_MS : false, - }); + // Asked again each second while a download runs. + const toolsQuery = useToolsStatus(); const tools: ToolsStatus | null = toolsQuery.data ?? null; const toolsError = toolsQuery.error ? `Could not look for the media tools. ${toolsQuery.error instanceof Error ? toolsQuery.error.message : String(toolsQuery.error)}`