diff --git a/CHANGELOG.md b/CHANGELOG.md
index db9cbd3f6..39e48c1e6 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -21,6 +21,18 @@ released versions carry their date on the heading.
### Features
+- 2026-10-08: **The Import form says when a program the import needs is
+ missing, with Try again.** A WhatsApp import needs wtsexporter, and
+ **Convert** and **Compress** need ffmpeg and ffprobe. When one hasn't been
+ downloaded or its download failed, the Import form names it and gives the
+ reason, with **Try again**, which downloads it at once, and a link to
+ Troubleshooting. A WhatsApp import can't start without wtsexporter. An
+ import with **Convert** or **Compress** still starts, as before. An import
+ started while its program is still downloading waits for it, and its
+ progress line says so, such as "Waiting for the wtsexporter download: 12 MB
+ of 30 MB (40%)". When that download fails, the run fails with the reason
+ and names the troubleshooting section (#1053).
+
- 2026-10-08: **The desktop app downloads ffmpeg, ffprobe and wtsexporter
by itself.** Each time it starts, it downloads in the background whatever
is missing from its Tools Directory, with no button and nothing to wait
diff --git a/docs/src/content/docs/docs/user/features/messages/attachments-and-media.md b/docs/src/content/docs/docs/user/features/messages/attachments-and-media.md
index dc61c4943..518f81530 100644
--- a/docs/src/content/docs/docs/user/features/messages/attachments-and-media.md
+++ b/docs/src/content/docs/docs/user/features/messages/attachments-and-media.md
@@ -82,10 +82,12 @@ These commands install both on `PATH`:
The desktop app finds the programs on `PATH`, else in its Tools Directory, and takes both from the same one.
[**Settings → System**](/docs/user/features/settings/system/#media) shows each under **Media**: `Found` with its path, its download's progress, why the download failed, `not found`, or `not used` with the reason.
-An Import Run looks for ffmpeg at the Staging Review, not when the run starts, because Staging copies the original files and needs neither program.
-When ffmpeg or ffprobe can't be used, the review names it, such as `Media can't use ffprobe.`, and the **Convert media** or **Compress media** button is disabled.
-The desktop app downloads both programs into the Tools Directory when it starts, so a program missing at the review is usually still downloading.
-When **Settings → System** says the download failed, the package manager commands above install ffmpeg and ffprobe instead.
+With **Convert** or **Compress** chosen, the Import form says when ffmpeg or ffprobe can't be used, and why, but the run still starts, because Staging copies the original files and needs neither program.
+At the Staging Review, the review names the program, such as `Media can't use ffprobe.`, and the **Convert media** or **Compress media** button is disabled.
+**Try again**, on the form and at the review, downloads the program into the Tools Directory at once.
+A program still downloading doesn't stop the review.
+Media waits for the download, and its progress line says so.
+When the download fails, the package manager commands above install ffmpeg and ffprobe instead.
The run keeps waiting at the review until the program is there.
## The size limit
diff --git a/docs/src/content/docs/docs/user/features/owner/troubleshooting.md b/docs/src/content/docs/docs/user/features/owner/troubleshooting.md
index 697a41ab3..0bf51cd11 100644
--- a/docs/src/content/docs/docs/user/features/owner/troubleshooting.md
+++ b/docs/src/content/docs/docs/user/features/owner/troubleshooting.md
@@ -98,6 +98,11 @@ Import continues once the field is empty.
A WhatsApp import runs a separate program, `wtsexporter`, which the desktop app downloads when it starts.
[**Settings → System**](/docs/user/features/settings/system/#media) shows whether it arrived, or why the download failed, and [WhatsApp](/docs/user/import-sources/whatsapp/#wtsexporter) says where it comes from.
+The Import form names the program and the reason, and its **Try again** downloads it at once instead of at the next start.
+
+The app has no `wtsexporter` download for some computers, such as Linux on ARM.
+There the Import form says so and offers no **Try again**, because nothing can be downloaded.
+A `wtsexporter` put in the Tools Directory by hand is used instead.
### A WhatsApp import doesn't stop when cancelled
@@ -112,6 +117,7 @@ Two of the **Attachments** choices on the Import form run `ffmpeg` and `ffprobe`
In the desktop app, **Settings → System** shows the Tools Directory under **Media**.
The app looks on the system `PATH`, then in the Tools Directory, and takes both programs from the same one.
The lines there report each program as found, with its path, not found, or not used, with the reason.
+When a download failed, **Try again** on the Import form, or at the Staging Review, downloads the program at once instead of at the next start.
## Convert
diff --git a/docs/src/content/docs/docs/user/features/settings/system.md b/docs/src/content/docs/docs/user/features/settings/system.md
index 29109b50f..8df124dad 100644
--- a/docs/src/content/docs/docs/user/features/settings/system.md
+++ b/docs/src/content/docs/docs/user/features/settings/system.md
@@ -66,13 +66,16 @@ One line per program reports the result:
- A cross with ` not found` and where to put it.
A missing ffmpeg or ffprobe goes beside the other one, or both go in the Tools Directory, because the two are used only from one place.
A missing wtsexporter goes in the Tools Directory.
+- A cross with ` not found, and the app has no download of it for this computer` and where to put a copy by hand.
+ The app has no wtsexporter download for Linux on ARM.
- A cross with ` not used` and the reason.
For ffmpeg and ffprobe, the reason is that they were found in two places, one only on `PATH` and the other only in the Tools Directory.
For wtsexporter, the reason is that it has no permission to run.
- A download arrow with `Downloading ` and how much has arrived, such as `12 MB of 29 MB (41%)`.
The line updates each second until the download ends.
-- A cross with ` download failed` and the reason: no connection to the download's server, the status the server answered, a checksum that didn't match, a file that couldn't be written to the Tools Directory, or a program in the Tools Directory that doesn't run on this computer.
- A failed download is tried again the next time the app starts.
+- A cross with ` download failed` and the reason: no connection to the download's server, the status the server answered, a checksum that didn't match, a file that couldn't be written to the Tools Directory, a download that was interrupted or couldn't be started, or a program in the Tools Directory that doesn't run on this computer.
+ A failed download is tried again the next time the app starts, or at once with **Try again** on the Import form.
+ The line ends with a link to the program's section in [Troubleshooting](/docs/user/features/owner/troubleshooting/#import).
A program that doesn't run is not downloaded again, because the download would be the same file.
For ffmpeg and ffprobe, the reason says to install them with a package manager, because the app uses the copy on `PATH`.
While an older copy is in place, the line shows that copy as found instead, because the old copy is still the one used.
diff --git a/docs/src/content/docs/docs/user/import-sources/whatsapp.md b/docs/src/content/docs/docs/user/import-sources/whatsapp.md
index 6fc82f57e..783b8b901 100644
--- a/docs/src/content/docs/docs/user/import-sources/whatsapp.md
+++ b/docs/src/content/docs/docs/user/import-sources/whatsapp.md
@@ -16,6 +16,8 @@ Two things make it hard:
The desktop app runs `wtsexporter`, from Message Crate's fork of the open-source [WhatsApp Chat Exporter](https://github.com/KnugiHK/WhatsApp-Chat-Exporter), [messagecrate/WhatsApp-Chat-Exporter](https://github.com/messagecrate/WhatsApp-Chat-Exporter), because only the fork records the ids a quoted reply is linked by.
Each time it starts, the desktop app downloads the pinned release into its Tools Directory when it isn't there, in the background, and [**Settings → System**](/docs/user/features/settings/system/#media) shows the program or how its download stands.
The app runs `wtsexporter` only from the Tools Directory, never from `PATH`.
+A WhatsApp import can't start without it, and the Import form names the reason and offers **Try again**.
+An import started while `wtsexporter` is still downloading waits for the download, and its progress line says so.
A program in the Tools Directory whose checksum is the pinned program's is kept, whoever put it there, and anything else under its name is replaced by the pinned program.
## WhatsApp on Android
diff --git a/src-tauri/src/commands/events.rs b/src-tauri/src/commands/events.rs
index ce7be2d89..aa65ce68b 100644
--- a/src-tauri/src/commands/events.rs
+++ b/src-tauri/src/commands/events.rs
@@ -8,6 +8,8 @@ use message_crate_core::{IssueSink, ProgressEvent, ProgressSink, RunIssue, Write
use serde::Serialize;
use tauri::{AppHandle, Emitter};
+use crate::tool_downloads::Program;
+
/// One log line for the UI's log panel. Payload: `String`.
pub const LOG: &str = "extract:log";
/// Progress-bar numbers. Payload: [`ExtractProgressEvent`].
@@ -92,6 +94,12 @@ pub struct ExtractProgressEvent {
/// Extra step status the UI shows. On `setup` it is the step's label.
#[serde(skip_serializing_if = "Option::is_none")]
pub status: Option,
+ /// The program the run waits for while it downloads (#1053). The event
+ /// then has no counts, `bytes_done` is how much has arrived and
+ /// `bytes_total` the file's size when the server said, and the window
+ /// says "Waiting for the download" with them.
+ #[serde(skip_serializing_if = "Option::is_none")]
+ pub waiting: Option,
}
impl ExtractProgressEvent {
@@ -104,6 +112,7 @@ impl ExtractProgressEvent {
bytes_done: None,
bytes_total: None,
status: None,
+ waiting: None,
}
}
}
diff --git a/src-tauri/src/commands/extract.rs b/src-tauri/src/commands/extract.rs
index 2c45fc60f..03a7cb8d7 100644
--- a/src-tauri/src/commands/extract.rs
+++ b/src-tauri/src/commands/extract.rs
@@ -40,6 +40,7 @@ use super::paths::{logs_dir, scratch_dir};
use crate::app_directories::RunLog;
use crate::run_directories::RunDirectories;
use crate::state::{AppState, JobName};
+use crate::tool_downloads::{Program, ToolDownloads};
/// Ask this process to stop the job that is running. Does nothing when no
/// job runs.
@@ -160,6 +161,7 @@ pub struct ExtractArgs {
pub fn extract(
state: tauri::State<'_, Arc>>,
directories: tauri::State<'_, RunDirectories>,
+ downloads: tauri::State<'_, ToolDownloads>,
app: tauri::AppHandle,
args: ExtractArgs,
) -> Result<(), String> {
@@ -201,7 +203,9 @@ pub fn extract(
let job = start_job(&state, JobName::Staging)?;
let app_handle = app.clone();
- config.cancel = Some(job.cancel_flag());
+ let cancel = job.cancel_flag();
+ config.cancel = Some(cancel.clone());
+ let downloads = downloads.inner().clone();
// Two channels, two jobs: log lines are for the person reading the log
// panel, progress events are for the bar. Nothing reads counts out of
// the prose.
@@ -217,6 +221,16 @@ pub fn extract(
config.issues = Some(events::run_issue_sink(&app_handle, &run_log));
spawn_job(app, job, move || {
+ // A program still downloading is waited for, and the wait is the
+ // Staging row's progress line (#1053).
+ super::tools::wait_for_downloads(
+ &app_handle,
+ &downloads,
+ programs_run_by(&config.source),
+ &cancel,
+ "setup",
+ )
+ .inspect_err(|error| run_log.error(error))?;
let run_result = run_staging(&config, &output_dir, &media_settings)
.inspect_err(|error| run_log.error(error))?;
let payload = finished_payload(&run_result);
@@ -494,6 +508,17 @@ fn build_exporter_config(
.map_err(|errors| errors.join("; "))
}
+/// The programs the Tools Directory holds that `source`'s exporter runs: a
+/// WhatsApp import runs wtsexporter. Staging copies attachments as they are
+/// whatever the person chose, so it never runs ffmpeg; the Media stage does
+/// (`transcode_staging`).
+fn programs_run_by(source: &SourceConfig) -> &'static [Program] {
+ match source {
+ SourceConfig::Whatsapp(_) => &[Program::Wtsexporter],
+ _ => &[],
+ }
+}
+
/// Call the exporter that matches `config.source`.
///
/// # Errors
diff --git a/src-tauri/src/commands/extract/tests.rs b/src-tauri/src/commands/extract/tests.rs
index 6eafd8dcb..55a2db6fc 100644
--- a/src-tauri/src/commands/extract/tests.rs
+++ b/src-tauri/src/commands/extract/tests.rs
@@ -742,3 +742,43 @@ fn a_staged_jsonl_attachment_that_is_not_utf8_does_not_fail_the_count() {
assert_eq!(counts, (1, 1));
}
+
+/// A WhatsApp import waits for wtsexporter's download, and no other source
+/// waits for a program: Staging runs ffmpeg for none of them (#1053).
+#[test]
+fn only_a_whatsapp_import_waits_for_a_download() {
+ use message_crate_core::{
+ AppleConfig, FormatConfig, GoSmsProConfig, ImazingConfig, OpenExtractConfig,
+ SmsBackupPlusConfig, SmsBackupRestoreConfig, WhatsappConfig,
+ };
+ let none: &[Program] = &[];
+ let sources = [
+ SourceConfig::GoSmsPro(GoSmsProConfig {
+ owner_phones: Vec::new(),
+ }),
+ SourceConfig::SmsBackupRestore(SmsBackupRestoreConfig {
+ owner_phones: Vec::new(),
+ }),
+ SourceConfig::SmsBackupPlus(SmsBackupPlusConfig {
+ owner_phones: Vec::new(),
+ owner_emails: Vec::new(),
+ phone_country: None,
+ verbose: false,
+ include_summary: false,
+ }),
+ SourceConfig::OpenExtract(OpenExtractConfig::default()),
+ SourceConfig::Imazing(ImazingConfig::default()),
+ SourceConfig::Apple(AppleConfig::default()),
+ SourceConfig::Format(FormatConfig::default()),
+ ];
+ for source in &sources {
+ assert_eq!(programs_run_by(source), none, "{source:?}");
+ }
+ for platform in [WhatsappPlatform::Android, WhatsappPlatform::Ios] {
+ let source = SourceConfig::Whatsapp(WhatsappConfig {
+ platform: Some(platform),
+ ..WhatsappConfig::default()
+ });
+ assert_eq!(programs_run_by(&source), [Program::Wtsexporter]);
+ }
+}
diff --git a/src-tauri/src/commands/staging.rs b/src-tauri/src/commands/staging.rs
index 3b16bd1f0..c635b79d7 100644
--- a/src-tauri/src/commands/staging.rs
+++ b/src-tauri/src/commands/staging.rs
@@ -38,6 +38,7 @@ use super::jobs::{spawn_job, start_job};
use crate::app_directories::RunLog;
use crate::run_directories::{self, RunDirectories, StagingRoot};
use crate::state::{AppState, JobName};
+use crate::tool_downloads::{Program, ToolDownloads};
/// The directory `summarize_staging`, `transcode_staging`, `delete_run_dir`
/// and the Import Run record commands act on.
@@ -158,6 +159,7 @@ pub async fn summarize_staging(
bytes_done: None,
bytes_total: None,
status: None,
+ waiting: None,
},
);
})
@@ -241,6 +243,7 @@ fn transcode_summary(report: &TranscodeReport) -> String {
pub fn transcode_staging(
state: tauri::State<'_, Arc>>,
directories: tauri::State<'_, RunDirectories>,
+ downloads: tauri::State<'_, ToolDownloads>,
app: tauri::AppHandle,
args: RunDirArgs,
) -> Result<(), String> {
@@ -255,8 +258,19 @@ pub fn transcode_staging(
let app_handle = app.clone();
let issues = events::run_issue_sink(&app, &run_log);
+ let downloads = downloads.inner().clone();
spawn_job(app, job, move || {
if media_stage_converts {
+ // ffmpeg or ffprobe still downloading is waited for, and the
+ // wait is the Media row's progress line (#1053).
+ super::tools::wait_for_downloads(
+ &app_handle,
+ &downloads,
+ &[Program::Ffmpeg, Program::Ffprobe],
+ &cancel,
+ "media",
+ )
+ .inspect_err(|error| run_log.error(error))?;
events::log_to_run(
&app_handle,
&run_log,
@@ -284,6 +298,7 @@ pub fn transcode_staging(
bytes_done: None,
bytes_total: None,
status: None,
+ waiting: None,
},
);
},
diff --git a/src-tauri/src/commands/tools.rs b/src-tauri/src/commands/tools.rs
index c0a187ab3..df2978266 100644
--- a/src-tauri/src/commands/tools.rs
+++ b/src-tauri/src/commands/tools.rs
@@ -8,8 +8,13 @@
//! where a program is run from.
use std::path::PathBuf;
+use std::sync::atomic::Ordering;
-use crate::tool_downloads::{DownloadState, Program, ToolDownloads};
+use message_crate_core::{CancelFlag, Cancelled};
+use tauri::AppHandle;
+
+use super::events::{self, ExtractProgressEvent};
+use crate::tool_downloads::{self, DownloadState, Program, ToolDownloads, WaitError};
/// Where one program is, as Settings shows it.
///
@@ -25,6 +30,11 @@ pub enum ToolStatus {
},
/// The program is on neither place it is looked for.
Missing,
+ /// The program is on neither place it is looked for, and the app has no
+ /// download of it for this computer (wtsexporter on Linux on ARM), so
+ /// Try again can't bring it. A copy put in the Tools Directory by hand
+ /// is used.
+ Unavailable,
/// The program was found but is not used, for `reason`.
Unusable {
/// Why it is not used.
@@ -38,7 +48,8 @@ pub enum ToolStatus {
total: Option,
},
/// The program's download failed, for `reason`, and the program is not
- /// found. It is tried again the next time the app starts.
+ /// found. It is tried again the next time the app starts, or with Try
+ /// again on the Import form.
DownloadFailed {
/// Why the download failed.
reason: String,
@@ -56,6 +67,15 @@ impl ToolStatus {
}
}
+ /// This status, with a missing program the app has no download for
+ /// (`pinned` is false) told apart as unavailable.
+ fn with_pin(self, pinned: bool) -> Self {
+ match self {
+ Self::Missing if !pinned => Self::Unavailable,
+ status => status,
+ }
+ }
+
/// This status as `download` changes it. A download in progress is
/// shown whatever was found, because the file is about to change. A
/// failed one is shown only when the program is missing: a program found
@@ -80,6 +100,12 @@ impl ToolStatus {
pub struct ToolsStatus {
/// The Tools Directory, `None` when the app has none (no home directory).
pub tools_dir: Option,
+ /// Whether a check of the Tools Directory runs in this process now, the
+ /// start-up check or Try again. A program it has not looked at yet shows
+ /// as missing, so the window keeps asking while this is true. Another
+ /// app's check on the same Tools Directory is not counted: Try again
+ /// reports it as [`RetryResult::AlreadyRunning`].
+ pub checking: bool,
/// ffmpeg, from `PATH` or the Tools Directory.
pub ffmpeg: ToolStatus,
/// ffprobe, from `PATH` or the Tools Directory.
@@ -92,11 +118,156 @@ pub struct ToolsStatus {
/// and say how each one's download stands.
#[tauri::command]
pub fn tools_status(downloads: tauri::State<'_, ToolDownloads>) -> ToolsStatus {
- tools_status_with(&downloads)
+ tools_status_with(&downloads, &pinned_programs())
+}
+
+/// The programs the app has a download of for this computer.
+fn pinned_programs() -> Vec {
+ tool_downloads::pinned_for(std::env::consts::OS, std::env::consts::ARCH)
+ .into_iter()
+ .map(|pin| pin.program)
+ .collect()
+}
+
+/// What Try again did, as [`retry_tool_downloads`] tells the window.
+#[derive(Debug, Clone, Copy, PartialEq, Eq, serde::Serialize)]
+#[serde(rename_all = "camelCase")]
+pub enum RetryResult {
+ /// A check started in this process, and `checking` is true until it ends.
+ Started,
+ /// Another check holds the lock on the Tools Directory: the start-up
+ /// check, an earlier Try again, or another app's check. Nothing started.
+ /// `checking` is true only when that check runs in this process, so the
+ /// window can't tell from the status when another app's check ends.
+ AlreadyRunning,
+ /// The app has no Tools Directory (no home directory), which the status
+ /// shows as `toolsDir: null`.
+ NoToolsDirectory,
+ /// The check's thread could not be started, and each program it would
+ /// have downloaded shows as a failed download with that reason.
+ CouldNotStart,
+}
+
+/// Check the Tools Directory again and download what is missing, as **Try
+/// again** on the Import form asks (#1053). Returns at once: the check runs
+/// on a thread of its own, and the window follows it through
+/// [`tools_status`].
+#[tauri::command]
+pub fn retry_tool_downloads(downloads: tauri::State<'_, ToolDownloads>) -> RetryResult {
+ let Some(dir) = media::tools_dir() else {
+ return RetryResult::NoToolsDirectory;
+ };
+ let status = tools_status_with(&downloads, &pinned_programs());
+ let not_found: Vec = [
+ (Program::Ffmpeg, &status.ffmpeg),
+ (Program::Ffprobe, &status.ffprobe),
+ (Program::Wtsexporter, &status.wtsexporter),
+ ]
+ .into_iter()
+ .filter(|(_, status)| !matches!(status, ToolStatus::Found { .. }))
+ .map(|(program, _)| program)
+ .collect();
+ let pinned = tool_downloads::pinned_for(std::env::consts::OS, std::env::consts::ARCH);
+ match tool_downloads::retry(
+ dir,
+ tool_downloads::GITHUB.to_string(),
+ pinned,
+ &downloads,
+ ¬_found,
+ ) {
+ // Nothing joins the check's thread: the window follows the check
+ // through `tools_status`.
+ tool_downloads::Retry::Started(_thread) => RetryResult::Started,
+ tool_downloads::Retry::AlreadyRunning => RetryResult::AlreadyRunning,
+ tool_downloads::Retry::CouldNotStart => RetryResult::CouldNotStart,
+ }
}
-/// [`tools_status`], with the downloads in `downloads`.
-fn tools_status_with(downloads: &ToolDownloads) -> ToolsStatus {
+/// How far a download waited for moves before the window hears of it again,
+/// so a download heard of at every 64 KiB read doesn't send an event each.
+const WAITING_STEP_BYTES: u64 = 256 * 1024;
+
+/// Wait while any of `programs` is downloading, before an import runs it
+/// (#1053). The wait goes to the window as `extract:progress` events on
+/// `step` ([`waiting_event`]): one when it starts, and one each time the
+/// program waited for changes or its download moves by
+/// [`WAITING_STEP_BYTES`]. The window writes the progress line from them,
+/// in the byte format the Import form uses.
+///
+/// # Errors
+///
+/// Returns [`Cancelled`] when the run is cancelled while it waits, and the
+/// failed download's reason when a download waited for fails.
+pub(crate) fn wait_for_downloads(
+ app: &AppHandle,
+ downloads: &ToolDownloads,
+ programs: &[Program],
+ cancel: &CancelFlag,
+ step: &str,
+) -> anyhow::Result<()> {
+ let mut last: Option<(Program, u64, Option)> = None;
+ downloads
+ .wait_for(
+ programs,
+ &|| cancel.load(Ordering::Relaxed),
+ &mut |program, received, total| {
+ let now = (program, received, total);
+ if !worth_sending(last, now) {
+ return;
+ }
+ events::emit(
+ app,
+ events::PROGRESS,
+ waiting_event(step, program, received, total),
+ );
+ last = Some(now);
+ },
+ )
+ .map_err(|err| match err {
+ WaitError::Cancelled => anyhow::Error::new(Cancelled),
+ failed @ WaitError::Failed { .. } => anyhow::Error::new(failed),
+ })
+}
+
+/// Whether the wait at `now` is news to the window, which last heard `last`.
+fn worth_sending(
+ last: Option<(Program, u64, Option)>,
+ now: (Program, u64, Option),
+) -> bool {
+ let Some((program, received, total)) = last else {
+ return true;
+ };
+ program != now.0 || total != now.2 || now.1.abs_diff(received) >= WAITING_STEP_BYTES
+}
+
+/// The `extract:progress` event of a run on `step` waiting for `program`'s
+/// download, `received` bytes in of `total`: marked by `waiting`, with no
+/// counts and the bytes in `bytes_done` and `bytes_total`.
+fn waiting_event(
+ step: &str,
+ program: Program,
+ received: u64,
+ total: Option,
+) -> ExtractProgressEvent {
+ ExtractProgressEvent {
+ step: step.to_string(),
+ done: 0,
+ total: 0,
+ bytes_done: Some(received),
+ bytes_total: total,
+ status: None,
+ waiting: Some(program),
+ }
+}
+
+/// [`tools_status`], with the downloads in `downloads` and a download for
+/// this computer of each of `pinned`.
+fn tools_status_with(downloads: &ToolDownloads, pinned: &[Program]) -> ToolsStatus {
+ let status = |program: Program, found: ToolStatus| {
+ found
+ .with_pin(pinned.contains(&program))
+ .with_download(downloads.get(program))
+ };
let (ffmpeg, ffprobe) = match media::ffmpeg_tools() {
Ok(tools) => (ToolStatus::of(tools.ffmpeg), ToolStatus::of(tools.ffprobe)),
// Found in two places: neither is used, and both say why.
@@ -115,9 +286,10 @@ fn tools_status_with(downloads: &ToolDownloads) -> ToolsStatus {
};
ToolsStatus {
tools_dir: media::tools_dir().map(|dir| dir.display().to_string()),
- ffmpeg: ffmpeg.with_download(downloads.get(Program::Ffmpeg)),
- ffprobe: ffprobe.with_download(downloads.get(Program::Ffprobe)),
- wtsexporter: wtsexporter.with_download(downloads.get(Program::Wtsexporter)),
+ checking: downloads.checking(),
+ ffmpeg: status(Program::Ffmpeg, ffmpeg),
+ ffprobe: status(Program::Ffprobe, ffprobe),
+ wtsexporter: status(Program::Wtsexporter, wtsexporter),
}
}
@@ -125,11 +297,69 @@ fn tools_status_with(downloads: &ToolDownloads) -> ToolsStatus {
mod tests {
use super::*;
+ /// A run waiting for a download is sent marked by the program, with its
+ /// bytes and no counts: the shape `progressDetail` in
+ /// `web/src/screens/import/useImportJob.ts` reads, whose tests feed the
+ /// same JSON.
+ #[test]
+ fn a_run_waiting_for_a_download_is_sent_as_the_program_and_its_bytes() {
+ assert_eq!(
+ serde_json::to_value(waiting_event(
+ "setup",
+ Program::Wtsexporter,
+ 12 * 1024 * 1024,
+ Some(30 * 1024 * 1024)
+ ))
+ .unwrap(),
+ serde_json::json!({
+ "step": "setup",
+ "done": 0,
+ "total": 0,
+ "bytes_done": 12_582_912,
+ "bytes_total": 31_457_280,
+ "waiting": "wtsexporter",
+ })
+ );
+ assert_eq!(
+ serde_json::to_value(waiting_event("media", Program::Ffmpeg, 0, None)).unwrap(),
+ serde_json::json!({
+ "step": "media",
+ "done": 0,
+ "total": 0,
+ "bytes_done": 0,
+ "waiting": "ffmpeg",
+ })
+ );
+ }
+
+ /// The window hears of a wait when it starts, when the program or the
+ /// size changes, and when the download moves by a step, not at every
+ /// read.
+ #[test]
+ fn a_wait_is_sent_when_it_moves_by_a_step() {
+ let first = (Program::Ffmpeg, 0, Some(10_000_000));
+ assert!(worth_sending(None, first));
+ assert!(!worth_sending(
+ Some(first),
+ (Program::Ffmpeg, 64 * 1024, Some(10_000_000))
+ ));
+ assert!(worth_sending(
+ Some(first),
+ (Program::Ffmpeg, WAITING_STEP_BYTES, Some(10_000_000))
+ ));
+ assert!(worth_sending(
+ Some(first),
+ (Program::Ffprobe, 0, Some(10_000_000))
+ ));
+ assert!(worth_sending(Some(first), (Program::Ffmpeg, 0, None)));
+ }
+
/// The window reads `state` and, for a found program, `path`.
#[test]
fn a_status_is_sent_as_its_state_and_path() {
let status = ToolsStatus {
tools_dir: Some("/home/sam/message-crate/tools".into()),
+ checking: true,
ffmpeg: ToolStatus::of(Some(PathBuf::from("/usr/bin/ffmpeg"))),
ffprobe: ToolStatus::of(Some(PathBuf::from("/usr/bin/ffprobe"))),
wtsexporter: ToolStatus::of(None),
@@ -138,6 +368,7 @@ mod tests {
serde_json::to_value(&status).unwrap(),
serde_json::json!({
"toolsDir": "/home/sam/message-crate/tools",
+ "checking": true,
"ffmpeg": { "state": "found", "path": "/usr/bin/ffmpeg" },
"ffprobe": { "state": "found", "path": "/usr/bin/ffprobe" },
"wtsexporter": { "state": "missing" },
@@ -170,12 +401,23 @@ mod tests {
let name = whatsapp_exporter::wtsexporter_file_name();
media::set_tools_dir(Some(tools.path().to_path_buf()));
let downloads = ToolDownloads::default();
- let missing = tools_status_with(&downloads).wtsexporter;
+ let pinned = [Program::Wtsexporter];
+ let missing = tools_status_with(&downloads, &pinned).wtsexporter;
+ let unavailable = tools_status_with(&downloads, &[]).wtsexporter;
media::testutil::write_with_mode(&tools.path().join(name), 0o755);
- let found = tools_status_with(&downloads).wtsexporter;
+ let found = tools_status_with(&downloads, &pinned).wtsexporter;
+ let found_by_hand = tools_status_with(&downloads, &[]).wtsexporter;
media::set_tools_dir(previous);
assert_eq!(missing, ToolStatus::Missing);
+ // No download for this computer: told apart, so the form offers no
+ // Try again; a copy put there by hand is found all the same.
+ assert_eq!(unavailable, ToolStatus::Unavailable);
+ assert_eq!(
+ serde_json::to_value(&unavailable).unwrap(),
+ serde_json::json!({ "state": "unavailable" })
+ );
+ assert_eq!(found_by_hand, found);
assert_eq!(
found,
ToolStatus::Found {
diff --git a/src-tauri/src/commands/upload.rs b/src-tauri/src/commands/upload.rs
index ccfe5e5cf..b3dc017e1 100644
--- a/src-tauri/src/commands/upload.rs
+++ b/src-tauri/src/commands/upload.rs
@@ -29,6 +29,7 @@ fn finished_upload_events(
bytes_done: None,
bytes_total: None,
status: None,
+ waiting: None,
};
let summary = serde_json::json!({
"summary": format!(
@@ -214,6 +215,7 @@ fn forward_upload_event(app: &tauri::AppHandle, event: ProgressEvent) {
bytes_done: None,
bytes_total: None,
status: None,
+ waiting: None,
},
);
}
diff --git a/src-tauri/src/main.rs b/src-tauri/src/main.rs
index ce5690ac0..dad8a6449 100644
--- a/src-tauri/src/main.rs
+++ b/src-tauri/src/main.rs
@@ -60,7 +60,7 @@ fn main() {
// nothing waits for it (`tool_downloads`).
if let Some(home) = dirs::home_dir() {
let tools = app_directories::use_tools_dir_in(&home);
- tool_downloads::start(tools, app.state::().inner().clone());
+ tool_downloads::start(tools, app.state::().inner());
}
let app_data_dir = app.path().app_data_dir()?;
let record = app_data_dir.join(run_directories::RECORD_FILE);
@@ -108,6 +108,7 @@ fn main() {
commands::staging::read_import_run_record,
commands::staging::save_import_run_record,
commands::tools::tools_status,
+ commands::tools::retry_tool_downloads,
]);
builder
diff --git a/src-tauri/src/tool_downloads.rs b/src-tauri/src/tool_downloads.rs
index dacd6fb1f..ab9d926e6 100644
--- a/src-tauri/src/tool_downloads.rs
+++ b/src-tauri/src/tool_downloads.rs
@@ -2,8 +2,11 @@
//! (#1053, `docs/adr/0019`).
//!
//! At every start the app checks the Tools Directory in the background and
-//! downloads what is missing or older than the release pinned here. Nothing
-//! waits for it, and a failed download is tried again at the next start.
+//! downloads what is missing or older than the release pinned here. Login
+//! and browsing don't wait for it, and a failed download is tried again at
+//! the next start, or when **Try again** on the Import form asks
+//! ([`retry`]). An import that needs a program still downloading waits for
+//! it ([`ToolDownloads::wait_for`]).
//!
//! - **ffmpeg and ffprobe** are not downloaded when both are on `PATH`,
//! because a person who installed ffmpeg chose it.
@@ -44,7 +47,9 @@ use std::fmt;
use std::fs::File;
use std::io::{self, Read, Seek, Write};
use std::path::{Path, PathBuf};
-use std::sync::{Arc, Mutex};
+use std::sync::atomic::{AtomicUsize, Ordering};
+use std::sync::{Arc, Condvar, Mutex, PoisonError};
+use std::thread::JoinHandle;
use std::time::Duration;
use sha2::{Digest, Sha256};
@@ -368,6 +373,11 @@ pub enum DownloadError {
CouldNotWrite(String),
/// The program passed its checksum and is in place, and does not run.
DoesNotRun(Program),
+ /// The check's thread could not be started, so nothing was downloaded.
+ CouldNotStart(String),
+ /// The check stopped partway through the download, such as on a panic
+ /// in its thread, so the download never ended.
+ Interrupted,
}
impl fmt::Display for DownloadError {
@@ -399,11 +409,12 @@ impl fmt::Display for DownloadError {
Install it with your package manager instead; the app uses the copy on PATH.",
program = program.name()
),
- Self::DoesNotRun(Program::Wtsexporter) => write!(
- f,
- "wtsexporter in the Tools Directory doesn't run on this computer. \
- See \"Import can't find wtsexporter\" in Troubleshooting at messagecrate.app."
- ),
+ // The run's error and Settings add the troubleshooting section.
+ Self::DoesNotRun(Program::Wtsexporter) => {
+ f.write_str("wtsexporter in the Tools Directory doesn't run on this computer.")
+ }
+ Self::CouldNotStart(why) => write!(f, "The download could not be started: {why}."),
+ Self::Interrupted => f.write_str("The download was interrupted."),
}
}
}
@@ -552,34 +563,216 @@ pub enum DownloadState {
},
}
-/// The download state of each program, shared by the start-up check and
-/// `tools_status`. A program with no entry is not downloading and has not
-/// failed since the app started.
+/// The download state of each program, shared by the checks, `tools_status`
+/// and an import waiting for a program. A program with no entry is not
+/// downloading and has not failed since the app started.
#[derive(Debug, Clone, Default)]
-pub struct ToolDownloads(Arc>>);
+pub struct ToolDownloads(Arc);
+
+/// The states, the signal a waiting import hears when one changes, and how
+/// many checks run in this process.
+#[derive(Debug, Default)]
+struct Shared {
+ states: Mutex>,
+ changed: Condvar,
+ checks: AtomicUsize,
+}
+
+/// One check in this process, from the moment it is started until it ends,
+/// owned by the thread that runs it.
+///
+/// While one exists [`ToolDownloads::checking`] says so, and the window
+/// keeps asking for the status, because a program the check has not yet
+/// looked at shows as missing. When it is dropped, at the end of the check
+/// or on a panic in its thread, every program it marked that is still
+/// downloading becomes failed with [`DownloadError::Interrupted`], so an
+/// import waiting for one wakes and the Import form offers Try again.
+#[derive(Debug)]
+struct CheckRun {
+ downloads: ToolDownloads,
+ marked: Vec,
+}
+
+impl CheckRun {
+ /// Show `program` as downloading, with nothing received yet.
+ fn mark_downloading(&mut self, program: Program) {
+ self.downloads.set(
+ program,
+ DownloadState::Downloading {
+ received: 0,
+ total: None,
+ },
+ );
+ if !self.marked.contains(&program) {
+ self.marked.push(program);
+ }
+ }
+}
+
+impl Drop for CheckRun {
+ fn drop(&mut self) {
+ {
+ let mut states = self.downloads.states();
+ for program in &self.marked {
+ if matches!(states.get(program), Some(DownloadState::Downloading { .. })) {
+ states.insert(
+ *program,
+ DownloadState::Failed {
+ reason: DownloadError::Interrupted.to_string(),
+ },
+ );
+ }
+ }
+ }
+ self.downloads.0.checks.fetch_sub(1, Ordering::SeqCst);
+ self.downloads.0.changed.notify_all();
+ }
+}
+
+/// How often an import waiting for a download looks at its cancel flag when
+/// no state changes.
+const WAIT_TICK: Duration = Duration::from_millis(250);
impl ToolDownloads {
+ fn states(&self) -> std::sync::MutexGuard<'_, HashMap> {
+ self.0.states.lock().unwrap_or_else(PoisonError::into_inner)
+ }
+
+ /// Whether a check runs in this process now, or has been started and not
+ /// yet taken the lock.
+ pub fn checking(&self) -> bool {
+ self.0.checks.load(Ordering::SeqCst) > 0
+ }
+
+ /// Start a check: [`Self::checking`] holds until the returned run is
+ /// dropped.
+ fn begin_check(&self) -> CheckRun {
+ self.0.checks.fetch_add(1, Ordering::SeqCst);
+ CheckRun {
+ downloads: self.clone(),
+ marked: Vec::new(),
+ }
+ }
+
/// Where `program`'s download stands, if anywhere.
pub fn get(&self, program: Program) -> Option {
- self.0
- .lock()
- .expect("tool downloads lock")
- .get(&program)
- .cloned()
+ self.states().get(&program).cloned()
}
fn set(&self, program: Program, state: DownloadState) {
- self.0
- .lock()
- .expect("tool downloads lock")
- .insert(program, state);
+ self.states().insert(program, state);
+ self.0.changed.notify_all();
}
fn clear(&self, program: Program) {
- self.0.lock().expect("tool downloads lock").remove(&program);
+ self.states().remove(&program);
+ self.0.changed.notify_all();
+ }
+
+ /// Wait while any of `programs` is downloading, so an import that needs
+ /// a program still downloading goes on once it has arrived (#1053).
+ ///
+ /// `waiting` hears the program waited for and its progress each time it
+ /// changes, for the import's progress line. `cancelled` is asked at
+ /// least every [`WAIT_TICK`], so a Cancel ends the wait.
+ ///
+ /// Returns at once when none of `programs` is downloading: the import
+ /// then finds the program or reports it missing as it always did.
+ ///
+ /// # Errors
+ ///
+ /// [`WaitError::Failed`] when a download waited for ended failed, with
+ /// its reason; [`WaitError::Cancelled`] when `cancelled` said so.
+ pub fn wait_for(
+ &self,
+ programs: &[Program],
+ cancelled: &dyn Fn() -> bool,
+ waiting: &mut dyn FnMut(Program, u64, Option),
+ ) -> Result<(), WaitError> {
+ let mut waited = Vec::new();
+ let mut states = self.states();
+ loop {
+ let downloading = programs
+ .iter()
+ .find_map(|&program| match states.get(&program) {
+ Some(DownloadState::Downloading { received, total }) => {
+ Some((program, *received, *total))
+ }
+ _ => None,
+ });
+ let Some((program, received, total)) = downloading else {
+ break;
+ };
+ if !waited.contains(&program) {
+ waited.push(program);
+ }
+ drop(states);
+ if cancelled() {
+ return Err(WaitError::Cancelled);
+ }
+ waiting(program, received, total);
+ states = self.states();
+ // Woken by any change, or after a tick to look at the cancel flag.
+ states = self
+ .0
+ .changed
+ .wait_timeout(states, WAIT_TICK)
+ .unwrap_or_else(PoisonError::into_inner)
+ .0;
+ }
+ for program in waited {
+ if let Some(DownloadState::Failed { reason }) = states.get(&program) {
+ return Err(WaitError::Failed {
+ program,
+ reason: reason.clone(),
+ });
+ }
+ }
+ Ok(())
}
}
+/// Why an import's wait for a download ended without the program.
+#[derive(Debug, Clone, PartialEq, Eq)]
+pub enum WaitError {
+ /// The download waited for failed, for `reason`.
+ Failed {
+ /// The program whose download failed.
+ program: Program,
+ /// Why, as [`DownloadError`] says it.
+ reason: String,
+ },
+ /// The import was cancelled while it waited.
+ Cancelled,
+}
+
+impl fmt::Display for WaitError {
+ fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
+ match self {
+ Self::Failed { program, reason } => write!(
+ f,
+ "The {} download failed. {reason} Try again on the Import form downloads it again. {}",
+ program.name(),
+ troubleshooting(*program)
+ ),
+ Self::Cancelled => f.write_str("cancelled"),
+ }
+ }
+}
+
+impl std::error::Error for WaitError {}
+
+/// The sentence that sends a person to the user guide's troubleshooting
+/// section for `program`, which says how to install it or where to put a
+/// copy by hand. The Import form and Settings link the same section.
+pub fn troubleshooting(program: Program) -> String {
+ let section = match program {
+ Program::Ffmpeg | Program::Ffprobe => "ffmpeg or ffprobe not found",
+ Program::Wtsexporter => "Import can't find wtsexporter",
+ };
+ format!("See \"{section}\" in Troubleshooting at messagecrate.app.")
+}
+
/// The state of `program` in place that does not run.
fn does_not_run(program: Program) -> DownloadState {
DownloadState::Failed {
@@ -606,27 +799,138 @@ fn lock_check(dir: &Path) -> io::Result