Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
485d442
feat(desktop): download ffmpeg, ffprobe and wtsexporter into the Tool…
mbeisser1 Oct 8, 2026
0b73638
docs(adr): 0019 keeps its decision text and records the fork in an am…
mbeisser1 Oct 8, 2026
3534888
refactor(desktop): the pinned releases and checksums live in tool_dow…
mbeisser1 Oct 8, 2026
a6dfae0
refactor(media): one function names the PATH searched, and tool_in_di…
mbeisser1 Oct 8, 2026
7f2800d
fix(desktop): one replacement rule for ffmpeg, ffprobe and wtsexporter
mbeisser1 Oct 8, 2026
4f3a42b
fix(desktop): the start-up check deletes what an interrupted download…
mbeisser1 Oct 8, 2026
7011a20
refactor(desktop): one failure loop, and the retry said once on the f…
mbeisser1 Oct 8, 2026
6639f3f
fix(desktop): every program to download shows as downloading from the…
mbeisser1 Oct 8, 2026
a33ec23
fix(desktop): a program not used keeps its reason over a failed download
mbeisser1 Oct 8, 2026
ef83f6e
test(desktop): the platform pin test no longer restates the release n…
mbeisser1 Oct 8, 2026
ac727bf
refactor(web): the tools status key is built in queryKeys.ts
mbeisser1 Oct 8, 2026
a107bc5
docs(changelog): the download entry is in reader terms
mbeisser1 Oct 8, 2026
cda5e87
docs(user): the WhatsApp, troubleshooting and media pages say the app…
mbeisser1 Oct 8, 2026
d017884
fix(desktop): ffmpeg and ffprobe are adopted by the program's checksum
mbeisser1 Oct 8, 2026
16d6ecb
fix(desktop): a pinned program that doesn't run is not downloaded again
mbeisser1 Oct 8, 2026
5532870
fix(desktop): the "doesn't run" failure names the program and what to do
mbeisser1 Oct 8, 2026
fe103ef
fix(desktop): one start-up check at a time in the Tools Directory
mbeisser1 Oct 8, 2026
220fc4a
docs(desktop): the pinned release tags are named in pins.rs, not repe…
mbeisser1 Oct 8, 2026
25756e0
docs(user): System and WhatsApp state the one Tools Directory rule alike
mbeisser1 Oct 8, 2026
04f63d6
docs(user): the Homebrew note says the app downloads its own ffmpeg
mbeisser1 Oct 8, 2026
36fef7d
docs(user): the media page says the app downloads ffmpeg, not to put …
mbeisser1 Oct 8, 2026
9698bd2
docs(adr): 0019's amendment names the change, not a date
mbeisser1 Oct 8, 2026
d8f8571
fix(desktop): a program that doesn't run is named by where it is, not…
mbeisser1 Oct 8, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,19 @@ released versions carry their date on the heading.

### Features

- 2026-10-08: **The desktop app downloads ffmpeg, ffprobe and wtsexporter
by itself.** Each time it starts, it downloads in the background whatever
is missing from its Tools Directory, with no button and nothing to wait
for. ffmpeg and ffprobe aren't downloaded when both are already installed
on the system path. Each program comes from a pinned release of its project,
wtsexporter from Message Crate's own copy of the WhatsApp Chat Exporter,
and every file is checked against a checksum the app carries. A file that
doesn't match is deleted. A program in the Tools Directory that isn't the
pinned one is replaced, but only once the new file has passed its check,
so with no internet the old one is still used. Settings → System shows
each download's progress, or why it failed, and a failed download is
tried again at the next start (#1053).

- 2026-10-08: **A phone number written without its country code can be given
its country.** The import form has a **Phone's country** under
**Processing Options (Advanced)**: every number the import writes without a
Expand Down
7 changes: 4 additions & 3 deletions CONTEXT.md
Original file line number Diff line number Diff line change
Expand Up @@ -537,9 +537,10 @@ The directory inside the Message Crate Directory where the desktop app keeps
the programs it downloads for itself: ffmpeg, ffprobe and wtsexporter. The
app owns its contents and replaces them when a release needs a newer
version. ffmpeg and ffprobe are looked for on `PATH` and then here,
wtsexporter only here, by the app and by the server it starts. The download
is decided, not built yet: #1053 tracks the work, and until it lands the app
downloads nothing.
wtsexporter only here, by the app and by the server it starts. The app
downloads what is missing here at every start, each file pinned to one
release and checksum, and keeps a record beside the programs of what it
wrote.
_Avoid_: Tools Folder, Bin Directory, ffmpeg directory

**Apple Messages Reader**:
Expand Down
4 changes: 2 additions & 2 deletions crates/libs/media/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -27,8 +27,8 @@ pub use process::{
transcode_file, transcode_file_as,
};
pub use tools::{
FfmpegTools, ffmpeg_available, ffmpeg_path, ffmpeg_tools, ffprobe_path, require_ffmpeg,
set_tools_dir, tools_dir,
FfmpegTools, ffmpeg_available, ffmpeg_on_path, ffmpeg_path, ffmpeg_tools, ffprobe_path,
require_ffmpeg, set_tools_dir, tool_in_dir, tools_dir,
};
pub use versions::{
THUMBNAIL_LONG_EDGE, browser_shows, make_preview, make_thumbnail, media_type_of,
Expand Down
11 changes: 11 additions & 0 deletions crates/libs/media/src/testutil.rs
Original file line number Diff line number Diff line change
Expand Up @@ -133,6 +133,17 @@ pub fn hide_ffmpeg() -> ToolsHidden {
}
}

/// Search `search_path` instead of the process's `PATH`, and `tools_dir`
/// as the Tools Directory, until the returned guard is dropped. Holds
/// [`tools_test_lock`], as [`hide_ffmpeg`] does.
#[must_use]
pub fn locate_tools(search_path: OsString, tools_dir: PathBuf) -> ToolsHidden {
let hidden = hide_ffmpeg();
set_search_path(Some(search_path));
set_tools_dir(Some(tools_dir));
hidden
}

impl Drop for ToolsHidden {
fn drop(&mut self) {
set_search_path(self.previous_search_path.take());
Expand Down
41 changes: 30 additions & 11 deletions crates/libs/media/src/tools.rs
Original file line number Diff line number Diff line change
Expand Up @@ -178,6 +178,27 @@ pub fn ffmpeg_available() -> bool {
resolve_tools().is_ok_and(|tools| tools.missing().is_empty())
}

/// True when ffmpeg and ffprobe are both on `PATH` and answer `-version`.
/// The desktop app downloads neither then, because a person who installed
/// ffmpeg chose it (`docs/adr/0019`).
pub fn ffmpeg_on_path() -> bool {
let search_path = searched_path();
["ffmpeg", "ffprobe"]
.iter()
.all(|name| find_on_path(name, search_path.as_deref()).is_some())
}

/// The `PATH` the lookup searches: the one a test put in place
/// ([`set_search_path`]), else the process's own.
fn searched_path() -> Option<OsString> {
tools_state()
.lock()
.expect("tools state lock")
.search_path
.clone()
.or_else(|| std::env::var_os("PATH"))
}

/// True when ffprobe is found.
pub(crate) fn ffprobe_available() -> bool {
ffprobe_path().is_some()
Expand Down Expand Up @@ -227,16 +248,14 @@ fn command_runs(bin: &Path, args: &[&str]) -> bool {
/// arrives, moves or leaves is seen at once without a restart; only the
/// `-version` answer of a file is kept ([`ToolsState::answered`]).
fn resolve_tools() -> Result<FfmpegTools> {
let (search_path, tools_dir) = {
let state = tools_state().lock().expect("tools state lock");
(state.search_path.clone(), state.tools_dir.clone())
};
let search_path = search_path.or_else(|| std::env::var_os("PATH"));
find_tools(search_path.as_deref(), tools_dir.as_deref())
let search_path = searched_path();
find_tools(search_path.as_deref(), tools_dir().as_deref())
}

/// The tool under `dir` if it is a file and runs.
fn find_tool_in_dir(dir: &Path, name: &str) -> Option<PathBuf> {
/// The program `name` in `dir`, `.exe` added on Windows, when it is a file
/// that answers `-version`. The desktop app also asks this of a program in
/// the Tools Directory, to know that the file it keeps there runs.
pub fn tool_in_dir(dir: &Path, name: &str) -> Option<PathBuf> {
let candidate = dir.join(executable_name(name));
let metadata = std::fs::metadata(&candidate).ok()?;
if !metadata.is_file() {
Expand Down Expand Up @@ -273,7 +292,7 @@ fn find_on_path(name: &str, search_path: Option<&OsStr>) -> Option<PathBuf> {
.into_iter()
.flat_map(std::env::split_paths)
.filter(|dir| !dir.as_os_str().is_empty())
.find_map(|dir| find_tool_in_dir(&dir, name))
.find_map(|dir| tool_in_dir(&dir, name))
}

/// Find ffmpeg and ffprobe on `search_path`, then in `tools_dir`, and
Expand All @@ -296,8 +315,8 @@ fn find_tools(search_path: Option<&OsStr>, tools_dir: Option<&Path>) -> Result<F
ffprobe: path_ffprobe,
});
}
let dir_ffmpeg = tools_dir.and_then(|dir| find_tool_in_dir(dir, "ffmpeg"));
let dir_ffprobe = tools_dir.and_then(|dir| find_tool_in_dir(dir, "ffprobe"));
let dir_ffmpeg = tools_dir.and_then(|dir| tool_in_dir(dir, "ffmpeg"));
let dir_ffprobe = tools_dir.and_then(|dir| tool_in_dir(dir, "ffprobe"));
if dir_ffmpeg.is_some() && dir_ffprobe.is_some() {
return Ok(FfmpegTools {
ffmpeg: dir_ffmpeg,
Expand Down
15 changes: 13 additions & 2 deletions docs/adr/0019-the-desktop-app-downloads-the-programs-it-needs.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,8 @@ release and one checksum, and a file that doesn't match is refused.
wtsexporter is always the app's own copy. ffmpeg is not downloaded when it is
on `PATH`.

Decided, not built yet: #1053 tracks the work, and until it lands the app
downloads nothing.
Built in #2001. The app records beside the programs which release and
checksum each file it wrote came from.

## Why

Expand Down Expand Up @@ -86,3 +86,14 @@ release and checksum for every platform, in one pull request.

ffmpeg from `PATH` is whatever version the person installed, so the app can
run a version no release was tested with.

## Amended 2026-10-08: wtsexporter comes from Message Crate's fork

The text above is kept as it was decided. Since #2001, the wtsexporter
the app downloads is a pinned release of Message Crate's fork,
`messagecrate/WhatsApp-Chat-Exporter`, and not of
`KnugiHK/WhatsApp-Chat-Exporter`. The fork adds to its JSON who sent each
message and who is in each group, because the original's JSON does not say
who sent a group message and the import needs to know. The fork's changes stay
in the fork, and the fork is archived once the original has everything
Message Crate needs (#1053). Nothing else in this decision changed.
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,9 @@ The second is a video that is already H.265, within the target resolution, at 12

## ffmpeg

**Convert** and **Compress & Convert** run the programs `ffmpeg` and `ffprobe`, which the desktop app doesn't include.
**Convert** and **Compress & Convert** run the programs `ffmpeg` and `ffprobe`.
The desktop app uses both from `PATH` when they are installed there, and otherwise downloads its own into its Tools Directory when it starts.
These commands install both on `PATH`:

| Windows | Linux | macOS |
|---|---|---|
Expand All @@ -78,11 +80,12 @@ The second is a video that is already H.265, within the target resolution, at 12
[ffmpeg.org](https://ffmpeg.org/download.html) has downloads for systems those commands don't cover.

The desktop app finds the programs on `PATH`, else in its Tools Directory, and takes both from the same one.
When they are somewhere else, both go in the Tools Directory, which [**Settings → System**](/docs/user/features/settings/system/#media) shows under **Media**.
There each program reads `Found` with its path, `not found`, or `not used` with the reason.
[**Settings → System**](/docs/user/features/settings/system/#media) shows each under **Media**: `Found` with its path, its download's progress, why the download failed, `not found`, or `not used` with the reason.

An Import Run looks for ffmpeg at the Staging Review, not when the run starts, because Staging copies the original files and needs neither program.
When ffmpeg or ffprobe can't be used, the review names it, such as `Media can't use ffprobe. Put it in the Tools Directory, which Settings → System shows with the reason, then come back to Import.`, and the **Convert media** or **Compress media** button is disabled.
When ffmpeg or ffprobe can't be used, the review names it, such as `Media can't use ffprobe.`, and the **Convert media** or **Compress media** button is disabled.
The desktop app downloads both programs into the Tools Directory when it starts, so a program missing at the review is usually still downloading.
When **Settings → System** says the download failed, the package manager commands above install ffmpeg and ffprobe instead.
The run keeps waiting at the review until the program is there.

## The size limit
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -96,8 +96,8 @@ Import continues once the field is empty.

### Import can't find wtsexporter

A WhatsApp import runs a separate program, `wtsexporter`, which the desktop app doesn't include.
[WhatsApp](/docs/user/import-sources/whatsapp/#install-wtsexporter) has the install command and says how the app finds the program.
A WhatsApp import runs a separate program, `wtsexporter`, which the desktop app downloads when it starts.
[**Settings → System**](/docs/user/features/settings/system/#media) shows whether it arrived, or why the download failed, and [WhatsApp](/docs/user/import-sources/whatsapp/#wtsexporter) says where it comes from.

### A WhatsApp import doesn't stop when cancelled

Expand Down
25 changes: 21 additions & 4 deletions docs/src/content/docs/docs/user/features/settings/system.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: System
description: What the System tab of Settings holds, the Staging Directory, remembered importer paths, where ffmpeg, ffprobe and wtsexporter were found, the Export Directory, the data directory of the app's own Message Crate, and the app's version.
description: What the System tab of Settings holds, the Staging Directory, remembered importer paths, where ffmpeg, ffprobe and wtsexporter were found and how their download stands, the Export Directory, the data directory of the app's own Message Crate, and the app's version.
---

The **System** tab of **Settings** holds the settings of one installed desktop app.
Expand Down Expand Up @@ -48,6 +48,16 @@ When it is on, Import restores the last backup path used for each import source.
`ffmpeg` and `ffprobe` convert and compress attachments, and are looked for on the system `PATH`, then in the Tools Directory.
`wtsexporter` reads WhatsApp backups, and is looked for in the Tools Directory only.

Each time it starts, the desktop app downloads into the Tools Directory the programs that are missing there, in the background.
Login and browsing don't wait for it.
`ffmpeg` and `ffprobe` are not downloaded when both are on `PATH`, because the copy installed there is the one used.
`ffmpeg` and `ffprobe` come from a pinned release of `eugeneware/ffmpeg-static`, and `wtsexporter` from a pinned release of Message Crate's fork, `messagecrate/WhatsApp-Chat-Exporter`.
The app carries the SHA-256 checksum of every file it downloads, and refuses and deletes a file that doesn't match.

The Tools Directory belongs to the app, and one rule holds for all three programs.
A program in the Tools Directory whose checksum is the pinned program's is kept, whoever put it there, and anything else under its name is replaced by the pinned program.
The old file is replaced only after the new one has arrived and passed its checksum, so with no internet, or after a failed download, the old one stays in use.

**Tools directory** shows the full path of the Tools Directory, `tools` in the Message Crate Directory, such as `/home/sam/message-crate/tools` on Linux.

One line per program reports the result:
Expand All @@ -59,9 +69,16 @@ One line per program reports the result:
- A cross with `<program> not used` and the reason.
For ffmpeg and ffprobe, the reason is that they were found in two places, one only on `PATH` and the other only in the Tools Directory.
For wtsexporter, the reason is that it has no permission to run.

On macOS, an app opened from the Dock or Finder doesn't see the `PATH` a terminal sets, so ffmpeg and ffprobe installed with Homebrew are not found.
Linking both into the Tools Directory makes them found, for example `ln -s /opt/homebrew/bin/ffmpeg /opt/homebrew/bin/ffprobe ~/message-crate/tools/` (`/usr/local/bin` on an Intel Mac).
- A download arrow with `Downloading <program>` and how much has arrived, such as `12 MB of 29 MB (41%)`.
The line updates each second until the download ends.
- A cross with `<program> download failed` and the reason: no connection to the download's server, the status the server answered, a checksum that didn't match, a file that couldn't be written to the Tools Directory, or a program in the Tools Directory that doesn't run on this computer.
A failed download is tried again the next time the app starts.
A program that doesn't run is not downloaded again, because the download would be the same file.
For ffmpeg and ffprobe, the reason says to install them with a package manager, because the app uses the copy on `PATH`.
While an older copy is in place, the line shows that copy as found instead, because the old copy is still the one used.

On macOS, an app opened from the Dock or Finder doesn't see the `PATH` Homebrew sets in a terminal, so it doesn't find ffmpeg and ffprobe installed with Homebrew.
It downloads its own copies into the Tools Directory instead, and nothing needs doing.

[Attachments and media](/docs/user/features/messages/attachments-and-media/) covers what ffmpeg and ffprobe are used for.

Expand Down
34 changes: 7 additions & 27 deletions docs/src/content/docs/docs/user/import-sources/whatsapp.md
Original file line number Diff line number Diff line change
@@ -1,42 +1,22 @@
---
title: WhatsApp
description: Import WhatsApp from an Android phone or an iPhone backup. Advanced, because it needs a separate program and, on Android, a decryption key.
description: Import WhatsApp from an Android phone or an iPhone backup. Advanced, because it runs a separate program the desktop app downloads and, on Android, needs a decryption key.
---

WhatsApp is the hardest source to import.
It belongs after a first import of [iPhone](/docs/user/your-messages/back-up-an-iphone/) or [Android](/docs/user/your-messages/back-up-an-android-phone/) messages has worked, not before.

Two things make it hard:

- The desktop app doesn't include the program that reads WhatsApp data. It must be installed separately ([issue 940](https://github.com/messagecrate/message-crate/issues/940)).
- The program that reads WhatsApp data, `wtsexporter`, isn't part of the desktop app. The app downloads it when it starts, so the first WhatsApp import needs the app to have started once with an internet connection.
- On Android, WhatsApp's backup file is encrypted, and Import needs the key.

## Install wtsexporter
## wtsexporter

The desktop app runs `wtsexporter`, from the open-source [WhatsApp Chat Exporter](https://github.com/KnugiHK/WhatsApp-Chat-Exporter) project.
Install it from Message Crate's fork, [messagecrate/WhatsApp-Chat-Exporter](https://github.com/messagecrate/WhatsApp-Chat-Exporter), at release `0.13.0-mc.2`, because only the fork records the ids a quoted reply is linked by.
It needs Python and [`pipx`](https://pipx.pypa.io/).

```bash title="Install wtsexporter"
pipx install --force "whatsapp-chat-exporter[android_backup,crypt15] @ https://github.com/messagecrate/WhatsApp-Chat-Exporter/archive/refs/tags/0.13.0-mc.2.tar.gz"
```

`wtsexporter --help` prints the program's options once it is installed.

The desktop app runs `wtsexporter` only from its Tools Directory, which [**Settings → System**](/docs/user/features/settings/system/#media) shows, and never from `PATH`.
After the install, a link in the Tools Directory points to the `pipx` install of `wtsexporter`:

```bash title="Linux and macOS"
mkdir -p ~/message-crate/tools
ln -sf "$(pipx environment --value PIPX_BIN_DIR)/wtsexporter" ~/message-crate/tools/
```

On Windows, `wtsexporter.exe` is copied there instead:

```powershell title="Windows"
New-Item -ItemType Directory -Force "$HOME\message-crate\tools"
Copy-Item "$(pipx environment --value PIPX_BIN_DIR)\wtsexporter.exe" "$HOME\message-crate\tools\"
```
The desktop app runs `wtsexporter`, from Message Crate's fork of the open-source [WhatsApp Chat Exporter](https://github.com/KnugiHK/WhatsApp-Chat-Exporter), [messagecrate/WhatsApp-Chat-Exporter](https://github.com/messagecrate/WhatsApp-Chat-Exporter), because only the fork records the ids a quoted reply is linked by.
Each time it starts, the desktop app downloads the pinned release into its Tools Directory when it isn't there, in the background, and [**Settings → System**](/docs/user/features/settings/system/#media) shows the program or how its download stands.
The app runs `wtsexporter` only from the Tools Directory, never from `PATH`.
A program in the Tools Directory whose checksum is the pinned program's is kept, whoever put it there, and anything else under its name is replaced by the pinned program.

## WhatsApp on Android

Expand Down
4 changes: 4 additions & 0 deletions src-tauri/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

8 changes: 8 additions & 0 deletions src-tauri/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,14 @@ message-crate-serve-protocol = { path = "../crates/libs/serve-protocol" }
# renamed onto the file the person chose (`commands::download`).
tempfile = "3"

# The start-up download of ffmpeg, ffprobe and wtsexporter into the Tools
# Directory (`tool_downloads`): the HTTP client, the SHA-256 each file is
# checked against, and the gzip the ffmpeg-static files are published in.
reqwest = { version = "0.12.25", default-features = false, features = ["blocking", "rustls-tls", "rustls-tls-native-roots"] }
sha2 = "0.11.0"
hex = "0.4.3"
flate2 = "1.1.9"

# `killpg`, to kill the server the app started with every process the server
# started (`local_server::process_tree`).
[target.'cfg(unix)'.dependencies]
Expand Down
Loading
Loading