feat(desktop): download ffmpeg, ffprobe and wtsexporter into the Tools Directory at start (#1053)

[mbeisser1]

Part of #1053 (step 3). It does not close the issue: step 4 (the Import form's message, Try again, and the wait for a download) and step 5 (the user guide) follow.

Feature Description

The desktop app downloads ffmpeg, ffprobe and wtsexporter into the Tools Directory (~/message-crate/tools) by itself. Each time it starts, a background thread checks the Tools Directory and downloads what is missing or older than the pinned release. Login and browsing don't wait. With no internet the app starts and works, the failure is shown in Settings → System → Media, and the download is tried again at the next start.

• ffmpeg and ffprobe come from eugeneware/ffmpeg-static release b6.1.1. Neither is downloaded when both are on PATH (the step-2 lookup).
• wtsexporter comes from messagecrate/WhatsApp-Chat-Exporter release 0.13.0-mc.2. It is never looked for on PATH.
• Each file is checked against a SHA-256 the app carries. A file that doesn't match is deleted, and the failure says the checksum did not match.
• A file is replaced only after its successor passed. The download goes to a temporary name in the Tools Directory, is checked, unpacked when gzipped, made executable on Unix, and then renamed over the old file.

User Story

As a person who installs the desktop app on a computer without ffmpeg or wtsexporter, I want the app to fetch them by itself, so that a WhatsApp import and Convert work with nothing else installed and nothing clicked.

Implementation Details

The record of what the app wrote. tools/manifest.json holds, per program, the release, asset, SHA-256 and the size of the file written. A file whose size matches its entry is the app's. It is current when the entry names the pinned release, so no program is read again at start-up, and a newer pin replaces it. A file with no entry, or whose size changed, was put there by someone else:

• ffmpeg or ffprobe that answers -version (media::tool_in_dir) is left in place, such as one linked in from Homebrew. One that doesn't run is downloaded over.
• wtsexporter is kept, and entered in the record, only when its SHA-256 is the pinned one. Any other wtsexporter is replaced. The issue said "unless it is wtsexporter older than the pinned release"; the release of a hand-placed wtsexporter can't be told from the file (a pipx shim, a renamed release file), and the import reads only the pinned release's JSON, so "not the pinned file" is the rule. A person offline who puts the pinned file there by hand gets it adopted. A different file stays in use until a download succeeds, because replacement waits for a passing download.

The source. Each pinned file's address is {base}/{repo}/releases/download/{tag}/{asset}. base is https://github.com in the app, and the tests point it at an httpmock server, which src-tauri already has as a dev-dependency.

The asset choice. The ffmpeg-static .gz files are downloaded rather than the plain ones, because they are a third of the size (19 to 30 MB against 45 to 83 MB). The checksum is over the .gz as it arrives, before it is unpacked with flate2. Windows on ARM gets the x64 ffmpeg files; whether they run there is unchecked, as the issue says. Linux on ARM gets no wtsexporter, because the fork publishes none for it.

The checksums. ffmpeg-static publishes no checksum file. Each b6.1.1 .gz asset for linux-x64, linux-arm64, darwin-x64, darwin-arm64 and win32-x64 was downloaded with curl -L on 2026-10-08 and hashed with sha256sum, and gzip -t passed on each. The wtsexporter checksums are copied from the release's SHA256SUMS asset, read on 2026-10-08. The app never fetches SHA256SUMS. All are constants in src-tauri/src/tool_downloads.rs, with a comment saying how they were made.

The HTTP client. reqwest 0.12 blocking, with the workspace's rustls features. The connect timeout is 30 seconds, and a 60-second timeout covers each read, so a stalled download fails rather than hanging. The User-Agent is MessageCrate/<build> (desktop app tool download).

The status. tools_status (tagged by state) gains downloading (received, and total or null) and downloadFailed (reason: no connection, the HTTP status with its reason phrase, checksum mismatch, or could not write). A download in progress is shown over whatever was found. A failed one is shown only when the program is not found, because a program found is still the one used. The Settings screen reads the status through TanStack Query (useQuery, keyed ["desktop", "tools-status"], not account-scoped because the programs belong to the computer) with a refetchInterval of one second while any program is downloading.

Checked by hand. A run of the downloader against the real GitHub releases for linux-x64, not committed, downloaded all three files, passed every checksum and wrote the record. The ffmpeg that arrived answered -version: the b6.1.1 linux-x64 file reports ffmpeg version 7.0.2-static (johnvansickle.com), not 6.1.1, which a reviewer may want to know. wtsexporter answered --help.

The GPL rule (ADR 0014) is not touched. The app downloads a file to the person's computer and links nothing.

macOS quarantine is unchecked. The app doesn't try to clear it, and whether a downloaded program runs on macOS without a security prompt is still unchecked, as #1053 says.

Key Files Changed

• src-tauri/src/tool_downloads.rs (new) - the pinned releases and checksums, the record, the decision per program, the streaming download with its checksum, the replace-after-success, and the background start.
• src-tauri/src/tool_downloads/tests.rs (new) - the checksum refusal, the replace-after-success, ffmpeg on PATH, hand-placed files, no network, and the platform table.
• src-tauri/src/commands/tools.rs - downloading and downloadFailed states, tools_status reads the shared ToolDownloads.
• src-tauri/src/main.rs - manages ToolDownloads and starts the check after the Tools Directory is named.
• crates/libs/media/src/tools.rs - ffmpeg_on_path() and tool_in_dir(); testutil::locate_tools for tests.
• web/src/screens/settings/SystemSection.tsx, web/src/lib/tauri.ts - the two new status lines, and useQuery with polling while a download runs.
• docs/.../settings/system.md, CONTEXT.md, docs/adr/0019 - the download is built; ADR 0019 names the fork as the wtsexporter source, and why.

HTTP API Changes

• None. The change is in a Tauri command, tools_status, which gains two states.

Database Changes

• No schema changes
• schema/sql/*.sql changed (every database rebuilds empty and re-imports; the fingerprint is computed, nothing to bump)

How to Test

1. cargo test --manifest-path src-tauri/Cargo.toml tool runs the downloader tests against a local httpmock server.
2. cd web && npx vitest run src/screens/settings/SystemSection.test.tsx covers the in-progress and failed lines and the polling.
3. In the desktop app on a computer without ffmpeg on PATH, empty ~/message-crate/tools and start the app. Settings → System → Media shows the download's progress, then the three programs found. tools/manifest.json names the releases.
4. With the network off, start the app with an empty Tools Directory. It starts, and Settings shows download failed with no connection as the reason.

Checklist

• Code follows the project's style guidelines
• Self-reviewed the code
• Added unit tests for new functionality
• Added integration tests where applicable
• Existing tests pass locally
• Updated documentation
• ./scripts/check-pr.sh passes and formatter rewrites are committed

Also run: cargo test --manifest-path src-tauri/Cargo.toml (182 passed), cargo test -p media, cd web && npx vitest run (2134 passed), cd docs && npx astro check (0 errors).

Related Issues

Part of #1053 (step 3).

Screenshots / Demo

No browser check. The status lines appear only in the desktop app, and the Vitest tests cover the in-progress and failed rows.

Deployment Notes

New src-tauri dependencies: reqwest 0.12 (blocking, rustls; already in the workspace lockfile), sha2 0.11, hex 0.4, flate2 1.1. Not tested: the download on macOS and Windows, and the rename over a running .exe on Windows, which would fail as "could not write" and be retried at the next start.

🤖 Generated with Claude Code
https://claude.ai/code/session_017gbWRfTT5AEWW1D6hTNDr1

[mbeisser1]

Review of 485d442 (up to date with main). Findings whose line is outside the diff follow; the rest are line comments.

• docs/src/content/docs/docs/user/import-sources/whatsapp.md — Standards · rule — whatsapp.md:11,26-38 is outside the diff and now describes the code wrongly: it says wtsexporter "must be installed separately" and tells the reader to link a pipx install into the Tools Directory. Under this PR's rule (a hand-placed wtsexporter is replaced unless it is the pinned file, system.md:62) a pipx link never matches the pinned checksum, so the app replaces it at every start. troubleshooting.md:99 ("which the desktop app doesn't include") and attachments-and-media.md:76-81 also predate the download. Step 5 rewrites these pages, but a page that now tells the reader to do something the app undoes is a broken instruction: cut the pipx step to a one-line note that the app downloads wtsexporter, or make the pages true in the minimal way.
• docs/adr/0019-the-desktop-app-downloads-the-programs-it-needs.md:84 — Standards · rule — The ADR says a newer pin "replaces what is there, whoever put it there". The code (need, tool_downloads.rs around line 380) and system.md:59 keep a hand-placed ffmpeg or ffprobe while it runs. The ADR and the code disagree and the PR changes neither to match the other. (Spec reports the same rule as reversed; the fix is in the code, see that thread.)

[mbeisser1]

Re-review of the fix commits (Standards and Correctness, step 4 of the review), pinned to d8f8571. Every finding below is fixed in this push.

Correctness · bug src-tauri/src/tool_downloads.rs:276: a program that downloads but does not run was recorded and downloaded again at every start, for good. Fixed in 16d6ecb: a recorded pinned program that doesn't run shows the failure without downloading; a new pin or a changed stamp starts a download. Test: a second start downloads nothing.
Correctness · bug tool_downloads.rs:621: ffmpeg and ffprobe could never be adopted (their pins are the .gz hashes), so a lost record or a changed stamp re-downloaded them at every start. Fixed in d017884: each pin and the record carry the unpacked program's SHA-256 (computed from the ten b6.1.1 assets with curl -L, gunzip -c, sha256sum); a file with no entry or a changed stamp is adopted when its hash matches; the download also checks the unpacked file. Tests for adoption, re-recording and an unpacked mismatch.
Correctness · bug tool_downloads.rs:87: two app instances deleted each other's temp files and raced on the record. Fixed in fe103ef: the check holds an exclusive lock on .check.lock in the Tools Directory (std File::try_lock); a second check returns at once; a filesystem without locks runs unlocked. Two-thread test.
Correctness · nit tool_downloads.rs:398,404: "was downloaded but doesn't run" was untrue for an adopted hand-placed file. Fixed in d8f8571: " in the Tools Directory doesn't run on this computer".
Standards · judgement system.md:59-60, whatsapp.md:18: the "one rule" overstated adoption and the two pages disagreed. Fixed in 25756e0 (after d017884 the rule is one rule for all three programs).
Standards · rule pins.rs:2-3: "a change to this file alone" was untrue. Fixed in 220fc4a: the user page and the doc text no longer name the tags; the comment names the places a wtsexporter release name still appears (the install hints, AGENTS.md, contributing.md).
Standards · judgement tool_downloads.rs:374: the reason had no contraction and said nothing a person can do. Fixed in 5532870 and d8f8571: ffmpeg and ffprobe point at the package manager (the app uses a copy on PATH), wtsexporter at Troubleshooting.
Standards · judgement system.md:81: the Homebrew sentence. Fixed in 04f63d6.
Standards · judgement attachments-and-media.md:86: stale "Put it in the Tools Directory" advice. Fixed in 36fef7d.
Standards · judgement ADR 0019:92: "Since 2026-10-02". Fixed in 9698bd2: "Since #2001".

[mbeisser1]

Review summary for d8f8571.

• Standards: 11 findings, all Fixed.
• Spec: 4 findings, all Fixed (the one replacement rule for all three programs; progress shown from the first moment; the ffmpeg version the b6.1.1 release really ships, checked by downloading the Linux asset: its checksum matches the one carried, the build is GPL with libx264 and libx265, and it reports ffmpeg 7.0.2; temp files of an interrupted download swept at start).
• Correctness: 2 findings, both Fixed.
• Re-review of the fixes: Standards 7 and Correctness 4 (a re-download loop, ffmpeg and ffprobe never adoptable, two app instances racing, and a wording nit), all Fixed, posted in the second review pinned to d8f8571.
• Merges of the base: none; the PR was up to date with main throughout.
• CI: the ci.yml run on d8f8571 ended in success, with no commit needed for it. The separate audit.yml "Audit npm deps" job fails on the docs site for the sharp advisory already filed as npm audit fails on the docs site: sharp below 0.35.5 (GHSA-wq5f-xc86-pv6w) #2000; this PR changes no lockfile.
• Leftover filed: Flaky desktop test: an_unreachable_server_is_reported_without_the_media_link can lose its free port #2002, a flaky desktop test (an_unreachable_server_is_reported_without_the_media_link) that predates this PR and failed once in four local runs.
• No user thread is open. Step 4 of The desktop app downloads ffmpeg and wtsexporter into the Tools Directory #1053 (the Import form's missing-program message, Try again, and the wait for a download) is next, as its own PR.