Repository navigation
TT-6276 show an estimated time while AI audio work is running - #766
Merged
Merged
Conversation
gtryus
marked this pull request as draft
October 15, 2026 16:14
Recognize Speech, Reduce Noise, and Convert Voice could run for a long time with no indication of how long they would take. Co-authored-by: Cursor <cursoragent@cursor.com>
gtryus
force-pushed
the
TT-6276-ai-time-estimate
branch
from
October 19, 2026 19:48
24b1454 to
b3f93e0
Compare
… each fails on the current code, then fixed the code. All 38 tests in the affected suites pass, typecheck passes, and lint is clean. I haven't committed anything.
I also found a fourth problem: the last commit (TT-6276) broke 5 existing AI tests in [WSAudioPlayer.hotkeys.test.tsx](web/src/components/WSAudioPlayer.hotkeys.test.tsx) with `Cannot read properties of undefined (reading 'replace')`. The test's mocked strings had no `aiEstimatedTime`. The real app does have that string, so only the tests were affected. I added it to the mock.
## The three findings
**1. Selected-region estimate uses the full recording.**
- **Cause:** `wsRegionBlob` sends only the selected region, but the overlay estimated from the whole recording's `duration`.
- **Fix:** I didn't reuse the player's `currentSegment` state, because `clearRegions` and region resizing can change the hook's current region without updating it. Instead, [useWaveSurfer.tsx](web/src/crud/useWaveSurfer.tsx) now has a small `regionSlice()` helper used by both `wsRegionBlob` and a new `wsRegionDuration()`, so the estimate and the upload always cover the same audio.
- In [WSAudioPlayer.tsx](web/src/components/WSAudioPlayer.tsx), `applyAudioAi` saves `wsRegionDuration()` as the estimate when the request starts, and `doingProcess(false)` clears it. Reduce Noise and Convert Voice both go through `applyAudioAi`, so they get the same value.
- Side benefit: Normalize uses the same overlay and used to show an AI estimate. It no longer does.
- **Test:** a 120 s recording with a 10 s region selected. Before the fix it showed `~1:15`; now it shows `~0:20`.
**2. Speech estimate doesn't update when the duration changes.**
- **Cause:** `findRecord` during render breaks CODING_STANDARDS.md, Orbit ("Reactive reads use `useOrbitData`"). In practice, the next 5-second poll would usually correct it.
- **Fix:** [AsrProgress.tsx](web/src/business/asr/AsrProgress.tsx) now reads `useOrbitData('mediafile')` and finds the duration by `mediaId` inside a `useMemo`.
- **Test:** in the new [AsrProgress.test.tsx](web/src/business/asr/AsrProgress.test.tsx), the record is changed in place the way Orbit does it, and the test's `useOrbitData` mock triggers the render. The duration goes from 60 to 120. Before the fix it stayed at `~0:45`; now it shows `~1:15`. The test starts at 60 rather than 0 so it checks only that the display updates, not the fix for #3.
**3. No fallback estimate when the stored duration is 0.**
- **Cause:** a stored duration of 0 can really happen; for example, offline creation does `data.duration || 0`. The `> 0` check hid the `~0:15` fallback in exactly that case.
- **Fix:** the estimate line now always shows. `AsrProgress` is only mounted while a run is starting or active, so there's no idle state where it would need hiding.
- I left the waveform overlay's `> 0` check out of this fix: it's gone anyway now, since the estimate shows whenever one has been set.
- **Test:** duration 0 and no transcription. The request is sent, and the line `Estimated time ~0:15` appears. Before the fix there was no line at all.
gtryus
marked this pull request as ready for review
October 19, 2026 20:58
sarahentzel
approved these changes
Oct 20, 2026
added 3 commits
October 20, 2026 11:28
…ook on that device. The progress bar, the countdown and the "taking longer than usual" message are all built. The tests I added or updated pass (73 across the five affected suites), and typecheck and lint are clean. I haven't run it in the app or committed anything. **How the estimate works** - **Before there's data:** until a device has 5 timed requests of one kind, it uses today's formula (half the audio length plus 15 seconds). - **After that:** it uses the last 50 requests of that kind to work out a fixed startup time plus time per second of audio. It then adds enough margin that 95% of those past requests would have finished in time. I used that instead of mean + 2 standard deviations because slow server startups make the times lopsided, and mean + 2 SD doesn't reliably cover 95% on that kind of data. - **Kinds of request:** noise removal and voice change each have their own history. Transcription has one per ASR model, so different models get separate estimates. - **What gets timed:** from the click until the result is applied, so upload and download are included. A request that times out is recorded too, so slow cases still count. Cancelled and failed requests are not. - **Where it's kept:** in the browser's local storage, per device, since upload speed depends on the device's connection. If storage is missing or corrupt, it falls back to the formula. **What the user sees** - The bar fills toward the estimate but stops at 95%, so it never looks finished before the result arrives. - While within the estimate the text reads "About m:ss remaining", counting down. - Past the estimate, the bar becomes an animated bar and the text reads "Taking longer than usual: m:ss elapsed". The existing 10-minute timeout for noise removal and voice change is unchanged. - For transcription, the real segment count still takes over once the server reports it. - A transcription with no stored duration, or one picked up again after a restart, shows only elapsed time and isn't recorded. **Main files** - [estimateAiTime.ts](web/src/control/estimateAiTime.ts) does the calculation and storage. - [AiTimeProgress.tsx](web/src/components/AiTimeProgress.tsx) is the shared bar, used by both [WSAudioPlayer.tsx](web/src/components/WSAudioPlayer.tsx) and [AsrProgress.tsx](web/src/business/asr/AsrProgress.tsx). **Other changes you'll see in the diff** - The "AI result timed out" error is now its own error type, so a timeout can be recorded and still reported as a failure. Its message is unchanged. - `aiEstimatedTime` is replaced by `aiTimeRemaining` and `aiTakingLonger` in the XLIFF, and the TypeScript and JSON were regenerated. I used a colon rather than a dash in "Taking longer than usual: …" because the English XLIFF is otherwise pure ASCII.
…n. It applies until a device has recorded 5 timings of its own. All 73 tests in the affected suites pass, typecheck passes, and lint shows no errors. I haven't committed anything. | Function | Old fallback | New fallback | 28 s clip | 157 s clip | |---|---|---|---|---| | Noise removal | 0.5 × length + 15 | **0.55 × length + 10** | 26 s (measured 21) | 96 s (measured 91) | | Voice conversion | 0.5 × length + 15 | **0.15 × length + 25** | 29 s (measured 13) | 49 s (measured 31) | | Transcription (every method) | 0.5 × length + 15 | unchanged | | | The new estimates deliberately run a bit above the measured times. That margin covers the extra wait from the app checking for results only every 10 s, plus slower-than-usual requests. Voice conversion has the most margin because its upload time varied the most. What changed: - **[estimateAiTime.ts](web/src/control/estimateAiTime.ts):** `fallbackEstimateSeconds` now takes the function as its first argument and looks up that function's formula. A comment records where the numbers came from and that cold starts are left to the "taking longer than usual" message. All transcription methods share one formula. - **[estimateAiTime.test.ts](web/src/control/estimateAiTime.test.ts):** the tests now cover each function's formula, plus the zero-length case. Three noise-removal tests that expected 75 s now expect 76 s. - **[WSAudioPlayer.hotkeys.test.tsx:961](web/src/components/WSAudioPlayer.hotkeys.test.tsx#L961):** the selected-region test now expects "About 0:16 remaining" instead of "0:20". It still checks that the estimate comes from the 10 s selection, not the 120 s recording.
… to `fallbackEstimateSeconds`. All 16 runs succeeded, and all 75 tests in the affected suites pass, as do typecheck and lint. Nothing is committed. **Transcription timings** (through the app's own route; the media files were already on the server, so there's no upload): | | Omnilingual (`zai`) | Whisper (`eng`) | |---|---|---| | Cold start, 28 s clip | 33 s | 15 s | | Same clip again | 16 s | 8 s | | Extra time from the cold start | +17 s | +8 s | | Warm requests (seconds) | 6.5 + 0.21 × audio length (close fit) | 4.9 + 0.065 × audio length | | 157 s clip | 40 s | 15 s | Whisper took about 6 s on the server for every clip from 29 to 69 s, so length barely matters below about a minute. Whisper was given English for Zapotec audio, though, and it may finish faster when the language doesn't match. I'd check its numbers against a real recording in a language Whisper supports before relying on them. **Fallback formulas now in [estimateAiTime.ts](web/src/control/estimateAiTime.ts):** | Function | Fallback (seconds) | |---|---| | Noise removal | 0.55 × length + 10 | | Voice conversion | 0.15 × length + 25 | | Transcription, omnilingual | 0.21 × length + 13 | | Transcription, whisper | 0.07 × length + 10 | | Transcription, other methods (mms, phonetic) | 0.5 × length + 15 (not measured, unchanged) | Each formula is the measured line plus a margin for the wait between the app's status checks, plus a margin for slower-than-usual requests. Cold starts are left to the "taking longer than usual" message. Across all four functions, the cold start cost noise removal far more (+145 s) than anything else.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
~m:ssshape as the download line.Estimated time ~1:15.Test plan
estimateAiTime.test.ts: 3 passed