SlidingWindowAsrManager: spurious text injected at chunk boundary when minContextForConfirmation/confirmationThreshold are zeroed

[janisbelozerovs-dev]

Summary

SlidingWindowAsrManager with CTC vocabulary boosting can inject spurious/duplicated text at a chunk boundary on multi-window audio, when minContextForConfirmation and confirmationThreshold are both set to 0. A plain (non-boosted) decode of the identical audio does not have this problem.

Environment

• FluidAudio 0.15.6 (revision 4dbf4f9)
• macOS, Apple Silicon
• AsrModelVersion TDT (parakeet-tdt-0.6b), CTC vocabulary boosting via CustomVocabularyContext

Repro

Config (mirrors SlidingWindowAsrConfig.default's 11s/2s/2s window, with the two confirmation knobs zeroed):

let config = SlidingWindowAsrConfig(
    chunkSeconds: 11.0,
    hypothesisChunkSeconds: 2.0,
    leftContextSeconds: 2.0,
    rightContextSeconds: 2.0,
    minContextForConfirmation: 0,
    confirmationThreshold: 0,
    language: language)

Zeroing those two is necessary for our use case: they exist to stabilize a live transcript view (don't promote text to "confirmed" until there's enough context/confidence), but we only read the final finish() result once, and CTC rescoring only ever applies to confirmed text (processWindow) — at the real 10s/0.85 defaults, a short dictation never reaches "confirmed" at all, so rescoring silently never fires.

Test audio (macOS say, reproducible by anyone):

say -v Samantha -o test.aiff "This is the first sentence and it should not disappear. Here comes the second sentence with some more words in it. Now we reach the third sentence which continues the thought further. This is the fourth sentence getting close to the end. Finally this is the fifth and last sentence of the test."

~15 seconds — long enough to cross one chunk boundary (chunkSeconds + rightContextSeconds = 13s), so appendSamplesAndProcess fires once during streaming and flushRemaining handles the tail.

Observed vs expected

Boosted decode (SlidingWindowAsrManager + CustomVocabularyContext, config above):

...This is the fourth sentence getting close to the end. Finally, this is
the first one fifth and last sentence of the test.

Plain decode (AsrManager.transcribe, same audio, no boosting):

...This is the fourth sentence getting close to the end. Finally this is
the fifth and last sentence of the test.

"the first one" appears out of nowhere in the boosted output, right at the chunk boundary — those words were never spoken. Not deterministic across all recordings: a longer 22s/3-window test with different sentence content didn't reproduce an analogous artifact at its own boundaries, so this looks timing/alignment-dependent (where a chunk seam happens to land relative to spoken words), not a guaranteed-every-time failure.

Suspected mechanism

Traced into updateTranscriptionState/processWindow in SlidingWindowAsrManager.swift. With minContextForConfirmation/confirmationThreshold both 0, shouldConfirm is true on essentially every window, so volatileTranscript gets promoted into confirmedTranscript on every call rather than only once real confidence/context thresholds are met. That's presumably fine for the live-transcript use case these knobs were designed for (display can always self-correct on the next update), but for a one-shot finish() read, premature promotion at a chunk seam looks like it can let a rescored/misaligned partial hypothesis get baked into the final concatenated text instead of being superseded by a later, more-complete window.

Workaround on our end

We gate boosting on the recording being short enough that a second window can never be constructed at all (duration < chunkSeconds + rightContextSeconds) — under that, SlidingWindowAsrManager always uses exactly one window via flushRemaining(), so this bug class is structurally unreachable. That's a real loss of the accuracy benefit for longer recordings though, not a fix.

Separately, we found UnifiedAsrManager (offline/batch design, same CTC vocabulary-boosting API) does not reproduce this on the same test cases — got the 15s case above, and a 22s/3-window case, word-for-word correct with no duration limit needed. That's what pointed us at the streaming confirm/promote logic specifically rather than the CTC rescoring itself as the likely source.

Question

Is zeroing minContextForConfirmation/confirmationThreshold an unsupported configuration for SlidingWindowAsrManager, or is this a real bug in the confirm/promote logic? If the former, is UnifiedAsrManager the intended manager for a one-shot/offline use case like ours (record complete audio, then transcribe once) rather than SlidingWindowAsrManager, which reads as built more for continuous/live transcription (matches most of the apps in this repo's own README showcase)? Happy to share more repro cases or a minimal reproduction project if useful.