fix(voice): place recorded audio where it actually played - #2327
Open
longcw wants to merge 2 commits into
Open
Conversation
RecorderIO rebuilt the agent channel by inference: it collected the frames it forwarded, placed them from the moment playbackFinished arrived, and padded the difference with silence. A sink that ran dry inside a segment, a late flush, audio discarded on pause(), or a microphone that stopped delivering each moved the audio, and none of them recorded the gap. AudioOutput now carries playbackProgressed next to playbackStarted and playbackFinished, and ParticipantAudioOutput reports one run at a time. RecorderIO places both channels on one absolute timeline and leaves unwritten time silent, falling back to the segment endpoints for a sink that reports nothing. That removes the pause reconstruction, the end-anchored padding, and the playbackPosition truncation. Ports livekit/agents#6921.
🦋 Changeset detectedLatest commit: 831e6b9 The changes in this PR will be included in the next version bump. This PR includes changesets to release 39 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
…amps Every existing case calls onPlaybackProgressed directly on the object under test, so the listener AudioOutput registers on nextInChain had no coverage. A broken registration there leaves the whole suite green and puts every recorder silently on the segment-endpoint fallback. The new case sends a report from a leaf two levels below the recorder and asserts the audio lands on the sink's own timestamp. The write task had no coverage either, so neither the stall timeout for a source that went quiet nor the pendingSince hold for a segment in flight was tested. WRITE_INTERVAL_MS and INPUT_STALL_TIMEOUT_MS are exported so the test reads the same constants the loop does; neither reaches the public entry point.
davidzhao
approved these changes
Sep 12, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Ports livekit/agents#6921, merged upstream as
d3621006.Problem
RecorderIO rebuilds the agent channel by inference.
It collects the frames it forwarded, places them from the moment
playbackFinishedarrives, and pads the difference with silence.Nothing measures where the audio really went.
Four conditions break that inference: a sink that runs dry mid-segment, a late flush, audio discarded on
pause(), and a microphone that stops.Each one moves the audio, and none of them records the gap.
A slow TTS or a muted participant can hit it.
Fix
A sink that owns its playback device now reports where its audio went.
AudioOutputgainsplaybackProgressednext toplaybackStartedandplaybackFinished, andParticipantAudioOutputreports one run at a time.Each report is past tense, so it is never revised, and a jump in its offset describes audio the sink discarded rather than played.
RecorderIO places both channels on one absolute timeline and leaves unwritten time silent.
When a sink reports nothing, the recorder uses its segment endpoints instead, which is all a remote avatar worker can know.
That removes the pause reconstruction, the end-anchored padding, and the
playbackPositiontruncation.Two behaviour changes follow from the absolute timeline:
close()now always writes the settled tail, so a silent session still produces a file, and the recording only extends as far as the wall clock does.The checked-in
agents.api.mdis stale onmain, so this change adds only the lines for the new event.