Repository navigation
Voices no longer sound under water: libopus conceals in every mode, and the playout is paced - #237
Merged
Merged
Conversation
…ong can be lined up with what the log says happened then The Logs window reads the level after the time, and still reads files written before it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…packets in every mode, and the playout is paced instead of looped - vendor/unsafe-libopus: c2rust turned silk_SAT16's lower bound into +32768, so SILK concealment and comfort noise came out at full scale. mediad played a repeated pitch period there instead, which is the under-water voice. Six constants fixed, marked "tinyclient patch". - A packet not here yet in mid-speech is carried on by libopus and fades over 60 ms, not looped. - The jitter buffer is sized from how late one packet in a hundred came over the last 10 s (a stall's backlog counts once), not only from three times the average. - A frame plays one pitch period longer when the buffer runs short and shorter when it holds more than it needs in mid-speech, so a stall's delay is given back while the person keeps talking. - Each voice logs what its playout made up every 10 s when anything was, and over the whole call. - e2e/scenarios/voice-playout: a Wi-Fi-shaped call through a real receive stream. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 of 11 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed and why
Other people's voices, in Tiny calls and in Discord calls through tinyclient, went "under water" at random. The cause was the pitch loop that stood in for packets that were lost or not here yet. It stood in for every speech-mode (SILK or hybrid) loss because
unsafe-libopus0.2.0 saturated there. That was a c2rust bug:silk_SAT16's lower bound came out as +32768. With the library patched, libopus conceals in every mode. The jitter buffer is also sized for Wi-Fi's stragglers, and its depth is kept by playing speech a pitch period longer or shorter, so it runs dry less and gives a stall's delay back while the person keeps talking. Every log line now has a timestamp, and each voice logs what its playout made up.silk/PLC.rsandsilk/CNG.rs, markedtinyclient patch. libopus 1.3.1'ssilk/PLC.c:374andsilk/CNG.c:176are the reference. Clean-packet decode is bit-identical to stock in every mode (compared against ffmpeg's real libopus). Concealment after SILK and hybrid no longer saturates: 50,000+ clipped samples per run went to 0.Arrivals::spread_ms). A stall's backlog counts once, and a pause is not lateness. It releases depth after 3 s calm instead of 5 s, with 10 ms of hysteresis so it doesn't hunt.pace.rs): one pitch period is added to a frame when the buffer runs short, or removed when it holds too much in mid-speech. This is what WebRTC's NetEq calls preemptive expand and accelerate. Quiet frames are still dropped whole as before.Type
Platforms
Proof
Scenario: e2e/scenarios/voice-playout
The scenario's build plays 3 s of speech-mode Opus through a real receive stream, arriving as over Wi-Fi: jitter, 1-in-50 stragglers 45 ms late, a 250 ms stall, losses including a double. It passes only if:
It looks for
voice-playout: provedon stderr, because any mediad exits 0 once its input closes.Locally on Windows:
The mediad suite passes: 290 library tests plus the integration tests.
Lab run: pending.
Compatibility
No wire change. The log line format grew a leading timestamp; the Logs window parses both forms.
Risk and rollback
a_silence_packet_after_speech_fades_in_instead_of_bangingholds it.Release note
Voices in calls no longer go "under water" on a shaky connection, and the logs say when a voice had to be made up.
🤖 Generated with Claude Code