Skip to content

Voices no longer sound under water: libopus conceals in every mode, and the playout is paced - #237

Merged
itz4blitz merged 3 commits into
mainfrom
fix/voice-playout-concealment
Nov 11, 2026
Merged

itz4blitz merged 3 commits into
mainfrom
fix/voice-playout-concealment

Conversation

@itz4blitz

Copy link
Copy Markdown
Collaborator

What changed and why

Other people's voices, in Tiny calls and in Discord calls through tinyclient, went "under water" at random. The cause was the pitch loop that stood in for packets that were lost or not here yet. It stood in for every speech-mode (SILK or hybrid) loss because unsafe-libopus 0.2.0 saturated there. That was a c2rust bug: silk_SAT16's lower bound came out as +32768. With the library patched, libopus conceals in every mode. The jitter buffer is also sized for Wi-Fi's stragglers, and its depth is kept by playing speech a pitch period longer or shorter, so it runs dry less and gives a stall's delay back while the person keeps talking. Every log line now has a timestamp, and each voice logs what its playout made up.

  • vendor/unsafe-libopus: six constants in silk/PLC.rs and silk/CNG.rs, marked tinyclient patch. libopus 1.3.1's silk/PLC.c:374 and silk/CNG.c:176 are the reference. Clean-packet decode is bit-identical to stock in every mode (compared against ffmpeg's real libopus). Concealment after SILK and hybrid no longer saturates: 50,000+ clipped samples per run went to 0.
  • Decoder: libopus conceals losses in every mode. In mid-speech, a packet that is not here yet is carried on by libopus and fades over 60 ms. The mode-switch reset workaround is gone, because it only existed for the same saturation. The pitch loop remains only as a fallback if libopus refuses.
  • Jitter buffer: the target is the larger of today's 3× average jitter and how late one packet in a hundred came over the last 10 s (Arrivals::spread_ms). A stall's backlog counts once, and a pause is not lateness. It releases depth after 3 s calm instead of 5 s, with 10 ms of hysteresis so it doesn't hunt.
  • Pacing (pace.rs): one pitch period is added to a frame when the buffer runs short, or removed when it holds too much in mid-speech. This is what WebRTC's NetEq calls preemptive expand and accelerate. Quiet frames are still dropped whole as before.
  • Logs:
    • Every line starts with its time in UTC. The Logs window reads the level after it and still reads older files.
    • Each voice writes a playout line every 10 s if anything was made up, and one over the whole call. It covers frames played, ticks waited for, losses made up by libopus or by the pitch loop, frames stretched, squeezed or dropped, the longest stand-in, the depth held, the spread and the jitter.

Type

  • feature
  • fix
  • perf
  • refactor
  • chore
  • docs

Platforms

  • windows
  • macos
  • linux
  • android
  • ios

Proof

Scenario: e2e/scenarios/voice-playout

The scenario's build plays 3 s of speech-mode Opus through a real receive stream, arriving as over Wi-Fi: jitter, 1-in-50 stragglers 45 ms late, a 250 ms stall, losses including a double. It passes only if:

  • the pitch loop never ran;
  • nothing saturated;
  • the stragglers didn't stall playback;
  • the stall's backlog was given back in mid-speech.

It looks for voice-playout: proved on stderr, because any mediad exits 0 once its input closes.

Locally on Windows:

  • This branch: passes. 10 ticks waited for, 8 of them during the stall. 2 losses concealed by libopus, 0 by the pitch loop. Peak 6710. 23 frames squeezed, ending at the 100 ms target.
  • Installed 0.2.65: fails.

The mediad suite passes: 290 library tests plus the integration tests.

Lab run: pending.

Compatibility

No wire change. The log line format grew a leading timestamp; the Logs window parses both forms.

Risk and rollback

  • Mode-switch splice: if the patched concealment misbehaved somewhere the tests don't reach, the SILK-to-CELT switch is the likeliest place. That's the silence packet after speech, and a_silence_packet_after_speech_fades_in_instead_of_banging holds it.
  • Pacing: it acts only on frames that repeat at a pitch period, or on quiet ones.
  • Rollback: revert the squash commit.

Release note

Voices in calls no longer go "under water" on a shaky connection, and the logs say when a voice had to be made up.

🤖 Generated with Claude Code

itz4blitz and others added 2 commits November 11, 2026 09:22
…ong can be lined up with what the log says happened then

The Logs window reads the level after the time, and still reads files written before it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…packets in every mode, and the playout is paced instead of looped

- vendor/unsafe-libopus: c2rust turned silk_SAT16's lower bound into +32768, so SILK concealment and comfort noise came out at full scale. mediad played a repeated pitch period there instead, which is the under-water voice. Six constants fixed, marked "tinyclient patch".
- A packet not here yet in mid-speech is carried on by libopus and fades over 60 ms, not looped.
- The jitter buffer is sized from how late one packet in a hundred came over the last 10 s (a stall's backlog counts once), not only from three times the average.
- A frame plays one pitch period longer when the buffer runs short and shorter when it holds more than it needs in mid-speech, so a stall's delay is given back while the person keeps talking.
- Each voice logs what its playout made up every 10 s when anything was, and over the whole call.
- e2e/scenarios/voice-playout: a Wi-Fi-shaped call through a real receive stream.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@itz4blitz itz4blitz mentioned this pull request Nov 11, 2026
1 of 11 tasks
@itz4blitz itz4blitz self-assigned this Nov 11, 2026
@itz4blitz
itz4blitz merged commit 6f5b1b6 into main Nov 11, 2026
7 checks passed
@itz4blitz
itz4blitz deleted the fix/voice-playout-concealment branch November 11, 2026 17:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant