Skip to content

fix(video): prime rotated muxers with the last muxed audio frame - #340

Merged
superuser404notfound merged 1 commit into
superuser404notfound:mainfrom
tschuegy:fix/live-rotation-audio-prime
Aug 9, 2026
Merged

fix(video): prime rotated muxers with the last muxed audio frame#340
superuser404notfound merged 1 commit into
superuser404notfound:mainfrom
tschuegy:fix/live-rotation-audio-prime

Conversation

@tschuegy

@tschuegy tschuegy commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

Problem

A live session with bridged E-AC-3 audio dies with muxerFailed whenever a mid-session muxer rotation (same-PID parameter-set change after a reconnect join, or an SSAI program switch) cuts its first segment before any post-seam audio packet has been muxed.

Field trace (live TV, reconnect join into a new encoder session — DTS jump + SPS/PPS change):

[HLSSegmentProducer] live video timeline rebase: jumpTicks=5287440972 ...
[HLSSegmentProducer] same-PID in-band parameter-set change #1 on video PID stream=0 (1920x1080, 51->43 B SPS/PPS); forcing a discontinuity cut + versioned init ...
[HLSSegmentProducer] muxer rotation (same-PID parameter-set change): seg-413 finalized on old init, fresh versioned init for seg-414 (1920x1080)
[HLSSegmentProducer] AE#222 seg-414.m4s cut deferred: audio sample entry needs a parsed packet and none has been muxed; scanning forward for a prime frame
[HLSSegmentProducer] seg-414.m4s final finalize failed; not adopted
[HLSSegmentProducer] pump finished: reason=muxerFailed packetsRead=39549 packetsWritten=24320 lastError=0

Why it happens:

  • Rotation allocates a brand-new MP4SegmentMuxer; E-AC-3/AC-3/TrueHD need a parsed packet for their mp4 sample entry (dec3/dac3/dmlp), so the fresh muxer starts un-primed.
  • The only moov-prime carry-in was the construction-time audioMoovPrimeFrame let — nil for every session that did not start audio-less.
  • Video leads audio across a reconnect seam and the bridge adds encoder latency, so the rotated muxer's first (0.5 s) cut arrives before any post-seam audio exists. The cut defers (AE#222), the producer converts the deferral into a pump exit, and teardown finalize fails on the same precondition.
  • The AE#222 exit-scan cannot convert the exit to .needsAudioSampleEntryPrime: it rightly excludes bridged sessions (a raw source frame cannot prime a bridge-encoded track), so the reason stays muxerFailed.

Fix

The producer retains the payload of the last audio frame a muxer accepted (copied before the write — movenc consumes the packet's data ref — and committed only on write success) and hands it to every later muxer allocation. A rotation is then always primed: audioPacketWritten/moovFlushed latch at init and the deferral cannot fire mid-session.

  • Scoped to AC-3/E-AC-3/TrueHD via the muxer's parsed-packet predicate, now exposed as a small static helper; AAC never copies.
  • One retained frame (~1–4 KB), pump-thread confined.
  • sessionAudioMoovPrimeFrame keeps covering the session-start case; the exit-scan and its bridge guard are unchanged.
  • For stream-copy audio, an upstream audio parameter change at the seam could make the retained sample entry describe the old config — the same exposure the construction-time scan already accepts, and strictly better than a dead pump. For bridged audio the encoder is engine-owned and unchanged across rotation, so the dec3 is exact.

Tests

LiveRotationAudioPrimeTests runs a real WAV → AudioBridge(.surroundCompat) → E-AC-3 → MP4SegmentMuxer pipeline:

  • Pins the incident: an un-primed rotated muxer over a bridge-encoder audio track defers its video-only first cut, and teardown finalize salvages nothing.
  • Pins the claim the fix rests on (not covered by the AE#222 source-frame tests): a bridge-output E-AC-3 frame is a valid dec3 prime — moov+dec3 at init, video-only cut completes, delivered segment is a plain fragment.

🤖 Generated with Claude Code

A mid-session muxer rotation (same-PID parameter-set change after a
live reconnect join, SSAI program switch) allocates a brand-new
MP4SegmentMuxer, and the only moov-prime carry-in was the construction
-time audioMoovPrimeFrame — nil for every session that did not start
audio-less. With bridged E-AC-3, video leads audio across the seam and
the bridge adds encoder latency, so the rotated muxer's first cut can
arrive before any post-seam audio packet: the cut defers awaiting a
sample entry (AE#222), the producer turns the deferral into a pump
exit, teardown finalize fails on the same precondition, and the live
session dies with muxerFailed. The AE#222 exit-scan cannot convert the
exit either — it rightly excludes bridged sessions, since a raw source
frame cannot prime a bridge-encoded track.

Have the producer retain the payload of the last audio frame a muxer
accepted (copied before the write — movenc consumes the data ref —
and committed only on write success) and hand it to every later muxer
allocation, so a rotation is always primed and the deferral cannot
fire mid-session. Scoped to AC-3/E-AC-3/TrueHD via the muxer's own
parsed-packet predicate, now exposed as a static helper; AAC never
copies. One retained frame, pump-thread confined.

Tests pin the incident shape (unprimed rotated muxer over a real
bridge-encoder track: video-only cut defers, finalize salvages
nothing) and the fix's load-bearing claim: a BRIDGE-OUTPUT E-AC-3
frame is a valid dec3 prime, not just a source frame as in AE#222.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@superuser404notfound
superuser404notfound merged commit 63319f7 into superuser404notfound:main Aug 9, 2026
3 checks passed
superuser404notfound added a commit that referenced this pull request Aug 9, 2026
…hable

Follow-up to #340. The rotation prime is right; three things around it were not
quite finished.

`lastMuxedAudioPrimeFrame` and `audioMoovPrimeFrame` were two fields for one
value: the retained frame is seeded from the construction-time prime and is only
ever assigned non-nil, so `lastMuxedAudioPrimeFrame ?? audioMoovPrimeFrame` at
the single use site could never take its right branch. One var now, documented
with both origins and with why it is safe unlocked (written from the two audio
write sites, read from allocateMuxer, and every allocation goes through
ensureMuxer on the pump thread).

The predicate now says what makes the fix safe for the codecs that reach it.
AC-3 and E-AC-3 prime from ANY frame, every frame is a complete syncframe, so an
arbitrary retained one is as good as the first. TrueHD would not: movenc latches
track->extradata from the first packet written and mov_write_dmlp_tag rejects it
without the major-sync word 0xF8726FBA, which most frames lack. It is
unreachable today (AudioCodecCompat.requiresBridge sends TrueHD through
AudioBridge, so the muxer's audio track is the bridge's E-AC-3 or FLAC encoder),
and that is now written down where someone would go to unblock it.

Plus a test pinning the predicate itself, since it is the gate the per-frame
retention hangs on and narrowing it would disarm the rotation prime silently,
and the house-style em-dashes removed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HX5zV3Fcf7Nzq4DYGdXvQE
superuser404notfound added a commit that referenced this pull request Aug 9, 2026
@superuser404notfound

Copy link
Copy Markdown
Owner

Merged. The branch was based on 6.15.2, so this went through a merge onto current main first: full suite green at 1688 tests, tvOS Simulator build clean, and the new suite holds as a pair (the unprimed case defers and finalizes to nothing, the primed one cuts a plain fragment). The two log lines in the trace line up with the code exactly: the deferral returns nil out of advanceMuxer, ensureMuxer turns that into muxerFailed, and the exit scan's audio.bridge == nil guard is why it could not convert the reason.

Two things pulled in on top in 09cabf5:

  • lastMuxedAudioPrimeFrame and audioMoovPrimeFrame collapsed into one field. The retained frame is seeded from the construction-time prime and is only ever assigned non-nil, so lastMuxedAudioPrimeFrame ?? audioMoovPrimeFrame at the single use site could never take its right branch.
  • The TrueHD arm of the predicate is now named as unreachable, because that is what makes an arbitrary retained frame safe. AC-3 and E-AC-3 prime from any frame (every frame is a complete syncframe), but movenc latches track->extradata from the first packet written and mov_write_dmlp_tag rejects it without the major-sync word, which most TrueHD frames lack. AudioCodecCompat.requiresBridge keeps TrueHD off a stream-copy muxer today, so it never comes up, and now it is written down where someone would go to unblock it.

Ships in the next release. Thanks for this one, the deferral to pump-exit chain in the trace was what made it readable end to end.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants