Skip to content

Sequential-origin sources: one unranged connection, native routing, and an append playlist with real durations - #346

Open
tschuegy wants to merge 4 commits into
superuser404notfound:mainfrom
tschuegy:fix/timeshift-sequential-origin
Open

Sequential-origin sources: one unranged connection, native routing, and an append playlist with real durations#346
tschuegy wants to merge 4 commits into
superuser404notfound:mainfrom
tschuegy:fix/timeshift-sequential-origin

Conversation

@tschuegy

@tschuegy tschuegy commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

Problem

IPTV timeshift/catch-up archives are served by origins that fabricate HTTP range answers: any Range: bytes=X-Y gets a plausible-looking 206 + Content-Range: bytes X-Y/total, but the body is positioned on a coarse internal chunk boundary rather than byte X. Live probes against a real panel showed a request for offset 524288 returning the bytes of offset 0, suffix ranges returning the first N bytes, and range ends ignored outright. Only byte 0 is addressable, deterministically.

Through the existing reader this produced three device-visible failures on catch-up playback:

  1. The 32 MB range rotations spliced ~1.9 s of misplaced content into every reconnect (mpegts Packet corrupt, negative audio packet durations, audio desync roughly once a minute).
  2. FFmpeg's tail-read duration estimate ran against fabricated offsets and probed a 135-min window as 9.5 h.
  3. Even with a clean sequential read, the static VOD plan's uniform EXTINF:4.000 was a lie for archives whose GOP cadence does not divide the cut target: a 1.92 s-GOP channel cut eleven 3.84 s segments and one 5.76 s segment per 48 s cycle, all advertised as 4.000, and AVPlayer on tvOS 26 visibly jumped ~1.8 s at every resync while every queue metric read clean. (A sibling channel whose keyframes happen to sit on the 4 s grid played the same pipeline flawlessly - that A/B is what finally pinned it.)

Change

Four commits, each standing on the previous:

  • fix(io) - LoadOptions.sequentialOrigin declares the shape (headers cannot expose it, so it is a caller declaration, not a probe): the reader routes onto its existing forward-only streaming mode - one long-lived unranged GET, no suffix/tail probes, no size probe, no detours, no byte-offset reconnects, pb non-seekable. A connection that dies short of its advisory Content-Length reports EIO instead of EOF (EOF reads as end-of-media, which hosts deliberately never retry), and the Loopback-HLS: the final segment of a long 4K DV title never completes at EOM — AVPlayer parks ~0.1s from the end (WaitingToMinimizeStalls) → -12889; resuming into the tail can't start a master #169 VOD readError revive is not admitted (its fresh demuxer could only reopen from byte 0 and then fails its anchor seek, burning a slot on connection-capped panels). The paired declaredDurationSeconds carries the caller-known window length with the highest precedence in the AE#105 duration chain. aetherctl play grows --sequential-origin / --declared-duration.
  • fix(routing) - a sequential origin keeps the native path: every reason the forward-only rule forces software (cue prewarm, segment seeks, size probes) is already answered for the explicit declaration, and the producer reads the archive linearly from byte 0 exactly like a live session does. Declared-interlaced archives still route software via the field-order policy (the Field-order routing sends progressive-in-interlaced-carriage (PsF) H.264 to software for no visual benefit — #220 reprices the stated cost #232 refute probe stays seekable-only: its sample cannot be rewound on a sequential reader).
  • fix(hls) - sequential sessions render their media playlist the way the live path always has: only finalized segments, each with the duration actually muxed, as an append-only EVENT playlist that completes with ENDLIST at true source EOF. The server holds the first playlist until the startup segments exist (mirrors the live gate). Two host-facing seams from the growing playlist are closed at the engine: a caller-declared duration outranks the item's, and a sequential session with no resume anchor queues a post-readiness seek to 0 through the iOS: quick app switches pay a full pipeline rebuild — grace period before paused-background teardown, warm reload, deferred pre-ready seeks, public readiness signal #127 replay (AVPlayer defaults an EVENT playlist to edge-minus-holdback and re-anchors there after the load-time seek).
  • fix(hls) - the finalize reports pair capture and duration order-independently (the muxer rotation can fire off an audio packet crossing the boundary before the first video packet of the new segment has recorded its start), plan indices a long GOP skipped report as zero-duration holes the renderer omits, and a small reorder buffer keeps the provider's strictly-contiguous append API satisfied.

Verification

  • SequentialOriginReaderTests (7 tests) against an extended ThrottledOriginServer (now logs Range-header presence - a range-less GET was indistinguishable from bytes=0- - and gains a serveThenDrop directive for the short-body EIO case): exactly one unranged request past the 32 MB rotation point, EIO on short body, EOF on clean end, profile plumbing, duration precedence.
  • Full suite green (1638 tests).
  • Device (tvOS 26.6, Apple TV 4K, real panel): a 720p50 archive and a 1080i-sourced 1.92 s-GOP archive both play clean end-to-end; the served playlist reads EXTINF:5.760/3.840 matching the muxed media exactly; start lands at 0; the scrubber shows the declared window length.

The sequential declaration deliberately leaves scrubber seeking unavailable (byte seeking never worked against a lying origin - it was silently wrong); the follow-up story for these sources is app-side re-request with a shifted Xtream start timestamp.

🤖 Generated with Claude Code

tschuegy and others added 4 commits August 9, 2026 10:05
IPTV timeshift/catch-up archives fabricate range answers: any Range:
bytes=X- gets a plausible 206 whose body actually sits on a coarse
internal chunk boundary, so every 32 MB range rotation spliced ~1.9 s of
misplaced content into the pump (mpegts "Packet corrupt", negative audio
packet durations, a once-a-minute audio desync on device). Headers
cannot expose the lie, so LoadOptions.sequentialOrigin lets the caller
declare it: the reader routes onto its existing forward-only streaming
mode (one long-lived unranged GET), skipping the suffix/tail prefetch,
the optimistic persistent open, the size probe and every detour, and the
pb stays non-seekable.

With the ranged tail read gone the container resolves no duration (the
fabricated tail data probed a 135-min window as 9.5 h), so the paired
LoadOptions.declaredDurationSeconds carries the caller-known duration
with the highest precedence in the AE#105 chain (declared > custom
reader > disc MPLS/IFO > container).

A lost connection cannot be resumed at an offset, so the streaming read
reports EIO instead of EOF when the body ends short of its advisory
Content-Length (EOF would read as end-of-media, which hosts deliberately
never retry), and the superuser404notfound#169 VOD readError revive is not admitted (its
fresh demuxer could only reopen from byte 0 and then fails its anchor
seek, burning a slot on connection-capped panels) - the pump loss goes
straight to onVODSourceFailed for the host to re-request. The
StreamingDelegate also gains the same redirect header preservation as
every other delegate in the file; IPTV origins 302 twice before the
archive host.

aetherctl play grows --sequential-origin / --declared-duration for
manual runs against a stub or real portal.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The forward-only rule forced every sequential-origin archive onto the
software path, trading AVPlayer's buffering and hardware decode for
nothing: the reasons the rule exists (cue prewarm, segment seeks, size
probes) are all already answered for an explicit sequential declaration
- caller-declared duration, uniform-stride plan, gated restarts, a
prewarm that fails fast on the non-seekable pb - and the producer reads
the archive linearly from byte 0 exactly like a live session does.
Device comparison on the same channel: the 720p50 live stream on the
native path plays clean while the forced-software replay of the same
provider visibly stutters with every queue metric reading healthy.

Declared-interlaced archives still route software via the field-order
policy (the superuser404notfound#232 refute probe stays seekable-only: its sample cannot be
rewound on a sequential reader), and the superuser404notfound#126 accidental forward-only
case (unknown-length HTTP MP4) keeps the software fallback - nothing
declared its timeline trustworthy.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…tions

The static VOD plan advertises uniform EXTINF values the muxer cannot
honor: cuts snap to real keyframes, so an archive whose GOP cadence does
not divide the cut target puts every segment's media outside its
advertised window (device trace: 1.92 s GOPs against the 4.0 s plan -
eleven 3.84 s segments and one 5.76 s segment per 48 s cycle, all
declared as 4.000; AVPlayer on tvOS 26 visibly jumped ~1.8 s at every
resync while every queue metric read clean; a channel whose keyframes
happen to sit on the 4 s grid played the same pipeline flawlessly).

A sequential-origin session now renders its media playlist the way the
live path always has - only finalized segments, each with the duration
actually muxed - as an append-only EVENT playlist that completes with
ENDLIST at true source EOF (a torn-down session never marks it). The
producer records each VOD segment's item-axis start at the superuser404notfound#65 ledger
site and reports real durations at rotation; the provider grows the
visible set contiguously and the server holds the first playlist until
the startup segments exist, mirroring the live gate.

Two host-facing seams from the growing playlist are closed at the
engine: a caller-declared duration outranks the item's (the scrubber
scales to the window length, not the produced span), and a sequential
session with no resume anchor queues a post-readiness seek to 0 through
the superuser404notfound#127 replay, because AVPlayer defaults an EVENT playlist to
edge-minus-holdback and re-anchors there after the load-time seek.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The first cut of the append playlist still advertised the plan's 4.000:
the muxer rotation can fire off an AUDIO packet crossing the boundary
before the first video packet of the new segment has recorded its start,
so the duration lookup at capture time found nothing and fell back to
the cut target on every single segment. Durations are now computed at
the ledger site (where the next start becomes known) and reports fire
once BOTH halves - capture and duration - are in, in either order.

Two more shapes the first version missed: a long GOP can skip plan
indices outright (a 5.76 s cut spans two 4 s boundaries), so skipped
holes report duration 0 and the playlist renderer omits zero-duration
entries; and since capture, holes and the EOF tail can resolve out of
order while the provider's append API is strictly contiguous, all
reports funnel through a small reorder buffer.

Verified against the failing archive: the served playlist now reads
EXTINF 5.760 / 3.840 matching the muxed media exactly, TARGETDURATION 6.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant