feat(replays): play video-backed sessions behind a replay engine seam

[Makisuo]

Why

The web player hardcoded new Replayer(...), so rrweb was the only thing it could ever play. Mobile sessions record H.264 segments wrapped in rrweb-shaped events — the chunk pipeline carries them untouched, but rrweb finds no DOM to rebuild and renders nothing.

What

Engine construction, the clock, seeking and letterboxing now sit behind a ReplayEngine interface (apps/web/src/components/replays/engine/), with two implementations:

• rrweb — today's Replayer moved over unchanged, including its config, the applyScale letterboxing math, and the negative-baselineTime clock guard (that quirk is rrweb's, not the player's, so it belongs in the engine).
• video — decodes each base64 segment to a Blob URL and drives one <video>. Since every segment opens on an IDR keyframe, an offset resolves to (segment, offset-within-segment) exactly; a seek landing in a gap between segments snaps forward to the next real footage. Object URLs are held for the current and next segment only — the base64 already sits in the loader's event array, so a decoded Blob is a second copy.

Nothing below the context changed. replay-player.tsx, replay-editor-timeline.tsx, replay-timeline.ts, session-events-panel.tsx, replay-range.ts and use-replay-chunk-loader.ts are byte-identical, and the existing player + transport tests pass as written — that's the evidence the refactor preserved behaviour, not just a claim.

Two bugs found along the way

• Idle derivation would have destroyed video playback. It treated every event as an instant, but a video session emits ~one event per 30s segment — so each quiet segment read as a 30s idle band, and skip-idle (on by default) collapsed real footage to ~1s. A segment now spans its duration, so only genuine gaps (the recorder stopping while the app is backgrounded) stay collapsible. Regression test included.
• Touch produced no click markers. deriveMeta only matched MouseInteractions.Click (2), never TouchStart (7). Also a latent fix for mobile-web rrweb sessions, which have never shown markers either.

Separately, normalizeEvents ran JSON.stringify over both sides of every adjacent pair; with an MP4 inside each event that serialized hundreds of KB per comparison on every range decode. It now rejects on timestamp + type first.

Format discriminator

maple.session.replay_format ("rrweb" | "video") rides the existing ResourceAttributes map, so there is no warehouse migration and no wire change: the detail query already ships the whole map as JSON and the route loader already prefetches it, so the engine is chosen before any chunk is fetched. Deliberately did not touch sessionReplaysListQuery — that would mean four duplicated projections plus a catalog.sql regeneration for a list badge nobody asked for.

An absent key means rrweb; every session recorded before the marker existed is a browser recording. The browser SDK stamps "rrweb" in buildSessionMetaRow's shared base object — placement matters, since session_replays is a ReplacingMergeTree that replaces the whole row.

Seeking needed no changes: is_checkpoint comes from the x-maple-is-checkpoint ingest header, so an SDK that flags every segment gets exact per-segment anchoring from checkpointAtOrBefore for free.

Scope checks

• apps/mobile is unaffected — it only calls /api/query-engine/* and /api/dashboards/, and never reads session_replays or any replay route.
• No new column: an earlier FirstEventMs attempt was reverted because the deployed cluster rejected it (warehouse_schema_drift) and it tripped the local-CLI schema gate. A map key avoids all of that.

Testing

144 tests across 13 replay files, plus browser-session (109) and the effect-sdk session test. Scoped typecheck clean on @maple/web and @maple/browser-session.

New coverage: segment extraction and offset→segment resolution (gaps, boundaries, past-the-end, negative), the video idle-band trap, touch vs mouse markers, replayFormat fallbacks, and the format marker on both active and ended meta rows.

The <video> DOM path is intentionally not unit-tested — jsdom implements neither URL.createObjectURL nor HTMLMediaElement.play/pause, so such a test would only exercise its own stubs. The segment math is pure and covered directly instead.

Not done

• Repo-wide bun run test / bun typecheck were skipped (authored late, scoped commands only) — worth a run in CI/morning.
• No live browser verification: it needs a real video-backed session in the warehouse. The eventsOverride provider prop injects a synthetic mobile chunk without touching the network.
• Portrait recordings letterbox inside the existing 16:9 surface and still show the browser-chrome header. Engine work only; portrait polish is a separate change.
• The mobile SDK must hold up its end: send maple.session.replay_format: "video" in its meta row and x-maple-is-checkpoint: 1 per segment. Unverified from this repo.

🤖 Generated with Claude Code

---

^{Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.}