vid-analysis — YouTube video composition analyzer

Breaks a YouTube video into a composition timeline so you can see how a host structures their content. Every second is classified into one of four categories:

• A-roll — host speaking, on camera (talking head)
• B-roll — host speaking over visuals (voiceover / animation / slides)
• Guest — a non-host speaker is talking
• Whitespace — no dialogue (music / b-roll only)

Language-agnostic: speaker diarization and face recognition are acoustic/visual, so multi-language videos work without per-language models. No transcription.

Pipeline (per video, in the backend)

1. Audio (download.py) — yt-dlp → ffmpeg → 16 kHz mono WAV (cached).
2. Diarization (diarize.py) — pyannote.audio 3.1 on GPU → who-spoke-when.
3. Cadence (analyze.py) — per-speaker shares; identifies the dominant "Host".
4. Visual (visual.py) — a low-res H.264 copy (cached) is decoded once; each ~1s window is labelled stale (host face on camera) vs active, using YuNet detection + SFace recognition. Robust to zoom/angle; thresholds adapt per-video (Otsu).
5. Composition (compose.py) — crosses speech × visual into the 4-category timeline (merged, despeckled) + rolling 1-min percentages + summary stats. This is the single source of truth the UI renders verbatim.
6. Viewer (viz.py) — a self-contained HTML QC tool: embedded player, composition chart, Host/Guest/Whitespace swimlanes, histogram, and metrics — all synced.

Setup

uv sync                              # torch (CUDA 12.1), pyannote, opencv, yt-dlp

Hugging Face token (one-time, for diarization)

pyannote models are license-gated. With a free HF account: accept the licenses for speaker-diarization-3.1 and segmentation-3.0, create a read token, then put it in .env:

HF_TOKEN=hf_xxxxxxxxxxxxxxxxxxxx

Usage

uv run vidcadence-serve              # web app at http://127.0.0.1:8731

• GET /<youtube-id> — redirects to the report if processed, else enqueues it and shows an auto-refreshing progress page. GET / has a paste-a-URL box.
• Jobs run through one background worker, so GPU diarization is serialized; the visual download is prefetched concurrently with diarization.
• Reports are cached as data/<id>.report.json (+ .report.html).

Lower-level entry points: vidcadence <url> (cadence report only) and vidcadence-viz <report.json> (rebuild the HTML from a report).