cohere: add long-form chunking

[rafarrw]

Summary

• window audio past the trained 35 second clip span, with 1 second of overlap and
  boundaries preferring low-energy points
• stitch windows in token space by matching the tail of the accumulated
  hypothesis against the head of the next window, then decode once at the end
• report Cohere input capacity as unbounded (max_audio_ms = 0) and move the
  family to the chunked bucket in docs/input-limits.md, keeping n_ctx as the
  per-window output budget
• route batches containing long inputs through the serial path, since the batched
  graph assumes one encoder pass per utterance

Motivation

Cohere Transcribe trains on clips of at most 35 seconds. The value lives in
max_audio_clip_s in the HF feature-extractor config and appears in the porting
intake, but the converter never writes to the GGUF. So the runtime gates only
against the encoder positional table, roughly 400 seconds, and accepts anything
in between as a single pass.

Past the trained span the decoder's cross-attention loses alignment. The decoder
emits EOS early and drops the rest of the audio. The run returns
TRANSCRIBE_OK with no WARN and no transcribe_was_truncated(), which
docs/input-limits.md states never happens.

On a 66.3 second recording (transcribe-cli, CPU, Q5_K_M) the single-pass path
returned 107 words against 146 for the windowed path. The single-pass output
dropped a whole paragraph from the middle and spliced the surrounding text
together. The decoder's 512 max-new-tokens cap does not explain this. Lowering
--n-ctx to 300 produced byte-identical output. Lowering to 100 truncated with
the expected WARN and a non-OK status, so detection works and its silence at
the default carries meaning.

This surfaced through Handy. Dictated paragraphs came back fluent, correctly
punctuated, and missing their middle.

Correctness

Windows overlap, so repeated speech is the adversarial case. A genuinely repeated
sentence must survive instead of matching the overlap and getting trimmed. A
first prototype used symmetric seam search and dropped one of four identical
repetitions.

The seam search now comes from canary_token_seam() in #112, including its
acceptance rules and asymmetric search widths derived from the model's frame
duration. Canary's other merge path assigns overlap words by timestamp midpoint.
Cohere advertises max_timestamp_kind = NONE, so only the token-space half
applies here. If #112 changes in review, this should follow rather than diverge.

Audio	Content	Result
44 s	samples/jfk.wav x4	4 of 4 repetitions preserved
66 s	six-checkpoint passage	all six checkpoints, complete tail

Known limitation. When no seam matches, the code appends the window whole and
logs a DEBUG line. Repeating a few words stays recoverable. Dropping speech
does not. The fallback does trigger. A capitalised word at a sentence start
tokenises differently from its lower-case form mid-sentence, so the 44 second
case leaves one duplicated word at one seam. Normalised text reconciliation would
remove the duplicate, but the approach diverges from the token-space strategy in
#112, so I would rather ask first.

Performance

Windowing runs faster than the single pass, not slower. Conformer attention
scales quadratically with sequence length, so three short windows cost less than
one long one.

On the 66.3 second clip, total time dropped from 20.1 s to 17.9 s, about 11%.
Encode alone dropped from 19.1 s to 16.7 s. Measured with transcribe-bench on
the identical binary, CPU backend, 1 warmup and 4 iterations each. The
per-iteration ranges never overlap (18.8 to 19.5 s against 16.3 to 17.0 s), so
this is not run-to-run noise.

Scope

• src/arch/cohere/model.cpp, src/arch/cohere/cohere.h. Windowing, seam
  search, batch routing, capability change.
• examples/cli/main.cpp. The limits printer treated max_audio_ms == 0 with a
  non-zero context as "context too small for audio + prompt". A family both
  windowed and context-capped breaks the assumption.
• tests/cohere_smoke.cpp, tests/cohere_e2e_smoke.cpp.
• docs/input-limits.md, docs/models/cohere-transcribe-03-2026.md,
  include/transcribe.h.

Out of scope: streaming, word timestamps, cross-window decoder context.

AI Assistance

AI assisted with the implementation, tests, and this description. I understand
and own the diff.

Validation

Environment: Windows 11 Pro 26200, MSVC 19.44, CPU backend,
cohere-transcribe-03-2026-Q5_K_M.gguf (HF revision dfa4adeb), on top of
223c9b0.

scripts/ci/clang-format.sh --check
# pass (pinned 22.1.5). git diff --check also clean.

ctest --test-dir build -C Release --output-on-failure
# 51 total: 33 passed, 17 skipped, 1 not run.
# The 17 skips are fixture and model gated. They include the Cohere real-model
# smokes, which need TRANSCRIBE_COHERE_GGUF. See the next command.
# The 1 not run is transcribe_sortformer_stream_ext_unit. The build never
# produced the binary, because the unchanged source calls POSIX setenv and
# unsetenv under MSVC. Same result at 223c9b0.
# Skip counts vary with which optional fixtures and models you have locally.

TRANSCRIBE_COHERE_GGUF=/path/to/cohere-transcribe-03-2026-Q5_K_M.gguf \
  ctest --test-dir build -C Release \
    -R '^transcribe_cohere_(smoke|real_smoke|e2e_smoke)$' \
    --output-on-failure
# 3/3 passed

transcribe_cohere_smoke adds 4 cases for boundary selection, 5 for the seam
search, and 1 for batch cancellation. The long-form e2e case requires all four
occurrences of a distinctive phrase and bounds total output length to 90 to 120%
of the expected transcript. So the test catches both missing repetitions and
runaway duplication.

Two assertions in cohere_e2e_smoke.cpp expected a rejected transcribe_run to
clear the previous result. src/transcribe.cpp documents the opposite as
intentional: "Caller-param rejections [...] preserve the previous snapshot". The
assertions were stale, not the library. They now check the previous result
survives. Bisection confirmed those failures predate this change.

Gates not applicable. Intake signoff, Preflight A, and Preflight B cover porting
a new family. This changes an existing one and touches no converter, GGUF,
manifest, or tolerance file.

Gate still open:

uv run scripts/validate.py all --family cohere
# blocked. The Cohere model is gated on Hugging Face. Unauthenticated fetch
# returns 401.

This is why the PR is a draft. Running the command with authenticated access
would close the remaining Cohere-specific validation gate. The Windows Sortformer
build failure noted above is separate and predates this change.

Also not covered. Unspaced scripts stay untested. Decoding once at the end
instead of concatenating strings should reproduce the tokenizer's own spacing for
Chinese and Japanese, but I have no audio in those languages to confirm.