handy-computer · rafarrw · Jul 31, 2026
@@ -57,6 +57,7 @@ value at the default context; for the effective limit under a lowered
 | Families | Limit | Behavior |
 | --- | --- | --- |
 | whisper, parakeet (all variants), voxtral_realtime | none (`max_audio_ms = 0`) | Long audio is windowed internally and stitched (whisper), processed by an unbounded/streaming encoder (parakeet, voxtral_realtime), or padded if short. No practical limit; all three **ignore `n_ctx`** (it cannot lower a limit they don't have). whisper and parakeet never error on length. voxtral_realtime has one exception — its absolute `dec_max_position` cap (~2.9 h, see below): a clip past it returns `INPUT_TOO_LONG` (one-shot/batch), and a stream that reaches it flags `was_truncated`. |
+| cohere | none (`max_audio_ms = 0`) | Windowed to the trained clip span (35 s) before the encoder, then the per-window transcripts are concatenated. Unlike the three above, cohere **does honor `n_ctx`**: it still bounds the decoder self-KV and therefore the per-window output budget, but no longer bounds input length. |
 
 Whisper slices audio into 30 s windows with prev-context stitching; parakeet's
 conformer is effectively unbounded (the encoder positional table is recomputed
@@ -67,11 +68,27 @@ constant-memory caches (`cache_last_channel` / `cache_last_time` + the decoder
 LSTM state) rather than a growing KV, so it is unbounded. These families do not
 need and do not have a length gate.
 
+Cohere is windowed for a different reason. Its encoder positional table spans
+~400 s, so a much longer clip than the model was trained on still encodes
+cleanly and an input gate keyed on that table lets it through. The decoder is
+what breaks: its cross-attention has no monotonicity constraint (unlike an
+RNN-T, which walks encoder frames in order), so past the trained clip span
+(`max_audio_clip_s`, 35 s) it loses alignment, emits EOS early, and drops the
+rest of the audio — returning `TRANSCRIBE_OK` with a silently incomplete
+transcript, which is exactly what this document promises never happens. So the
+family windows to the trained span and stitches the windows back together.
+Windows overlap by one second and are joined in token space by matching the tail
+of the accumulated hypothesis against the head of the next window, so a boundary
+landing mid-word costs nothing — the word survives whole in the next window and
+the duplicate is trimmed at the join. Boundaries still prefer a detected pause,
+which makes that match easier to find. When no join is identified the window is
+appended whole, repeating a few words rather than dropping any.
+
 ### 2. Hard context cap — reject up front
 
 | Families | Limit source | Behavior |
 | --- | --- | --- |
-| qwen3_asr, canary_qwen, funasr_nano, granite, granite_nar, voxtral, cohere, canary | decoder context window (`dec_max_position_embeddings` / `dec_max_seq`), or the encoder positional table (`enc_pos_emb_max_len`, for cohere/canary) — all from GGUF | KV cache grows to fit, clamped to the model's true max. Over-length input is **rejected before the decode** (or before the encoder, where the encoder table is the binding limit) with `TRANSCRIBE_ERR_INPUT_TOO_LONG`. |
+| qwen3_asr, canary_qwen, funasr_nano, granite, granite_nar, voxtral, canary | decoder context window (`dec_max_position_embeddings` / `dec_max_seq`), or the encoder positional table (`enc_pos_emb_max_len`, for canary) — all from GGUF | KV cache grows to fit, clamped to the model's true max. Over-length input is **rejected before the decode** (or before the encoder, where the encoder table is the binding limit) with `TRANSCRIBE_ERR_INPUT_TOO_LONG`. |
 
 These families wrap an LLM-style decoder whose context window
 (`audio_tokens + prompt + generation`) is the binding constraint. The number of
@@ -138,16 +155,21 @@ reports the model's default-context ceiling (`n_ctx == 0`); it is not re-derived
 for a session that narrows `n_ctx`. A session that lowers `n_ctx` may therefore
 reject audio shorter than the advertised `max_audio_ms`.
 
-Encoder-bound families are different. For cohere and canary, the input-audio
-limit is the encoder positional table, while `n_ctx` only bounds the decoder
-self-KV / output budget. In those families `transcribe_session_get_limits()`
-reports a smaller `effective_n_ctx` and `max_kv_bytes` when `n_ctx` is lowered,
-but `effective_max_audio_ms` stays pinned to the encoder input bound.
-
-**Chunked / unbounded families (bucket 1) ignore `n_ctx` entirely.** whisper,
-parakeet, and voxtral_realtime have no lowerable context ceiling, so a non-zero
-`n_ctx` is a documented no-op and `transcribe_session_get_limits()` keeps
-reporting them unbounded. voxtral_realtime is the subtle case: it *does* have an
+Encoder-bound families are different. For canary, the input-audio limit is the
+encoder positional table, while `n_ctx` only bounds the decoder self-KV / output
+budget. There `transcribe_session_get_limits()` reports a smaller
+`effective_n_ctx` and `max_kv_bytes` when `n_ctx` is lowered, but
+`effective_max_audio_ms` stays pinned to the encoder input bound.
+
+Cohere is the hybrid: it windows its input (bucket 1, `max_audio_ms = 0`) but
+its decoder self-KV is still `n_ctx`-bounded, so a lowered `n_ctx` shrinks the
+per-window output budget without changing how much audio it accepts.
+
+**Most chunked / unbounded families (bucket 1) ignore `n_ctx` entirely** —
+cohere, described just above, is the exception. whisper, parakeet, and
+voxtral_realtime have no lowerable context ceiling, so a non-zero `n_ctx` is a
+documented no-op and `transcribe_session_get_limits()` keeps reporting them
+unbounded. voxtral_realtime is the subtle case: it *does* have an
 absolute decoder position cap (`dec_max_position`, ~2.9 h of audio), but that is
 the model's true RoPE wall — not a memory ceiling a caller can lower — so
 `n_ctx` does not narrow it (its decoder KV is a constant-memory sliding ring;

@@ -21,10 +21,24 @@ pinned 2026-04-16.
 
 ## Input limits
 
-Accepts up to about **6.7 minutes (400 s)** of 16 kHz mono audio per call — the
-encoder's positional table is the binding limit. Longer audio is rejected up
-front with `TRANSCRIBE_ERR_INPUT_TOO_LONG` rather than silently truncated; split
-it into shorter segments. See the [input-length contract](../input-limits.md).
+**No practical limit** (`max_audio_ms = 0`): audio longer than the model's
+trained clip span (`max_audio_clip_s`, 35 s) is windowed internally and the
+per-window transcripts are concatenated, so a caller never has to segment input
+by hand.
+
+Windowing is required, not an optimization. The encoder's positional table
+spans ~400 s, so a much longer clip than the model ever saw in training still
+encodes cleanly; the decoder is what fails. Its cross-attention has no
+monotonicity constraint, so past the trained span it loses alignment, emits EOS
+early, and drops the rest of the audio — returning `TRANSCRIBE_OK` with a
+silently incomplete transcript. Windows overlap by one second and are joined in
+token space, so a boundary that lands mid-word is harmless — the word survives
+whole in the next window and the duplicate is trimmed at the join. Boundaries
+still prefer a detected pause, which makes that match easier to find.
+
+`n_ctx` still applies: it bounds the decoder self-KV, and therefore the output
+budget *per window*, but no longer bounds input length. See the
+[input-length contract](../input-limits.md).
 
 ## Download
 

@@ -1166,6 +1166,16 @@ int main(int argc, char ** argv) {
         {
             struct transcribe_session_limits lim;
             transcribe_session_limits_init(&lim);
+
+            // A model that advertises no input ceiling of its own is unbounded
+            // however small its decoder context is -- the family windows the
+            // audio internally. Without this, a family that is both windowed
+            // and context-capped (cohere) reads as "context too small".
+            struct transcribe_capabilities lim_caps;
+            transcribe_capabilities_init(&lim_caps);
+            const bool model_is_unbounded =
+                transcribe_model_get_capabilities(model, &lim_caps) == TRANSCRIBE_OK && lim_caps.max_audio_ms == 0;
+
             if (transcribe_session_get_limits(ctx, &lim) == TRANSCRIBE_OK) {
                 if (lim.effective_max_audio_ms > 0) {
                     std::printf("  max audio:  %.1f s", (double) lim.effective_max_audio_ms / 1000.0);
@@ -1174,7 +1184,7 @@ int main(int argc, char ** argv) {
                                     (long long) (lim.max_kv_bytes >> 20));
                     }
                     std::printf("\n");
-                } else if (lim.effective_n_ctx > 0) {
+                } else if (lim.effective_n_ctx > 0 && !model_is_unbounded) {
                     // Capped family whose context is too small to fit any audio
                     // plus a prompt (e.g. an aggressively low --n-ctx).
                     std::printf(

@@ -243,7 +243,7 @@ typedef enum {
      * Returned by transcribe_run / transcribe_run_batch when the input
      * audio is longer than the loaded model can process in a single
      * decode. Hard-context-cap families (LLM-style decoders: qwen3_asr,
-     * canary_qwen, funasr_nano, granite, granite_nar, voxtral, cohere,
+     * canary_qwen, funasr_nano, granite, granite_nar, voxtral,
      * canary) reject an over-length clip UP FRONT — before the decode
      * (and, where the binding limit is the encoder's positional table,
      * before the encoder) — by comparing the audio's prefill token count

@@ -33,6 +33,30 @@ namespace transcribe::cohere {
 
 void apply_family_invariants(transcribe_model & model);
 
+// Long-form window boundary picker: searches back from `target` over `search`
+// samples for a genuine pause to cut on, and returns -1 when the region is
+// continuous speech and no candidate qualifies. Declared here so the boundary
+// logic can be unit-tested without a model; see model.cpp for the full
+// contract and its limitations.
+int pick_split(const float * pcm, int n_samples, int target, int search);
+
+// Where two overlapping windows should be joined, in token space.
+struct TokenSeam {
+    int  previous_keep;  // tokens to keep from the accumulated hypothesis
+    int  current_skip;   // tokens to drop from the front of the new window
+    bool matched;        // false when no overlap was identified
+};
+
+// Find the join point between consecutive windows by matching the tail of
+// `previous` against the head of `current`. Timestamp-free, so it works for a
+// family that emits no alignment data. Ported from the canary long-form path
+// (see PR #112) so both families stitch the same way. Declared here for unit
+// testing; see model.cpp for the acceptance rules.
+TokenSeam token_seam(const std::vector<int> & previous,
+                     const std::vector<int> & current,
+                     int                      previous_search,
+                     int                      current_search);
+
 // KV cache for the autoregressive decoder. Flat 1D K/V tensors (whisper.cpp
 // pattern), per-layer slices via views. Self cache grows per step;
 // cross cache is computed once from encoder output, then reused.
@@ -172,6 +196,10 @@ struct CohereSession final : public transcribe_session {
     bool encoder_use_flash = true;
     bool decoder_use_flash = true;
 
+    // Token ids the last decoded window produced, kept so the long-form path
+    // can stitch windows in token space instead of re-parsing decoded text.
+    std::vector<int> window_ids;
+
     CohereSession() = default;
     ~CohereSession() override;
 };