From baec9bf05479b48aeba2ac953f48ec871bc74a51 Mon Sep 17 00:00:00 2001 From: barckcode Date: Mon, 21 Sep 2026 13:04:49 +0200 Subject: [PATCH] docs(models): document the reasoning-only stream limit Publishes the platform-side limit that closes reasoning-only streams (issue #90 follow-up, alongside the estimated usage chunk shipped in nan-devops#550): 60,000 reasoning characters or 420 seconds with no visible output, closed with finish_reason=length plus an estimated, not-billed usage chunk when include_usage was requested. Sits in the same section as the reasoning_effort contract, EN + ES mirrors, with the disarm-on-first-output behavior and the mitigation levers members can act on. --- src/content/docs-es/models.mdx | 32 ++++++++++++++++++++++++++++++++ src/content/docs/models.mdx | 30 ++++++++++++++++++++++++++++++ 2 files changed, 62 insertions(+) diff --git a/src/content/docs-es/models.mdx b/src/content/docs-es/models.mdx index fdf5e87..d230870 100644 --- a/src/content/docs-es/models.mdx +++ b/src/content/docs-es/models.mdx @@ -341,4 +341,36 @@ curl https://api.nan.builders/v1/chat/completions \ }' ``` +## Límite de stream solo-razonamiento. + +Los prompts que exigen mucho razonamiento pueden dejar al modelo pensando +mucho tiempo antes de escribir nada. Cuando un turno en streaming produjo +solo razonamiento — sin `content` ni tool calls — durante **60.000 caracteres +de razonamiento** (unos 15.000 tokens) o **420 segundos**, la plataforma +cierra el turno en lugar de dejarlo correr hacia un callejón sin salida: +recibís `finish_reason: "length"` sobre un delta vacío, seguido de un chunk +de usage. + +Ese chunk de usage es una estimación de lo que consumió el intento hasta el +corte (tokens de prompt y de razonamiento), marcada con +`"estimated": true, "billed": false` — los turnos cerrados así no se cobran +de tu cuota — y lleva un marcador `nan_truncation` para que los clientes +puedan distinguir este cierre de un corte real por longitud de contexto. +Pedilo con `stream_options: {"include_usage": true}`. + +Dos cosas que conviene saber: + +- El límite no acota tu output. El primer token visible de contenido o + cualquier tool call lo desarma por el resto del turno: las respuestas que + escriben sobre la marcha nunca se cortan, por largas que sean. +- Si un turno se cierra así, el modelo seguía planificando cuando tocó el + techo. Bajá `reasoning_effort` donde el modelo lo soporte (tabla de + arriba), o reestructurá el prompt para que empiece a escribir temprano. + +El límite existe para acotar un modo de fallo conocido de los modelos con +thinking: a veces planifican sin converger durante 10–25 minutos y queman +todo su presupuesto de output sin output visible. Está por encima del +razonamiento legítimo más grande que medimos (~44.000 caracteres) y por +debajo de toda corrida estancada. + diff --git a/src/content/docs/models.mdx b/src/content/docs/models.mdx index e1493bf..8e1c146 100644 --- a/src/content/docs/models.mdx +++ b/src/content/docs/models.mdx @@ -339,4 +339,34 @@ curl https://api.nan.builders/v1/chat/completions \ }' ``` +## Reasoning-only stream limit. + +Reasoning-heavy prompts can keep a model thinking for a long time before it +writes anything. When a streaming turn has produced only reasoning — no +`content`, no tool calls — for **60,000 reasoning characters** (roughly +15,000 tokens) or **420 seconds**, the platform closes the turn instead of +letting it run into a dead end: you receive `finish_reason: "length"` on an +empty delta, followed by a usage chunk. + +That usage chunk is an estimate of what the attempt consumed up to the cut +(prompt and reasoning tokens), marked `"estimated": true, "billed": false` — +turns closed this way are not charged against your quota — and carries a +`nan_truncation` marker so clients can tell this closure apart from a real +context-length cut. Request it with `stream_options: {"include_usage": true}`. + +Two things worth knowing: + +- The limit does not cap your output. The first visible token of content or + any tool call disarms it for the rest of the turn: answers that write as + they go are never cut, however long they run. +- If a turn closes this way, the model was still planning when it hit the + ceiling. Lower `reasoning_effort` where the model supports it (table + above), or restructure the prompt so the model starts writing early. + +The limit exists to bound a known failure mode of thinking models: they +occasionally plan without converging for 10–25 minutes and burn their whole +output budget with zero visible output. It sits above the largest legitimate +reasoning trace we have measured (~44,000 characters) and below every +stalled run. +