feat(llm): background-mode OpenAI calls, call metadata, cancellation, bounded retries

AGENTS.md

@@ -118,6 +118,12 @@ PY
 - Write `slug` explicitly as the descriptive human-readable identifier. Slugs must be unique, but may be changed before or after publication when naming conventions improve.
 - Use `filename_safe_name` when a model-run key needs to become part of a filename; do not hand-roll lossy replacements for characters such as `/` or `*`.
 - Put every runtime call option in the `ModelRun` declaration; do not add hidden defaults elsewhere.
+  Transport mechanics are the exception: how a provider class talks to its API (the Anthropic
+  provider streams; the OpenAI provider uses background mode and polls) is not a model option,
+  lives in the provider, and is not part of any run's options or fingerprint.
+  For OpenAI runs the `timeout` option is the budget for the whole background response
+  (cancelled server-side when it runs out), not an HTTP request timeout; size it for the
+  slowest expected response, not for a single request.
 - Use exact provider option names and values as they are passed to `get_response`.
 - If an option affects performance and should appear in human-readable slugs, filenames, or forecast labels, make the handwritten `slug` reflect it.
 - Unit tests validate declared top-level provider option names against installed provider SDK

README.md

@@ -62,6 +62,37 @@ human-readable slug `claude-sonnet-4-6-1024`. Use immutable `model_run_key` valu
 references. Human-readable slugs are available for display and convenience lookups but while the
 `model_run_key` should be used for stable lookups
 
+## Call metadata, cancellation and fetching by id
+
+`get_response_with_metadata(prompt, context=..., **options)` makes the same call as
+`get_response` but returns a `utils.llm.types.LLMResponse`: the text plus the provider's
+response id, final status, token usage (`input_tokens`, `output_tokens`, `reasoning_tokens`,
+`tool_calls`) and the number of attempts the retry wrapper made. A `CallContext` carries a
+`threading.Event` that cancels the call between retries and, for OpenAI, cancels the
+server-side response (or, with `cancel_server_side=False`, stops waiting and leaves the
+response running so it can be fetched later), and an `on_response_id` hook that receives
+the provider's id as soon as the request is accepted. `model_run.fetch_response(id)`
+returns an earlier OpenAI response by id without a new call, and
+`model_run.wait_for_response(id)` waits for one that is still running.
+
+OpenAI calls run in the Responses API background mode: the request returns immediately and
+the result is polled, so no connection sits idle while the model works and a finished
+response is never lost to a dropped connection. This is how the provider talks to the API,
+not a model option, so it is not part of any run's declared options.
+
+In background mode a run's `timeout` option is not sent to the API as an HTTP timeout. It
+is the budget for the whole response, from the request until the response finishes; when
+it runs out the response is cancelled server-side and the call raises `TimeoutError`. Runs
+without a `timeout` get 30 minutes. A failed poll is polled again for as long as the budget
+lasts and never starts a new generation.
+
+Failed attempts are retried up to five attempts in total for OpenAI (in background mode a
+retry only follows a failure that was not billed) and two for every other provider (a
+failed attempt there is usually billed and lost), except errors that would only be billed
+again: a cancelled call, a call that ran out of its budget, a refused (HTTP 403)
+OpenAI request, and an incomplete response (`LLMResponseIncomplete`, usually the
+output-token cap). Every failed attempt is logged at WARNING, since it may be billed.
+
 ## LLM Structured Output
 
 `get_response` returns the model's text. When you ask a provider for structured output, that

tests/unit/test_llm_model_runs.py

@@ -1602,3 +1602,68 @@ def test_get_model_run_by_slug_is_convenience_lookup_for_human_slugs():
     assert model_runs.get_model_run_by_slug("gpt-5.5-2026-04-23-high") is run
     with pytest.raises(KeyError, match="missing-slug"):
         model_runs.get_model_run_by_slug("missing-slug")
+
+
+def test_model_run_get_response_with_metadata_routes_context_and_returns_metadata():
+    """The metadata entry point passes the caller's context through and returns the record."""
+    from utils.llm import model_registry, model_runs
+    from utils.llm.types import CallContext, LLMResponse, LLMUsage
+
+    run = model_runs.ModelRun(
+        model_run_key="gpt-5.5-2026-04-23-run-variant-02",
+        slug="gpt-5.5-2026-04-23-high",
+        model=model_registry.MODELS_BY_KEY["gpt-5.5-2026-04-23"],
+        options={"reasoning": {"effort": "high"}},
+    )
+    context = CallContext()
+    expected = LLMResponse(
+        text="forecast",
+        provider_response_id="resp_1",
+        status="completed",
+        usage=LLMUsage(input_tokens=10, output_tokens=5, reasoning_tokens=3, tool_calls=0),
+        attempts=2,
+        duration_s=1.5,
+    )
+
+    with patch(
+        "utils.llm.model_registry.get_response_with_metadata", return_value=expected
+    ) as get_response_with_metadata:
+        response = run.get_response_with_metadata("prompt", context=context, max_output_tokens=10)
+
+    assert response == expected
+    get_response_with_metadata.assert_called_once_with(
+        provider=PROVIDERS["OpenAI"],
+        model_id="gpt-5.5-2026-04-23",
+        prompt="prompt",
+        options={"reasoning": {"effort": "high"}, "max_output_tokens": 10},
+        context=context,
+    )
+    # The text-only entry point is unchanged for callers that want just the text.
+    with patch("utils.llm.model_registry.get_response", return_value="forecast"):
+        assert run.get_response("prompt") == "forecast"
+
+
+def test_model_run_wait_for_response_uses_the_run_timeout_as_budget():
+    """Waiting on an earlier response is bounded like a fresh call of the same run."""
+    from utils.llm import model_registry, model_runs
+    from utils.llm.types import LLMResponse
+
+    run = model_runs.ModelRun(
+        model_run_key="gpt-5.5-2026-04-23-run-variant-02",
+        slug="gpt-5.5-2026-04-23-high",
+        model=model_registry.MODELS_BY_KEY["gpt-5.5-2026-04-23"],
+        options={"reasoning": {"effort": "high"}, "timeout": 900},
+    )
+    expected = LLMResponse(text="forecast", provider_response_id="resp_old", status="completed")
+
+    with patch(
+        "utils.llm.model_registry.wait_for_response", return_value=expected
+    ) as wait_for_response:
+        assert run.wait_for_response("resp_old") == expected
+
+    wait_for_response.assert_called_once_with(
+        provider=PROVIDERS["OpenAI"],
+        provider_response_id="resp_old",
+        budget=900,
+        context=None,
+    )