Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,7 @@ Some examples require extra dependencies. See each sample's directory for specif
without wrapping them in a workflow.
* [open_telemetry](open_telemetry) - Trace workflows with OpenTelemetry.
* [openai_agents](openai_agents) - Run OpenAI Agents SDK agents as durable Temporal workflows.
* [openrouter](openrouter) - Call OpenRouter from Activities: fan out a prompt batch, and pause instead of failing when the budget or credits run out.
* [patching](patching) - Alter workflows safely with `patch` and `deprecate_patch`.
* [polling](polling) - Recommended implementation of an activity that needs to periodically poll an external resource waiting its successful completion.
* [prometheus](prometheus) - Configure Prometheus metrics on clients/workers.
Expand Down
17 changes: 17 additions & 0 deletions openai_agents/model_providers/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,23 @@ The example uses Anthropic Claude by default but can be modified to use other Li

Find more LiteLLM providers at: https://docs.litellm.ai/docs/providers

#### OpenRouter
Uses [OpenRouter](https://openrouter.ai/) as the model provider, so the agent can run on any of the hundreds of models OpenRouter serves through one API key. OpenRouter speaks the OpenAI Chat Completions API, so the stock `OpenAIProvider` works once it is pointed at OpenRouter's base URL.

Start the OpenRouter worker:
```bash
export OPENROUTER_API_KEY="your_openrouter_api_key"

uv run openai_agents/model_providers/run_openrouter_worker.py
```

Then run the example in a separate terminal:
```bash
uv run openai_agents/model_providers/run_openrouter_workflow.py
```

The workflow uses `openai/gpt-4o-mini`; change `OPENROUTER_MODEL` in [workflows/openrouter_workflow.py](workflows/openrouter_workflow.py) to any OpenRouter model slug, or to `openrouter/auto` to let OpenRouter pick. See the [openrouter](../../openrouter) sample for calling OpenRouter directly from Activities with cost tracking and budgets.

### Extra

#### GPT-OSS with Ollama
Expand Down
78 changes: 78 additions & 0 deletions openai_agents/model_providers/run_openrouter_worker.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
import asyncio
import logging
import os
from datetime import timedelta

from agents import OpenAIProvider, set_tracing_disabled
from openai import AsyncOpenAI
from temporalio.client import Client
from temporalio.contrib.openai_agents import ModelActivityParameters, OpenAIAgentsPlugin
from temporalio.worker import Worker

from openai_agents.model_providers.workflows.openrouter_workflow import (
OpenRouterAgentWorkflow,
)


# @@@SNIPSTART python-openai-agents-openrouter-provider
def openrouter_provider() -> OpenAIProvider:
"""OpenAI Agents SDK model provider backed by OpenRouter.

OpenRouter speaks the OpenAI Chat Completions API, so the stock provider
works once it is pointed at OpenRouter's base URL. Client retries are off:
the plugin runs each model call as a Temporal Activity, and Temporal owns
the retries.
"""
default_headers: dict[str, str] = {}
# Optional app attribution for OpenRouter's rankings.
if referer := os.getenv("OPENROUTER_HTTP_REFERER"):
default_headers["HTTP-Referer"] = referer
if title := os.getenv("OPENROUTER_APP_TITLE"):
default_headers["X-OpenRouter-Title"] = title

client = AsyncOpenAI(
base_url="https://openrouter.ai/api/v1",
api_key=os.environ["OPENROUTER_API_KEY"],
max_retries=0,
default_headers=default_headers or None,
)
# Chat Completions is OpenRouter's primary endpoint; the Agents SDK
# defaults to the Responses API, which OpenRouter offers only in beta.
return OpenAIProvider(openai_client=client, use_responses=False)


# @@@SNIPEND


async def main():
# Disable Agents SDK tracing: the default exporter sends traces to OpenAI's
# backend, which needs an OpenAI API key that this sample does not have.
set_tracing_disabled(disabled=True)

logging.basicConfig(level=logging.WARNING)
logging.getLogger("temporalio.workflow").setLevel(logging.DEBUG)

client = await Client.connect(
"localhost:7233",
plugins=[
OpenAIAgentsPlugin(
model_params=ModelActivityParameters(
start_to_close_timeout=timedelta(seconds=60)
),
model_provider=openrouter_provider(),
),
],
)

worker = Worker(
client,
task_queue="openai-agents-model-providers-task-queue",
workflows=[
OpenRouterAgentWorkflow,
],
)
await worker.run()


if __name__ == "__main__":
asyncio.run(main())
29 changes: 29 additions & 0 deletions openai_agents/model_providers/run_openrouter_workflow.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
import asyncio

from temporalio.client import Client
from temporalio.contrib.openai_agents import OpenAIAgentsPlugin

from openai_agents.model_providers.workflows.openrouter_workflow import (
OpenRouterAgentWorkflow,
)


async def main():
client = await Client.connect(
"localhost:7233",
plugins=[
OpenAIAgentsPlugin(),
],
)

result = await client.execute_workflow(
OpenRouterAgentWorkflow.run,
"What's the weather in Tokyo?",
id="openai-agents-openrouter-workflow-id",
task_queue="openai-agents-model-providers-task-queue",
)
print(f"Result: {result}")


if __name__ == "__main__":
asyncio.run(main())
28 changes: 28 additions & 0 deletions openai_agents/model_providers/workflows/openrouter_workflow.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
from __future__ import annotations

from agents import Agent, Runner, function_tool
from temporalio import workflow

# Any OpenRouter model slug works here. A fixed, tool-capable model keeps the
# sample reproducible; swap in "openrouter/auto" to let OpenRouter choose.
OPENROUTER_MODEL = "openai/gpt-4o-mini"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we default to openrouter/auto since the README says and picks providers and models per request ?



@workflow.defn
class OpenRouterAgentWorkflow:
@workflow.run
async def run(self, prompt: str) -> str:
@function_tool
def get_weather(city: str):
workflow.logger.debug(f"Getting weather for {city}")
return f"The weather in {city} is sunny."

agent = Agent(
name="Assistant",
instructions="You only respond in haikus. When asked about the weather always use the tool to get the current weather.",
model=OPENROUTER_MODEL,
tools=[get_weather],
)

result = await Runner.run(agent, prompt)
return result.final_output
75 changes: 75 additions & 0 deletions openrouter/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
# OpenRouter

These samples call [OpenRouter](https://openrouter.ai/) from Temporal Activities. OpenRouter serves hundreds of models from many providers behind one OpenAI-compatible API and one API key, and picks providers and models per request. Temporal handles everything around those calls: retries with backoff, fan-out with bounded concurrency, crash recovery, pausing for a human, and a durable per-attempt record of what was called and what it cost.

| Sample | Description |
|--------|-------------|
| [prompt_batch](prompt_batch) | Fan one OpenRouter call out per prompt with OpenRouter's Auto Router, and collect answer, model, and cost per prompt. Shows Temporal-owned retries, `Retry-After` handling, and retries served for free from OpenRouter's response cache. Start here. |
| [budget_gate](budget_gate) | The same batch, but it pauses instead of failing when money runs out, whether a soft budget in the Workflow or OpenRouter's own "insufficient credits" error, and resumes on a `raise_budget` Update. |

For OpenRouter as the model provider behind the [OpenAI Agents SDK plugin](../openai_agents), see [openai_agents/model_providers](../openai_agents/model_providers#openrouter).

## Prerequisites

1. Follow the [repository prerequisites](../README.md), then install this sample's dependencies:

```bash
uv sync --group openrouter
```

2. Start a local dev server with the [Temporal CLI](https://docs.temporal.io/cli):

```bash
temporal server start-dev
```

3. Set an [OpenRouter API key](https://openrouter.ai/settings/keys) in the Worker's environment. A few cents of credit is enough for these samples.

```bash
export OPENROUTER_API_KEY="sk-or-v1-..."
```

Optional: set `OPENROUTER_HTTP_REFERER` and `OPENROUTER_APP_TITLE` for [app attribution](https://openrouter.ai/docs/app-attribution) in OpenRouter's rankings.

The API key stays in the Worker process. Prompts, answers, models, and costs go through the Workflow and are recorded in Event History; the key never does.

## Running a sample

Each sample has a Worker and a starter. Run them in separate terminals:

```bash
# Terminal 1
uv run --group openrouter openrouter/prompt_batch/run_worker.py

# Terminal 2
uv run --group openrouter openrouter/prompt_batch/run_workflow.py "Explain retries in one sentence." "Write a haiku about databases."
```

## How the Activity calls OpenRouter

[activities.py](activities.py) uses the `openai` SDK pointed at `https://openrouter.ai/api/v1`, which is the setup OpenRouter documents for OpenAI-compatible clients. OpenRouter-specific fields go in `extra_body`. Four things matter for durable execution:

- **Temporal owns retries.** The client is created with `max_retries=0`, so every attempt is one HTTP call and shows up in Event History. If you use OpenRouter's official `openrouter` package instead, pass `retry_config=RetryConfig("none", ...)`: by default it retries 5xx and connection errors for up to an hour, invisibly.
- **Errors are classified.** 408, 429, and 5xx raise a retryable `ApplicationError`; 400, 401, 402 (out of credits), 403 (moderation), and other 4xx raise a non-retryable one. A `Retry-After` header becomes the next retry delay. OpenRouter can also return HTTP 200 with an `error` body and no `choices`; the Activity checks for that.
- **Retries are free when the first call succeeded.** The Activity sends `X-OpenRouter-Cache: true`, so if a Worker dies after OpenRouter answered but before Temporal recorded the result, the retried, byte-identical request is served from OpenRouter's response cache and billed at $0. Nothing per-attempt goes in the request body, so attempts stay identical.
- **Heartbeats.** The Activity heartbeats so a dead Worker is detected after `heartbeat_timeout` (10s) rather than after the full `start_to_close_timeout`.

Each result carries the concrete model OpenRouter chose, OpenRouter's reported `usage.cost`, the generation id, and the cache status.

## What Temporal does and does not guarantee

Activities are at-least-once. If a Worker dies mid-call, the retry re-sends the request; within the cache TTL that retry costs nothing, but two identical requests in flight at the same time both miss the cache and both bill. Completed Activities are never re-run, so a restarted batch resumes at the first unfinished prompt.

OpenRouter decides which provider and model serve a request, in milliseconds (Auto Router, `models` fallback lists, provider preferences). Temporal decides what happens over time: waiting out a rate limit, surviving a Worker crash, pausing for hours until a human acts, and keeping the audit trail.

## Batch size

Each Activity adds a few events to the Workflow's Event History, and every answer is part of the Workflow result. These samples cap a batch at 100 prompts. For larger batches, use one Workflow per slice, or the pattern in [batch_sliding_window](../batch_sliding_window) with continue-as-new.

## Tests

The tests replace OpenRouter with a fake HTTP transport and the Activity with a fake, so they need no API key and make no network calls:

```bash
uv run --group openrouter pytest tests/openrouter
```
Empty file added openrouter/__init__.py
Empty file.
Loading
Loading