Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 47 additions & 21 deletions mkdocs/docs/concepts/presets.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ Presets offer a toolkit that streamlines agent-based model inference optimizatio
??? info "Prerequisites"
Before using presets, make sure you’ve [installed](../installation.md) the server and CLI, and created a [fleet](fleets.md).

Creating a preset requires the `claude` CLI to be installed on the machine where you create a preset.
Creating a preset requires the `claude` or `codex` CLI to be installed on the machine where you create a preset.

## Apply a configuration

Expand Down Expand Up @@ -68,7 +68,7 @@ Create the preset dsv4-flash? [y/n]: y
> optimization is done against that hardware. Point `dstack apply` to a fleet configured
> correspondingly, via `fleets` inside the preset configuration or via `--fleet` in the CLI.

The command executes entirely locally and uses the locally installed `claude` CLI along with `dstack`'s bundled skills. The agent uses a `dstack` task to find the best serving configuration for the available fleet offers, then submits it as a `dstack` service for a final benchmark.
The command executes entirely locally and uses the locally installed `claude` or `codex` CLI along with `dstack`'s bundled skills (see [Agent](#agent)). The agent uses a `dstack` task to find the best serving configuration for the available fleet offers, then submits it as a `dstack` service for a final benchmark.

You can stop watching with `Ctrl`+`C` at any time. The agent keeps running, and `dstack preset logs -f` follows it again. Resume an interrupted creation with `dstack preset resume`:

Expand All @@ -84,24 +84,6 @@ When resuming, the configuration and constraints are read from the original sess

To stop a creation and its runs, use `dstack preset stop`.

??? info "Claude configuration"
By default, preset creation uses the existing `claude` login. To use an Anthropic API key instead, set:

```shell
export DSTACK_AGENT_ANTHROPIC_API_KEY=...
```

By default, the agent sets neither a model nor an effort level, so the `claude` CLI's built-in defaults apply. To override them, set:

```shell
export DSTACK_AGENT_ANTHROPIC_MODEL=claude-fable-5-1
export DSTACK_AGENT_CLAUDE_EFFORT=max
```

See the [Models overview](https://platform.claude.com/docs/en/models/overview) for the available models and their IDs.

Supported effort levels are `low`, `medium`, `high`, `xhigh`, and `max`.

??? info "Presets directory"
The verified presets are saved locally under `~/.dstack/presets`, and `dstack preset` reads them from there. Presets aren't stored on the server.

Expand All @@ -116,6 +98,50 @@ Alternatively, pass `--fleet` to `dstack apply`.
> Profile settings such as `spot_policy`, `max_price`, and `backends` are ignored during preset
> creation. Configure them on the fleet instead.

### Agent

Set `agent` to choose which agent CLI creates the preset. Optionally, pin the model and the reasoning effort; otherwise the CLI's defaults apply.

=== "Claude"

```yaml
agent:
provider: claude
model: claude-fable-5-1
effort: max
```

Model IDs are listed in the [Models overview](https://platform.claude.com/docs/en/models/overview). Supported effort levels are `low`, `medium`, `high`, `xhigh`, and `max`.

To set the defaults for all presets, use `DSTACK_AGENT_PROVIDER=claude`, `DSTACK_AGENT_ANTHROPIC_MODEL`, and `DSTACK_AGENT_CLAUDE_EFFORT`.

??? info "Authentication"
By default, the agent uses your existing `claude` login. To use an Anthropic API key instead, set:

```shell
export DSTACK_AGENT_ANTHROPIC_API_KEY=...
```

=== "Codex"

```yaml
agent:
provider: codex
model: gpt-6-astra
effort: xhigh
```

Run `codex debug models` to list model IDs. Supported effort levels are `low`, `medium`, `high`, and `xhigh`.

To set the defaults for all presets, use `DSTACK_AGENT_PROVIDER=codex`, `DSTACK_AGENT_OPENAI_MODEL`, and `DSTACK_AGENT_CODEX_EFFORT`.

??? info "Authentication"
By default, the agent uses your existing `codex` login and configuration, without its MCP servers. To use an OpenAI API key instead, set:

```shell
export DSTACK_AGENT_OPENAI_API_KEY=...
```

### Model

=== "Base"
Expand Down Expand Up @@ -354,7 +380,7 @@ $ dstack preset delete c83375b4

## Protips

Under the hood, presets run an agent as a subprocess, using the local `claude` CLI. This process writes a real-time trace to `~/.dstack/presets/<id>/trace.jsonl`. The subprocess is launched with a built-in harness: how to run trials, submit runs, benchmark, verify presets, and use `dstack`.
Under the hood, presets run an agent as a subprocess, using the local `claude` or `codex` CLI. This process writes a real-time trace to `~/.dstack/presets/<id>/trace.jsonl`. The subprocess is launched with a built-in harness: how to run trials, submit runs, benchmark, verify presets, and use `dstack`.

At the same time, it's recommended to create presets using your own agent — either via a CLI such as Claude Code, or inside your IDE. Your agent helps you design the preset configuration, formulate hypotheses, and — most importantly — analyze the session's traces as well as the trial results (stored under `~/.dstack/presets/<id>/trials/<n>/trial.json`), to decide what the next session can be and what instructions to give it via `prompt`.

Expand Down
13 changes: 10 additions & 3 deletions mkdocs/docs/reference/cli/dstack/preset.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,15 +43,22 @@ $ dstack preset create --help

##### Agent settings

Preset creation uses the existing `claude` login unless
`DSTACK_AGENT_ANTHROPIC_API_KEY` is set.
Presets are created with the `claude` CLI by default, or with `codex` when the
configuration's `agent` or `DSTACK_AGENT_PROVIDER` selects it. The configuration
takes precedence over the variables below. Each CLI uses its existing login unless
its API key variable is set.

| Variable | Description |
| --- | --- |
| `DSTACK_AGENT_ANTHROPIC_API_KEY` | Anthropic API key used by the agent. |
| `DSTACK_AGENT_PROVIDER` | The agent CLI: `claude` (default) or `codex`. |
| `DSTACK_AGENT_ANTHROPIC_API_KEY` | Anthropic API key used by the `claude` agent. |
| `DSTACK_AGENT_CLAUDE_PATH` | `claude` executable name or path. Defaults to `claude` from `PATH`. |
| `DSTACK_AGENT_ANTHROPIC_MODEL` | Claude model used by the agent. If unset, the `claude` CLI's built-in default is used. |
| `DSTACK_AGENT_CLAUDE_EFFORT` | Claude effort level: `low`, `medium`, `high`, `xhigh`, or `max`. If unset, the `claude` CLI default is used. |
| `DSTACK_AGENT_OPENAI_API_KEY` | OpenAI API key used by the `codex` agent. |
| `DSTACK_AGENT_CODEX_PATH` | `codex` executable name or path. Defaults to `codex` from `PATH`. |
| `DSTACK_AGENT_OPENAI_MODEL` | Model used by the `codex` agent. If unset, the `codex` CLI's built-in default is used. |
| `DSTACK_AGENT_CODEX_EFFORT` | Codex reasoning effort: `low`, `medium`, `high`, or `xhigh`. If unset, the `codex` CLI default is used. |

Agent progress is written to `agent.log` under `~/.dstack/presets/<preset-id>/`,
alongside the effective configuration (`preset.dstack.yml`), the recorded
Expand Down
6 changes: 6 additions & 0 deletions mkdocs/docs/reference/dstack.yml/preset.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,12 @@ used to create or apply a [preset](../../concepts/presets.md).
type:
required: true

### `agent`

#SCHEMA# dstack._internal.core.models.configurations.PresetAgentConfig
overrides:
show_root_heading: false

### `model`

=== "Base model"
Expand Down
7 changes: 6 additions & 1 deletion mkdocs/docs/reference/env.md
Original file line number Diff line number Diff line change
Expand Up @@ -217,7 +217,12 @@ $ find ~/.dstack/logs/cli/
</div>

- `DSTACK_PROJECT`{ #DSTACK_PROJECT } – Has the same effect as `--project`. Defaults to `None`.
- `DSTACK_AGENT_ANTHROPIC_API_KEY`{ #DSTACK_AGENT_ANTHROPIC_API_KEY } – The Anthropic API key used by the preset agent. If unset, the existing `claude` login is used.
- `DSTACK_AGENT_PROVIDER`{ #DSTACK_AGENT_PROVIDER } – The agent CLI that creates presets: `claude` (default) or `codex`. The preset configuration's `agent` block overrides the `DSTACK_AGENT_*` variables.
- `DSTACK_AGENT_ANTHROPIC_API_KEY`{ #DSTACK_AGENT_ANTHROPIC_API_KEY } – The Anthropic API key used by the `claude` preset agent. If unset, the existing `claude` login is used.
- `DSTACK_AGENT_CLAUDE_PATH`{ #DSTACK_AGENT_CLAUDE_PATH } – The `claude` executable name or path used by the preset agent. Defaults to `claude` from `PATH`.
- `DSTACK_AGENT_ANTHROPIC_MODEL`{ #DSTACK_AGENT_ANTHROPIC_MODEL } – The Claude model used by the preset agent. If unset, the `claude` CLI's built-in default is used.
- `DSTACK_AGENT_CLAUDE_EFFORT`{ #DSTACK_AGENT_CLAUDE_EFFORT } – The Claude effort level used by the preset agent. Can be `low`, `medium`, `high`, `xhigh`, or `max`. If unset, the `claude` CLI default is used.
- `DSTACK_AGENT_OPENAI_API_KEY`{ #DSTACK_AGENT_OPENAI_API_KEY } – The OpenAI API key used by the `codex` preset agent. If unset, the existing `codex` login is used.
- `DSTACK_AGENT_CODEX_PATH`{ #DSTACK_AGENT_CODEX_PATH } – The `codex` executable name or path used by the preset agent. Defaults to `codex` from `PATH`.
- `DSTACK_AGENT_OPENAI_MODEL`{ #DSTACK_AGENT_OPENAI_MODEL } – The model used by the `codex` preset agent. If unset, the `codex` CLI's built-in default is used.
- `DSTACK_AGENT_CODEX_EFFORT`{ #DSTACK_AGENT_CODEX_EFFORT } – The reasoning effort used by the `codex` preset agent. Can be `low`, `medium`, `high`, or `xhigh`. If unset, the `codex` CLI default is used.
1 change: 1 addition & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ dependencies = [
"requests",
"requests-unixsocket>=0.4.1",
"typing-extensions>=4.0.0",
"tomli>=2.0; python_version < '3.11'",
"cryptography",
"packaging",
"python-dateutil",
Expand Down
51 changes: 41 additions & 10 deletions src/dstack/_internal/cli/models/preset_agent.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
)

from dstack._internal.core.models.common import CoreModel
from dstack._internal.core.models.configurations import ServiceConfiguration
from dstack._internal.core.models.configurations import PresetAgentProvider, ServiceConfiguration
from dstack._internal.core.models.presets import PresetBenchmark


Expand Down Expand Up @@ -98,12 +98,15 @@ class PresetSessionRun(CoreModel):

workspace: PresetSessionWorkspace
finalize: PresetSessionFinalize
# Only known when this CLI launched the agent; a follower leaves it as is.
claude_model: Optional[str]
# None between claude process attempts and after a detach outlives them.
agent: Optional[PresetSessionProcess]
# None until the agent's stream reveals it.
claude_session_id: Optional[str]
# A resume launches the same agent CLI the session started with.
agent_provider: PresetAgentProvider
# The model pinned by the user; None leaves the choice to the agent CLI.
agent_model: Optional[str]
# The agent process; None between attempts and after a detach outlives them.
session_process: Optional[PresetSessionProcess]
# The agent CLI's own session, which its resume continues; None until the
# agent's stream reveals it.
session_id: Optional[str]


class PresetSessionState(CoreModel):
Expand All @@ -126,14 +129,15 @@ class PresetSessionState(CoreModel):


class PresetAgentInfo(CoreModel):
"""`agent.json`: how the claude agent was launched. A debug record."""
"""`agent.json`: how the agent CLI was launched. A debug record."""

provider: PresetAgentProvider
executable: str
version: Optional[str]
auth_status: str
# None is the claude CLI's default.
# None is the agent CLI's default.
effort: Optional[str]
# Reported by claude on its init line; None until then.
# Reported by the agent CLI once it starts; None until then.
model: Optional[str]


Expand Down Expand Up @@ -169,3 +173,30 @@ class ClaudeResultEvent(ClaudeStreamEvent):
AnyClaudeStreamEvent = Annotated[
Union[ClaudeResultEvent, ClaudeStreamEvent], Field(union_mode="left_to_right")
]


class CodexStreamItem(CoreModel):
# "agent_message", "command_execution", "web_search", "reasoning", and others.
type: str
# The message text of an "agent_message"; the final one is the JSON report.
text: Optional[str] = None


class CodexStreamError(CoreModel):
message: str


class CodexStreamEvent(CoreModel):
"""One line of `codex exec --json`. Not our format: unknown fields are dropped
and omitted fields default."""

# "thread.started", "turn.started", "item.started", "item.completed",
# "turn.completed", "turn.failed", "error", and whatever a newer codex adds.
type: str
# On "thread.started": the id `codex exec resume` takes.
thread_id: Optional[str] = None
item: Optional[CodexStreamItem] = None
# On "turn.failed".
error: Optional[CodexStreamError] = None
# On "error": a stream or API failure outside a turn.
message: Optional[str] = None
Loading
Loading