Skip to content
Merged

Dev #61

Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,10 @@ OLI_API_HOST="0.0.0.0"
OLI_API_PORT=9734
OLI_API_PROFILE="default"
OLI_API_MODE="agent"
# WebSocket permission policy: "prompt" asks the client, "auto" auto-approves.
# REST endpoints always auto-approve (no interactive channel).
OLI_API_PERMISSION_MODE="prompt"
OLI_API_PERMISSION_TIMEOUT=120

# Whether to use OAuth for authentication with third party MCP servers
OLI_USE_OAUTH=false
23 changes: 12 additions & 11 deletions AGENTS.md

Large diffs are not rendered by default.

25 changes: 17 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -125,6 +125,13 @@ can pick up where you left off.
| `/clear` | Clear the conversation |
| `/home` | Return to the home screen |
| `Ctrl+Q` / `Ctrl+L` / `Ctrl+Y` | Quit / Clear / Copy last message |
| `Ctrl+I` (or `Esc` while generating) | Interrupt the current response; the prompt is restored to the input box |

## Interrupting a response

Press `Ctrl+I` (or `Esc`) while the assistant is streaming or running tools to stop the turn. The partial response stays on screen marked `(interrupted)`, the originating user message is removed from history, and its text is restored to the input box so you can edit and resubmit. Shell commands started by the turn are killed, and a delegated `dispatch` fan-out is cancelled rather than left running in the background.

> The Transformers backend cannot currently be interrupted mid-generation: its synchronous `TextIteratorStreamer` loop only observes cancellation once the current generation finishes. Local Transformers runs therefore stop at the next checkpoint, not instantly. Ollama, OpenAI, and HuggingFace backends cancel promptly.

## Voice mode (experimental)

Expand Down Expand Up @@ -207,9 +214,9 @@ It listens on `0.0.0.0:9734` by default (override with `OLI_API_HOST`/`OLI_API_P
| `WS /v1/chat` | Stateful WebSocket — per-connection history, relays every agent event as a typed JSON frame |
| `GET /health` | Liveness probe |

REST conversations are **stateless** (like real OpenAI): each `/v1/chat/completions` request carries its full message history. The server holds a single process-private `Agent` instance (backend, tool registrations, MCP wiring) shared across requests, and serializes concurrent in-flight requests in-process. Because there is no human to prompt at permission time, the API auto-allows permission scopes for the current request; offline and dry-run gating from `AppConfig` still apply.
REST conversations are **stateless** (like real OpenAI): each `/v1/chat/completions` request carries its full message history. The server holds a single process-private `Agent` instance (backend, tool registrations, MCP wiring) shared across requests, and serializes concurrent in-flight requests in-process. Because a stateless REST request has no interactive channel to prompt at permission time, REST auto-allows permission scopes for the current request; offline and dry-run gating from `AppConfig` still apply.

The `WS /v1/chat` WebSocket is the stateful counterpart for real-time browser UIs: the server keeps a per-connection `messages` history, so a client sends each next turn as `{"content": "..."}` and receives every `AgentEvent` back as a typed JSON frame (`text_chunk`/`thinking`/`tool_call_executing`/`tool_call_result`/`assistant_response`/`usage`/`error`/`done`), plus `sub_agent_started`/`sub_agent_progress`/`sub_agent_completed` and `todo` frames for delegated runs and live task lists; `{"action": "clear"}` resets the history. See [docs/API_SERVER.md](docs/API_SERVER.md) for the full frame reference.
The `WS /v1/chat` WebSocket is the stateful counterpart for real-time browser UIs: the server keeps a per-connection `messages` history, so a client sends each next turn as `{"content": "..."}` and receives every `AgentEvent` back as a typed JSON frame (`text_chunk`/`thinking`/`tool_call_executing`/`tool_call_result`/`assistant_response`/`usage`/`error`/`done`), plus `sub_agent_started`/`sub_agent_progress`/`sub_agent_completed` and `todo` frames for delegated runs and live task lists. Unlike REST, the WebSocket **prompts for permission** by default: the server sends a `permission_request` frame and waits for `{"action": "permission_response", "id": ..., "decision": "once"|"session"|"deny"}` (120 s timeout → deny). Set `OLI_API_PERMISSION_MODE=auto` to restore the legacy auto-approve behaviour. `{"action": "clear"}` resets the history; `{"action": "interrupt"}` cancels the in-flight turn (the server emits `interrupted` then `done`, and rolls the partial user turn back out of history). See [docs/API_SERVER.md](docs/API_SERVER.md) for the full frame reference.

### curl

Expand Down Expand Up @@ -256,12 +263,14 @@ The `openai` SDK needs an `api_key`; pass a placeholder — the server does not

### API configuration

| Env var | Default | Description |
| ----------------- | --------- | ------------------------------------------- |
| `OLI_API_HOST` | `0.0.0.0` | Bind address |
| `OLI_API_PORT` | `9734` | Listen port |
| `OLI_API_PROFILE` | `default` | Agent profile to load |
| `OLI_API_MODE` | `agent` | Agent mode (`agent`, `ask`, `plan`, `chat`) |
| Env var | Default | Description |
| --------------------------- | --------- | ----------------------------------------------------------------- |
| `OLI_API_HOST` | `0.0.0.0` | Bind address |
| `OLI_API_PORT` | `9734` | Listen port |
| `OLI_API_PROFILE` | `default` | Agent profile to load |
| `OLI_API_MODE` | `agent` | Agent mode (`agent`, `ask`, `plan`, `chat`) |
| `OLI_API_PERMISSION_MODE` | `prompt` | WebSocket permission policy: `prompt` (ask the client) or `auto` |
| `OLI_API_PERMISSION_TIMEOUT`| `120` | Seconds to wait for a WebSocket permission reply before denying |

`max_tokens` / `temperature` request fields are accepted but best-effort: the agent tool loop reads them from `AppConfig` (configured via `OLI_*` env or `settings.json`).

Expand Down
78 changes: 68 additions & 10 deletions docs/API_SERVER.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,10 +72,13 @@ with the same `OLI_*` env vars / `settings.json` the TUI uses.
across `await` points. Concurrent requests queue. (A threading lock would not
work: it is per-thread reentrant, and all coroutines sharing the event loop
run on the same thread.)
- **No human in the loop** — the permission confirm-callback auto-defaults to
`"session"` for every scope (the REST equivalent of the TUI's "Allow for
session"). Offline mode and dry-run gating from `AppConfig` still apply
unchanged.
- **Permission handling** — REST has no interactive channel, so its confirm
callback auto-defaults to `"session"` for every scope (the REST equivalent of
the TUI's "Allow for session"). The stateful WebSocket does have a channel, so
it prompts the client by default (`permission_request` → `permission_response`)
and denies on timeout. Offline mode and dry-run gating from `AppConfig` still
apply unchanged on both surfaces. Set `OLI_API_PERMISSION_MODE=auto` to restore
auto-approval for WebSocket clients too.

## Endpoints

Expand Down Expand Up @@ -251,6 +254,9 @@ A stateful WebSocket for real-time browser UIs. The server maintains a
per-connection `messages` history, so a client sends each next user turn as
`{"content": "..."}` and receives every `AgentEvent` back as a typed JSON frame.
`{"action": "clear"}` resets the per-connection history.
`{"action": "interrupt"}` cancels the in-flight turn: the server emits an
`interrupted` frame followed by `done`, and rolls the partial user turn back out
of the history so the client can resubmit it.

The server sends a `{"type": "connected"}` frame on accept, then one frame per
inbound message:
Expand All @@ -265,13 +271,47 @@ inbound message:
| `token_count` | `{"prompt_tokens", "completion_tokens", "total_tokens", "estimated", "phase"}` — live context snapshot (`phase` is `prompt`/`thinking`/`round`/`tool_result`/`final`) |
| `usage` | `{"prompt_tokens": ..., "completion_tokens": ..., ...}` |
| `error` | `{"message": "..."}` |
| `done` | `{"full_text": "..."}` — run finished |
| `interrupted` | `{"full_text": "..."}` — partial text when a turn is interrupted |
| `done` | `{"full_text": "...", "interrupted": true?}` — run finished |
| `cleared` | `{}` — history reset (reply to `action: clear`) |
| `permission_request` | `{"id", "description", "options", "default", "timeout"}` — reply required |
| `permission_timeout` | `{"id"}` — the prompt timed out and resolved to deny |
| `sub_agent_started` | `{"task_id", "agent_name", "pool_name", "task"}` |
| `sub_agent_progress` | `{"task_id", "agent_name", "activity", "status"}` |
| `sub_agent_completed` | `{"task_id", "agent_name", "status", "full_text"}` |
| `todo` | `{"todos": [...]}` (root) or with `task_id`/`agent_name` (sub) |

#### Permissions

By default (`api_server.permission_mode = "prompt"`), a permission-gated tool
sends:

```json
{
"type": "permission_request",
"data": {
"id": "perm-…",
"description": "Write to /path/file.txt",
"options": ["once", "session", "deny"],
"default": "deny",
"timeout": 120.0
}
}
```

The client replies with:

```json
{"action": "permission_response", "id": "perm-…", "decision": "once"}
```

`once` allows just this call, `session` grants the scope for the rest of the
server process, and `deny` (the timeout default) returns a permission error to
the model. An unknown/expired `id` or an invalid decision produces an `error`
frame. Set `OLI_API_PERMISSION_MODE=auto` to skip prompts entirely. The pending
prompt holds the process-wide agent lock, so all other turns queue until the
client answers or the timeout elapses.

Sub-agent activity is relayed live while a `dispatch` tool call is in flight:
frames carry the same shape as above (`text_chunk`, `tool_call_executing`,
…) with `task_id` and `agent_name` attached so the client can demux by run.
Expand All @@ -294,9 +334,18 @@ async def main():
break
if frame["type"] == "text_chunk":
print(frame["data"]["text"], end="")
elif frame["type"] == "permission_request":
prompt = frame["data"]
print("\nPERMISSION:", prompt["description"])
await ws.send(json.dumps({
"action": "permission_response",
"id": prompt["id"],
"decision": "deny", # or "once" / "session"
}))
elif frame["type"] == "error":
print("ERROR:", frame["data"]["message"])
# clear with: await ws.send(json.dumps({"action": "clear"}))
# interrupt with: await ws.send(json.dumps({"action": "interrupt"}))

asyncio.run(main())
```
Expand All @@ -314,7 +363,8 @@ field shape (`openai_model`, `truncation_max_chars_small`, `api_port`, ...).
`GET /v1/config` returns the current flat config — one key per
`settings.json` leaf, so every setting the TUI's `/config` screen edits is
reachable here (`session_*`, `workspace_max_workspaces`, `ollama_context_window`,
`first_chunk_timeout`, `openai_responses_enabled`, ...):
`first_chunk_timeout`, `openai_responses_enabled`, `api_permission_mode`,
`api_permission_timeout`, ...):

```bash
curl http://localhost:9734/v1/config
Expand All @@ -325,7 +375,9 @@ curl http://localhost:9734/v1/config
"backend": "ollama",
"ollama_model": "llama3.1",
"temperature": 0.7,
"api_port": 9734
"api_port": 9734,
"api_permission_mode": "prompt",
"api_permission_timeout": 120.0
}
```

Expand Down Expand Up @@ -448,9 +500,10 @@ is what the TUI's `/profile load` does.

- **Workspace** — the agent's workspace defaults to the process CWD unless the
CWD is a sensitive path (see `is_sensitive_path` in
[`sessions.py`](../oli_bot/sessions.py)). File and shell tools that touch paths
outside the workspace are auto-approved in API mode but still respect
offline/dry-run gating.
[`sessions.py`](../oli_bot/sessions.py)). REST requests auto-approve file and
shell tools that touch paths outside the workspace; WebSocket clients receive
a `permission_request` instead (unless `api_permission_mode = "auto"`). Both
surfaces still respect offline/dry-run gating.
- **Modes** — in `ask` / `plan` modes only read-only tools are exposed (plus
MCP limits per mode); in `chat` mode no tools are offered at all. The mode also
shapes the system prompt, exactly as in the TUI.
Expand All @@ -461,5 +514,10 @@ is what the TUI's `/profile load` does.
- **Concurrency** — requests are serialized on a process-wide lock; there is no
horizontal multiplexing. For concurrent consumers, run multiple `oli-server`
processes.
- **Interrupts** — `{"action": "interrupt"}` cancels the in-flight turn, kills
shell subprocesses started by it, and cancels a delegated `dispatch` fan-out.
The partial user turn is rolled back and the client receives `interrupted`
followed by `done` (`interrupted: true`). The local Transformers backend only
observes cancellation after its current generation pass finishes.
- **Logging** — the server writes NDJSON to `AppConfig.log_file` via the shared
logger in [`logger.py`](../oli_bot/logger.py).
12 changes: 11 additions & 1 deletion docs/CONFIGURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,8 @@ If `~/.config/oli/settings.json` does not exist, it is auto-created on first loa
| `request_timeout` | `30.0` | `OLI_REQUEST_TIMEOUT` | HTTP request timeout (seconds) |
| `max_messages` | `100` | `OLI_MAX_MESSAGES` | Auto-prune chat history when message count exceeds this |
| `max_tool_iterations` | `25` | `OLI_MAX_TOOL_ITERATIONS` | Max tool-calling rounds per response |
| `stream_timeout` | `240.0` | `OLI_STREAM_TIMEOUT` | Seconds before streaming response times out |
| `stream_timeout` | `300.0` | `OLI_STREAM_TIMEOUT` | Seconds of inactivity before a streaming response times out |
| `first_chunk_timeout` | `600.0` | `OLI_FIRST_CHUNK_TIMEOUT` | Seconds to wait for the first streamed chunk (large local models can cold-start slowly) |
| `model_filters` | `""` | `OLI_MODEL_FILTERS` | Comma-separated model name exclusion filters |
| `truncation_max_chars_small` | `4000` | `OLI_TRUNCATION_SMALL` | Max chars for tool results under the small model tier |
| `truncation_max_chars_large` | `100000` | `OLI_TRUNCATION_LARGE` | Max chars for tool results under the large model tier |
Expand All @@ -66,6 +67,15 @@ If `~/.config/oli/settings.json` does not exist, it is auto-created on first loa
| `voice_vad_aggressiveness` | `2` | `OLI_VOICE_VAD_AGGRESSIVENESS` | WebRTC VAD aggressiveness `0`–`3`; higher rejects more noise |
| `voice_silence_timeout_ms` | `800` | `OLI_VOICE_SILENCE_TIMEOUT_MS` | Stop recording after this much consecutive silence |
| `voice_max_record_seconds` | `15` | `OLI_VOICE_MAX_RECORD_SECONDS` | Hard cap on a single recording |
| `api_host` | `0.0.0.0` | `OLI_API_HOST` | `oli-server` bind address |
| `api_port` | `9734` | `OLI_API_PORT` | `oli-server` listen port |
| `api_profile` | `default` | `OLI_API_PROFILE` | Profile loaded by `oli-server` |
| `api_mode` | `agent` | `OLI_API_MODE` | Agent mode for `oli-server` (`agent`, `ask`, `plan`, `chat`) |
| `api_permission_mode` | `prompt` | `OLI_API_PERMISSION_MODE` | WebSocket permission policy: `prompt` asks the client; `auto` keeps the legacy auto-approve behaviour. REST always auto-approves. |
| `api_permission_timeout` | `120.0` | `OLI_API_PERMISSION_TIMEOUT` | Seconds to wait for a WebSocket `permission_response` before denying |

> The `/config` screen renders every setting above, and `GET/PUT /v1/config`
> exposes the same flat field names for browser clients.

## Quick examples

Expand Down
Loading
Loading