The API server lives in the oli_bot/api/ package (a FastAPI
app factory in app.py, route modules in routers/, plus harness.py,
runner.py, session_service.py, errors.py, convert.py, and deps.py).
oli_bot/api_server.py is a compatibility shim that
re-exports the same public surface as the historical single module. It exposes
the same Agent
harness that powers the TUI over an OpenAI-compatible REST API plus a stateful
WebSocket. Any client that speaks the OpenAI wire protocol — the openai Python
SDK, curl, or a custom HTTP client — can drive the full tool-calling agent,
including built-in tools, MCP servers, profiles, and modes.
The oli-server console script starts the server:
oli-serverOn startup it prints a banner with the resolved backend, model, mode, profile,
and URL. By default it listens on 0.0.0.0:9734.
The server uses the same AppConfig as the TUI (see
CONFIGURE.md) for backend, model, offline/dry-run, and profile
settings. Four fields are specific to the server:
| Setting (env var) | Default | Description |
|---|---|---|
api_host (OLI_API_HOST) |
0.0.0.0 |
Bind address |
api_port (OLI_API_PORT) |
9734 |
Listen port |
api_profile (OLI_API_PROFILE) |
default |
Profile the API server loads at startup (independent of the TUI's OLI_DEFAULT_PROFILE) |
api_mode (OLI_API_MODE) |
agent |
Mode: agent / ask / chat / plan |
Examples:
OLI_API_HOST=127.0.0.1 OLI_API_PORT=9000 OLI_API_MODE=ask oli-serveror via the api_server section of ~/.config/oli/settings.json:
{
"api_server": {
"host": "127.0.0.1",
"port": 9000,
"profile": "researcher",
"mode": "ask"
}
}Note the server has no --model / --url CLI flags; set the backend and model
with the same OLI_* env vars / settings.json the TUI uses.
- Stateless REST — every
POST /v1/chat/completionsrequest carries the fullmessageshistory and is answered with a complete completion, mirroring real OpenAI semantics. The server does not persist conversation state between requests. - Shared agent — a single process-private
Agent(backend, tool registrations, MCP wiring, profile) is built once (bymain()or the app lifespan'sinit_state()) and reused across requests, so connections are never rebuilt per call. - Serialized runs — the shared
Agentis not concurrent-safe, so in-flightAgent.process()runs are serialized process-wide with anasyncio.Lockheld acrossawaitpoints. Concurrent requests queue. (A threading lock would not work: it is per-thread reentrant, and all coroutines sharing the event loop run on the same thread.) - Permission handling — REST has no interactive channel, so its confirm
callback auto-defaults to
"session"for every scope (the REST equivalent of the TUI's "Allow for session"). The stateful WebSocket does have a channel, so it prompts the client by default (permission_request→permission_response) and denies on timeout. Offline mode and dry-run gating fromAppConfigstill apply unchanged on both surfaces. SetOLI_API_PERMISSION_MODE=autoto restore auto-approval for WebSocket clients too.
Liveness probe. Returns {"status": "ok"}.
Lists the single active model.
curl http://localhost:9734/v1/models{
"object": "list",
"data": [
{
"id": "oli-bot-<model>",
"object": "model",
"created": 1730000000,
"owned_by": "oli"
}
]
}Non-streaming completion:
curl http://localhost:9734/v1/chat/completions \
-H 'Content-Type: application/json' \
-d '{
"model": "gpt-4o",
"messages": [{"role": "user", "content": "List the files in this repo"}]
}'Request body mirrors the OpenAI schema. model is accepted but optional; the
server always uses its configured model. Recognized fields:
messages(required) — array of{role, content, name?}messagesstream—truefor SSE streamingtemperature,max_tokens,max_completion_tokens,top_p,stop,n— accepted for compatibility (the agent loop applies its own config-driven sampler defaults)
content may be a plain string or a list of parts for multimodality:
{
"messages": [
{
"role": "user",
"content": [
{ "type": "text", "text": "What is in this image?" },
{
"type": "image_url",
"image_url": { "url": "data:image/png;base64,iVBORw0KGgo=..." }
}
]
}
]
}image_url parts with data: URIs become ImageAttachments and ride through the
tool loop for vision-capable backends. Non-data: image URLs degrade to a
textual [image: <url>] placeholder.
Non-streaming response:
{
"id": "chatcmpl-<hex>",
"object": "chat.completion",
"created": 1730000000,
"model": "gpt-4o",
"choices": [
{
"index": 0,
"message": { "role": "assistant", "content": "The repo contains ..." },
"finish_reason": "stop"
}
],
"usage": {
"prompt_tokens": 18,
"completion_tokens": 42,
"total_tokens": 60
}
}usage is the summed per-call usage reported by the backend for the whole
agent run (prompt + completion across every tool-loop round). When the backend
reports no counts, it falls back to a completion-only ~chars/4 estimate.
Errors are returned in OpenAI format with HTTP 500:
{
"error": {
"message": "The agent produced no response.",
"type": "server_error",
"code": "empty_response"
}
}A failed run (backend error, tool error, empty reply) surfaces as
code: "agent_error"; an empty completion uses code: "empty_response".
Set "stream": true. The server returns text/event-stream SSE:
curl -N http://localhost:9734/v1/chat/completions \
-H 'Content-Type: application/json' \
-d '{"stream": true, "messages": [{"role": "user", "content": "hi"}]}'data: {"id":"chatcmpl-...","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"role":"assistant","content":""}}]}
data: {"id":"chatcmpl-...","choices":[{"index":0,"delta":{"content":"Hello"}}]}
...
data: {"id":"chatcmpl-...","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}
data: [DONE]
Streaming errors cannot change the HTTP status (the stream has already begun), so
a failed run emits one final SSE frame whose payload is {"error": {...}} before
data: [DONE].
from openai import OpenAI
client = OpenAI(base_url="http://localhost:9734/v1", api_key="unused")
resp = client.chat.completions.create(
model="gpt-4o",
messages=[{"role": "user", "content": "Explain the tool loop"}],
)
print(resp.choices[0].message.content)Streaming works unchanged:
stream = client.chat.completions.create(
model="gpt-4o",
messages=[{"role": "user", "content": "Count to ten"}],
stream=True,
)
for chunk in stream:
if chunk.choices and chunk.choices[0].delta.content:
print(chunk.choices[0].delta.content, end="")A stateful WebSocket for real-time browser UIs. The server maintains a
per-connection messages history, so a client sends each next user turn as
{"content": "..."} and receives every AgentEvent back as a typed JSON frame.
{"action": "clear"} resets the per-connection history.
{"action": "interrupt"} cancels the in-flight turn: the server emits an
interrupted frame followed by done, and rolls the partial user turn back out
of the history so the client can resubmit it.
The server sends a {"type": "connected"} frame on accept, then one frame per
inbound message:
| Frame type | Payload |
|---|---|
text_chunk |
{"text": "..."} — streamed assistant text |
thinking |
{"text": "..."} — model reasoning block |
tool_call_executing |
{"name": "...", "parameters": {...}} |
tool_call_result |
{"name": "...", "result": "..."} |
assistant_response |
{"content": "..."} — final assembled assistant text |
token_count |
{"prompt_tokens", "completion_tokens", "total_tokens", "estimated", "phase"} — live context snapshot (phase is prompt/thinking/round/tool_result/final) |
usage |
{"prompt_tokens": ..., "completion_tokens": ..., ...} |
error |
{"message": "..."} |
interrupted |
{"full_text": "..."} — partial text when a turn is interrupted |
done |
{"full_text": "...", "interrupted": true?} — run finished |
cleared |
{} — history reset (reply to action: clear) |
permission_request |
{"id", "description", "options", "default", "timeout"} — reply required |
permission_timeout |
{"id"} — the prompt timed out and resolved to deny |
sub_agent_started |
{"task_id", "agent_name", "pool_name", "task"} |
sub_agent_progress |
{"task_id", "agent_name", "activity", "status"} |
sub_agent_completed |
{"task_id", "agent_name", "status", "full_text"} |
todo |
{"todos": [...]} (root) or with task_id/agent_name (sub) |
By default (api_server.permission_mode = "prompt"), a permission-gated tool
sends:
{
"type": "permission_request",
"data": {
"id": "perm-…",
"description": "Write to /path/file.txt",
"options": ["once", "session", "deny"],
"default": "deny",
"timeout": 120.0
}
}The client replies with:
{"action": "permission_response", "id": "perm-…", "decision": "once"}once allows just this call, session grants the scope for the rest of the
server process, and deny (the timeout default) returns a permission error to
the model. An unknown/expired id or an invalid decision produces an error
frame. Set OLI_API_PERMISSION_MODE=auto to skip prompts entirely. The pending
prompt holds the process-wide agent lock, so all other turns queue until the
client answers or the timeout elapses.
Sub-agent activity is relayed live while a dispatch tool call is in flight:
frames carry the same shape as above (text_chunk, tool_call_executing,
…) with task_id and agent_name attached so the client can demux by run.
todo frames are emitted whenever builtin__todowrite updates a task list —
the root list has no task_id; a sub-agent's list adds task_id/agent_name.
Example client:
import asyncio, json
import websockets
async def main():
async with websockets.connect("ws://localhost:9734/v1/chat") as ws:
print(await ws.recv()) # {"type": "connected", ...}
await ws.send(json.dumps({"content": "List the todos"}))
while True:
frame = json.loads(await ws.recv())
if frame["type"] == "done":
break
if frame["type"] == "text_chunk":
print(frame["data"]["text"], end="")
elif frame["type"] == "permission_request":
prompt = frame["data"]
print("\nPERMISSION:", prompt["description"])
await ws.send(json.dumps({
"action": "permission_response",
"id": prompt["id"],
"decision": "deny", # or "once" / "session"
}))
elif frame["type"] == "error":
print("ERROR:", frame["data"]["message"])
# clear with: await ws.send(json.dumps({"action": "clear"}))
# interrupt with: await ws.send(json.dumps({"action": "interrupt"}))
asyncio.run(main())Invalid JSON payloads, non-object payloads, and empty messages are answered with
{"type": "error", "data": {"message": "..."}} and the connection stays open.
Management endpoints backing the browser UI's settings view. They read and write
~/.config/oli/settings.json through the same SettingsManager the TUI uses,
translating between the nested settings format and a flat key-per-AppConfig-
field shape (openai_model, truncation_max_chars_small, api_port, ...).
GET /v1/config returns the current flat config — one key per
settings.json leaf, so every setting the TUI's /config screen edits is
reachable here (session_*, workspace_max_workspaces, ollama_context_window,
first_chunk_timeout, openai_responses_enabled, api_permission_mode,
api_permission_timeout, ...):
curl http://localhost:9734/v1/config{
"backend": "ollama",
"ollama_model": "llama3.1",
"temperature": 0.7,
"api_port": 9734,
"api_permission_mode": "prompt",
"api_permission_timeout": 120.0
}PUT /v1/config takes a partial flat dict, overlays only the recognized keys
onto the existing settings (so secrets and env-driven values the UI doesn't
touch are preserved), validates them by round-tripping through AppConfig, and
persists the result:
curl -X PUT http://localhost:9734/v1/config \
-H 'Content-Type: application/json' \
-d '{"temperature": 0.2, "max_tool_iterations": 10}'On validation failure it returns HTTP 422 with {"error": {"message": "Invalid config: ..."}}.
Most keys here require a restart: the write updates settings.json and the
in-memory AppConfig, but the running agent is not rebuilt, so values baked
into it at startup (backend selection, model, MCP wiring) keep their old values
until the process restarts.
The one exception is api_profile, which this endpoint only persists. To
actually switch the live agent's profile use PUT /v1/profiles/{name} below,
which takes effect on the next turn without a restart.
CRUD for the MCP server configuration persisted to mcp_servers.json, backing
the browser UI's MCP server management view. Operates on the shared
MCPClientManager (app.state.agent.mcp_manager); all responses return the
updated list of server configs.
GET /v1/mcp lists the configured servers:
curl http://localhost:9734/v1/mcp[
{
"name": "github",
"transport": "stdio",
"command": "npx",
"args": ["-y", "@modelcontextprotocol/server-github"],
"env": { "GITHUB_TOKEN": "..." },
"url": ""
}
]POST /v1/mcp registers a new server (409 on duplicate name), DELETE /v1/mcp/{name} removes one (404 if unknown), and PUT /v1/mcp/{name} updates
the server whose stored name matches the path parameter (404 if not found).
Unknown transports, missing command for stdio, and missing url for http
return HTTP 422. The request body is the same shape as the list entries above.
curl -X POST http://localhost:9734/v1/mcp \
-H 'Content-Type: application/json' \
-d '{"name": "filesystem", "command": "/usr/local/bin/mcp-fs"}'Lists the agent profiles found in the profiles directory and selects which one
the shared agent runs as. Backs the browser UI's profile chip in the status bar
(click :: default to pick a persona) and mirrors the TUI's /profile list and
/profile load <name> commands.
GET /v1/profiles returns one entry per profile directory, each with its
manifest metadata and an active flag that is true for exactly one of them:
curl http://localhost:9734/v1/profiles[
{
"name": "coder",
"active": true,
"description": "",
"version": "0.1.0",
"default_model_tier": "large",
"allow_tools": ["builtin__*"],
"deny_tools": []
}
]A profile whose manifest can't be resolved (e.g. a circular base chain) is
still listed, with its description set to unavailable: <reason> and empty
metadata, so one bad directory doesn't break the menu.
PUT /v1/profiles/{name} makes that profile active and returns the same summary
shape with active: true:
curl -X PUT http://localhost:9734/v1/profiles/coderThe swap is serialized on the same process-wide lock as the chat endpoints, so a
profile never changes mid-stream, and it takes effect on the next turn — the
agent reads its system prompt from the loaded profile per turn rather than
caching it at construction, so no restart is needed. The selection is written to
settings.json under api_server.profile, so a restart resumes on the same
profile and a plain PUT /v1/config {"api_profile": ...} is unnecessary. An
unknown name returns HTTP 404 with the usual {"error": {"message": ...}} body
and leaves the active profile untouched.
Note the agent is a process-global singleton, so a switch applies to every
connected client, not just the one that requested it. Switching also leaves any
in-flight conversation history intact; clients that want a clean slate under
the new persona should follow up with {"action": "clear"} on /v1/chat, which
is what the TUI's /profile load does.
- Workspace — the agent's workspace defaults to the process CWD unless the
CWD is a sensitive path (see
is_sensitive_pathinsessions.py). REST requests auto-approve file and shell tools that touch paths outside the workspace; WebSocket clients receive apermission_requestinstead (unlessapi_permission_mode = "auto"). Both surfaces still respect offline/dry-run gating. - Modes — in
ask/planmodes only read-only tools are exposed (plus MCP limits per mode); inchatmode no tools are offered at all. The mode also shapes the system prompt, exactly as in the TUI. - Profiles —
PUT /v1/profiles/{name}mutates the single sharedAgent, so the new system prompt and tool permissions apply process-wide from the next turn on, across every open WebSocket and in-flight request queue. There is no per-session profile isolation. - Concurrency — requests are serialized on a process-wide lock; there is no
horizontal multiplexing. For concurrent consumers, run multiple
oli-serverprocesses. - Interrupts —
{"action": "interrupt"}cancels the in-flight turn, kills shell subprocesses started by it, and cancels a delegateddispatchfan-out. The partial user turn is rolled back and the client receivesinterruptedfollowed bydone(interrupted: true). The local Transformers backend only observes cancellation after its current generation pass finishes. - Logging — the server writes NDJSON to
AppConfig.log_filevia the shared logger inlogger.py.