Skip to content

Commit b01bd47

Browse files
TheGreatAxiosTheGreatAxios
authored
Merge pull request #687 from corbitsdev/cl-6923-harbor-agent-adapter-wrapping-corbits-exec
Add Harbor adapter that wraps headless Corbits exec
2 parents d22dc0c + 4115f84 commit b01bd47

14 files changed

Lines changed: 1155 additions & 1 deletion

File tree

‎evals/harbor/README.md‎

Lines changed: 142 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,142 @@
1+
# Harbor adapter for Corbits Code
2+
3+
Thin Harbor `BaseInstalledAgent` that runs headless product Corbits:
4+
5+
```text
6+
corbits exec --cwd … --config … --provider … --model …
7+
--dangerously-skip-permissions <instruction>
8+
```
9+
10+
No second agent loop. Credentials are translated from Harbor kwargs/env into a
11+
temporary `settings.json` passed with `--config` — product Corbits still does
12+
not read API keys from the environment.
13+
14+
## Layout
15+
16+
| Path | Role |
17+
| ----------------------- | ------------------------------------------------------ |
18+
| `argv.py` | Pure settings + argv builders (unit-tested, no Harbor) |
19+
| `session.py` | Parse harvested `~/.corbits` usage (unit-tested) |
20+
| `agent.py` | `Corbits` installed agent (requires Harbor at import) |
21+
| `tasks/trivial/` | Minimal smoke task (`hello.txt`) |
22+
| `tests/test_argv.py` | Argv/settings unit tests |
23+
| `tests/test_session.py` | Session usage harvest unit tests |
24+
25+
## Prerequisites
26+
27+
1. **Harbor CLI** installed in the host Python env (`pip install harbor` / uv).
28+
2. **Linux ELF `corbits` binary** for the task container (Darwin host builds
29+
will not run inside Linux Docker). Acquire one of:
30+
- Build on Linux: `bun run build:bin` → `dist/corbits`
31+
- Release / CI tarball that unpacks a `corbits` binary
32+
3. **git** inside the task image (adapter also installs it via Harbor system
33+
packages). Corbits storage requires git — there is no git-less fallback.
34+
4. **Provider API key** for the model under test.
35+
5. **Provider `base_url`** — stock Harbor `-m xai/…` infers `https://api.x.ai/v1`. Override only when the catalog default is wrong. The adapter fail-closes if nothing resolves.
36+
37+
## Secrets / credentials
38+
39+
Pass a key through Harbor agent kwargs or env. The adapter writes it into the
40+
temp settings file only:
41+
42+
| Source | Notes |
43+
| ----------------------- | ---------------------------------------------- |
44+
| `api_key=` agent kwarg | Preferred for one-off runs |
45+
| `CORBITS_API_KEY` | Generic adapter env |
46+
| `{PROVIDER}_API_KEY` | Only the selected provider, e.g. `XAI_API_KEY` |
47+
| Harbor model connection | Falls back to `model_connection.api_key` |
48+
49+
### Required base URL
50+
51+
`providers.<provider>.baseURL` is always written. Resolve it via one of:
52+
53+
| Source | Notes |
54+
| ----------------------- | ----------------------------------------------------------------------------------------- |
55+
| `base_url=` agent kwarg | Preferred for one-off runs |
56+
| Harbor configured URL | `model_connection.configured_base_url` (`CORBITS_BASE_URL` / `XAI_BASE_URL` / …) |
57+
| Harbor inferred catalog | `model_connection.base_url` — e.g. xAI → `https://api.x.ai/v1` when an API key is present |
58+
| `CORBITS_BASE_URL` | Adapter env (also feeds Harbor configured URL) |
59+
60+
Stock `harbor run -m xai/grok-4.5` with `XAI_API_KEY` therefore does **not** need an explicit `base_url`. The adapter still fail-closes if nothing resolves.
61+
62+
Do **not** point Harbor cells at the grok-cli OAuth proxy (`https://cli-chat-proxy.grok.com/v1`). That URL only accepts OAuth tokens from `~/.corbits/xai-auth.json`, which Harbor containers do not have. Profile-qualified product cells (`xai/<profile>`, `codex/<profile>`) are OAuth; use an API-key provider name (`xai`, `openai`, …) inside Harbor.
63+
64+
Example values:
65+
66+
| Cell | Example `base_url` |
67+
| ----------------------------- | ------------------------------------------------------- |
68+
| xAI API key | `https://api.x.ai/v1` (Harbor infers this) |
69+
| Product OAuth / grok-cli path | Not usable in Harbor containers |
70+
| OpenAI-compatible | e.g. `https://api.openai.com/v1` or your cell's gateway |
71+
72+
Optional: `shell_timeout_ms=` → `shell.timeoutMs` in settings.
73+
74+
Default provider/model when Harbor does not pass `provider/model`: **xai** /
75+
**grok-4.5**. Codex cells typically use `--model openai/<id>` (or pass
76+
`provider=` / `model=` kwargs).
77+
78+
## Linux binary acquisition
79+
80+
The adapter installs the binary onto PATH from **one** of:
81+
82+
| Kwarg | Behavior |
83+
| --------------------- | ------------------------------------------------- |
84+
| `corbits_binary_path` | Upload a host file into `/usr/local/bin/corbits` |
85+
| `corbits_binary_url` | `curl` a raw binary URL |
86+
| `corbits_tarball_url` | `curl` + extract; expects a `corbits` file inside |
87+
88+
Archive vs raw binary is detected from the URL suffix (`.tar.gz`, `.tgz`,
89+
`.tar`) — the install script does not call `file(1)`.
90+
91+
If none are set and `corbits` is not already on PATH in the environment,
92+
`install()` raises with this README pointer.
93+
94+
## Invoke
95+
96+
From the repo root (so `evals.harbor.agent` is importable):
97+
98+
```bash
99+
# Unit tests (no Harbor package required; pytest may be absent)
100+
PYTHONPATH=. python3 -m unittest evals.harbor.tests.test_argv evals.harbor.tests.test_session -v
101+
102+
# Dry-run trivial task (needs Harbor CLI + Linux binary + API key)
103+
export CORBITS_API_KEY=… # or XAI_API_KEY=…
104+
harbor run \
105+
-p evals/harbor/tasks/trivial \
106+
-a evals.harbor.agent:Corbits \
107+
-m xai/grok-4.5 \
108+
--ak corbits_binary_path=/absolute/path/to/linux/corbits
109+
```
110+
111+
Equivalent kwargs via Harbor job config:
112+
113+
```yaml
114+
agents:
115+
- name: evals.harbor.agent:Corbits
116+
kwargs:
117+
provider: xai
118+
model: grok-4.5
119+
api_key: ${CORBITS_API_KEY}
120+
base_url: https://api.x.ai/v1
121+
corbits_binary_path: /absolute/path/to/linux/corbits
122+
# or: corbits_tarball_url: https://…/corbits-linux.tar.gz
123+
```
124+
125+
Harbor's own trial/job timeout is the only wall-clock cap. Do **not** wrap
126+
`harbor run` in a parent tool timeout — killing the CLI mid-trial wastes the
127+
tokens already spent and leaves no verifier result.
128+
129+
After each trial Harbor copies `/logs/agent`. The adapter tars `~/.corbits`
130+
there as `corbits-home.tar.gz` and fills `AgentContext` token fields from
131+
session `metadata.json`. `cost_usd` is set only when the harvested
132+
`models-pricing.json` cache has that model; unknown cells stay `null` rather
133+
than inventing a rate. Tool traces are under
134+
`agent/corbits-home/.corbits/projects/…/turns-*.jsonl`.
135+
136+
## Known gaps
137+
138+
- Trivial Harbor dry-run has been run locally (`hello.txt`, verifier reward 1.0)
139+
with OpenCode Go `mimo-v2.5`. Cost-capped Terminal-Bench smoke is the parent
140+
ticket **CL-6922** (the old CL-6924 slice ticket was canceled).
141+
- This adapter does not convert Corbits trajectories into Harbor ATIF. Usage
142+
and the session tar are enough to audit a trial without that conversion.

‎evals/harbor/__init__.py‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
"""Harbor installed-agent adapter for Corbits Code (CL-6923).
2+
3+
Import the agent class as ``evals.harbor.agent:Corbits`` when running Harbor.
4+
Pure helpers live in ``argv`` and ``session`` and are unit-tested without
5+
the Harbor package.
6+
"""
7+
8+
__all__ = ["argv", "session"]

0 commit comments

Comments
 (0)