Skip to content

Commit c91eda8

Browse files
TheGreatAxiosTheGreatAxios
committed
Add Harbor adapter that wraps headless Corbits exec
Harbor needs a thin installed-agent bridge so Terminal-Bench and similar cells can drive product Corbits without a second agent loop. Credentials and baseURL go through a temporary settings.json for --config only; end-to-end dry-run remains deferred until a Linux binary and Harbor CLI are available on a suitable host.
1 parent 7a3b997 commit c91eda8

12 files changed

Lines changed: 650 additions & 1 deletion

File tree

‎evals/harbor/README.md‎

Lines changed: 129 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,129 @@
1+
# Harbor adapter for Corbits Code
2+
3+
Thin Harbor `BaseInstalledAgent` that runs headless product Corbits:
4+
5+
```text
6+
corbits exec --cwd … --config … --provider … --model …
7+
--dangerously-skip-permissions --force <instruction>
8+
```
9+
10+
No second agent loop. Credentials are translated from Harbor kwargs/env into a
11+
temporary `settings.json` passed with `--config` — product Corbits still does
12+
not read API keys from the environment.
13+
14+
## Layout
15+
16+
| Path | Role |
17+
| -------------------- | ------------------------------------------------------ |
18+
| `argv.py` | Pure settings + argv builders (unit-tested, no Harbor) |
19+
| `agent.py` | `Corbits` installed agent (requires Harbor at import) |
20+
| `tasks/trivial/` | Minimal smoke task (`hello.txt`) |
21+
| `tests/test_argv.py` | Argv/settings unit tests |
22+
23+
## Prerequisites
24+
25+
1. **Harbor CLI** installed in the host Python env (`pip install harbor` / uv).
26+
2. **Linux ELF `corbits` binary** for the task container (Darwin host builds
27+
will not run inside Linux Docker). Acquire one of:
28+
- Build on Linux: `bun run build:bin` → `dist/corbits`
29+
- Release / CI tarball that unpacks a `corbits` binary
30+
3. **git** inside the task image (adapter also installs it via Harbor system
31+
packages). Corbits storage requires git — there is no git-less fallback.
32+
4. **Provider API key** for the model under test.
33+
5. **Provider `base_url`** (required — see below). The adapter fail-closes if
34+
none is resolved; it does not invent a default.
35+
36+
## Secrets / credentials
37+
38+
Pass a key through Harbor agent kwargs or env. The adapter writes it into the
39+
temp settings file only:
40+
41+
| Source | Notes |
42+
| ----------------------- | ---------------------------------------- |
43+
| `api_key=` agent kwarg | Preferred for one-off runs |
44+
| `CORBITS_API_KEY` | Generic adapter env |
45+
| `{PROVIDER}_API_KEY` | e.g. `XAI_API_KEY`, `OPENAI_API_KEY` |
46+
| Harbor model connection | Falls back to `model_connection.api_key` |
47+
48+
### Required base URL
49+
50+
`providers.<provider>.baseURL` is always written. Resolve it via one of:
51+
52+
| Source | Notes |
53+
| ------------------------------ | -------------------------------------------------- |
54+
| `base_url=` agent kwarg | Preferred for one-off runs |
55+
| `CORBITS_BASE_URL` | Adapter env |
56+
| Harbor model connection | `model_connection.configured_base_url` when set |
57+
58+
If none are set, the adapter raises before writing settings.
59+
60+
Example values:
61+
62+
| Cell | Example `base_url` |
63+
| ---------------------------- | -------------------------------------- |
64+
| xAI API key | `https://api.x.ai/v1` |
65+
| Product OAuth / grok-cli path | `https://cli-chat-proxy.grok.com/v1` (`XAI_BASE_URL` in `src/auth/xai/constants.ts`) |
66+
| OpenAI-compatible | e.g. `https://api.openai.com/v1` or your cell's gateway |
67+
68+
Optional: `shell_timeout_ms=` → `shell.timeoutMs` in settings.
69+
70+
Default provider/model when Harbor does not pass `provider/model`: **xai** /
71+
**grok-4.5**. Codex cells typically use `--model openai/<id>` (or pass
72+
`provider=` / `model=` kwargs).
73+
74+
## Linux binary acquisition
75+
76+
The adapter installs the binary onto PATH from **one** of:
77+
78+
| Kwarg | Behavior |
79+
| --------------------- | ------------------------------------------------- |
80+
| `corbits_binary_path` | Upload a host file into `/usr/local/bin/corbits` |
81+
| `corbits_binary_url` | `curl` a raw binary URL |
82+
| `corbits_tarball_url` | `curl` + extract; expects a `corbits` file inside |
83+
84+
Archive vs raw binary is detected from the URL suffix (`.tar.gz`, `.tgz`,
85+
`.tar`) — the install script does not call `file(1)`.
86+
87+
If none are set and `corbits` is not already on PATH in the environment,
88+
`install()` raises with this README pointer.
89+
90+
## Invoke
91+
92+
From the repo root (so `evals.harbor.agent` is importable):
93+
94+
```bash
95+
# Unit tests (no Harbor package required; pytest may be absent)
96+
PYTHONPATH=. python3 -m unittest evals.harbor.tests.test_argv -v
97+
98+
# Dry-run trivial task (needs Harbor CLI + Linux binary + API key + base URL)
99+
export CORBITS_API_KEY=… # or XAI_API_KEY=…
100+
export CORBITS_BASE_URL=https://api.x.ai/v1
101+
harbor run \
102+
-p evals/harbor/tasks/trivial \
103+
-a evals.harbor.agent:Corbits \
104+
-m xai/grok-4.5 \
105+
--ae corbits_binary_path=/absolute/path/to/linux/corbits
106+
```
107+
108+
Equivalent kwargs via Harbor job config:
109+
110+
```yaml
111+
agents:
112+
- name: evals.harbor.agent:Corbits
113+
kwargs:
114+
provider: xai
115+
model: grok-4.5
116+
api_key: ${CORBITS_API_KEY}
117+
base_url: https://api.x.ai/v1
118+
corbits_binary_path: /absolute/path/to/linux/corbits
119+
# or: corbits_tarball_url: https://…/corbits-linux.tar.gz
120+
```
121+
122+
## Known gaps (CL-6924)
123+
124+
- Full Harbor dry-run + Terminal-Bench smoke are **not** claimed by this change.
125+
They need Harbor CLI, a Linux ELF binary, Docker, and provider credentials on
126+
a machine that can run the harness end-to-end — tracked as **CL-6924**.
127+
- This adapter does not parse Corbits trajectories into Harbor ATIF; exit
128+
metadata is limited to `context.metadata["exit_code"]` plus tee'd stdout in
129+
`/logs/agent/corbits.txt`.

‎evals/harbor/__init__.py‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
"""Harbor installed-agent adapter for Corbits Code (CL-6923).
2+
3+
Import the agent class as ``evals.harbor.agent:Corbits`` when running Harbor.
4+
Pure helpers live in ``argv`` and are unit-tested without the Harbor package.
5+
"""
6+
7+
__all__ = ["argv"]

0 commit comments

Comments
 (0)