Add Harbor adapter that wraps headless Corbits exec - #687
Merged
TheGreatAxios merged 5 commits intoSep 5, 2026
Conversation
TheGreatAxios
force-pushed
the
main
branch
2 times, most recently
from
August 27, 2026 23:46
ab4779a to
8b115bf
Compare
TheGreatAxios
force-pushed
the
cl-6923-harbor-agent-adapter-wrapping-corbits-exec
branch
from
August 28, 2026 02:23
c91eda8 to
5fd886d
Compare
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
TheGreatAxios
force-pushed
the
cl-6923-harbor-agent-adapter-wrapping-corbits-exec
branch
from
August 28, 2026 19:14
f1273f0 to
746297d
Compare
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
2 tasks
TheGreatAxios
marked this pull request as draft
September 2, 2026 19:46
TheGreatAxios
marked this pull request as ready for review
September 5, 2026 05:34
TheGreatAxios
commented
Sep 5, 2026
TheGreatAxios
left a comment
TheGreatAxios
left a comment
Collaborator
Author
There was a problem hiding this comment.
Review · Approve
Harbor installed-agent under evals/harbor/ runs product corbits exec --dangerously-skip-permissions with a temp --config, then copies ~/.corbits into trial logs and fills Harbor token fields.
No findings that block merge.
Notes
- Scope is
evals/harbor/plus a one-line pointer inevals/public/README.md(git diff origin/main...origin/cl-6923-harbor-agent-adapter-wrapping-corbits-exec --stat: 14 files, no unexpected binaries). - Subjects pass
git log origin/main..origin/cl-6923-harbor-agent-adapter-wrapping-corbits-exec --format='%s' | awk 'length > 72'(empty). Two existing body lines wrap past 72 (3b109ed7,746297d3); not rewriting history for that. PYTHONPATH=. python3 -m unittest evals.harbor.tests.test_argv evals.harbor.tests.test_session -vis the adapter's unit gate; I did not re-run it in this review. CI on the PR is green (prettier, eslint, typecheck, build-and-test).
Harbor needs a thin installed-agent bridge so Terminal-Bench and similar cells can drive product Corbits without a second agent loop. Credentials and baseURL go through a temporary settings.json for --config only; end-to-end dry-run remains deferred until a Linux binary and Harbor CLI are available on a suitable host.
The CLI has no --version flag, so install verification now uses --help and the version probe records the help banner. Run the exec command through environment.exec so the real non-zero exit code lands in context metadata instead of a hardcoded 1. Drop the cross-provider API key fallback and the no-op --force exec flag.
Harbor 0.22 infers provider URLs when an API key is present. The adapter only read configured_base_url, so stock harbor run -m xai/... fail-closed without an explicit override.
Harbor left n_input_tokens and cost_usd null because the adapter never copied ~/.corbits out of the container. Failed trials could not be audited from corbits.txt alone.
TheGreatAxios
force-pushed
the
cl-6923-harbor-agent-adapter-wrapping-corbits-exec
branch
from
September 5, 2026 20:58
47a64f8 to
4115f84
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
BaseInstalledAgentunderevals/harbor/that runs productcorbits execwith--dangerously-skip-permissions --force(no second agent loop).--configsettings.json (requiredbaseURL+ API key); ships argv unit tests and a trivial Harbor task scaffold for later dry-run.Verification
PYTHONPATH=. python3 -m unittest evals.harbor.tests.test_argv -vpassessrc/changes; full Harbor CLI dry-run still needs a Linux ELF binary + Harbor install (tracked separately)toward CL-6923