configure: add --skip-unavailable so --agents can configure the available subset - #308
configure: add --skip-unavailable so --agents can configure the available subset#308Edwinhe03 wants to merge 3 commits into
Conversation
`--agents` treats an explicit list as all-or-nothing: if any named agent isn't available on the workspace, the run raises before configuring any of them. On a workspace whose AI Gateway exposes no OpenAI models, `--agents claude,codex,pi` therefore configures nothing, even though claude and pi are both usable there. Keep that strict default — naming agents explicitly is a request for a specific outcome, and a run that silently configures a subset would exit 0 while leaving a caller believing codex works, surfacing the failure later at launch time instead. Add an opt-in `--skip-unavailable` for callers that want the tolerant behavior: available agents are configured and the rest are skipped with a warning, preserving the requested order. The run still exits non-zero when none of the requested agents are available, so the flag can't turn a completely unusable workspace into a silent success. `--skip-unavailable` requires `--agents`: the interactive picker already offers only available agents, and `--agent` names a single agent whose absence is the whole answer. The strict error message now points at the flag. Co-authored-by: Isaac
Live-tested against a real workspaceVerified end-to-end (not just the mocked unit tests) against
Also confirms Pi configures correctly without codex — the Pre-existing bug found while testing (not fixed here)The "none available" case prints I've left it out to keep this PR scoped. Worth noting for CI users: with |
default_model() returned None for workspaces whose codex_models list contains only gpt-oss-* ids (e.g. system.ai.gpt-oss-120b), causing codex to configure with a stale or missing model. The restriction was intended to exclude non-GPT ids (e.g. moonshotai/kimi-k2.5) that the responses gateway would reject, but gpt-oss-* ids come from the codex bucket in UC model-services and expose the responses API — confirmed live against az-dogfood, HTTP 200, tokens consumed. Fall back to the first available model when no versioned GPT parses, instead of returning None. Versioned GPT ids (gpt-5, gpt-5-6-luna, ...) still win when present. Co-authored-by: Isaac
Two regressions from the previous commit: 1. default_model() fallback was too broad — it returned any first model when no versioned GPT parsed, including non-GPT ids like moonshotai/kimi-k2.5 that the responses gateway would reject. Tighten the fallback to gpt-* prefixed models only via a new _is_gpt_family() helper. gpt-oss-* stays usable; non-GPT ids stay excluded. 2. test_skip_unavailable_forwarded_with_agents used --profiles DEFAULT --use-pat which reads ~/.databrickscfg — that file doesn't exist on CI runners. Switch to --workspaces https://... which doesn't touch the filesystem. Co-authored-by: Isaac
| # No versioned GPT found — fall back to the first available model. Any id in | ||
| # codex_models comes from UC model-services' codex bucket and exposes the | ||
| # responses API, so it's routable (e.g. system.ai.gpt-oss-120b). | ||
| return codex_models[0] if codex_models else None |
There was a problem hiding this comment.
@tt-le added this fallback for oss models for codex on azure
Problem
--agentstreats an explicit list as all-or-nothing. If any named agent isn't available on the workspace,configure_workspace_commandraises before configuring any of them:So on a workspace whose AI Gateway exposes no OpenAI models (
codex discovery: no endpoint exposes api_type openai/v1/responses):configures nothing and exits 1 — even though
claudeandpiare both usable there (pineeds only one of claude/codex/gemini). For CI running one command across heterogeneous workspaces, one unavailable model family takes down the whole setup.Why not just make
--agentsfail openThe default stays strict, because the two directions aren't symmetric:
--agents claude,codexexiting 0 having configured only claude leaves the pipeline believing codex works — the failure resurfaces later atucode codexlaunch time, far from the cause.Naming agents explicitly is a request for a specific outcome, so it should keep meaning "all of these, or tell me why not."
Change
Add an opt-in
--skip-unavailable:--agents.Skipping agent(s) not available on this workspace: Codex.), on top of the existing per-source discovery diagnostics.if not available_on_workspace: return 1covers this, since that list is built only fromselected_tools. The flag can't turn a completely unusable workspace into a silent success.--agents: the interactive picker already offers only available agents, and--agentnames a single agent whose absence is the whole answer.... Pass --skip-unavailable to configure the available ones instead.Behavior with the flag absent is unchanged.
Tests
Added to
tests/test_cli.py(the existingtest_unavailable_selected_tool_errors_before_configurestill guards the strict default):test_skip_unavailable_configures_available_subset— the azure/no-GPT case: requestingclaude,codex,piconfigures["claude", "pi"], installs only those binaries, and warns about Codex.test_skip_unavailable_still_fails_when_none_available— returns 1, never callsconfigure_selected_tools.test_strict_error_mentions_skip_unavailable— discoverability of the flag from the error.test_skip_unavailable_requires_agents/_forwarded_with_agents/_absent_by_default— CLI wiring.uv run pytest: 1690 passed, 1 failed. The one failure (test_managed_wizard.py::TestCliWiring::test_successful_apply_exits_zero) reproduces on pristineorigin/mainin a clean worktree with none of these changes — pre-existing and unrelated.uv run ruff check .andruff format --checkare clean.Docs
README gets the strict-vs-tolerant distinction next to
--agents, plus a quick-reference row.