Skip to content

configure: add --skip-unavailable so --agents can configure the available subset - #308

Open
Edwinhe03 wants to merge 3 commits into
databricks:mainfrom
Edwinhe03:edwin-he/agents-skip-unavailable
Open

configure: add --skip-unavailable so --agents can configure the available subset#308
Edwinhe03 wants to merge 3 commits into
databricks:mainfrom
Edwinhe03:edwin-he/agents-skip-unavailable

Conversation

@Edwinhe03

Copy link
Copy Markdown
Collaborator

Problem

--agents treats an explicit list as all-or-nothing. If any named agent isn't available on the workspace, configure_workspace_command raises before configuring any of them:

if unavailable_tools:
    _print_discovery_diagnostics(state)
    raise RuntimeError(f"Requested agent(s) not available on this workspace: {displays}.")

So on a workspace whose AI Gateway exposes no OpenAI models (codex discovery: no endpoint exposes api_type openai/v1/responses):

ucode configure --profiles DEFAULT --agents claude,codex,pi --use-pat --skip-validate --skip-upgrade

configures nothing and exits 1 — even though claude and pi are both usable there (pi needs only one of claude/codex/gemini). For CI running one command across heterogeneous workspaces, one unavailable model family takes down the whole setup.

Why not just make --agents fail open

The default stays strict, because the two directions aren't symmetric:

  • Strict default: a caller who wants tolerance opts in with a flag.
  • Fail-open default: a caller who wants a guarantee has no way to get it back. --agents claude,codex exiting 0 having configured only claude leaves the pipeline believing codex works — the failure resurfaces later at ucode codex launch time, far from the cause.

Naming agents explicitly is a request for a specific outcome, so it should keep meaning "all of these, or tell me why not."

Change

Add an opt-in --skip-unavailable:

ucode configure --agents claude,codex,pi --skip-unavailable
  • Configures the available subset, preserving the order given in --agents.
  • Skips the rest with a warning (Skipping agent(s) not available on this workspace: Codex.), on top of the existing per-source discovery diagnostics.
  • Still exits non-zero when none of the requested agents are available — the pre-existing if not available_on_workspace: return 1 covers this, since that list is built only from selected_tools. The flag can't turn a completely unusable workspace into a silent success.
  • Requires --agents: the interactive picker already offers only available agents, and --agent names a single agent whose absence is the whole answer.
  • The strict error now points at the flag: ... Pass --skip-unavailable to configure the available ones instead.

Behavior with the flag absent is unchanged.

Tests

Added to tests/test_cli.py (the existing test_unavailable_selected_tool_errors_before_configure still guards the strict default):

  • test_skip_unavailable_configures_available_subset — the azure/no-GPT case: requesting claude,codex,pi configures ["claude", "pi"], installs only those binaries, and warns about Codex.
  • test_skip_unavailable_still_fails_when_none_available — returns 1, never calls configure_selected_tools.
  • test_strict_error_mentions_skip_unavailable — discoverability of the flag from the error.
  • test_skip_unavailable_requires_agents / _forwarded_with_agents / _absent_by_default — CLI wiring.

uv run pytest: 1690 passed, 1 failed. The one failure (test_managed_wizard.py::TestCliWiring::test_successful_apply_exits_zero) reproduces on pristine origin/main in a clean worktree with none of these changes — pre-existing and unrelated. uv run ruff check . and ruff format --check are clean.

Docs

README gets the strict-vs-tolerant distinction next to --agents, plus a quick-reference row.

`--agents` treats an explicit list as all-or-nothing: if any named agent
isn't available on the workspace, the run raises before configuring any
of them. On a workspace whose AI Gateway exposes no OpenAI models,
`--agents claude,codex,pi` therefore configures nothing, even though
claude and pi are both usable there.

Keep that strict default — naming agents explicitly is a request for a
specific outcome, and a run that silently configures a subset would exit
0 while leaving a caller believing codex works, surfacing the failure
later at launch time instead. Add an opt-in `--skip-unavailable` for
callers that want the tolerant behavior: available agents are configured
and the rest are skipped with a warning, preserving the requested order.

The run still exits non-zero when none of the requested agents are
available, so the flag can't turn a completely unusable workspace into a
silent success. `--skip-unavailable` requires `--agents`: the interactive
picker already offers only available agents, and `--agent` names a single
agent whose absence is the whole answer. The strict error message now
points at the flag.

Co-authored-by: Isaac
@Edwinhe03

Copy link
Copy Markdown
Collaborator Author

Live-tested against a real workspace

Verified end-to-end (not just the mocked unit tests) against ai-oss-ecosystem-integration-testing with --dry-run: real OAuth, real AI Gateway verification, real model discovery. To reproduce the "GPT not allowed" workspace, GPT ids were filtered out of the model-services response and discover_codex_models was stubbed to return the real-world empty result ([], "no endpoint exposes api_type \openai/v1/responses`")` — everything else (auth, gateway, Claude/Gemini/OSS discovery from 51 live model-services ids) ran unmodified.

Case Result Exit
--agents claude,codex,pi --skip-unavailable, no GPT Skipping agent(s) not available on this workspace: Codex.Claude Code + Pi configured 0
--agents claude,codex,pi, no GPT (strict default) ERROR Requested agent(s) not available on this workspace: Codex. Pass --skip-unavailable…, nothing configured 1
--agents codex --skip-unavailable, no GPT (none available) ERROR No coding agents are available on this workspace., nothing configured see note
--agents claude,codex,pi --skip-unavailable, all available (control) all three configured, no spurious skip 0

Also confirms Pi configures correctly without codex — the if codex_models guard in agents/pi.py drops the databricks-openai provider rather than erroring.

Pre-existing bug found while testing (not fixed here)

The "none available" case prints ERROR No coding agents are available on this workspace. but exits 0. Cause: configure_workspace_command returns 1 there, but all seven call sites in configure() discard the return value — nothing converts it into typer.Exit(1). Reproduced on pristine origin/main with none of this branch's changes, so it predates this PR and affects the existing strict path too.

I've left it out to keep this PR scoped. Worth noting for CI users: with --skip-unavailable, a workspace where no requested agent is available currently exits 0 with an error on stderr. Happy to fix it here or in a follow-up — say which you'd prefer.

tt-le
tt-le previously approved these changes Aug 12, 2026
default_model() returned None for workspaces whose codex_models list
contains only gpt-oss-* ids (e.g. system.ai.gpt-oss-120b), causing
codex to configure with a stale or missing model. The restriction was
intended to exclude non-GPT ids (e.g. moonshotai/kimi-k2.5) that the
responses gateway would reject, but gpt-oss-* ids come from the codex
bucket in UC model-services and expose the responses API — confirmed
live against az-dogfood, HTTP 200, tokens consumed.

Fall back to the first available model when no versioned GPT parses,
instead of returning None. Versioned GPT ids (gpt-5, gpt-5-6-luna, ...)
still win when present.

Co-authored-by: Isaac
Two regressions from the previous commit:

1. default_model() fallback was too broad — it returned any first model
   when no versioned GPT parsed, including non-GPT ids like
   moonshotai/kimi-k2.5 that the responses gateway would reject. Tighten
   the fallback to gpt-* prefixed models only via a new _is_gpt_family()
   helper. gpt-oss-* stays usable; non-GPT ids stay excluded.

2. test_skip_unavailable_forwarded_with_agents used --profiles DEFAULT
   --use-pat which reads ~/.databrickscfg — that file doesn't exist on CI
   runners. Switch to --workspaces https://... which doesn't touch the
   filesystem.

Co-authored-by: Isaac
Comment thread src/ucode/agents/codex.py Outdated
# No versioned GPT found — fall back to the first available model. Any id in
# codex_models comes from UC model-services' codex bucket and exposes the
# responses API, so it's routable (e.g. system.ai.gpt-oss-120b).
return codex_models[0] if codex_models else None

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tt-le added this fallback for oss models for codex on azure

@Edwinhe03
Edwinhe03 requested a review from tt-le August 12, 2026 22:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants