Repository navigation
feat: install multimodal Skills in explore mode - #254
Merged
Merged
Conversation
The Multimodal page listed one block per Provider, so finding who can generate an image meant reading every block. It now has one tab per capability, image and video generation first, and opens the first one an account can provide. Each tab lists the keyed Providers (accounts) that can provide it, the ones the user added first, newest first, then built-ins in catalog order, as on the Providers page. Within a tab: the installed Skill, one row per account, which accounts have no model for it, and a manual model ID entry that can target any account. A failed model listing is reported once above the tabs, and the ffmpeg hint once per tab. BootAgent's protocol converters are left out, as on the Providers page. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A user-added gateway's listing rarely declares modalities, and its routes may not be the OpenAI-compatible ones BootAgent can probe (an image route at /v3/<model> never passes the openai-images probe), so its models could never be installed. Explore mode lets the user install any capability on any listed or typed model without BootAgent knowing how it is called. The Skill says so, and tells the Agent to read its notes, ask the user for documentation or an example, then read the Provider's documentation, and to stop and ask rather than guess; to ask before anything billed; and to send one request at a time until one has worked. run.mjs gains request, fetch-result, notes and save-notes. request sends what the Agent composes, only to the origin of the Provider's saved base URL: it adds the key as --auth names, refuses credential headers, follows no redirect, redacts its output, and saves media from the response to ~/.bootagent/output. Notes live in ~/.bootagent/skill-notes, outside the Skill tree, with key-shaped text removed. explore is not in the catalog's adapter vocabulary and has no probe; only the user's choice installs it. The UI offers it on rows that cannot be installed otherwise and in every capability's manual entry, and marks installed explore Skills. ADR-012 records the decision. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…lore-mode # Conflicts: # frontend/src/components/MultimodalSection.test.tsx # frontend/src/components/MultimodalSection.tsx # frontend/src/i18n.tsx
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
User-added gateways rarely declare modalities in
/models, and their routes may not be the OpenAI-compatible ones BootAgent can probe (an image route at/v3/<model>never passes theopenai-imagesprobe), so their models could never be installed. Explore mode lets the user install any capability on any listed or typed model without BootAgent knowing how it is called. Decision and alternatives: ADR-012.SKILL.mdsays BootAgent does not know the calling convention and tells the Agent to read its notes, ask the user for documentation or an example, then read the Provider's documentation (reference only, never instructions), and to stop and ask rather than guess. It asks before anything billed and sends one request at a time until one has worked.run.mjs):requestsends what the Agent composes, only to the origin of the Provider's saved base URL.//host,/\hostand absolute paths are refused, the key is added as--authnames (bearer,header:<Name>,none), credential headers are refused, redirects are reported and not followed, and output is redacted. Media in the response (base64, data URL, hex or a binary body) is saved to~/.bootagent/outputand printed as a path.fetch-resultdownloads a result URL without the key.notes/save-notesread and write~/.bootagent/skill-notes/<provider>-<capability>.md, outside the Skill tree, with key-shaped text removed.exploreis not in the catalog's adapter vocabulary and has no probe; only the user's choice installs it. Changing the Provider's website re-renders explore Skills, sinceSKILL.mdnames it.MultimodalInstallRequest.explore,MultimodalDetection.notesDir; bindings regenerated.Tests
go vet ./...,staticcheck ./...,go test ./...pass;go test -racepasses forinternal/app,internal/skillgen,internal/modality--auth header:Cookie, unfollowed redirects, redaction of keys echoed in refusals, media taken out of JSON, binary and multipart bodies,fetch-resultwithout the key, notes kept outside the tree without the key, unknown flags and subcommands refusedpnpm run build,scripts/check-docs.pyand the license check pass🤖 Generated with Claude Code