From 3ebc4dd3cd5f788a35a2d23527414626dd5fb35c Mon Sep 17 00:00:00 2001 From: Bill Murdock Date: Mon, 24 Aug 2026 16:32:19 -0400 Subject: [PATCH 01/34] Add RFC-0011: Agent Registry (first draft) First draft covering the introductory material and user journeys: record-level registry for AI agents with Agent/AgentVersion entities, composition (BOM) with definitional anchors, A2A card as canonical-when-present payload, protocol-typed access bindings, and agent-centric traces and evaluations via a new trace destination. Detailed design and later sections are TBD. Co-Authored-By: Claude Fable 5 --- .../0011-agent-registry.md | 777 ++++++++++++++++++ 1 file changed, 777 insertions(+) create mode 100644 rfcs/0011-agent-registry/0011-agent-registry.md diff --git a/rfcs/0011-agent-registry/0011-agent-registry.md b/rfcs/0011-agent-registry/0011-agent-registry.md new file mode 100644 index 00000000..a311f2f4 --- /dev/null +++ b/rfcs/0011-agent-registry/0011-agent-registry.md @@ -0,0 +1,777 @@ +# RFC 0011: Agent Registry + +| start_date | 2026-08-24 | +| :----------- | :--------- | +| mlflow_issue | | +| rfc_pr | | + +| Author(s) | [Bill Murdock](https://github.com/jwm4) (Red Hat) | +| :--------------------- | :-- | +| **Date Last Modified** | 2026-08-24 | +| **AI Assistant(s)** | Claude Code | + +**Table of contents** + +- [Summary](#summary) +- [Basic example](#basic-example) +- [Motivation](#motivation) + - [The problem](#the-problem) + - [User journeys](#user-journeys) + - [Out of scope](#out-of-scope) +- [Detailed design](#detailed-design) +- [Drawbacks](#drawbacks) +- [Alternatives](#alternatives) +- [Adoption strategy](#adoption-strategy) +- [Open questions](#open-questions) + +# Summary + +Add an Agent Registry to MLflow: a governed, record-level registry for +AI agents. The registry catalogs the agents an organization has built. +It records each agent's identity, ownership, composition, source +provenance, and lifecycle status, and it anchors the agent's traces +and evaluation results. It answers "what agents exist, who owns them, +what are they made of, and how well do they work?" + +The Agent Registry is the third registry in a series, following the +[MCP Server Registry +(RFC-0004)](https://github.com/mlflow/rfcs/blob/main/rfcs/0004-mcp-registry/0004-mcp-registry.md) +and the [Skill Registry +(RFC-0008)](https://github.com/mlflow/rfcs/blob/main/rfcs/0008-mvp-skill-registry/0008-mvp-skill-registry.md). +It completes a progression: MLflow can govern the tools an agent calls +(MCP servers), the expertise it carries (skills), and the models it +invokes (Model Registry), but the agent itself, the thing that acts, +has no registry entry. This RFC adds one. + +**The registry is record-level, not runtime-aware.** It stores what an +agent is: who built it, what components it is made of, where its +source lives, and what lifecycle state it is in. It does not store +whether the agent is running, healthy, or scaled. Runtime state +belongs to the serving platform. A platform that deploys registered +agents can join its own runtime inventory against registry records at +query time; the registry supplies the governance half of that join. +This is the same division of responsibility RFC-0004 draws between +the MCP registry and a gateway. + +The registry manages two primary entities under the `mlflow.genai` +SDK namespace, following the pattern of RFC-0004 and RFC-0008: + +- **Agent**: a named, owned entity with DNS-style naming + (`org/agent-name`), in the spirit of the namespaced names the MCP + and Skill registries use; exact alignment with RFC-0008's + `{workspace, organization, name}` coordinates is a detailed-design + point. +- **AgentVersion**: an immutable snapshot of the agent's composition, + its **bill of materials (BOM)**: skill references, MCP server + references, model references (registry models or external model + identifiers such as `gpt-4o`), and, for agents that run as + configurations of a packaged harness, a harness reference (a + proposed axis; see [Open questions](#open-questions)). Each + version also carries at least one **definitional anchor**: source + provenance (Git URI plus commit), an immutable configuration + snapshot, or an A2A Agent Card. Each change to composition is a + new version. + +BOM entries are soft references, structured values rather than +foreign keys. They resolve against the Skill Registry, MCP Server +Registry, and Model Registry when matching entries exist, and they +remain valid when they do not. This is what makes cross-registry +questions ("which agents use skill X?") answerable as registry +queries while keeping registration decoupled from registration order. + +**The BOM is a component inventory, not a complete recipe.** Its +structured axes exist because corresponding registries or identifier +conventions exist, so it is bounded by MLflow's governance surface +rather than by agent anatomy: an agent's prompt strategy or memory +configuration has no axis because nothing governs one. Three layers +share the job of describing an agent. Structured BOM references are +selective but queryable across agents. Definitional anchors (a source +commit, a configuration snapshot) are complete but opaque: they +capture everything about one agent without supporting cross-agent +queries. Free-form tags are the catch-all for facts that fit neither, +and there will always be some, because agents acquire new kinds of +parts faster than any schema anticipates. New structured axes are +expected as the governance surface grows. + +**A2A Agent Cards are the canonical interchange payload when +present.** The [A2A protocol](https://a2a-protocol.org/) defines the +Agent Card schema that much of the industry has converged on for +describing an agent's identity, capabilities, and endpoint. Following +the hybrid storage pattern of RFC-0004 (`server_json`) and RFC-0008 +(`plugin.json`), an agent version registered from an Agent Card +preserves the complete card immutably, with MLflow-managed fields +(BOM, lifecycle, tags, aliases) kept outside the canonical payload. +The card is optional: agents that do not speak A2A register with +plain metadata, and gain a card later if they adopt one. Whether the +card should instead be required or synthesized is an open question. + +**Endpoints are access bindings, not version fields.** Some agents +are reachable at a URL (A2A agents inherently; deployed agents +generally), and recording that URL lets the registry drive tracing +and evaluation for agents whose code the user cannot run. But +endpoints change independently of composition, and agent versions are +immutable. Following RFC-0004's `MCPAccessBinding` model, approved +endpoints are separate mutable binding records that target a version +or alias, created and deleted as connectivity changes without +touching version history. Where an MCP binding's protocol is always +MCP, an agent binding declares its protocol: `a2a`, `mcp` (for +agents exposed as MCP servers), or `other`. Registration accepts an +optional endpoint as a convenience that creates a binding. + +**Agents become the primary anchor for GenAI traces and +evaluations.** Today traces and evaluation runs attach to +experiments, an abstraction that fits model training but not the +agent development loop. This RFC makes an agent a trace +destination: a new `MlflowAgentLocation` joins the existing +`MlflowExperimentLocation`, usable wherever MLflow accepts a trace +destination today, and evaluation and trace-search APIs gain agent +identity alongside experiment identity. Traces and eval results +appear on the agent's registry page, filterable by version. The +change is additive: under the hood an agent owns a default +experiment, the agent location resolves to it, and experiment-based +workflows continue unchanged. + +**Relationship to other RFCs.** RFC-0004 establishes the canonical +payload and access binding patterns this RFC reuses. RFC-0008 defines +the skills and agent plugins that agent BOMs reference; note that an +*agent plugin* (a package of components installed into a harness) is +not an *agent* (an application that acts); this registry governs the +latter. [RFC-0009: Skill +Tracing](https://github.com/mlflow/rfcs/pull/37) annotates spans +within a trace with skill coordinates; this RFC associates whole +traces with an agent version. The two compose: an agent-linked trace +containing `SKILL` spans shows which parts of the agent's BOM were +exercised in a given run. +[RFC-0010: Extended Agent Plugins](https://github.com/mlflow/rfcs/pull/27) +extends plugin composition and is complementary. + +# Basic example + +The API shapes below are illustrative sketches; exact signatures +belong to the detailed design. + +## Register an agent + +```python +import mlflow +from mlflow.genai import GitSource + +mlflow.genai.register_agent( + name="acme/billing-agent", + description="Answers customer billing questions.", + source=GitSource( + url="https://github.com/acme/billing-agent.git", + ref="8f4e2a1", + ), + skills=["skills:/billing-policy/1", "skills:/refund-rules/2"], + mcp_servers=["mcp-servers:/acme.internal/payments-db/2.0.0"], + models=["models:/acme-billing-llm/3", "gpt-4o"], +) +``` + +This creates the `Agent` (if new) and an `AgentVersion` with status +`draft`. + +## Register an agent from an A2A Agent Card + +```python +mlflow.genai.register_agent( + name="acme/travel-agent", + a2a_endpoint="https://agents.acme.internal/travel", + skills=["skills:/itinerary-planning/4"], +) +``` + +Given an endpoint, the client SDK fetches the Agent Card from the +endpoint's well-known path, stores it immutably as the canonical +payload and the version's definitional anchor (supplying +description and capability metadata), and creates an `a2a` access +binding for the endpoint. The registry `name` is always chosen by the +registrant: an Agent Card's `name` is a free-form display string, +not an identity, so it feeds presentation rather than naming. +Following RFC-0004's pattern for `server_json["title"]`, the UI +falls back to the card's name when the mutable MLflow-managed +`display_name` is unset, and to the registry `name` after that. The card is whatever the running agent serves; it +need not exist as a static file anywhere. The fetch always happens in +the client: the registry server never fetches user-supplied URLs, +consistent with RFC-0008. A caller that wants to inspect or adjust +card metadata before registering can fetch the card itself and pass +it via `a2a_card=` instead. + +## Register a harness-based agent + +```python +mlflow.genai.register_agent( + name="acme/oncall-helper", + description="On-call assistant run in OpenCode.", + harness="opencode", + harness_version="0.5.3", + skills=["skills:/runbook-triage/2"], + mcp_servers=["mcp-servers:/acme.internal/pagerduty/1.2.0"], + models=["claude-sonnet-5"], + config_snapshot="./opencode.json", +) +``` + +There is no user source repository: the agent is the harness plus its +configuration, so the configuration snapshot serves as the +definitional anchor. `config_snapshot` names a file or directory on +the caller's local disk; the client reads it and uploads the content +to MLflow artifact storage as an immutable artifact on the version. +The registry server never fetches user-supplied locations itself, +consistent with RFC-0008, which also means any redaction of secrets +must happen client-side before upload. + +## Trace and evaluate against the agent + +```python +mlflow.genai.set_active_agent("acme/billing-agent", version=3) + +with mlflow.start_span(name="answer-question"): + result = agent.run(question) + +mlflow.genai.evaluate( + data=eval_dataset, + scorers=[correctness_scorer], + agent_id="acme/billing-agent", +) +``` + +# Motivation + +## The problem + +MLflow can already govern most of the components an agent is built +from, and it can already trace what agents do. What is missing is the +agent itself as a governed entity, and that gap shows up in four +ways. + +1. **Agents have no record.** An agent is built, deployed, and + iterated on with no formal registration: no owner, no lifecycle + state, no accountability chain. Models, MCP servers, and skills + each have registry entries; the agent that composes them, the + thing that acts autonomously on behalf of the organization, has + none. When an agent misbehaves, "who is responsible for this?" is + answered by asking around. + +2. **Composition is untracked.** An agent is a composition of skills, + MCP servers, and models, each versioned independently. No record + captures which versions of which components a given build of the + agent used. When behavior changes, "something changed and the + agent broke; what was it?" requires reconstructing the composition + from memory, commit history, and luck. + +3. **Experiments do not map to agents.** MLflow traces and evaluation + runs attach to experiments. Experiments fit training workflows, + where a run is an attempt at producing a model. They fit the agent + development loop badly: a developer looking at an agent wants its + traces and eval results immediately, not by way of an experiment + wrapper created as ceremony. There is no first-class way to say + "show me this agent's traces, for this version." + +4. **Cross-registry questions require manual inspection.** The Skill + and MCP registries can say a component version is deprecated or + compromised, but nothing records which agents carry it. "Skill X + is compromised; which agents are affected?" is answered today by + inspecting every deployment individually. The registries hold the + component half of the answer; the consumer half is missing. + +These problems compound in regulated environments, where the +accountability and blast-radius gaps are commonly reported as +blockers for putting agents into production. + +## User journeys + +These journeys define the MVP scope. They are written against the +SDK, CLI, and UI surfaces the registry will offer; exact API shapes +are illustrative. + +### Register an agent + +A developer (or a CI/CD pipeline) has built an agent and wants it on +the record. + +1. Register the agent, supplying identity, provenance, and + composition: + ```python + mlflow.genai.register_agent( + name="acme/billing-agent", + description="Answers customer billing questions.", + source=GitSource( + url="https://github.com/acme/billing-agent.git", + ref="8f4e2a1", + ), + skills=["skills:/billing-policy/1"], + mcp_servers=["mcp-servers:/acme.internal/payments-db/2.0.0"], + models=["models:/acme-billing-llm/3"], + ) + ``` + Required: name, description, and at least one definitional + anchor: source provenance, a configuration snapshot, or an A2A + Agent Card. Composition (the BOM) is required for source- and + config-anchored registrations and may be empty or partial for + card-anchored ones (see below). Optional: tags and an endpoint + (a URL plus protocol; the endpoint journey below covers + bindings). +2. MLflow creates an `AgentVersion` record with initial status + `draft`. +3. The agent appears in the registry listing for its workspace, with + its BOM entries linked to the Skill, MCP, and Model registry pages + where matching entries exist. +4. **A2A path:** an agent that has an Agent Card registers by + importing it. The UI registration form offers two modes, "import + from A2A card" and "manual"; the import mode pre-fills + descriptive and capability fields from the card, stores the card + as the immutable canonical payload, and creates an `a2a` access + binding for the card's endpoint. The registry `name` is supplied by the + registrant on this path too: the card's free-form name is + presentation, not identity, and reaches the UI through the + display-name fallback chain (RFC-0004's `title` pattern). In the SDK and CLI, the client fetches the + card from the agent's endpoint; in the UI, the card is fetched by + the browser when the endpoint permits it, or pasted, since the + registry server never fetches user-supplied URLs (consistent with + RFC-0008). The BOM is supplied alongside the card, since the card + schema does not carry component version pins. +5. **CI path:** the same call runs from a pipeline, registering a new + version on each release build with the source ref set to the + build's commit. +6. **Harness path:** an agent that runs as a configuration of a + packaged harness (Claude Code, OpenCode, Goose, and similar; see + [RFC-0009](https://github.com/mlflow/rfcs/pull/37) for the + harness/framework distinction) typically has no source repository + of its own: the agent *is* the harness plus its configuration. It + registers with a harness reference in place of user source, and + attaches the configuration that defines it: + ```python + mlflow.genai.register_agent( + name="acme/oncall-helper", + description="On-call assistant run in OpenCode.", + harness="opencode", + harness_version="0.5.3", + skills=["skills:/runbook-triage/2"], + mcp_servers=["mcp-servers:/acme.internal/pagerduty/1.2.0"], + models=["claude-sonnet-5"], + config_snapshot="./opencode.json", + ) + ``` + The configuration snapshot is read from the caller's local disk + and stored as an immutable artifact on the version. It is what + distinguishes this agent from every other installation of the + same harness: enabled tools, overridden defaults (for example, a + non-standard shell for the harness's shell tool), and behavioral + settings live there and nowhere else. + + A harness's configuration surface is not always a single file: + Claude Code, for example, spreads behavioral configuration across + a settings file, instruction files, and subagent definitions. + `config_snapshot` therefore accepts a file or a directory. What it + should capture is the configuration the registry does not + otherwise represent; content that BOM references already govern, + such as installed skill directories or MCP server definitions, + should stay out of the snapshot, because an embedded copy is + invisible to cross-registry queries and can silently drift from + the declared references. + +Composition is required wherever it is knowable, because the BOM is +the value: a registry record without composition is just a name in +a list. A registrant anchoring on source or a configuration +snapshot has the composition in front of them, so the requirement +holds there. A card-anchored registration is different: the +registrant of a vendor or partner agent cannot know which skills, +servers, and models are inside a black box, and forcing a +declaration would invite invented BOMs that pollute cross-registry +queries. For card-anchored registrations the BOM may therefore be +empty or partial, and an absent BOM is recorded as *undeclared* +composition, not as an empty dependency list. Everything else is +progressive enrichment. The endpoint URL is optional; the +trace-and-eval journey below explains which agents need one. When +provided, the URL becomes an access binding rather than a field on +the immutable version. + +The harness path is the newest part of this design and the least +settled (see [Open questions](#open-questions)). For framework-built +and custom agents, the source commit is the natural complete record +and remains the expected anchor. For harness-based agents, requiring +source would force registrations that point at the harness vendor's +repository, which identifies nothing about the specific agent, while +the configuration that actually defines it went unrecorded. +Configuration files frequently embed secrets and environment-specific +values, so the snapshot mechanism needs redaction guidance at +minimum. + +### Publish and maintain an agent's endpoint + +A platform operator deploys a registered agent and needs the +registry to say where, and how, it can be reached, without +disturbing the immutable version history. + +1. The agent is already registered (any path above) and a version + has been promoted. The platform team deploys it. +2. The operator creates an access binding for the deployment, + targeting a version or an alias and declaring the endpoint's + protocol: + ```python + mlflow.genai.create_agent_access_binding( + agent="acme/billing-agent", + target_alias="production", + endpoint_url="https://agents.acme.internal/billing", + protocol="a2a", + ) + ``` + The endpoint accepted at registration time is sugar for creating + a binding; the A2A registration path creates an `a2a` binding + automatically. +3. The agent's detail page lists its bindings. Bindings whose + protocol is self-describing (`a2a`, `mcp`) are actionable: they + are the entry points for the endpoint-driven tracing and + evaluation in the trace-and-eval journey below. An `other` + binding is a documented pointer. +4. The deployment moves to a new URL. The operator updates the + binding; no version record changes. +5. The deployment is retired. The operator deletes the binding; the + agent, its versions, and its history remain untouched. + +The protocol field is where agent bindings depart from RFC-0004, +whose bindings are always MCP and vary only by transport. Agent +endpoints speak different protocols, and the field is limited to +values that tell a caller something actionable: `a2a` and `mcp` are +self-describing (an Agent Card at the well-known path; the MCP +handshake), so URL plus protocol is enough to connect. Labels like +REST or gRPC name a transport without telling anyone how to call +the agent, so they are deliberately collapsed into `other`, which +records where an agent lives without claiming MLflow can invoke it. +This keeps the registry open to custom protocols without pretending +to understand them. As in RFC-0004, a binding that targets an alias +such as `production` follows the alias as it moves between +versions. + +### Version an agent and compare bills of materials + +A developer iterating on an agent changes its composition and needs +to see exactly what changed between two builds. + +1. The developer updates the agent: bumps a skill version, adds an + MCP server, or swaps a model. +2. They register the updated composition, producing a new + `AgentVersion`. Each version is an immutable BOM snapshot; there + is no in-place edit of composition. +3. In the UI, they open the agent's detail page, select two versions + on the Versions tab, and choose Compare. +4. The comparison shows a side-by-side BOM diff: for example, + `billing-policy` skill `1` → `2`, model `llama-3.1-70b` → + `llama-3.1-405b`, one MCP server added. When both versions carry + configuration snapshots, those diff as content: for example, v4 + switched the harness's shell configuration. +5. Changed components link to their Skill Registry and MCP Server + Registry entries, where their own version histories and changelogs + live. + +Immutable versions are what make the diff trustworthy: the comparison +reflects what was registered, not what a mutable record has drifted +into. Combined with the evaluation comparison in the next journey, +this answers the two-part question every regression investigation +starts with: what changed, and did it matter? + +### Develop against agent-centric traces and evaluations + +A developer evaluating agent quality wants traces and eval results +organized by agent and version, not by experiment. + +1. Log traces against the agent instead of an experiment: + ```python + mlflow.genai.set_active_agent("acme/billing-agent", version=3) + + with mlflow.start_span(name="answer-question"): + result = agent.run(question) + ``` + `set_active_agent` is convenience over MLflow's existing trace + destination mechanism: it wraps + `mlflow.tracing.set_destination(MlflowAgentLocation(...))`, the + way `set_experiment` establishes an experiment destination + today. The location can also be passed per span through + `start_span`'s existing `trace_destination` parameter. Framework + and harness autologgers respect the active destination, so + instrumented applications need only state which agent they are. +2. Run evaluations against the agent: + ```python + mlflow.genai.evaluate(data=eval_dataset, + scorers=[correctness_scorer], + agent_id="acme/billing-agent") + ``` +3. Open the agent's detail page. A Traces tab shows the agent's + traces, filterable by version; an Evaluations tab shows eval runs; + a summary card shows latest eval score and trace volume. +4. Compare versions: select v2 and v3, see score deltas and regressed + cases, and drill from a regressed case into its trace to identify + the cause, cross-referencing the BOM diff from the previous + journey. + +Backward compatibility is preserved by construction: an agent owns a +default experiment under the hood, and the agent location resolves +to that experiment's ID. Where MLflow already accepts a typed +destination or location (trace destinations, `search_traces` +locations), agent identity becomes a new accepted value; where it +does not (`evaluate`), `agent_id` is new, optional surface. Existing +experiment-based workflows (including model training and +fine-tuning) continue unchanged. The change is additive, not a data +model rewrite. + +No endpoint is needed for any of this when the developer has the +agent's code: the agent runs locally or in CI, autologging captures +traces during execution, and evaluation scores outputs against a test +dataset. This is why code-only registration supports the full +trace-and-eval workflow. The exception is agents whose code the user +cannot run: another team's A2A agent, a vendor agent, a partner +service. For those, the endpoint is the only execution surface, and +tracing and evaluation work by invoking the agent's access binding +with test inputs and observing responses. This track requires a +binding whose protocol MLflow can speak: `a2a` (invoked through the +card's declared interface) or `mcp` (through the MCP handshake). An +`other` binding records where the agent lives but does not by +itself tell MLflow how to call it, so it does not enable +endpoint-driven tracing or evaluation. The registry supports both +tracks; the optional endpoint exists largely for the second. + +Where [RFC-0009](https://github.com/mlflow/rfcs/pull/37) annotates +spans inside a trace with the skill that produced them, this journey +attaches the whole trace to the agent that ran. Together they give +component-level attribution within agent-level organization: from an +agent's trace list, the `SKILL` spans inside a trace show which BOM +entries were actually exercised. + +### Manage an agent's lifecycle + +An agent owner or platform team needs agents to carry an explicit, +auditable lifecycle state. + +1. An agent version starts as `draft`: visible in the registry while + the owner iterates. +2. The owner runs evaluations and reviews scores and traces (previous + journey). +3. Satisfied with quality, the owner promotes the version to + `active`, manually or from CI. Promotion is informed by evals but + not gated on them; the registry records the decision, it does not + make it. +4. When a version is superseded or found vulnerable, it transitions + to `deprecated`: still visible, marked as superseded, discouraged + from new use. +5. Every transition is recorded as an auditable event with a + timestamp and an actor, whether the actor is a human or a CI/CD + identity. + +This is the same core `draft` → `active` → `deprecated` lifecycle +the MCP and Skill registries use (their soft-delete `deleted` state +and transition rules carry over as well), applied to the agent +itself. The +auditable transition history is the accountability chain that problem +1 identifies as missing: for any agent, the registry can say who +promoted it, when, and what its evaluation evidence looked like at +the time. + +### Assess the blast radius of a compromised component + +A security engineer learns a component is compromised and must find +every affected agent without inspecting deployments one by one. + +1. The skill `k8s-troubleshooter` version `1` is flagged as + compromised. +2. The engineer transitions that skill version to `deprecated` in the + Skill Registry. +3. They query the Agent Registry for consumers of the compromised + version: + ```python + versions = mlflow.genai.search_agent_versions( + filter_string=( + "bom.skill.name = 'k8s-troubleshooter' " + "AND bom.skill.version = 1" + ) + ) + ``` + Dropping the version clause widens the query to consumers of any + version, useful when every version of the skill is suspect or when + assembling the full consumer list before deciding who is affected. +4. The registry returns the affected agent versions with their owners: + for example, three agents across two teams. +5. The engineer contacts the owning teams, who ship new agent + versions with the skill removed or upgraded, and deprecate the + affected versions (previous journey). + +The same query works for the other BOM axes: "which agents use MCP +server Y whose tool schema changed?" and "which agents call model Z +being retired?" Because BOM entries are structured soft references, +the query is a registry lookup with exact-match semantics rather than +a scan of running infrastructure. Because a BOM holds many refs per +axis, the name and version predicates must bind to the same BOM +entry; matching a name on one skill ref and a version on another +would produce false positives. That same-entry matching requirement +is a store-level obligation for the detailed design, in the way +exact matching on span attributes is for RFC-0009. The registry +answers with consumers and owners; automated notification of those +owners is deliberately not in the MVP (see Out of scope). + +Coverage has two limits a reader of the results should keep in mind: +the query sees only the component types the registry tracks, and +within those, only what registrants declared. Agents with +undeclared composition (black-box, card-anchored registrations) can +never match, so results should surface them alongside matches: "3 +agents declare the compromised skill; 12 more have undeclared +composition." The query replaces infrastructure inspection for the +governed axes; it does not claim completeness beyond them. + +## Out of scope + +The following are explicitly out of scope for this RFC. Several are +natural follow-ons; their exclusion here is sequencing, not +rejection. + +- **Runtime state.** Health, liveness, deployment status, scaling, + and placement are the serving platform's domain. The registry + stores no runtime state and performs no polling or health checks. A + platform's runtime view can join registry records against its own + inventory at query time; the full "which *running* agents are + affected?" question is that join, with this registry supplying the + consumer-and-owner half. +- **Deployment and orchestration.** The registry does not deploy + agents. Access bindings record where an approved endpoint is; they + do not create it. +- **Registry synchronization from deployments.** Auto-registering + agents when they deploy, and keeping BOMs fresh when composition + changes at deploy time, calls for platform-side glue (for example a + Kubernetes controller) pushing to the registry APIs this RFC + defines. Deferred. +- **Auto-discovery of composition.** BOMs are developer-asserted in + the MVP. Inferring actual composition from traces (for example, + from RFC-0009 `SKILL` spans) and notifying owners when assertion + and observation disagree is a compelling follow-on, and the span + data this RFC and RFC-0009 produce is designed to enable it. +- **Detection of unregistered agents.** Surfacing "shadow" agents + running without registry entries requires runtime scanning, + which is platform work built on top of this registry. +- **Automated notifications.** The blast-radius journey ends with the + registry naming owners; notifying them is left to the organization + in the MVP. +- **Agent-to-agent runtime discovery.** A programmatic "find me an + agent that can do X and call it" surface for running agents is a + gateway concern, as is any request routing. +- **Cost attribution.** Per-agent token cost is an observability + rollup over agent-linked traces, not registry metadata. +- **Cross-workspace federation.** Discovery across registries is + future work, potentially via A2A. +- **Discovery for reuse as a first-class journey.** Registry listings + are workspace-scoped and searchable, which gives teams a working + answer to "what do we have?", but curated cross-team browsing + experiences are not an MVP goal. + +# Detailed design + +TBD. + +# Drawbacks + +TBD. + +# Alternatives + +TBD. + +# Adoption strategy + +TBD. + +# Open questions + +- **How canonical is the A2A Agent Card?** This RFC treats the card + as canonical when present and optional overall. Arguments for + requiring it: industry convergence on the schema, and one canonical + payload instead of two metadata shapes. Arguments against: A2A + exists to let independent agents discover and interoperate with + agents they have never met, and a large share of production agents + today are single-agent applications or closed teams of agents that + already know each other, for which the card is a heavy dependency + whose primary purpose does not apply. Requiring it would also force + synthesized cards for manual registrations, and the card schema + carries no BOM, so MLflow-managed composition metadata exists + either way. Part of the same question is how thin a card-only + record may be: this RFC lets a card-anchored registration omit the + BOM (recorded as undeclared composition), since a black-box + agent's components are unknowable to its registrant. Is + canonical-when-present the right landing, and should registration + be able to synthesize a valid card from manual metadata for + export? + +- **Should agent-centric traces and evaluations be a separate RFC?** + The experiments bridge (`agent_id` resolving to a default + experiment) touches tracing APIs, evaluation APIs, and UI surface + area well beyond the registry itself, and there is precedent for + splitting: RFC-0008 defined the Skill Registry and RFC-0009 + followed with skill tracing. The counter-argument is that + agent-anchored traces and evals are the registry's core value; a + registry without them is a list of names. This RFC keeps the + journey in scope on the additive framing above, but if reviewers + prefer a narrower registry RFC, the journey splits cleanly along + the RFC-0008/0009 seam. + +- **Do endpoint records belong in MLflow at all?** The access binding + model resolves the mechanical objections to endpoints (mutability + against immutable versions, staleness on version records), and + RFC-0004 sets the precedent. The remaining objection is + architectural: an endpoint could be considered runtime metadata, and + the record/runtime boundary could place all of it on the platform + side. The position taken here is that "where an approved endpoint + for this agent is" belongs to the record, while "whether anything + answers there" belongs to the platform, and that without endpoints + the registry cannot trace or evaluate agents whose code the user + cannot run. This boundary needs explicit review. A related + sub-question: when an agent is exposed as an MCP server, should an + `mcp` binding cross-reference the MCP Registry entry for the same + endpoint instead of duplicating it? + +- **How should harness-based agents be described?** Agents that run + as configurations of a packaged harness (Claude Code, OpenCode, + Goose) have no user source repository; the agent is the harness + plus its configuration. This RFC proposes three mechanisms for + them, going beyond the source-centric design the journeys otherwise + follow: a harness reference axis in the BOM (an external + identifier, like external model references), an immutable + configuration snapshot stored as a version artifact to serve as the + definitional anchor, and a relaxation of the required fields from + "source" to "at least one definitional anchor" (source, + configuration snapshot, or A2A card). Sub-questions: + + - Do configuration snapshots invite secret leakage badly enough to + need enforced redaction rather than guidance? + - Should a harness axis wait for some notion of harness identity + governance? + - What are the identifier semantics of a harness reference? The + options are an opaque asserted name/version pair; an open + vocabulary in which free-text names are always accepted but + names matching MLflow's known harness integrations are + recognized and normalized; or a resolvable package identity. The + open vocabulary is the likely landing: fragmented spellings + would undermine cross-agent queries, while no single package + ecosystem could serve as an authority given how heterogeneously + harnesses are distributed. + - Should the set of files that constitutes a harness's + configuration surface be defined by per-harness integrations + (the harness integrations contemplated by RFC-0009 would be a + natural home) rather than hand-picked by each registrant, given + that the surface must also exclude content the BOM already + governs? + - Does configuration-as-artifact belong in this RFC or a + follow-on? + +- **What is the BOM reference format?** The journeys sketch URI-style + references (`skills:/billing-policy/1`, + `mcp-servers:/acme.internal/payments-db/2.0.0`, + `models:/acme-billing-llm/3`). The skill scheme follows RFC-0008's + member references. RFC-0004 defines no MCP URI scheme, so the MCP + scheme adopts the `mcp-servers:/` proposal from RFC-0010; the refs + here respect RFC-0004's reverse-DNS server names and semantic + versions. Bare identifiers cover external models (`gpt-4o`). BOM + references are soft (string-resolved, valid when + the target is unregistered), which URI syntax may misleadingly + suggest otherwise. Alternatives include structured + `{registry, name, version}` objects. Relatedly: should the BOM be + able to reference agent plugins (RFC-0008/RFC-0010) as composed + units, and should it be able to reference other agents to represent + multi-agent systems? From 56b7bf633e4d159fd168a92d721eeb504f7880c2 Mon Sep 17 00:00:00 2001 From: Bill Murdock Date: Mon, 24 Aug 2026 16:32:53 -0400 Subject: [PATCH 02/34] Fill in rfc_pr link for RFC-0011 Co-Authored-By: Claude Fable 5 --- rfcs/0011-agent-registry/0011-agent-registry.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rfcs/0011-agent-registry/0011-agent-registry.md b/rfcs/0011-agent-registry/0011-agent-registry.md index a311f2f4..53307717 100644 --- a/rfcs/0011-agent-registry/0011-agent-registry.md +++ b/rfcs/0011-agent-registry/0011-agent-registry.md @@ -3,7 +3,7 @@ | start_date | 2026-08-24 | | :----------- | :--------- | | mlflow_issue | | -| rfc_pr | | +| rfc_pr | https://github.com/mlflow/rfcs/pull/39 | | Author(s) | [Bill Murdock](https://github.com/jwm4) (Red Hat) | | :--------------------- | :-- | From 8541375f982fdb7c6e9f9491570d81bcb3b977f4 Mon Sep 17 00:00:00 2001 From: Bill Murdock Date: Tue, 25 Aug 2026 13:51:29 -0400 Subject: [PATCH 03/34] Promote agent plugin references to a BOM axis Plugins are referenced as composed units and expand through their RFC-0008-recorded members for cross-registry queries, so blast-radius lookups find agents that consume a skill via a plugin. Previously held as an open question; resolved by team direction and confirmed by the registry UI prototype's "Link plugin" flow. Co-Authored-By: Claude Fable 5 --- .../0011-agent-registry.md | 25 ++++++++++++------- 1 file changed, 16 insertions(+), 9 deletions(-) diff --git a/rfcs/0011-agent-registry/0011-agent-registry.md b/rfcs/0011-agent-registry/0011-agent-registry.md index 53307717..b11b991f 100644 --- a/rfcs/0011-agent-registry/0011-agent-registry.md +++ b/rfcs/0011-agent-registry/0011-agent-registry.md @@ -62,7 +62,9 @@ SDK namespace, following the pattern of RFC-0004 and RFC-0008: `{workspace, organization, name}` coordinates is a detailed-design point. - **AgentVersion**: an immutable snapshot of the agent's composition, - its **bill of materials (BOM)**: skill references, MCP server + its **bill of materials (BOM)**: skill references, agent plugin + references (a plugin is referenced as a composed unit and expands + through its registered members for queries), MCP server references, model references (registry models or external model identifiers such as `gpt-4o`), and, for agents that run as configurations of a packaged harness, a harness reference (a @@ -73,7 +75,8 @@ SDK namespace, following the pattern of RFC-0004 and RFC-0008: new version. BOM entries are soft references, structured values rather than -foreign keys. They resolve against the Skill Registry, MCP Server +foreign keys. They resolve against the Skill Registry (which +RFC-0008 defines for both skills and agent plugins), MCP Server Registry, and Model Registry when matching entries exist, and they remain valid when they do not. This is what makes cross-registry questions ("which agents use skill X?") answerable as registry @@ -164,6 +167,7 @@ mlflow.genai.register_agent( ref="8f4e2a1", ), skills=["skills:/billing-policy/1", "skills:/refund-rules/2"], + agent_plugins=["agent-plugins:/billing-workflow/1.2.0"], mcp_servers=["mcp-servers:/acme.internal/payments-db/2.0.0"], models=["models:/acme-billing-llm/3", "gpt-4o"], ) @@ -316,8 +320,8 @@ the record. 2. MLflow creates an `AgentVersion` record with initial status `draft`. 3. The agent appears in the registry listing for its workspace, with - its BOM entries linked to the Skill, MCP, and Model registry pages - where matching entries exist. + its BOM entries linked to the skill, agent plugin, MCP server, + and model registry pages where matching entries exist. 4. **A2A path:** an agent that has an Agent Card registers by importing it. The UI registration form offers two modes, "import from A2A card" and "manual"; the import mode pre-fills @@ -598,7 +602,11 @@ every affected agent without inspecting deployments one by one. The same query works for the other BOM axes: "which agents use MCP server Y whose tool schema changed?" and "which agents call model Z -being retired?" Because BOM entries are structured soft references, +being retired?" Agent plugin references expand through their +members: RFC-0008 plugin versions immutably record which registered +skills they contain, so "which agents use skill X" also finds +agents that consume X through a plugin, rather than leaving a +blind spot behind composed units. Because BOM entries are structured soft references, the query is a registry lookup with exact-match semantics rather than a scan of running infrastructure. Because a BOM holds many refs per axis, the name and version predicates must bind to the same BOM @@ -771,7 +779,6 @@ TBD. references are soft (string-resolved, valid when the target is unregistered), which URI syntax may misleadingly suggest otherwise. Alternatives include structured - `{registry, name, version}` objects. Relatedly: should the BOM be - able to reference agent plugins (RFC-0008/RFC-0010) as composed - units, and should it be able to reference other agents to represent - multi-agent systems? + `{registry, name, version}` objects. Relatedly: should the BOM + also be able to reference other agents, to represent multi-agent + systems? From d7dfe85d4de463847ee6de6dd4a13dfa4e17d57a Mon Sep 17 00:00:00 2001 From: Bill Murdock Date: Tue, 25 Aug 2026 13:52:57 -0400 Subject: [PATCH 04/34] Generalize source to RFC-0008 typed source pointers Source provenance now covers Git repos, OCI images, and archives via the typed source pointers RFC-0008 defines, matching how agents are actually distributed, instead of Git-only. Co-Authored-By: Claude Fable 5 --- rfcs/0011-agent-registry/0011-agent-registry.md | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/rfcs/0011-agent-registry/0011-agent-registry.md b/rfcs/0011-agent-registry/0011-agent-registry.md index b11b991f..2ccd492a 100644 --- a/rfcs/0011-agent-registry/0011-agent-registry.md +++ b/rfcs/0011-agent-registry/0011-agent-registry.md @@ -70,7 +70,8 @@ SDK namespace, following the pattern of RFC-0004 and RFC-0008: configurations of a packaged harness, a harness reference (a proposed axis; see [Open questions](#open-questions)). Each version also carries at least one **definitional anchor**: source - provenance (Git URI plus commit), an immutable configuration + provenance (a typed source pointer as in RFC-0008: a Git repo and + ref, an OCI image, or an archive), an immutable configuration snapshot, or an A2A Agent Card. Each change to composition is a new version. @@ -89,7 +90,7 @@ rather than by agent anatomy: an agent's prompt strategy or memory configuration has no axis because nothing governs one. Three layers share the job of describing an agent. Structured BOM references are selective but queryable across agents. Definitional anchors (a source -commit, a configuration snapshot) are complete but opaque: they +pointer, a configuration snapshot) are complete but opaque: they capture everything about one agent without supporting cross-agent queries. Free-form tags are the catch-all for facts that fit neither, and there will always be some, because agents acquire new kinds of @@ -174,7 +175,9 @@ mlflow.genai.register_agent( ``` This creates the `Agent` (if new) and an `AgentVersion` with status -`draft`. +`draft`. `GitSource` is one of the typed source pointers shared +with RFC-0008; an agent distributed as a container image or an +archive registers with the corresponding source type instead. ## Register an agent from an A2A Agent Card @@ -394,8 +397,8 @@ the immutable version. The harness path is the newest part of this design and the least settled (see [Open questions](#open-questions)). For framework-built -and custom agents, the source commit is the natural complete record -and remains the expected anchor. For harness-based agents, requiring +and custom agents, the registered source is the natural complete +record and remains the expected anchor. For harness-based agents, requiring source would force registrations that point at the harness vendor's repository, which identifies nothing about the specific agent, while the configuration that actually defines it went unrecorded. From ae905396bf1704bded2c313eec0f4eb2e5408dbe Mon Sep 17 00:00:00 2001 From: Bill Murdock Date: Tue, 25 Aug 2026 13:56:40 -0400 Subject: [PATCH 05/34] State the version identity policy for agent versions Agent versions are registry-minted serial numbers, per the policy from the earlier registries: adopt the artifact's inherent version when its format defines one, mint serial numbers when it does not. No standard agent artifact defines one; an A2A card's provider-defined version string is preserved as metadata, not adopted as identity. Co-Authored-By: Claude Fable 5 --- rfcs/0011-agent-registry/0011-agent-registry.md | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/rfcs/0011-agent-registry/0011-agent-registry.md b/rfcs/0011-agent-registry/0011-agent-registry.md index 2ccd492a..33706732 100644 --- a/rfcs/0011-agent-registry/0011-agent-registry.md +++ b/rfcs/0011-agent-registry/0011-agent-registry.md @@ -75,6 +75,18 @@ SDK namespace, following the pattern of RFC-0004 and RFC-0008: snapshot, or an A2A Agent Card. Each change to composition is a new version. +Agent versions are registry-minted serial numbers (1, 2, 3), +following the policy the earlier registries establish: a registry +entry adopts the underlying artifact's own version when its format +defines one (RFC-0004's `server_json`, RFC-0008's plugin manifests) +and mints serial numbers when it does not (RFC-0008's skills, whose +format defines no version field). No standard agent artifact defines +an inherent agent version. An A2A Agent Card carries a +provider-defined `version` string, but the card is optional and the +A2A specification leaves that field's format to the provider, so it +is preserved in the canonical payload as descriptive metadata rather +than adopted as version identity. + BOM entries are soft references, structured values rather than foreign keys. They resolve against the Skill Registry (which RFC-0008 defines for both skills and agent plugins), MCP Server From 6bf4e1564ea1f5d328f3c3023d55f79d12d1b743 Mon Sep 17 00:00:00 2001 From: Bill Murdock Date: Tue, 25 Aug 2026 15:40:35 -0400 Subject: [PATCH 06/34] Allow multiple typed sources per agent version A version's source provenance is now a list of typed pointers, so an agent built from a Git repo and shipped as an OCI image records both, and source entries join the queryable axes ("which agents ship image X?"). Deliberate divergence from RFC-0008's one-source-per-version rule, with the rationale (agents have no content digest to reconcile per-source duplicates) stated in Open questions. Prompted by Ann Marie's review question. Co-Authored-By: Claude Fable 5 --- .../0011-agent-registry.md | 48 ++++++++++++++----- 1 file changed, 35 insertions(+), 13 deletions(-) diff --git a/rfcs/0011-agent-registry/0011-agent-registry.md b/rfcs/0011-agent-registry/0011-agent-registry.md index 33706732..cd3bd402 100644 --- a/rfcs/0011-agent-registry/0011-agent-registry.md +++ b/rfcs/0011-agent-registry/0011-agent-registry.md @@ -70,9 +70,9 @@ SDK namespace, following the pattern of RFC-0004 and RFC-0008: configurations of a packaged harness, a harness reference (a proposed axis; see [Open questions](#open-questions)). Each version also carries at least one **definitional anchor**: source - provenance (a typed source pointer as in RFC-0008: a Git repo and - ref, an OCI image, or an archive), an immutable configuration - snapshot, or an A2A Agent Card. Each change to composition is a + provenance (one or more typed source pointers as in RFC-0008: a + Git repo and ref, an OCI image, an archive), an immutable + configuration snapshot, or an A2A Agent Card. Each change to composition is a new version. Agent versions are registry-minted serial numbers (1, 2, 3), @@ -175,10 +175,13 @@ from mlflow.genai import GitSource mlflow.genai.register_agent( name="acme/billing-agent", description="Answers customer billing questions.", - source=GitSource( - url="https://github.com/acme/billing-agent.git", - ref="8f4e2a1", - ), + sources=[ + GitSource( + url="https://github.com/acme/billing-agent.git", + ref="8f4e2a1", + ), + OciSource("quay.io/acme/billing-agent@sha256:9f2c1e"), + ], skills=["skills:/billing-policy/1", "skills:/refund-rules/2"], agent_plugins=["agent-plugins:/billing-workflow/1.2.0"], mcp_servers=["mcp-servers:/acme.internal/payments-db/2.0.0"], @@ -187,9 +190,12 @@ mlflow.genai.register_agent( ``` This creates the `Agent` (if new) and an `AgentVersion` with status -`draft`. `GitSource` is one of the typed source pointers shared -with RFC-0008; an agent distributed as a container image or an -archive registers with the corresponding source type instead. +`draft`. `GitSource` and `OciSource` are typed source pointers as +in RFC-0008. A version may record multiple sources; here, the Git +repo the agent is built from and the container image it ships as. +Unlike skill versions, which carry exactly one source, agent +versions allow several (see [Open questions](#open-questions) for +the rationale). ## Register an agent from an A2A Agent Card @@ -316,10 +322,10 @@ the record. mlflow.genai.register_agent( name="acme/billing-agent", description="Answers customer billing questions.", - source=GitSource( + sources=[GitSource( url="https://github.com/acme/billing-agent.git", ref="8f4e2a1", - ), + )], skills=["skills:/billing-policy/1"], mcp_servers=["mcp-servers:/acme.internal/payments-db/2.0.0"], models=["models:/acme-billing-llm/3"], @@ -617,7 +623,9 @@ every affected agent without inspecting deployments one by one. The same query works for the other BOM axes: "which agents use MCP server Y whose tool schema changed?" and "which agents call model Z -being retired?" Agent plugin references expand through their +being retired?" Source entries are queryable the same way: "which +agents ship OCI image X?" is the container-CVE variant of this +journey. Agent plugin references expand through their members: RFC-0008 plugin versions immutably record which registered skills they contain, so "which agents use skill X" also finds agents that consume X through a plugin, rather than leaving a @@ -783,6 +791,20 @@ TBD. - Does configuration-as-artifact belong in this RFC or a follow-on? +- **Should a version record multiple sources?** This RFC says yes: a + version's source provenance is a list of typed pointers, so an + agent built from a Git repo and shipped as an OCI image records + both, and source entries are queryable like BOM axes ("which + agents ship image X?"). This deliberately diverges from RFC-0008, + where each skill version has exactly one source and the content + digest reconciles identical content registered from different + sources. Agents have no defined content bundle to digest, so the + skill pattern applied to agents would produce irreconcilable + duplicate versions of what is really one agent. The list is + currently an unordered set of asserted pointers. Reviewers who + weigh cross-registry consistency + heavily should push back here if the divergence is not worth it. + - **What is the BOM reference format?** The journeys sketch URI-style references (`skills:/billing-policy/1`, `mcp-servers:/acme.internal/payments-db/2.0.0`, From 9ffdbb6904ce4a467ff1226d7dda0227b28d0631 Mon Sep 17 00:00:00 2001 From: Bill Murdock Date: Tue, 25 Aug 2026 16:36:53 -0400 Subject: [PATCH 07/34] Add per-agent version schemes with monotonic default An Agent chooses its version scheme at creation: monotonic (registry-assigned serial numbers, the default when no version is supplied), semver, or freeform, with possible autodetection from the first registration. Keeps the simple path as simple as the Skill Registry while letting provider-versioned agents keep their own versioning. Proposed by Matthew Prahl in review discussion. Co-Authored-By: Claude Fable 5 --- .../0011-agent-registry.md | 37 ++++++++++++------- 1 file changed, 24 insertions(+), 13 deletions(-) diff --git a/rfcs/0011-agent-registry/0011-agent-registry.md b/rfcs/0011-agent-registry/0011-agent-registry.md index cd3bd402..405f812b 100644 --- a/rfcs/0011-agent-registry/0011-agent-registry.md +++ b/rfcs/0011-agent-registry/0011-agent-registry.md @@ -75,17 +75,26 @@ SDK namespace, following the pattern of RFC-0004 and RFC-0008: configuration snapshot, or an A2A Agent Card. Each change to composition is a new version. -Agent versions are registry-minted serial numbers (1, 2, 3), -following the policy the earlier registries establish: a registry -entry adopts the underlying artifact's own version when its format -defines one (RFC-0004's `server_json`, RFC-0008's plugin manifests) -and mints serial numbers when it does not (RFC-0008's skills, whose -format defines no version field). No standard agent artifact defines -an inherent agent version. An A2A Agent Card carries a -provider-defined `version` string, but the card is optional and the -A2A specification leaves that field's format to the provider, so it -is preserved in the canonical payload as descriptive metadata rather -than adopted as version identity. +Version identity is a per-agent choice made when the agent is +created: `monotonic` (registry-assigned serial numbers: 1, 2, 3), +`semver` (registrant-supplied semantic versions), or `freeform` +(registrant-supplied opaque strings). `monotonic` is the default +when registration supplies no version, so the simple path stays as +simple as the Skill Registry's: registrants who never think about +versioning get serial numbers automatically, while an agent that +already carries its own versioning (a team's release train, a +provider-versioned A2A agent) can keep it. The scheme may also be +autodetected from the first registration's input. This refines the +policy the earlier registries establish, where an entry adopts the +underlying artifact's version when its format defines one +(RFC-0004's `server_json`, RFC-0008's plugin manifests) and mints +serial numbers when it does not (RFC-0008's skills): no standard +agent artifact defines an inherent agent version, so minting is the +default rather than the rule. An A2A card's provider-defined +`version` string is preserved in the canonical payload either way; +an agent registered under `semver` or `freeform` may mirror it. +Aliases and `latest` resolution are well defined for `monotonic` +and `semver`; `freeform` versions order by registration time. BOM entries are soft references, structured values rather than foreign keys. They resolve against the Skill Registry (which @@ -335,8 +344,10 @@ the record. anchor: source provenance, a configuration snapshot, or an A2A Agent Card. Composition (the BOM) is required for source- and config-anchored registrations and may be empty or partial for - card-anchored ones (see below). Optional: tags and an endpoint - (a URL plus protocol; the endpoint journey below covers + card-anchored ones (see below). Optional: tags, an explicit + version where the agent's version scheme takes one (the default + `monotonic` scheme assigns versions automatically), and an + endpoint (a URL plus protocol; the endpoint journey below covers bindings). 2. MLflow creates an `AgentVersion` record with initial status `draft`. From bfd3abb1801a67300711edcf7c10cfba27e74d4b Mon Sep 17 00:00:00 2001 From: Bill Murdock Date: Wed, 26 Aug 2026 10:57:59 -0400 Subject: [PATCH 08/34] Fetch Agent Cards on demand instead of storing them The agent's endpoint is the card's system of record: registration imports descriptive metadata (card name seeds display_name) and creates the a2a binding, the UI renders the card read-only by fetching through the binding, and no card payload is persisted. This deliberately departs from the RFC-0004/0008 canonical-payload pattern. Endpoint-only registrations become interface-only records with no definitional anchor, marked as such. The canonicity open question is replaced by whether registration and lifecycle events should capture card copies as audit evidence. Direction from design-doc review comments, confirmed by Bill. Co-Authored-By: Claude Fable 5 --- .../0011-agent-registry.md | 175 ++++++++++-------- 1 file changed, 97 insertions(+), 78 deletions(-) diff --git a/rfcs/0011-agent-registry/0011-agent-registry.md b/rfcs/0011-agent-registry/0011-agent-registry.md index 405f812b..24ec86a4 100644 --- a/rfcs/0011-agent-registry/0011-agent-registry.md +++ b/rfcs/0011-agent-registry/0011-agent-registry.md @@ -71,9 +71,10 @@ SDK namespace, following the pattern of RFC-0004 and RFC-0008: proposed axis; see [Open questions](#open-questions)). Each version also carries at least one **definitional anchor**: source provenance (one or more typed source pointers as in RFC-0008: a - Git repo and ref, an OCI image, an archive), an immutable - configuration snapshot, or an A2A Agent Card. Each change to composition is a - new version. + Git repo and ref, an OCI image, an archive) and/or an immutable + configuration snapshot. An agent registered from an A2A endpoint + alone is an interface-only record with no anchor (see below). + Each change to composition is a new version. Version identity is a per-agent choice made when the agent is created: `monotonic` (registry-assigned serial numbers: 1, 2, 3), @@ -91,7 +92,7 @@ underlying artifact's version when its format defines one serial numbers when it does not (RFC-0008's skills): no standard agent artifact defines an inherent agent version, so minting is the default rather than the rule. An A2A card's provider-defined -`version` string is preserved in the canonical payload either way; +`version` string lives on the live card rather than in the registry; an agent registered under `semver` or `freeform` may mirror it. Aliases and `latest` resolution are well defined for `monotonic` and `semver`; `freeform` versions order by registration time. @@ -118,17 +119,27 @@ and there will always be some, because agents acquire new kinds of parts faster than any schema anticipates. New structured axes are expected as the governance surface grows. -**A2A Agent Cards are the canonical interchange payload when -present.** The [A2A protocol](https://a2a-protocol.org/) defines the -Agent Card schema that much of the industry has converged on for -describing an agent's identity, capabilities, and endpoint. Following -the hybrid storage pattern of RFC-0004 (`server_json`) and RFC-0008 -(`plugin.json`), an agent version registered from an Agent Card -preserves the complete card immutably, with MLflow-managed fields -(BOM, lifecycle, tags, aliases) kept outside the canonical payload. -The card is optional: agents that do not speak A2A register with -plain metadata, and gain a card later if they adopt one. Whether the -card should instead be required or synthesized is an open question. +**A2A Agent Cards are fetched, not stored.** The +[A2A protocol](https://a2a-protocol.org/) makes an agent's endpoint +the authoritative home of its Agent Card: the live card is always +one GET away at the endpoint's well-known path, and every A2A +client reads it from there. The registry follows suit. Registering +from an endpoint imports the card's descriptive metadata +(description, capabilities, and its free-form name, which seeds the +mutable MLflow-managed `display_name`) into ordinary registry +fields and creates an `a2a` access binding; the card content itself +is not persisted. The UI renders the card read-only by fetching it +through the binding at view time, so what MLflow displays can never +drift from what the agent serves. This deliberately departs from +the canonical-payload pattern of RFC-0004 (`server_json`) and +RFC-0008 (`plugin.json`): MLflow is the system of record for those +payloads, while an Agent Card's system of record is the agent +itself. An agent registered from an endpoint alone, with no source +and no configuration snapshot, is an **interface-only record**: the +registry captures the claim surface (identity, imported metadata, +endpoint) and marks that it holds no definitional anchor. Whether +registration and lifecycle events should optionally capture a card +copy as audit evidence is an open question. **Endpoints are access bindings, not version fields.** Some agents are reachable at a URL (A2A agents inherently; deployed agents @@ -156,8 +167,9 @@ change is additive: under the hood an agent owns a default experiment, the agent location resolves to it, and experiment-based workflows continue unchanged. -**Relationship to other RFCs.** RFC-0004 establishes the canonical -payload and access binding patterns this RFC reuses. RFC-0008 defines +**Relationship to other RFCs.** RFC-0004 establishes the access +binding pattern this RFC reuses (its canonical-payload pattern is +deliberately not applied to Agent Cards; see above). RFC-0008 defines the skills and agent plugins that agent BOMs reference; note that an *agent plugin* (a package of components installed into a harness) is not an *agent* (an application that acts); this registry governs the @@ -217,20 +229,22 @@ mlflow.genai.register_agent( ``` Given an endpoint, the client SDK fetches the Agent Card from the -endpoint's well-known path, stores it immutably as the canonical -payload and the version's definitional anchor (supplying -description and capability metadata), and creates an `a2a` access -binding for the endpoint. The registry `name` is always chosen by the -registrant: an Agent Card's `name` is a free-form display string, -not an identity, so it feeds presentation rather than naming. -Following RFC-0004's pattern for `server_json["title"]`, the UI -falls back to the card's name when the mutable MLflow-managed -`display_name` is unset, and to the registry `name` after that. The card is whatever the running agent serves; it -need not exist as a static file anywhere. The fetch always happens in -the client: the registry server never fetches user-supplied URLs, -consistent with RFC-0008. A caller that wants to inspect or adjust -card metadata before registering can fetch the card itself and pass -it via `a2a_card=` instead. +endpoint's well-known path, imports its descriptive metadata +(description, capabilities, and its free-form name, which seeds the +mutable MLflow-managed `display_name`), and creates an `a2a` access +binding for the endpoint. The card content is not persisted: the +endpoint is the card's system of record, and the UI renders the +card read-only by fetching it through the binding at view time. The +registry `name` is always chosen by the registrant, since a card's +`name` is a display string, not an identity. The card is whatever +the running agent serves; it need not exist as a static file +anywhere. Fetches always happen in the client or the browser: the +registry server never fetches user-supplied URLs, consistent with +RFC-0008. A caller that wants to inspect or adjust the imported +metadata before registering can fetch the card itself and pass it +via `a2a_card=` instead. A version registered this way, with no +source and no configuration snapshot, is an interface-only record +(see the register journey). ## Register a harness-based agent @@ -340,11 +354,14 @@ the record. models=["models:/acme-billing-llm/3"], ) ``` - Required: name, description, and at least one definitional - anchor: source provenance, a configuration snapshot, or an A2A - Agent Card. Composition (the BOM) is required for source- and - config-anchored registrations and may be empty or partial for - card-anchored ones (see below). Optional: tags, an explicit + Required: name, description, and at least one of: a definitional + anchor (source provenance or a configuration snapshot) or an A2A + endpoint. These combine freely; a first-party A2A agent registers + with both source and endpoint. Only a registration with an + endpoint and no anchor produces an interface-only record (see + below). Composition (the BOM) is required when an anchor is + present and may be empty or partial for interface-only records + (see below). Optional: tags, an explicit version where the agent's version scheme takes one (the default `monotonic` scheme assigns versions automatically), and an endpoint (a URL plus protocol; the endpoint journey below covers @@ -354,20 +371,21 @@ the record. 3. The agent appears in the registry listing for its workspace, with its BOM entries linked to the skill, agent plugin, MCP server, and model registry pages where matching entries exist. -4. **A2A path:** an agent that has an Agent Card registers by - importing it. The UI registration form offers two modes, "import +4. **A2A path:** an agent that serves an Agent Card registers from + its endpoint. The UI registration form offers two modes, "import from A2A card" and "manual"; the import mode pre-fills - descriptive and capability fields from the card, stores the card - as the immutable canonical payload, and creates an `a2a` access - binding for the card's endpoint. The registry `name` is supplied by the - registrant on this path too: the card's free-form name is - presentation, not identity, and reaches the UI through the - display-name fallback chain (RFC-0004's `title` pattern). In the SDK and CLI, the client fetches the - card from the agent's endpoint; in the UI, the card is fetched by - the browser when the endpoint permits it, or pasted, since the - registry server never fetches user-supplied URLs (consistent with - RFC-0008). The BOM is supplied alongside the card, since the card - schema does not carry component version pins. + descriptive and capability fields from the card (its free-form + name seeds the mutable `display_name`; the registry `name` is + supplied by the registrant) and creates an `a2a` access binding + for the endpoint. The card content is not persisted: the agent + detail page renders the card read-only by fetching it through + the binding, so the display never drifts from what the agent + serves. In the SDK and CLI, the client fetches the card at + import; in the UI, the browser fetches it when the endpoint + permits, or the user pastes it, since the registry server never + fetches user-supplied URLs (consistent with RFC-0008). The BOM + is supplied alongside, since the card schema does not carry + component version pins. 5. **CI path:** the same call runs from a pipeline, registering a new version on each release build with the source ref set to the build's commit. @@ -412,13 +430,16 @@ Composition is required wherever it is knowable, because the BOM is the value: a registry record without composition is just a name in a list. A registrant anchoring on source or a configuration snapshot has the composition in front of them, so the requirement -holds there. A card-anchored registration is different: the -registrant of a vendor or partner agent cannot know which skills, -servers, and models are inside a black box, and forcing a -declaration would invite invented BOMs that pollute cross-registry -queries. For card-anchored registrations the BOM may therefore be -empty or partial, and an absent BOM is recorded as *undeclared* -composition, not as an empty dependency list. Everything else is +holds there. An interface-only registration (an A2A endpoint with +no source or configuration snapshot) is different: the registrant +of a vendor or partner agent cannot know which skills, servers, and +models are inside a black box, and forcing a declaration would +invite invented BOMs that pollute cross-registry queries. For +interface-only records the BOM may therefore be empty or partial, +an absent BOM is recorded as *undeclared* composition, not as an +empty dependency list, and the record itself is marked as holding +no definitional anchor: the registry knows the agent's claim +surface, not its contents. Everything else is progressive enrichment. The endpoint URL is optional; the trace-and-eval journey below explains which agents need one. When provided, the URL becomes an access binding rather than a field on @@ -427,7 +448,8 @@ the immutable version. The harness path is the newest part of this design and the least settled (see [Open questions](#open-questions)). For framework-built and custom agents, the registered source is the natural complete -record and remains the expected anchor. For harness-based agents, requiring +record and remains the expected anchor. For harness-based agents, +requiring source would force registrations that point at the harness vendor's repository, which identifies nothing about the specific agent, while the configuration that actually defines it went unrecorded. @@ -640,7 +662,8 @@ journey. Agent plugin references expand through their members: RFC-0008 plugin versions immutably record which registered skills they contain, so "which agents use skill X" also finds agents that consume X through a plugin, rather than leaving a -blind spot behind composed units. Because BOM entries are structured soft references, +blind spot behind composed units. Because BOM entries are +structured soft references, the query is a registry lookup with exact-match semantics rather than a scan of running infrastructure. Because a BOM holds many refs per axis, the name and version predicates must bind to the same BOM @@ -654,8 +677,9 @@ owners is deliberately not in the MVP (see Out of scope). Coverage has two limits a reader of the results should keep in mind: the query sees only the component types the registry tracks, and within those, only what registrants declared. Agents with -undeclared composition (black-box, card-anchored registrations) can -never match, so results should surface them alongside matches: "3 +undeclared composition (black-box, interface-only registrations) +can never match, so results should surface them alongside matches: +"3 agents declare the compromised skill; 12 more have undeclared composition." The query replaces infrastructure inspection for the governed axes; it does not claim completeness beyond them. @@ -722,24 +746,19 @@ TBD. # Open questions -- **How canonical is the A2A Agent Card?** This RFC treats the card - as canonical when present and optional overall. Arguments for - requiring it: industry convergence on the schema, and one canonical - payload instead of two metadata shapes. Arguments against: A2A - exists to let independent agents discover and interoperate with - agents they have never met, and a large share of production agents - today are single-agent applications or closed teams of agents that - already know each other, for which the card is a heavy dependency - whose primary purpose does not apply. Requiring it would also force - synthesized cards for manual registrations, and the card schema - carries no BOM, so MLflow-managed composition metadata exists - either way. Part of the same question is how thin a card-only - record may be: this RFC lets a card-anchored registration omit the - BOM (recorded as undeclared composition), since a black-box - agent's components are unknowable to its registrant. Is - canonical-when-present the right landing, and should registration - be able to synthesize a valid card from manual metadata for - export? +- **Should events capture Agent Card evidence?** This RFC treats + the agent's endpoint as the card's system of record: cards are + fetched for display and imported for metadata, never persisted. + That leaves governance without any record of what an agent + claimed at the moment somebody acted on the claim. Should + registration and lifecycle transitions (for example, promotion to + `active`) optionally capture a copy of the card as audit evidence + attached to the event? That would give regulated environments + decision-time evidence without reintroducing a stored payload + that pretends to define the version. Relatedly, how thin may an + interface-only record be before it is not worth registering: is + identity, imported metadata, a binding, and undeclared + composition enough? - **Should agent-centric traces and evaluations be a separate RFC?** The experiments bridge (`agent_id` resolving to a default From 2ad2c82a379a73eed0b4817575e5ad48df46ebaa Mon Sep 17 00:00:00 2001 From: Bill Murdock Date: Wed, 26 Aug 2026 12:41:20 -0400 Subject: [PATCH 09/34] Record the agent version on evaluation runs Both evaluate sketches now pass agent_version alongside agent_id; version-to-version eval comparison (J3) needs the version captured at evaluation time. From design-doc review feedback. Co-Authored-By: Claude Fable 5 --- rfcs/0011-agent-registry/0011-agent-registry.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/rfcs/0011-agent-registry/0011-agent-registry.md b/rfcs/0011-agent-registry/0011-agent-registry.md index 24ec86a4..6221b2b8 100644 --- a/rfcs/0011-agent-registry/0011-agent-registry.md +++ b/rfcs/0011-agent-registry/0011-agent-registry.md @@ -282,6 +282,7 @@ mlflow.genai.evaluate( data=eval_dataset, scorers=[correctness_scorer], agent_id="acme/billing-agent", + agent_version=3, ) ``` @@ -554,7 +555,8 @@ organized by agent and version, not by experiment. ```python mlflow.genai.evaluate(data=eval_dataset, scorers=[correctness_scorer], - agent_id="acme/billing-agent") + agent_id="acme/billing-agent", + agent_version=3) ``` 3. Open the agent's detail page. A Traces tab shows the agent's traces, filterable by version; an Evaluations tab shows eval runs; From 5b19e673dc6dc35123ea1cce23d4c6c000b9ad0f Mon Sep 17 00:00:00 2001 From: Bill Murdock Date: Wed, 26 Aug 2026 12:48:09 -0400 Subject: [PATCH 10/34] Drop the card-evidence open question The case for capturing card copies as decision evidence was weak: no customer signal, promotion decisions rest on eval evidence, and registration already imports the card's claim surface into registry fields. The open question is now solely about how thin an interface-only record may be. Co-Authored-By: Claude Fable 5 --- .../0011-agent-registry.md | 23 ++++++------------- 1 file changed, 7 insertions(+), 16 deletions(-) diff --git a/rfcs/0011-agent-registry/0011-agent-registry.md b/rfcs/0011-agent-registry/0011-agent-registry.md index 6221b2b8..2f5f1393 100644 --- a/rfcs/0011-agent-registry/0011-agent-registry.md +++ b/rfcs/0011-agent-registry/0011-agent-registry.md @@ -137,9 +137,7 @@ payloads, while an Agent Card's system of record is the agent itself. An agent registered from an endpoint alone, with no source and no configuration snapshot, is an **interface-only record**: the registry captures the claim surface (identity, imported metadata, -endpoint) and marks that it holds no definitional anchor. Whether -registration and lifecycle events should optionally capture a card -copy as audit evidence is an open question. +endpoint) and marks that it holds no definitional anchor. **Endpoints are access bindings, not version fields.** Some agents are reachable at a URL (A2A agents inherently; deployed agents @@ -748,19 +746,12 @@ TBD. # Open questions -- **Should events capture Agent Card evidence?** This RFC treats - the agent's endpoint as the card's system of record: cards are - fetched for display and imported for metadata, never persisted. - That leaves governance without any record of what an agent - claimed at the moment somebody acted on the claim. Should - registration and lifecycle transitions (for example, promotion to - `active`) optionally capture a copy of the card as audit evidence - attached to the event? That would give regulated environments - decision-time evidence without reintroducing a stored payload - that pretends to define the version. Relatedly, how thin may an - interface-only record be before it is not worth registering: is - identity, imported metadata, a binding, and undeclared - composition enough? +- **How thin may an interface-only record be?** A black-box A2A + agent registers with a name, metadata imported from its card, an + access binding, undeclared composition, and no definitional + anchor. Is that enough of a record to be worth governing, and + should the registry require anything more of it before such a + record can be promoted to `active`? - **Should agent-centric traces and evaluations be a separate RFC?** The experiments bridge (`agent_id` resolving to a default From c918c25a7cf2cce31516102c53730b547db169be Mon Sep 17 00:00:00 2001 From: Bill Murdock Date: Wed, 26 Aug 2026 13:22:36 -0400 Subject: [PATCH 11/34] Note that version-vs-new-agent is the registrant's call Two sentences in the versioning journey: the choice is the same judgment developers make for any software, the registry enforces no rule, and the consequence is that version comparison exists only within one agent. From design-doc review discussion. Co-Authored-By: Claude Fable 5 --- rfcs/0011-agent-registry/0011-agent-registry.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/rfcs/0011-agent-registry/0011-agent-registry.md b/rfcs/0011-agent-registry/0011-agent-registry.md index 2f5f1393..5b1f4142 100644 --- a/rfcs/0011-agent-registry/0011-agent-registry.md +++ b/rfcs/0011-agent-registry/0011-agent-registry.md @@ -529,6 +529,13 @@ into. Combined with the evaluation comparison in the next journey, this answers the two-part question every regression investigation starts with: what changed, and did it matter? +Whether a change is a new version or a new agent is the +registrant's call, the same judgment developers already make for +any software: is this a new release of the same application, or a +different application? The registry enforces no rule; the practical +consequence of the choice is that version comparison exists only +within one agent. + ### Develop against agent-centric traces and evaluations A developer evaluating agent quality wants traces and eval results From b7ae6df75c0117f10f5a3d15f557dd494218499a Mon Sep 17 00:00:00 2001 From: Bill Murdock Date: Wed, 26 Aug 2026 13:26:44 -0400 Subject: [PATCH 12/34] Reframe the versioning journey to avoid overpromising The journey no longer claims the diff reveals what broke and then walks it back. The investigator is framed from the start as someone other than the change author, the diff is a shared record of declared changes ("what differs in the declarations for two versions"), and it gives an investigation its starting facts rather than answers. From design-doc review discussion. Co-Authored-By: Claude Fable 5 --- .../0011-agent-registry.md | 21 ++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-) diff --git a/rfcs/0011-agent-registry/0011-agent-registry.md b/rfcs/0011-agent-registry/0011-agent-registry.md index 5b1f4142..68bf9a0d 100644 --- a/rfcs/0011-agent-registry/0011-agent-registry.md +++ b/rfcs/0011-agent-registry/0011-agent-registry.md @@ -504,8 +504,10 @@ versions. ### Version an agent and compare bills of materials -A developer iterating on an agent changes its composition and needs -to see exactly what changed between two builds. +A developer iterating on an agent registers each change as a new +version. Later, when behavior shifts, whoever is investigating, and +it is often not the person who made the change, compares two +versions to see exactly what differs. 1. The developer updates the agent: bumps a skill version, adds an MCP server, or swaps a model. @@ -523,11 +525,16 @@ to see exactly what changed between two builds. Registry entries, where their own version histories and changelogs live. -Immutable versions are what make the diff trustworthy: the comparison -reflects what was registered, not what a mutable record has drifted -into. Combined with the evaluation comparison in the next journey, -this answers the two-part question every regression investigation -starts with: what changed, and did it matter? +Immutable versions are what make the diff trustworthy: the +comparison reflects what was registered, not what a mutable record +has drifted into. The diff is a shared record of declared changes: +its value is that a teammate, reviewer, or security engineer can +see what differs in the declarations for two versions without +depending on the change author's memory or availability. Combined +with the evaluation comparison in the next journey, it gives an +investigation its starting facts: what changed, and did it matter? +Discovering changes that nobody declared belongs to the deferred +auto-discovery work (see Out of scope). Whether a change is a new version or a new agent is the registrant's call, the same judgment developers already make for From 5f34e0079d8a5b4e34341f7119d7f5070c5d0dc3 Mon Sep 17 00:00:00 2001 From: Bill Murdock Date: Wed, 26 Aug 2026 15:16:10 -0400 Subject: [PATCH 13/34] Scope trace destinations per deployment New J3 paragraph: the default experiment is a default, not a router. Deployments log wherever their own configuration points; scale-out replicas aggregate by sharing config; deployments needing separation override the destination (destinations, not tags, because permissions are experiment-scoped). Proposes, as not yet settled, that such deployments notify the registry so the experiment ID is recorded on their access binding. Sync bullet gains trace-location link upkeep. From the deployment/experiment-mapping review discussions. Co-Authored-By: Claude Fable 5 --- .../0011-agent-registry.md | 29 ++++++++++++++++--- 1 file changed, 25 insertions(+), 4 deletions(-) diff --git a/rfcs/0011-agent-registry/0011-agent-registry.md b/rfcs/0011-agent-registry/0011-agent-registry.md index 68bf9a0d..bc651ae2 100644 --- a/rfcs/0011-agent-registry/0011-agent-registry.md +++ b/rfcs/0011-agent-registry/0011-agent-registry.md @@ -588,6 +588,26 @@ experiment-based workflows (including model training and fine-tuning) continue unchanged. The change is additive, not a data model rewrite. +The default experiment is a default, not a router. A running agent +logs traces to whatever destination its own deployment +configuration sets; the registry is not in the call path. When +nothing is set, traces land in the agent's default experiment, +which is the right behavior for the development loop and +registry-driven evaluations. Scale-out replicas of one deployment +share its configuration, so their traces aggregate without further +arrangement. A deployment that needs its traces kept separate from +other deployments of the same agent (a different owner or user +base) overrides the destination in its own configuration; because +permissions are experiment-scoped, separation is done with +destinations, not trace tags. To keep such traces findable from the +agent's registry page, this RFC proposes that a deployment using a +non-default experiment notify the registry, which records the +experiment ID on that deployment's access binding. A deployment +with no binding would need another place to record the link, which +is one reason this mechanism is a proposal rather than settled. +Automating the setup and upkeep of these links at deploy time +belongs to the deferred registry synchronization glue. + No endpoint is needed for any of this when the developer has the agent's code: the agent runs locally or in CI, autologging captures traces during execution, and evaluation scores outputs against a test @@ -715,10 +735,11 @@ rejection. agents. Access bindings record where an approved endpoint is; they do not create it. - **Registry synchronization from deployments.** Auto-registering - agents when they deploy, and keeping BOMs fresh when composition - changes at deploy time, calls for platform-side glue (for example a - Kubernetes controller) pushing to the registry APIs this RFC - defines. Deferred. + agents when they deploy, keeping BOMs fresh when composition + changes at deploy time, and maintaining deployment trace-location + links (see the trace journey) calls for platform-side glue (for + example a Kubernetes controller) pushing to the registry APIs + this RFC defines. Deferred. - **Auto-discovery of composition.** BOMs are developer-asserted in the MVP. Inferring actual composition from traces (for example, from RFC-0009 `SKILL` spans) and notifying owners when assertion From 708f9640561e0b7c8f92f9b8b51d0d26621d6004 Mon Sep 17 00:00:00 2001 From: Bill Murdock Date: Wed, 26 Aug 2026 15:23:11 -0400 Subject: [PATCH 14/34] Explain why versions share the agent's experiment One sentence in the J3 deployment paragraph: a version is an analysis dimension recorded on every trace, not an access boundary, and per-version experiments would break the longitudinal view across upgrades. Co-Authored-By: Claude Fable 5 --- rfcs/0011-agent-registry/0011-agent-registry.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/rfcs/0011-agent-registry/0011-agent-registry.md b/rfcs/0011-agent-registry/0011-agent-registry.md index bc651ae2..9a678770 100644 --- a/rfcs/0011-agent-registry/0011-agent-registry.md +++ b/rfcs/0011-agent-registry/0011-agent-registry.md @@ -599,7 +599,11 @@ arrangement. A deployment that needs its traces kept separate from other deployments of the same agent (a different owner or user base) overrides the destination in its own configuration; because permissions are experiment-scoped, separation is done with -destinations, not trace tags. To keep such traces findable from the +destinations, not trace tags. Versions share the agent's experiment +for the same reason seen from the other side: a version is an +analysis dimension recorded on every trace, not an access boundary, +and per-version experiments would break the longitudinal view of an +agent's behavior across upgrades. To keep such traces findable from the agent's registry page, this RFC proposes that a deployment using a non-default experiment notify the registry, which records the experiment ID on that deployment's access binding. A deployment From ec342cd00b85521abd5c649a21652f0b9acecd8a Mon Sep 17 00:00:00 2001 From: Bill Murdock Date: Wed, 26 Aug 2026 17:45:38 -0400 Subject: [PATCH 15/34] Open the per-agent vs per-version default experiment question The RFC keeps its per-agent position but now acknowledges the fork as an open question with both sides stated, since a reviewer holds the per-version position and the author is not settled. Whichever default is chosen, the other camp overrides per deployment, so the question is which behavior makes the better default. Co-Authored-By: Claude Fable 5 --- rfcs/0011-agent-registry/0011-agent-registry.md | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/rfcs/0011-agent-registry/0011-agent-registry.md b/rfcs/0011-agent-registry/0011-agent-registry.md index 9a678770..3dc9376c 100644 --- a/rfcs/0011-agent-registry/0011-agent-registry.md +++ b/rfcs/0011-agent-registry/0011-agent-registry.md @@ -785,6 +785,20 @@ TBD. # Open questions +- **Is the default experiment per agent or per agent-version?** This + RFC says per agent: a version is an analysis dimension recorded on + every trace, one experiment preserves the longitudinal view of an + agent's behavior across upgrades, and the agent-to-experiment + mapping stays one-to-one, with no version required to resolve a + destination. The opposing position, held by at least one reviewer, + is one default experiment per version, which partitions each + version's traces physically, at the cost of making cross-upgrade + monitoring a cross-experiment query and making the version a + required argument wherever a destination is resolved. Whichever + default is chosen, users who want the other behavior override it + per deployment, so the question is which behavior makes the better + default, not which is possible. + - **How thin may an interface-only record be?** A black-box A2A agent registers with a name, metadata imported from its card, an access binding, undeclared composition, and no definitional From 3efd3cf3d4d1e9702edf98da40b926d8938e8f3c Mon Sep 17 00:00:00 2001 From: Bill Murdock Date: Thu, 27 Aug 2026 10:09:39 -0400 Subject: [PATCH 16/34] Clarify that destinations are agent-only and versions are metadata The API sketches implied the version participates in choosing the default experiment. Now stated explicitly in the Summary, basic example, and trace journey: a destination identifies the agent only and resolves to its one default experiment; the version is recorded on every trace and evaluation run as metadata. set_active_agent is defined as convenience over two separable pieces, with get_default_experiment_id exposed as a public agent-only lookup (partially adopting Ann Marie's two-call review suggestion), and deployment overrides swap only the destination piece. Co-Authored-By: Claude Fable 5 --- .../0011-agent-registry.md | 46 +++++++++++++++---- 1 file changed, 36 insertions(+), 10 deletions(-) diff --git a/rfcs/0011-agent-registry/0011-agent-registry.md b/rfcs/0011-agent-registry/0011-agent-registry.md index 3dc9376c..5c229c09 100644 --- a/rfcs/0011-agent-registry/0011-agent-registry.md +++ b/rfcs/0011-agent-registry/0011-agent-registry.md @@ -159,8 +159,13 @@ agent development loop. This RFC makes an agent a trace destination: a new `MlflowAgentLocation` joins the existing `MlflowExperimentLocation`, usable wherever MLflow accepts a trace destination today, and evaluation and trace-search APIs gain agent -identity alongside experiment identity. Traces and eval results -appear on the agent's registry page, filterable by version. The +identity alongside experiment identity. A destination identifies +the agent only and resolves to the agent's one default experiment; +the version is never part of the destination and is instead +recorded on every trace and evaluation run as metadata, which is +what per-version filtering and comparison use. Traces and eval +results appear on the agent's registry page, filterable by +version. The change is additive: under the hood an agent owns a default experiment, the agent location resolves to it, and experiment-based workflows continue unchanged. @@ -284,6 +289,12 @@ mlflow.genai.evaluate( ) ``` +`set_active_agent` does two separable things: it points the trace +destination at the agent's one default experiment (the version +plays no part in that), and it records the agent and version as +metadata on everything emitted while it is active. The trace +journey shows the underlying pieces. + # Motivation ## The problem @@ -555,14 +566,25 @@ organized by agent and version, not by experiment. with mlflow.start_span(name="answer-question"): result = agent.run(question) ``` - `set_active_agent` is convenience over MLflow's existing trace - destination mechanism: it wraps - `mlflow.tracing.set_destination(MlflowAgentLocation(...))`, the - way `set_experiment` establishes an experiment destination - today. The location can also be passed per span through - `start_span`'s existing `trace_destination` parameter. Framework - and harness autologgers respect the active destination, so - instrumented applications need only state which agent they are. + `set_active_agent` is convenience over two separable pieces. + First, it sets the trace destination to the agent's one default + experiment, equivalent to calling `mlflow.set_experiment` on the + result of + `mlflow.genai.get_default_experiment_id("acme/billing-agent")`, + where `get_default_experiment_id` is a public lookup that takes + only the agent: the version is never part of the destination. (An + `MlflowAgentLocation` naming the agent can likewise be passed + anywhere MLflow accepts a trace destination, including + `start_span`'s existing `trace_destination` parameter.) Second, + it records the agent and version as trace-level metadata, the way + session and user metadata are recorded today; this is what + per-version filtering and comparison use. A deployment that + overrides its destination (below) swaps only the first piece, + pointing `set_experiment` at its own experiment, and still + declares the agent and version so its traces stay labeled. + Framework and harness autologgers respect the active destination + and metadata, so instrumented applications need only state which + agent they are. 2. Run evaluations against the agent: ```python mlflow.genai.evaluate(data=eval_dataset, @@ -570,6 +592,10 @@ organized by agent and version, not by experiment. agent_id="acme/billing-agent", agent_version=3) ``` + As with tracing, `agent_id` determines where the results land + (the agent's default experiment, unless overridden) and + `agent_version` is recorded on the evaluation run for filtering + and comparison. 3. Open the agent's detail page. A Traces tab shows the agent's traces, filterable by version; an Evaluations tab shows eval runs; a summary card shows latest eval score and trace volume. From b2b9a388920e697853122c35c6d45b9291ccbce3 Mon Sep 17 00:00:00 2001 From: Bill Murdock Date: Thu, 27 Aug 2026 13:13:21 -0400 Subject: [PATCH 17/34] Conciseness pass Cut redundancy that accreted across incremental revisions: rules stated in multiple places (server-never-fetches, card-not-persisted, default-experiment mechanics) now appear once where they matter most, the blast-radius rationale is consolidated, and digressions are trimmed. No design content changed. Co-Authored-By: Claude Fable 5 --- .../0011-agent-registry.md | 276 ++++++++---------- 1 file changed, 118 insertions(+), 158 deletions(-) diff --git a/rfcs/0011-agent-registry/0011-agent-registry.md b/rfcs/0011-agent-registry/0011-agent-registry.md index 5c229c09..38aa5efa 100644 --- a/rfcs/0011-agent-registry/0011-agent-registry.md +++ b/rfcs/0011-agent-registry/0011-agent-registry.md @@ -26,11 +26,10 @@ # Summary -Add an Agent Registry to MLflow: a governed, record-level registry for -AI agents. The registry catalogs the agents an organization has built. -It records each agent's identity, ownership, composition, source -provenance, and lifecycle status, and it anchors the agent's traces -and evaluation results. It answers "what agents exist, who owns them, +Add an Agent Registry to MLflow: a governed, record-level registry +that catalogs each agent's identity, ownership, composition, source +provenance, and lifecycle status, and anchors the agent's traces and +evaluation results. It answers "what agents exist, who owns them, what are they made of, and how well do they work?" The Agent Registry is the third registry in a series, following the @@ -43,15 +42,12 @@ It completes a progression: MLflow can govern the tools an agent calls invokes (Model Registry), but the agent itself, the thing that acts, has no registry entry. This RFC adds one. -**The registry is record-level, not runtime-aware.** It stores what an -agent is: who built it, what components it is made of, where its -source lives, and what lifecycle state it is in. It does not store -whether the agent is running, healthy, or scaled. Runtime state -belongs to the serving platform. A platform that deploys registered -agents can join its own runtime inventory against registry records at -query time; the registry supplies the governance half of that join. -This is the same division of responsibility RFC-0004 draws between -the MCP registry and a gateway. +**The registry is record-level, not runtime-aware.** It stores what +an agent is, not whether it is running, healthy, or scaled. Runtime +state belongs to the serving platform, which can join its own +runtime inventory against registry records at query time. This is +the same division of responsibility RFC-0004 draws between the MCP +registry and a gateway. The registry manages two primary entities under the `mlflow.genai` SDK namespace, following the pattern of RFC-0004 and RFC-0008: @@ -80,11 +76,10 @@ Version identity is a per-agent choice made when the agent is created: `monotonic` (registry-assigned serial numbers: 1, 2, 3), `semver` (registrant-supplied semantic versions), or `freeform` (registrant-supplied opaque strings). `monotonic` is the default -when registration supplies no version, so the simple path stays as -simple as the Skill Registry's: registrants who never think about -versioning get serial numbers automatically, while an agent that -already carries its own versioning (a team's release train, a -provider-versioned A2A agent) can keep it. The scheme may also be +when registration supplies no version: registrants who never think +about versioning get serial numbers automatically, while an agent +that already carries its own versioning (a provider-versioned A2A +agent, for example) can keep it. The scheme may also be autodetected from the first registration's input. This refines the policy the earlier registries establish, where an entry adopts the underlying artifact's version when its format defines one @@ -101,9 +96,9 @@ BOM entries are soft references, structured values rather than foreign keys. They resolve against the Skill Registry (which RFC-0008 defines for both skills and agent plugins), MCP Server Registry, and Model Registry when matching entries exist, and they -remain valid when they do not. This is what makes cross-registry -questions ("which agents use skill X?") answerable as registry -queries while keeping registration decoupled from registration order. +remain valid when they do not. This makes cross-registry questions +("which agents use skill X?") answerable as registry queries +without constraining registration order. **The BOM is a component inventory, not a complete recipe.** Its structured axes exist because corresponding registries or identifier @@ -114,16 +109,15 @@ share the job of describing an agent. Structured BOM references are selective but queryable across agents. Definitional anchors (a source pointer, a configuration snapshot) are complete but opaque: they capture everything about one agent without supporting cross-agent -queries. Free-form tags are the catch-all for facts that fit neither, -and there will always be some, because agents acquire new kinds of -parts faster than any schema anticipates. New structured axes are -expected as the governance surface grows. +queries. Free-form tags are the catch-all for facts that fit +neither. New structured axes are expected as the governance surface +grows. **A2A Agent Cards are fetched, not stored.** The [A2A protocol](https://a2a-protocol.org/) makes an agent's endpoint -the authoritative home of its Agent Card: the live card is always -one GET away at the endpoint's well-known path, and every A2A -client reads it from there. The registry follows suit. Registering +the authoritative home of its Agent Card: every A2A client reads +the live card from the endpoint's well-known path, and the registry +follows suit. Registering from an endpoint imports the card's descriptive metadata (description, capabilities, and its free-form name, which seeds the mutable MLflow-managed `display_name`) into ordinary registry @@ -164,11 +158,9 @@ the agent only and resolves to the agent's one default experiment; the version is never part of the destination and is instead recorded on every trace and evaluation run as metadata, which is what per-version filtering and comparison use. Traces and eval -results appear on the agent's registry page, filterable by -version. The -change is additive: under the hood an agent owns a default -experiment, the agent location resolves to it, and experiment-based -workflows continue unchanged. +results appear on the agent's registry page, filterable by version. +The change is additive: the default experiment exists under the +hood, and experiment-based workflows continue unchanged. **Relationship to other RFCs.** RFC-0004 establishes the access binding pattern this RFC reuses (its canonical-payload pattern is @@ -239,15 +231,12 @@ binding for the endpoint. The card content is not persisted: the endpoint is the card's system of record, and the UI renders the card read-only by fetching it through the binding at view time. The registry `name` is always chosen by the registrant, since a card's -`name` is a display string, not an identity. The card is whatever -the running agent serves; it need not exist as a static file -anywhere. Fetches always happen in the client or the browser: the -registry server never fetches user-supplied URLs, consistent with -RFC-0008. A caller that wants to inspect or adjust the imported -metadata before registering can fetch the card itself and pass it -via `a2a_card=` instead. A version registered this way, with no -source and no configuration snapshot, is an interface-only record -(see the register journey). +`name` is a display string, not an identity. Fetches happen in the +client, never in the registry server, consistent with RFC-0008; a +caller can also fetch the card itself and pass it via `a2a_card=` +to inspect the imported metadata first. A version registered this +way, with no source and no configuration snapshot, is an +interface-only record (see the register journey). ## Register a harness-based agent @@ -264,14 +253,13 @@ mlflow.genai.register_agent( ) ``` -There is no user source repository: the agent is the harness plus its -configuration, so the configuration snapshot serves as the +There is no user source repository: the agent is the harness plus +its configuration, so the configuration snapshot serves as the definitional anchor. `config_snapshot` names a file or directory on -the caller's local disk; the client reads it and uploads the content -to MLflow artifact storage as an immutable artifact on the version. -The registry server never fetches user-supplied locations itself, -consistent with RFC-0008, which also means any redaction of secrets -must happen client-side before upload. +the caller's local disk; the client reads it and uploads the +content to MLflow artifact storage as an immutable artifact on the +version, which means any redaction of secrets must happen +client-side before upload. ## Trace and evaluate against the agent @@ -387,15 +375,11 @@ the record. descriptive and capability fields from the card (its free-form name seeds the mutable `display_name`; the registry `name` is supplied by the registrant) and creates an `a2a` access binding - for the endpoint. The card content is not persisted: the agent - detail page renders the card read-only by fetching it through - the binding, so the display never drifts from what the agent - serves. In the SDK and CLI, the client fetches the card at - import; in the UI, the browser fetches it when the endpoint - permits, or the user pastes it, since the registry server never - fetches user-supplied URLs (consistent with RFC-0008). The BOM - is supplied alongside, since the card schema does not carry - component version pins. + for the endpoint. In the SDK and CLI, the client fetches the + card at import; in the UI, the browser fetches it when the + endpoint permits, or the user pastes it, since the server never + fetches user-supplied URLs. The BOM is supplied alongside, since + the card schema does not carry component version pins. 5. **CI path:** the same call runs from a pipeline, registering a new version on each release build with the source ref set to the build's commit. @@ -421,51 +405,42 @@ the record. The configuration snapshot is read from the caller's local disk and stored as an immutable artifact on the version. It is what distinguishes this agent from every other installation of the - same harness: enabled tools, overridden defaults (for example, a - non-standard shell for the harness's shell tool), and behavioral + same harness: enabled tools, overridden defaults, and behavioral settings live there and nowhere else. - A harness's configuration surface is not always a single file: - Claude Code, for example, spreads behavioral configuration across - a settings file, instruction files, and subagent definitions. - `config_snapshot` therefore accepts a file or a directory. What it - should capture is the configuration the registry does not - otherwise represent; content that BOM references already govern, - such as installed skill directories or MCP server definitions, - should stay out of the snapshot, because an embedded copy is - invisible to cross-registry queries and can silently drift from - the declared references. + A harness's configuration surface is not always a single file + (Claude Code, for example, spreads it across a settings file, + instruction files, and subagent definitions), so + `config_snapshot` accepts a file or a directory. It should + capture only configuration the registry does not otherwise + represent: content that BOM references already govern, such as + installed skill directories or MCP server definitions, stays + out, because an embedded copy is invisible to cross-registry + queries and can drift from the declared references. Composition is required wherever it is knowable, because the BOM is the value: a registry record without composition is just a name in a list. A registrant anchoring on source or a configuration -snapshot has the composition in front of them, so the requirement -holds there. An interface-only registration (an A2A endpoint with -no source or configuration snapshot) is different: the registrant -of a vendor or partner agent cannot know which skills, servers, and -models are inside a black box, and forcing a declaration would -invite invented BOMs that pollute cross-registry queries. For -interface-only records the BOM may therefore be empty or partial, -an absent BOM is recorded as *undeclared* composition, not as an -empty dependency list, and the record itself is marked as holding -no definitional anchor: the registry knows the agent's claim -surface, not its contents. Everything else is -progressive enrichment. The endpoint URL is optional; the -trace-and-eval journey below explains which agents need one. When -provided, the URL becomes an access binding rather than a field on -the immutable version. +snapshot has the composition in front of them. The registrant of a +black-box vendor or partner agent does not, and forcing a +declaration would invite invented BOMs that pollute cross-registry +queries; for interface-only records the BOM may therefore be empty +or partial, an absent BOM is recorded as *undeclared* composition +rather than an empty dependency list, and the record is marked as +holding no definitional anchor: the registry knows the agent's +claim surface, not its contents. Everything else is progressive +enrichment. The endpoint is optional; the trace-and-eval journey +below explains which agents need one. The harness path is the newest part of this design and the least settled (see [Open questions](#open-questions)). For framework-built and custom agents, the registered source is the natural complete -record and remains the expected anchor. For harness-based agents, -requiring -source would force registrations that point at the harness vendor's -repository, which identifies nothing about the specific agent, while -the configuration that actually defines it went unrecorded. -Configuration files frequently embed secrets and environment-specific -values, so the snapshot mechanism needs redaction guidance at -minimum. +record and remains the expected anchor; for harness-based agents, +requiring source would force registrations that point at the +harness vendor's repository, which identifies nothing about the +specific agent. Configuration files frequently embed secrets and +environment-specific values, so the snapshot mechanism needs +redaction guidance at minimum. ### Publish and maintain an agent's endpoint @@ -500,17 +475,15 @@ disturbing the immutable version history. agent, its versions, and its history remain untouched. The protocol field is where agent bindings depart from RFC-0004, -whose bindings are always MCP and vary only by transport. Agent -endpoints speak different protocols, and the field is limited to -values that tell a caller something actionable: `a2a` and `mcp` are -self-describing (an Agent Card at the well-known path; the MCP -handshake), so URL plus protocol is enough to connect. Labels like -REST or gRPC name a transport without telling anyone how to call -the agent, so they are deliberately collapsed into `other`, which -records where an agent lives without claiming MLflow can invoke it. -This keeps the registry open to custom protocols without pretending -to understand them. As in RFC-0004, a binding that targets an alias -such as `production` follows the alias as it moves between +whose bindings are always MCP and vary only by transport. The field +is limited to values that tell a caller something actionable: `a2a` +and `mcp` are self-describing (an Agent Card at the well-known +path; the MCP handshake), so URL plus protocol is enough to +connect. Labels like REST or gRPC name a transport without telling +anyone how to call the agent, so they are deliberately collapsed +into `other`, which records where an agent lives without claiming +MLflow can invoke it. As in RFC-0004, a binding that targets an +alias such as `production` follows the alias as it moves between versions. ### Version an agent and compare bills of materials @@ -573,9 +546,8 @@ organized by agent and version, not by experiment. `mlflow.genai.get_default_experiment_id("acme/billing-agent")`, where `get_default_experiment_id` is a public lookup that takes only the agent: the version is never part of the destination. (An - `MlflowAgentLocation` naming the agent can likewise be passed - anywhere MLflow accepts a trace destination, including - `start_span`'s existing `trace_destination` parameter.) Second, + `MlflowAgentLocation` naming the agent works anywhere MLflow + accepts a trace destination.) Second, it records the agent and version as trace-level metadata, the way session and user metadata are recorded today; this is what per-version filtering and comparison use. A deployment that @@ -604,11 +576,10 @@ organized by agent and version, not by experiment. the cause, cross-referencing the BOM diff from the previous journey. -Backward compatibility is preserved by construction: an agent owns a -default experiment under the hood, and the agent location resolves -to that experiment's ID. Where MLflow already accepts a typed -destination or location (trace destinations, `search_traces` -locations), agent identity becomes a new accepted value; where it +Backward compatibility is preserved by construction. Where MLflow +already accepts a typed destination or location (trace +destinations, `search_traces` locations), agent identity becomes a +new accepted value; where it does not (`evaluate`), `agent_id` is new, optional surface. Existing experiment-based workflows (including model training and fine-tuning) continue unchanged. The change is additive, not a data @@ -626,10 +597,10 @@ other deployments of the same agent (a different owner or user base) overrides the destination in its own configuration; because permissions are experiment-scoped, separation is done with destinations, not trace tags. Versions share the agent's experiment -for the same reason seen from the other side: a version is an -analysis dimension recorded on every trace, not an access boundary, -and per-version experiments would break the longitudinal view of an -agent's behavior across upgrades. To keep such traces findable from the +for the same reason: a version is an analysis dimension recorded on +every trace, not an access boundary, and per-version experiments +would break the longitudinal view of an agent's behavior across +upgrades. So that overridden deployments stay findable from the agent's registry page, this RFC proposes that a deployment using a non-default experiment notify the registry, which records the experiment ID on that deployment's access binding. A deployment @@ -640,9 +611,8 @@ belongs to the deferred registry synchronization glue. No endpoint is needed for any of this when the developer has the agent's code: the agent runs locally or in CI, autologging captures -traces during execution, and evaluation scores outputs against a test -dataset. This is why code-only registration supports the full -trace-and-eval workflow. The exception is agents whose code the user +traces during execution, and evaluation scores outputs against a +test dataset. The exception is agents whose code the user cannot run: another team's A2A agent, a vendor agent, a partner service. For those, the endpoint is the only execution surface, and tracing and evaluation work by invoking the agent's access binding @@ -684,8 +654,8 @@ auditable lifecycle state. This is the same core `draft` → `active` → `deprecated` lifecycle the MCP and Skill registries use (their soft-delete `deleted` state and transition rules carry over as well), applied to the agent -itself. The -auditable transition history is the accountability chain that problem +itself. The auditable transition history is the accountability +chain that problem 1 identifies as missing: for any agent, the registry can say who promoted it, when, and what its evaluation evidence looked like at the time. @@ -718,35 +688,25 @@ every affected agent without inspecting deployments one by one. versions with the skill removed or upgraded, and deprecate the affected versions (previous journey). -The same query works for the other BOM axes: "which agents use MCP -server Y whose tool schema changed?" and "which agents call model Z -being retired?" Source entries are queryable the same way: "which -agents ship OCI image X?" is the container-CVE variant of this -journey. Agent plugin references expand through their -members: RFC-0008 plugin versions immutably record which registered -skills they contain, so "which agents use skill X" also finds -agents that consume X through a plugin, rather than leaving a -blind spot behind composed units. Because BOM entries are -structured soft references, -the query is a registry lookup with exact-match semantics rather than -a scan of running infrastructure. Because a BOM holds many refs per -axis, the name and version predicates must bind to the same BOM -entry; matching a name on one skill ref and a version on another -would produce false positives. That same-entry matching requirement -is a store-level obligation for the detailed design, in the way -exact matching on span attributes is for RFC-0009. The registry -answers with consumers and owners; automated notification of those -owners is deliberately not in the MVP (see Out of scope). - -Coverage has two limits a reader of the results should keep in mind: -the query sees only the component types the registry tracks, and -within those, only what registrants declared. Agents with -undeclared composition (black-box, interface-only registrations) -can never match, so results should surface them alongside matches: -"3 -agents declare the compromised skill; 12 more have undeclared -composition." The query replaces infrastructure inspection for the -governed axes; it does not claim completeness beyond them. +The same query works for the other BOM axes ("which agents use MCP +server Y whose tool schema changed?", "which agents call model Z +being retired?") and for source entries: "which agents ship OCI +image X?" is the container-CVE variant. Agent plugin references +expand through their members: RFC-0008 plugin versions immutably +record which registered skills they contain, so the query also +finds agents that consume a skill through a plugin. The query has +exact-match semantics, and the name and version predicates must +bind to the same BOM entry (a store-level obligation for the +detailed design, like exact span-attribute matching in RFC-0009). +The registry answers with consumers and owners; automated +notification of those owners is deliberately not in the MVP (see +Out of scope). + +Coverage has two limits: the query sees only the component types +the registry tracks, and within those, only what registrants +declared. Agents with undeclared composition can never match, so +results should surface them alongside matches: "3 agents declare +the compromised skill; 12 more have undeclared composition." ## Out of scope @@ -773,8 +733,8 @@ rejection. - **Auto-discovery of composition.** BOMs are developer-asserted in the MVP. Inferring actual composition from traces (for example, from RFC-0009 `SKILL` spans) and notifying owners when assertion - and observation disagree is a compelling follow-on, and the span - data this RFC and RFC-0009 produce is designed to enable it. + and observation disagree is a follow-on that the span data from + this RFC and RFC-0009 is designed to enable. - **Detection of unregistered agents.** Surfacing "shadow" agents running without registry entries requires runtime scanning, which is platform work built on top of this registry. @@ -904,8 +864,8 @@ TBD. skill pattern applied to agents would produce irreconcilable duplicate versions of what is really one agent. The list is currently an unordered set of asserted pointers. Reviewers who - weigh cross-registry consistency - heavily should push back here if the divergence is not worth it. + weigh cross-registry consistency heavily should push back here if + the divergence is not worth it. - **What is the BOM reference format?** The journeys sketch URI-style references (`skills:/billing-policy/1`, @@ -915,9 +875,9 @@ TBD. scheme adopts the `mcp-servers:/` proposal from RFC-0010; the refs here respect RFC-0004's reverse-DNS server names and semantic versions. Bare identifiers cover external models (`gpt-4o`). BOM - references are soft (string-resolved, valid when - the target is unregistered), which URI syntax may misleadingly - suggest otherwise. Alternatives include structured + references are soft (string-resolved, valid when the target is + unregistered), which URI syntax may misleadingly suggest + otherwise. Alternatives include structured `{registry, name, version}` objects. Relatedly: should the BOM also be able to reference other agents, to represent multi-agent systems? From f464589ae41abc8f6e6cfb7710d7f693a5e47787 Mon Sep 17 00:00:00 2001 From: Bill Murdock Date: Thu, 27 Aug 2026 13:34:33 -0400 Subject: [PATCH 18/34] Move design positions out of the Summary The Summary is now a summary: opening, lineage, a one-paragraph tour of the positions, and RFC relationships. The full position statements move intact to a Design positions subsection under Detailed design, where they seed the eventual full design. No content changed beyond the pointer sentences. Co-Authored-By: Claude Fable 5 --- .../0011-agent-registry.md | 260 ++++++++++-------- 1 file changed, 139 insertions(+), 121 deletions(-) diff --git a/rfcs/0011-agent-registry/0011-agent-registry.md b/rfcs/0011-agent-registry/0011-agent-registry.md index 38aa5efa..6411b5aa 100644 --- a/rfcs/0011-agent-registry/0011-agent-registry.md +++ b/rfcs/0011-agent-registry/0011-agent-registry.md @@ -19,6 +19,7 @@ - [User journeys](#user-journeys) - [Out of scope](#out-of-scope) - [Detailed design](#detailed-design) + - [Design positions](#design-positions) - [Drawbacks](#drawbacks) - [Alternatives](#alternatives) - [Adoption strategy](#adoption-strategy) @@ -42,129 +43,22 @@ It completes a progression: MLflow can govern the tools an agent calls invokes (Model Registry), but the agent itself, the thing that acts, has no registry entry. This RFC adds one. -**The registry is record-level, not runtime-aware.** It stores what -an agent is, not whether it is running, healthy, or scaled. Runtime -state belongs to the serving platform, which can join its own -runtime inventory against registry records at query time. This is -the same division of responsibility RFC-0004 draws between the MCP -registry and a gateway. - -The registry manages two primary entities under the `mlflow.genai` -SDK namespace, following the pattern of RFC-0004 and RFC-0008: - -- **Agent**: a named, owned entity with DNS-style naming - (`org/agent-name`), in the spirit of the namespaced names the MCP - and Skill registries use; exact alignment with RFC-0008's - `{workspace, organization, name}` coordinates is a detailed-design - point. -- **AgentVersion**: an immutable snapshot of the agent's composition, - its **bill of materials (BOM)**: skill references, agent plugin - references (a plugin is referenced as a composed unit and expands - through its registered members for queries), MCP server - references, model references (registry models or external model - identifiers such as `gpt-4o`), and, for agents that run as - configurations of a packaged harness, a harness reference (a - proposed axis; see [Open questions](#open-questions)). Each - version also carries at least one **definitional anchor**: source - provenance (one or more typed source pointers as in RFC-0008: a - Git repo and ref, an OCI image, an archive) and/or an immutable - configuration snapshot. An agent registered from an A2A endpoint - alone is an interface-only record with no anchor (see below). - Each change to composition is a new version. - -Version identity is a per-agent choice made when the agent is -created: `monotonic` (registry-assigned serial numbers: 1, 2, 3), -`semver` (registrant-supplied semantic versions), or `freeform` -(registrant-supplied opaque strings). `monotonic` is the default -when registration supplies no version: registrants who never think -about versioning get serial numbers automatically, while an agent -that already carries its own versioning (a provider-versioned A2A -agent, for example) can keep it. The scheme may also be -autodetected from the first registration's input. This refines the -policy the earlier registries establish, where an entry adopts the -underlying artifact's version when its format defines one -(RFC-0004's `server_json`, RFC-0008's plugin manifests) and mints -serial numbers when it does not (RFC-0008's skills): no standard -agent artifact defines an inherent agent version, so minting is the -default rather than the rule. An A2A card's provider-defined -`version` string lives on the live card rather than in the registry; -an agent registered under `semver` or `freeform` may mirror it. -Aliases and `latest` resolution are well defined for `monotonic` -and `semver`; `freeform` versions order by registration time. - -BOM entries are soft references, structured values rather than -foreign keys. They resolve against the Skill Registry (which -RFC-0008 defines for both skills and agent plugins), MCP Server -Registry, and Model Registry when matching entries exist, and they -remain valid when they do not. This makes cross-registry questions -("which agents use skill X?") answerable as registry queries -without constraining registration order. - -**The BOM is a component inventory, not a complete recipe.** Its -structured axes exist because corresponding registries or identifier -conventions exist, so it is bounded by MLflow's governance surface -rather than by agent anatomy: an agent's prompt strategy or memory -configuration has no axis because nothing governs one. Three layers -share the job of describing an agent. Structured BOM references are -selective but queryable across agents. Definitional anchors (a source -pointer, a configuration snapshot) are complete but opaque: they -capture everything about one agent without supporting cross-agent -queries. Free-form tags are the catch-all for facts that fit -neither. New structured axes are expected as the governance surface -grows. - -**A2A Agent Cards are fetched, not stored.** The -[A2A protocol](https://a2a-protocol.org/) makes an agent's endpoint -the authoritative home of its Agent Card: every A2A client reads -the live card from the endpoint's well-known path, and the registry -follows suit. Registering -from an endpoint imports the card's descriptive metadata -(description, capabilities, and its free-form name, which seeds the -mutable MLflow-managed `display_name`) into ordinary registry -fields and creates an `a2a` access binding; the card content itself -is not persisted. The UI renders the card read-only by fetching it -through the binding at view time, so what MLflow displays can never -drift from what the agent serves. This deliberately departs from -the canonical-payload pattern of RFC-0004 (`server_json`) and -RFC-0008 (`plugin.json`): MLflow is the system of record for those -payloads, while an Agent Card's system of record is the agent -itself. An agent registered from an endpoint alone, with no source -and no configuration snapshot, is an **interface-only record**: the -registry captures the claim surface (identity, imported metadata, -endpoint) and marks that it holds no definitional anchor. - -**Endpoints are access bindings, not version fields.** Some agents -are reachable at a URL (A2A agents inherently; deployed agents -generally), and recording that URL lets the registry drive tracing -and evaluation for agents whose code the user cannot run. But -endpoints change independently of composition, and agent versions are -immutable. Following RFC-0004's `MCPAccessBinding` model, approved -endpoints are separate mutable binding records that target a version -or alias, created and deleted as connectivity changes without -touching version history. Where an MCP binding's protocol is always -MCP, an agent binding declares its protocol: `a2a`, `mcp` (for -agents exposed as MCP servers), or `other`. Registration accepts an -optional endpoint as a convenience that creates a binding. - -**Agents become the primary anchor for GenAI traces and -evaluations.** Today traces and evaluation runs attach to -experiments, an abstraction that fits model training but not the -agent development loop. This RFC makes an agent a trace -destination: a new `MlflowAgentLocation` joins the existing -`MlflowExperimentLocation`, usable wherever MLflow accepts a trace -destination today, and evaluation and trace-search APIs gain agent -identity alongside experiment identity. A destination identifies -the agent only and resolves to the agent's one default experiment; -the version is never part of the destination and is instead -recorded on every trace and evaluation run as metadata, which is -what per-version filtering and comparison use. Traces and eval -results appear on the agent's registry page, filterable by version. -The change is additive: the default experiment exists under the -hood, and experiment-based workflows continue unchanged. +In brief, the design takes these positions, each stated in full in +[Design positions](#design-positions): the registry is +record-level, not runtime-aware; an agent's versions are immutable +snapshots of its composition (a bill of materials of skills, agent +plugins, MCP servers, and models) plus at least one definitional +anchor (source pointers and/or a configuration snapshot); A2A Agent +Cards are fetched from the agent's endpoint, never stored; +endpoints are mutable, protocol-typed access bindings rather than +version fields; and an agent is a trace destination with one +default experiment, with the version recorded on every trace and +evaluation run. **Relationship to other RFCs.** RFC-0004 establishes the access binding pattern this RFC reuses (its canonical-payload pattern is -deliberately not applied to Agent Cards; see above). RFC-0008 defines +deliberately not applied to Agent Cards; see +[Design positions](#design-positions)). RFC-0008 defines the skills and agent plugins that agent BOMs reference; note that an *agent plugin* (a package of components installed into a harness) is not an *agent* (an application that acts); this registry governs the @@ -755,7 +649,131 @@ rejection. # Detailed design -TBD. +## Design positions + +The following positions are settled enough to draft against; the +remainder of the detailed design is TBD. + +**The registry is record-level, not runtime-aware.** It stores what +an agent is, not whether it is running, healthy, or scaled. Runtime +state belongs to the serving platform, which can join its own +runtime inventory against registry records at query time. This is +the same division of responsibility RFC-0004 draws between the MCP +registry and a gateway. + +The registry manages two primary entities under the `mlflow.genai` +SDK namespace, following the pattern of RFC-0004 and RFC-0008: + +- **Agent**: a named, owned entity with DNS-style naming + (`org/agent-name`), in the spirit of the namespaced names the MCP + and Skill registries use; exact alignment with RFC-0008's + `{workspace, organization, name}` coordinates is a detailed-design + point. +- **AgentVersion**: an immutable snapshot of the agent's composition, + its **bill of materials (BOM)**: skill references, agent plugin + references (a plugin is referenced as a composed unit and expands + through its registered members for queries), MCP server + references, model references (registry models or external model + identifiers such as `gpt-4o`), and, for agents that run as + configurations of a packaged harness, a harness reference (a + proposed axis; see [Open questions](#open-questions)). Each + version also carries at least one **definitional anchor**: source + provenance (one or more typed source pointers as in RFC-0008: a + Git repo and ref, an OCI image, an archive) and/or an immutable + configuration snapshot. An agent registered from an A2A endpoint + alone is an interface-only record with no anchor (see below). + Each change to composition is a new version. + +Version identity is a per-agent choice made when the agent is +created: `monotonic` (registry-assigned serial numbers: 1, 2, 3), +`semver` (registrant-supplied semantic versions), or `freeform` +(registrant-supplied opaque strings). `monotonic` is the default +when registration supplies no version: registrants who never think +about versioning get serial numbers automatically, while an agent +that already carries its own versioning (a provider-versioned A2A +agent, for example) can keep it. The scheme may also be +autodetected from the first registration's input. This refines the +policy the earlier registries establish, where an entry adopts the +underlying artifact's version when its format defines one +(RFC-0004's `server_json`, RFC-0008's plugin manifests) and mints +serial numbers when it does not (RFC-0008's skills): no standard +agent artifact defines an inherent agent version, so minting is the +default rather than the rule. An A2A card's provider-defined +`version` string lives on the live card rather than in the registry; +an agent registered under `semver` or `freeform` may mirror it. +Aliases and `latest` resolution are well defined for `monotonic` +and `semver`; `freeform` versions order by registration time. + +BOM entries are soft references, structured values rather than +foreign keys. They resolve against the Skill Registry (which +RFC-0008 defines for both skills and agent plugins), MCP Server +Registry, and Model Registry when matching entries exist, and they +remain valid when they do not. This makes cross-registry questions +("which agents use skill X?") answerable as registry queries +without constraining registration order. + +**The BOM is a component inventory, not a complete recipe.** Its +structured axes exist because corresponding registries or identifier +conventions exist, so it is bounded by MLflow's governance surface +rather than by agent anatomy: an agent's prompt strategy or memory +configuration has no axis because nothing governs one. Three layers +share the job of describing an agent. Structured BOM references are +selective but queryable across agents. Definitional anchors (a source +pointer, a configuration snapshot) are complete but opaque: they +capture everything about one agent without supporting cross-agent +queries. Free-form tags are the catch-all for facts that fit +neither. New structured axes are expected as the governance surface +grows. + +**A2A Agent Cards are fetched, not stored.** The +[A2A protocol](https://a2a-protocol.org/) makes an agent's endpoint +the authoritative home of its Agent Card: every A2A client reads +the live card from the endpoint's well-known path, and the registry +follows suit. Registering from an endpoint imports the card's +descriptive metadata (description, capabilities, and its free-form +name, which seeds the mutable MLflow-managed `display_name`) into +ordinary registry fields and creates an `a2a` access binding; the +card content itself is not persisted. The UI renders the card +read-only by fetching it through the binding at view time, so what +MLflow displays can never drift from what the agent serves. This +deliberately departs from the canonical-payload pattern of RFC-0004 +(`server_json`) and RFC-0008 (`plugin.json`): MLflow is the system +of record for those payloads, while an Agent Card's system of +record is the agent itself. An agent registered from an endpoint +alone, with no source and no configuration snapshot, is an +**interface-only record**: the registry captures the claim surface +(identity, imported metadata, endpoint) and marks that it holds no +definitional anchor. + +**Endpoints are access bindings, not version fields.** Some agents +are reachable at a URL (A2A agents inherently; deployed agents +generally), and recording that URL lets the registry drive tracing +and evaluation for agents whose code the user cannot run. But +endpoints change independently of composition, and agent versions +are immutable. Following RFC-0004's `MCPAccessBinding` model, +approved endpoints are separate mutable binding records that target +a version or alias, created and deleted as connectivity changes +without touching version history. Where an MCP binding's protocol +is always MCP, an agent binding declares its protocol: `a2a`, `mcp` +(for agents exposed as MCP servers), or `other`. Registration +accepts an optional endpoint as a convenience that creates a +binding. + +**Agents become the primary anchor for GenAI traces and +evaluations.** Today traces and evaluation runs attach to +experiments, an abstraction that fits model training but not the +agent development loop. This RFC makes an agent a trace +destination: a new `MlflowAgentLocation` joins the existing +`MlflowExperimentLocation`, usable wherever MLflow accepts a trace +destination today, and evaluation and trace-search APIs gain agent +identity alongside experiment identity. A destination identifies +the agent only and resolves to the agent's one default experiment; +the version is never part of the destination and is instead +recorded on every trace and evaluation run as metadata, which is +what per-version filtering and comparison use. Traces and eval +results appear on the agent's registry page, filterable by version. +The change is additive: the default experiment exists under the +hood, and experiment-based workflows continue unchanged. # Drawbacks From 4c85d1c721276960a04b46d2981b9f6121fc83a7 Mon Sep 17 00:00:00 2001 From: Bill Murdock Date: Wed, 2 Sep 2026 16:18:28 -0400 Subject: [PATCH 19/34] Scope plugin expansion to recorded member types Blast-radius expansion through plugin members covers what plugin versions record: skill members today, with more member types arriving via the extended agent plugins proposal. From review feedback. Numeric RFC cross-references avoided in new text pending the numbering-scheme clarification. Co-Authored-By: Claude Fable 5 --- rfcs/0011-agent-registry/0011-agent-registry.md | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/rfcs/0011-agent-registry/0011-agent-registry.md b/rfcs/0011-agent-registry/0011-agent-registry.md index 6411b5aa..535146b2 100644 --- a/rfcs/0011-agent-registry/0011-agent-registry.md +++ b/rfcs/0011-agent-registry/0011-agent-registry.md @@ -586,9 +586,12 @@ The same query works for the other BOM axes ("which agents use MCP server Y whose tool schema changed?", "which agents call model Z being retired?") and for source entries: "which agents ship OCI image X?" is the container-CVE variant. Agent plugin references -expand through their members: RFC-0008 plugin versions immutably -record which registered skills they contain, so the query also -finds agents that consume a skill through a plugin. The query has +expand through their members: plugin versions immutably record +which registered skills they contain, so the query also finds +agents that consume a skill through a plugin. Expansion covers the +member types plugin versions record: skill members today, with more +member types (MCP servers, subagents) extending the expansion as +the extended agent plugins proposal lands. The query has exact-match semantics, and the name and version predicates must bind to the same BOM entry (a store-level obligation for the detailed design, like exact span-attribute matching in RFC-0009). From 273dd75598d742f7684f8898599db6b964c3e5f1 Mon Sep 17 00:00:00 2001 From: Bill Murdock Date: Thu, 3 Sep 2026 10:28:51 -0400 Subject: [PATCH 20/34] Add agent references as a BOM axis and state the call convention Agents an agent calls are now a BOM axis: pinned to a version when the referencing team controls the callee's deployment (agents versioned and deployed together as one application), name-level when the callee is independently managed. Blast radius gains "which agents call the compromised agent?". The trace journey states the convention for calls: annotated at the call site with the callee's identity; in-process callees nest in the caller's trace, remote callees produce linked traces. Replaces the multi-agent open sub-question. From review feedback. Co-Authored-By: Claude Fable 5 --- .../0011-agent-registry.md | 29 +++++++++++++++---- 1 file changed, 23 insertions(+), 6 deletions(-) diff --git a/rfcs/0011-agent-registry/0011-agent-registry.md b/rfcs/0011-agent-registry/0011-agent-registry.md index 535146b2..272886f3 100644 --- a/rfcs/0011-agent-registry/0011-agent-registry.md +++ b/rfcs/0011-agent-registry/0011-agent-registry.md @@ -47,7 +47,8 @@ In brief, the design takes these positions, each stated in full in [Design positions](#design-positions): the registry is record-level, not runtime-aware; an agent's versions are immutable snapshots of its composition (a bill of materials of skills, agent -plugins, MCP servers, and models) plus at least one definitional +plugins, MCP servers, models, and other agents it calls) plus at +least one definitional anchor (source pointers and/or a configuration snapshot); A2A Agent Cards are fetched from the agent's endpoint, never stored; endpoints are mutable, protocol-typed access bindings rather than @@ -94,6 +95,7 @@ mlflow.genai.register_agent( ], skills=["skills:/billing-policy/1", "skills:/refund-rules/2"], agent_plugins=["agent-plugins:/billing-workflow/1.2.0"], + agents=["agents:/acme/records-agent/2"], mcp_servers=["mcp-servers:/acme.internal/payments-db/2.0.0"], models=["models:/acme-billing-llm/3", "gpt-4o"], ) @@ -525,6 +527,17 @@ component-level attribution within agent-level organization: from an agent's trace list, the `SKILL` spans inside a trace show which BOM entries were actually exercised. +Calls to other agents follow the same idea. Every call to another +agent is annotated at the call site with the callee's identity, so +the callee's registry page can find traces it participated in. +Where the callee's own work lands follows from where it runs: an +in-process callee (an agent used as a tool inside the caller's +process) nests as spans in the caller's trace, like nested skills; +a remote callee (delegation over A2A to a separately running agent) +produces its own trace in its own destination, linked to the +caller's trace through propagated trace context, using the span +links MLflow already supports for OpenTelemetry. + ### Manage an agent's lifecycle An agent owner or platform team needs agents to carry an explicit, @@ -585,7 +598,9 @@ every affected agent without inspecting deployments one by one. The same query works for the other BOM axes ("which agents use MCP server Y whose tool schema changed?", "which agents call model Z being retired?") and for source entries: "which agents ship OCI -image X?" is the container-CVE variant. Agent plugin references +image X?" is the container-CVE variant. Agent references close the +highest-impact case: "which agents call the compromised agent?" is +a query over the same axis. Agent plugin references expand through their members: plugin versions immutably record which registered skills they contain, so the query also finds agents that consume a skill through a plugin. Expansion covers the @@ -677,7 +692,11 @@ SDK namespace, following the pattern of RFC-0004 and RFC-0008: references (a plugin is referenced as a composed unit and expands through its registered members for queries), MCP server references, model references (registry models or external model - identifiers such as `gpt-4o`), and, for agents that run as + identifiers such as `gpt-4o`), references to other agents it calls + (pinned to a version when the referencing team controls the + callee's deployment, as with a set of agents versioned and + deployed together as one application, and name-level when the + callee is independently managed), and, for agents that run as configurations of a packaged harness, a harness reference (a proposed axis; see [Open questions](#open-questions)). Each version also carries at least one **definitional anchor**: source @@ -899,6 +918,4 @@ TBD. references are soft (string-resolved, valid when the target is unregistered), which URI syntax may misleadingly suggest otherwise. Alternatives include structured - `{registry, name, version}` objects. Relatedly: should the BOM - also be able to reference other agents, to represent multi-agent - systems? + `{registry, name, version}` objects. From 7ab79b10666c9446949dcce8d50b608b82009298 Mon Sep 17 00:00:00 2001 From: Bill Murdock Date: Thu, 3 Sep 2026 13:32:22 -0400 Subject: [PATCH 21/34] Link the MLflow feature-request issue Co-Authored-By: Claude Fable 5 --- rfcs/0011-agent-registry/0011-agent-registry.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rfcs/0011-agent-registry/0011-agent-registry.md b/rfcs/0011-agent-registry/0011-agent-registry.md index 272886f3..f37d8b93 100644 --- a/rfcs/0011-agent-registry/0011-agent-registry.md +++ b/rfcs/0011-agent-registry/0011-agent-registry.md @@ -2,7 +2,7 @@ | start_date | 2026-08-24 | | :----------- | :--------- | -| mlflow_issue | | +| mlflow_issue | https://github.com/mlflow/mlflow/issues/25572 | | rfc_pr | https://github.com/mlflow/rfcs/pull/39 | | Author(s) | [Bill Murdock](https://github.com/jwm4) (Red Hat) | From 20c0522f2c0cbe16329f0bdec4e1e8dcb351ed86 Mon Sep 17 00:00:00 2001 From: Bill Murdock Date: Thu, 3 Sep 2026 17:15:00 -0400 Subject: [PATCH 22/34] Reference harness configuration through source pointers Configuration snapshots are removed. A harness-based agent's configuration is now its source, referenced through any of the Skill Registry's source types (git, oci, zip, or mlflow artifact storage where the deployment serves artifacts) rather than uploaded by the registry. MLflow points at configuration instead of holding it, secrets custody stays where the configuration lives, and the harness open question loses two sub-questions. From review feedback. Co-Authored-By: Claude Fable 5 --- .../0011-agent-registry.md | 123 +++++++++--------- 1 file changed, 65 insertions(+), 58 deletions(-) diff --git a/rfcs/0011-agent-registry/0011-agent-registry.md b/rfcs/0011-agent-registry/0011-agent-registry.md index f37d8b93..0280d798 100644 --- a/rfcs/0011-agent-registry/0011-agent-registry.md +++ b/rfcs/0011-agent-registry/0011-agent-registry.md @@ -48,8 +48,8 @@ In brief, the design takes these positions, each stated in full in record-level, not runtime-aware; an agent's versions are immutable snapshots of its composition (a bill of materials of skills, agent plugins, MCP servers, models, and other agents it calls) plus at -least one definitional -anchor (source pointers and/or a configuration snapshot); A2A Agent +least one definitional anchor (typed source pointers, which for a +harness-based agent point at its configuration); A2A Agent Cards are fetched from the agent's endpoint, never stored; endpoints are mutable, protocol-typed access bindings rather than version fields; and an agent is a trace destination with one @@ -131,7 +131,7 @@ registry `name` is always chosen by the registrant, since a card's client, never in the registry server, consistent with RFC-0008; a caller can also fetch the card itself and pass it via `a2a_card=` to inspect the imported metadata first. A version registered this -way, with no source and no configuration snapshot, is an +way, with no source, is an interface-only record (see the register journey). ## Register a harness-based agent @@ -145,17 +145,22 @@ mlflow.genai.register_agent( skills=["skills:/runbook-triage/2"], mcp_servers=["mcp-servers:/acme.internal/pagerduty/1.2.0"], models=["claude-sonnet-5"], - config_snapshot="./opencode.json", + sources=[GitSource( + url="https://github.com/acme/oncall-config.git", + ref="c41d9e0", + )], ) ``` -There is no user source repository: the agent is the harness plus -its configuration, so the configuration snapshot serves as the -definitional anchor. `config_snapshot` names a file or directory on -the caller's local disk; the client reads it and uploads the -content to MLflow artifact storage as an immutable artifact on the -version, which means any redaction of secrets must happen -client-side before upload. +There is no agent code of the user's own: the agent is the harness +plus its configuration, so the source pointer points at the +configuration and serves as the definitional anchor. Any of the +Skill Registry's source types works: a Git repo, an OCI image, a +zip archive, or direct MLflow artifact storage (`mlflow`), the last +only in deployments where MLflow serves artifacts. Configuration +files often embed secrets, so the `mlflow` type carries the same +caveat it does for skill content: what is uploaded is what is +stored. ## Trace and evaluate against the agent @@ -249,7 +254,7 @@ the record. ) ``` Required: name, description, and at least one of: a definitional - anchor (source provenance or a configuration snapshot) or an A2A + anchor (one or more typed source pointers) or an A2A endpoint. These combine freely; a first-party A2A agent registers with both source and endpoint. Only a registration with an endpoint and no anchor produces an interface-only record (see @@ -282,10 +287,10 @@ the record. 6. **Harness path:** an agent that runs as a configuration of a packaged harness (Claude Code, OpenCode, Goose, and similar; see [RFC-0009](https://github.com/mlflow/rfcs/pull/37) for the - harness/framework distinction) typically has no source repository - of its own: the agent *is* the harness plus its configuration. It - registers with a harness reference in place of user source, and - attaches the configuration that defines it: + harness/framework distinction) has no agent code of its own: the + agent *is* the harness plus its configuration. It registers with + a harness reference and a source pointer to the configuration + that defines it: ```python mlflow.genai.register_agent( name="acme/oncall-helper", @@ -295,29 +300,35 @@ the record. skills=["skills:/runbook-triage/2"], mcp_servers=["mcp-servers:/acme.internal/pagerduty/1.2.0"], models=["claude-sonnet-5"], - config_snapshot="./opencode.json", + sources=[GitSource( + url="https://github.com/acme/oncall-config.git", + ref="c41d9e0", + )], ) ``` - The configuration snapshot is read from the caller's local disk - and stored as an immutable artifact on the version. It is what - distinguishes this agent from every other installation of the - same harness: enabled tools, overridden defaults, and behavioral - settings live there and nowhere else. + The configuration is what distinguishes this agent from every + other installation of the same harness: enabled tools, overridden + defaults, and behavioral settings live there and nowhere else. It + is referenced, not stored. Any of the Skill Registry's source + types serves (Git, OCI, zip, or direct MLflow artifact storage + where the deployment serves artifacts), so the registry treats + configuration the way it treats skill content, pointing at where + it belongs rather than becoming its home. A harness's configuration surface is not always a single file (Claude Code, for example, spreads it across a settings file, - instruction files, and subagent definitions), so - `config_snapshot` accepts a file or a directory. It should - capture only configuration the registry does not otherwise - represent: content that BOM references already govern, such as - installed skill directories or MCP server definitions, stays - out, because an embedded copy is invisible to cross-registry - queries and can drift from the declared references. + instruction files, and subagent definitions), which every source + type accommodates as a tree. The configuration source should hold + only configuration the registry does not otherwise represent: + content that BOM references already govern, such as installed + skill directories or MCP server definitions, stays out, because + an embedded copy is invisible to cross-registry queries and can + drift from the declared references. Composition is required wherever it is knowable, because the BOM is the value: a registry record without composition is just a name in -a list. A registrant anchoring on source or a configuration -snapshot has the composition in front of them. The registrant of a +a list. A registrant anchoring on a source has the composition in +front of them. The registrant of a black-box vendor or partner agent does not, and forcing a declaration would invite invented BOMs that pollute cross-registry queries; for interface-only records the BOM may therefore be empty @@ -335,8 +346,9 @@ record and remains the expected anchor; for harness-based agents, requiring source would force registrations that point at the harness vendor's repository, which identifies nothing about the specific agent. Configuration files frequently embed secrets and -environment-specific values, so the snapshot mechanism needs -redaction guidance at minimum. +environment-specific values; pointing at configuration rather than +storing it keeps that custody outside MLflow, and the `mlflow` +source type carries the same caveat it does for skills. ### Publish and maintain an agent's endpoint @@ -398,9 +410,8 @@ versions to see exactly what differs. on the Versions tab, and choose Compare. 4. The comparison shows a side-by-side BOM diff: for example, `billing-policy` skill `1` → `2`, model `llama-3.1-70b` → - `llama-3.1-405b`, one MCP server added. When both versions carry - configuration snapshots, those diff as content: for example, v4 - switched the harness's shell configuration. + `llama-3.1-405b`, one MCP server added, or the configuration + source moved from one ref to another. 5. Changed components link to their Skill Registry and MCP Server Registry entries, where their own version histories and changelogs live. @@ -700,9 +711,11 @@ SDK namespace, following the pattern of RFC-0004 and RFC-0008: configurations of a packaged harness, a harness reference (a proposed axis; see [Open questions](#open-questions)). Each version also carries at least one **definitional anchor**: source - provenance (one or more typed source pointers as in RFC-0008: a - Git repo and ref, an OCI image, an archive) and/or an immutable - configuration snapshot. An agent registered from an A2A endpoint + provenance, as one or more typed source pointers of the kinds the + Skill Registry supports (a Git repo and ref, an OCI image, a zip + archive, or direct MLflow artifact storage), which for a + harness-based agent point at its configuration. An agent + registered from an A2A endpoint alone is an interface-only record with no anchor (see below). Each change to composition is a new version. @@ -740,8 +753,8 @@ conventions exist, so it is bounded by MLflow's governance surface rather than by agent anatomy: an agent's prompt strategy or memory configuration has no axis because nothing governs one. Three layers share the job of describing an agent. Structured BOM references are -selective but queryable across agents. Definitional anchors (a source -pointer, a configuration snapshot) are complete but opaque: they +selective but queryable across agents. Definitional anchors (source +pointers) are complete but opaque: they capture everything about one agent without supporting cross-agent queries. Free-form tags are the catch-all for facts that fit neither. New structured axes are expected as the governance surface @@ -762,7 +775,7 @@ deliberately departs from the canonical-payload pattern of RFC-0004 (`server_json`) and RFC-0008 (`plugin.json`): MLflow is the system of record for those payloads, while an Agent Card's system of record is the agent itself. An agent registered from an endpoint -alone, with no source and no configuration snapshot, is an +alone, with no source, is an **interface-only record**: the registry captures the claim surface (identity, imported metadata, endpoint) and marks that it holds no definitional anchor. @@ -862,17 +875,13 @@ TBD. - **How should harness-based agents be described?** Agents that run as configurations of a packaged harness (Claude Code, OpenCode, Goose) have no user source repository; the agent is the harness - plus its configuration. This RFC proposes three mechanisms for - them, going beyond the source-centric design the journeys otherwise + plus its configuration. This RFC proposes two mechanisms for them, + going beyond the code-centric design the journeys otherwise follow: a harness reference axis in the BOM (an external - identifier, like external model references), an immutable - configuration snapshot stored as a version artifact to serve as the - definitional anchor, and a relaxation of the required fields from - "source" to "at least one definitional anchor" (source, - configuration snapshot, or A2A card). Sub-questions: - - - Do configuration snapshots invite secret leakage badly enough to - need enforced redaction rather than guidance? + identifier, like external model references), and treating the + configuration as the version's source, pointed at through the same + typed source pointers used for skill content. Sub-questions: + - Should a harness axis wait for some notion of harness identity governance? - What are the identifier semantics of a harness reference? The @@ -886,12 +895,10 @@ TBD. harnesses are distributed. - Should the set of files that constitutes a harness's configuration surface be defined by per-harness integrations - (the harness integrations contemplated by RFC-0009 would be a - natural home) rather than hand-picked by each registrant, given - that the surface must also exclude content the BOM already - governs? - - Does configuration-as-artifact belong in this RFC or a - follow-on? + (the harness integrations contemplated by the skill tracing + proposal would be a natural home) rather than assembled by each + registrant, given that the surface must also exclude content the + BOM already governs? - **Should a version record multiple sources?** This RFC says yes: a version's source provenance is a list of typed pointers, so an From 2c15cd83b61ef4f573ba435663e9de8c6afee20b Mon Sep 17 00:00:00 2001 From: Bill Murdock Date: Wed, 9 Sep 2026 08:47:02 -0400 Subject: [PATCH 23/34] Register trace locations on the agent; fixed default experiment Overrides remain (separate audiences need separate experiments), and discoverability is handled by rule rather than proposal: each agent has a registered set of trace locations (its fixed default experiment plus any a deployment registers when it overrides), the agent's page searches across them, and overriding deployments must register their location. The default is never re-pointed. Adds the OpenTelemetry export-header convention for agent identity. From review feedback. Co-Authored-By: Claude Fable 5 --- .../0011-agent-registry.md | 34 ++++++++++++------- 1 file changed, 21 insertions(+), 13 deletions(-) diff --git a/rfcs/0011-agent-registry/0011-agent-registry.md b/rfcs/0011-agent-registry/0011-agent-registry.md index 0280d798..7603becb 100644 --- a/rfcs/0011-agent-registry/0011-agent-registry.md +++ b/rfcs/0011-agent-registry/0011-agent-registry.md @@ -463,7 +463,9 @@ organized by agent and version, not by experiment. declares the agent and version so its traces stay labeled. Framework and harness autologgers respect the active destination and metadata, so instrumented applications need only state which - agent they are. + agent they are. Agents that export traces through OpenTelemetry + without the MLflow SDK pass the same agent identity and version + in export headers, mirroring the existing experiment-ID header. 2. Run evaluations against the agent: ```python mlflow.genai.evaluate(data=eval_dataset, @@ -507,14 +509,16 @@ destinations, not trace tags. Versions share the agent's experiment for the same reason: a version is an analysis dimension recorded on every trace, not an access boundary, and per-version experiments would break the longitudinal view of an agent's behavior across -upgrades. So that overridden deployments stay findable from the -agent's registry page, this RFC proposes that a deployment using a -non-default experiment notify the registry, which records the -experiment ID on that deployment's access binding. A deployment -with no binding would need another place to record the link, which -is one reason this mechanism is a proposal rather than settled. -Automating the setup and upkeep of these links at deploy time -belongs to the deferred registry synchronization glue. +upgrades. Overrides do not make traces hard to find, because the +registry keeps the list: each agent has a set of registered trace +locations, consisting of its default experiment plus any experiment +a deployment registers when it overrides, and the agent's page +enumerates and searches across all of them. A deployment that +overrides its destination must register that location with the +agent; the default experiment itself is fixed when the agent is +created and is never re-pointed, so locations are added rather than +moved. Automating the registration and upkeep of these locations at +deploy time belongs to the deferred registry synchronization glue. No endpoint is needed for any of this when the developer has the agent's code: the agent runs locally or in CI, autologging captures @@ -805,10 +809,14 @@ identity alongside experiment identity. A destination identifies the agent only and resolves to the agent's one default experiment; the version is never part of the destination and is instead recorded on every trace and evaluation run as metadata, which is -what per-version filtering and comparison use. Traces and eval -results appear on the agent's registry page, filterable by version. -The change is additive: the default experiment exists under the -hood, and experiment-based workflows continue unchanged. +what per-version filtering and comparison use. The default +experiment is fixed when the agent is created; a deployment that +sends its traces elsewhere registers that experiment as an +additional trace location on the agent, so the agent's page always +knows where its traces are. Traces and eval results appear on the +agent's registry page, filterable by version. The change is +additive: the default experiment exists under the hood, and +experiment-based workflows continue unchanged. # Drawbacks From fe0f6f9c4cf2e2ae6427b626b53d634ccffcc93b Mon Sep 17 00:00:00 2001 From: Bill Murdock Date: Wed, 9 Sep 2026 10:16:17 -0400 Subject: [PATCH 24/34] Note why the default experiment is never re-pointed Re-pointing the default would be redundant with the override mechanism, so trace locations are added rather than moved. Co-Authored-By: Claude Fable 5 --- rfcs/0011-agent-registry/0011-agent-registry.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/rfcs/0011-agent-registry/0011-agent-registry.md b/rfcs/0011-agent-registry/0011-agent-registry.md index 7603becb..dcf91117 100644 --- a/rfcs/0011-agent-registry/0011-agent-registry.md +++ b/rfcs/0011-agent-registry/0011-agent-registry.md @@ -516,7 +516,8 @@ a deployment registers when it overrides, and the agent's page enumerates and searches across all of them. A deployment that overrides its destination must register that location with the agent; the default experiment itself is fixed when the agent is -created and is never re-pointed, so locations are added rather than +created and is never re-pointed, since re-pointing it would be +redundant with overriding it, so locations are added rather than moved. Automating the registration and upkeep of these locations at deploy time belongs to the deferred registry synchronization glue. From 26acba81248fe72adf7fe384e6afebb42556126d Mon Sep 17 00:00:00 2001 From: Bill Murdock Date: Wed, 9 Sep 2026 10:21:53 -0400 Subject: [PATCH 25/34] Give examples of an agent having several trace audiences Co-Authored-By: Claude Fable 5 --- rfcs/0011-agent-registry/0011-agent-registry.md | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/rfcs/0011-agent-registry/0011-agent-registry.md b/rfcs/0011-agent-registry/0011-agent-registry.md index dcf91117..e9590dff 100644 --- a/rfcs/0011-agent-registry/0011-agent-registry.md +++ b/rfcs/0011-agent-registry/0011-agent-registry.md @@ -505,7 +505,12 @@ arrangement. A deployment that needs its traces kept separate from other deployments of the same agent (a different owner or user base) overrides the destination in its own configuration; because permissions are experiment-scoped, separation is done with -destinations, not trace tags. Versions share the agent's experiment +destinations, not trace tags. The same agent can have several such +audiences: two teams each running it for their own users, +per-tenant or per-customer deployments whose prompts and data must +not cross, production deployments whose traces carry stricter +access than non-production ones, or deployments split by region or +jurisdiction. Versions share the agent's experiment for the same reason: a version is an analysis dimension recorded on every trace, not an access boundary, and per-version experiments would break the longitudinal view of an agent's behavior across From 9e62019501d1a801481b66c84916da4461b0d55d Mon Sep 17 00:00:00 2001 From: Bill Murdock Date: Wed, 9 Sep 2026 13:35:39 -0400 Subject: [PATCH 26/34] Add a framework axis and fix harness/framework vocabulary What runs an agent is now recorded as either a harness reference (packaged application) or a framework reference (agent framework), each with a version. Values come from a set of well-known identifiers shipped with MLflow plus an "other" escape hatch with a free-text name, the same shape as the binding protocol field, so spelling variants cannot fragment queries. Resolves two harness open-question sub-questions. From review feedback. Co-Authored-By: Claude Fable 5 --- .../0011-agent-registry.md | 32 +++++++++---------- 1 file changed, 15 insertions(+), 17 deletions(-) diff --git a/rfcs/0011-agent-registry/0011-agent-registry.md b/rfcs/0011-agent-registry/0011-agent-registry.md index e9590dff..a047146f 100644 --- a/rfcs/0011-agent-registry/0011-agent-registry.md +++ b/rfcs/0011-agent-registry/0011-agent-registry.md @@ -47,9 +47,10 @@ In brief, the design takes these positions, each stated in full in [Design positions](#design-positions): the registry is record-level, not runtime-aware; an agent's versions are immutable snapshots of its composition (a bill of materials of skills, agent -plugins, MCP servers, models, and other agents it calls) plus at -least one definitional anchor (typed source pointers, which for a -harness-based agent point at its configuration); A2A Agent +plugins, MCP servers, models, other agents it calls, and the harness +or framework that runs it) plus at least one definitional anchor +(typed source pointers, which for a harness-based agent point at its +configuration); A2A Agent Cards are fetched from the agent's endpoint, never stored; endpoints are mutable, protocol-typed access bindings rather than version fields; and an agent is a trace destination with one @@ -98,6 +99,8 @@ mlflow.genai.register_agent( agents=["agents:/acme/records-agent/2"], mcp_servers=["mcp-servers:/acme.internal/payments-db/2.0.0"], models=["models:/acme-billing-llm/3", "gpt-4o"], + framework="langgraph", + framework_version="0.3.1", ) ``` @@ -717,9 +720,15 @@ SDK namespace, following the pattern of RFC-0004 and RFC-0008: (pinned to a version when the referencing team controls the callee's deployment, as with a set of agents versioned and deployed together as one application, and name-level when the - callee is independently managed), and, for agents that run as - configurations of a packaged harness, a harness reference (a - proposed axis; see [Open questions](#open-questions)). Each + callee is independently managed), and a reference to what runs + the agent: a harness reference for agents that run as + configurations of a packaged application (OpenCode, Claude Code), + or a framework reference for agents built on an agent framework + (LangGraph, CrewAI), each with a version. Harness and framework + values come from a set of well-known identifiers shipped with + MLflow, with `other` plus a free-text name as the escape hatch, + the same shape as the binding protocol field, so that spelling + variants of well-known names cannot fragment queries. Each version also carries at least one **definitional anchor**: source provenance, as one or more typed source pointers of the kinds the Skill Registry supports (a Git repo and ref, an OCI image, a zip @@ -896,17 +905,6 @@ TBD. configuration as the version's source, pointed at through the same typed source pointers used for skill content. Sub-questions: - - Should a harness axis wait for some notion of harness identity - governance? - - What are the identifier semantics of a harness reference? The - options are an opaque asserted name/version pair; an open - vocabulary in which free-text names are always accepted but - names matching MLflow's known harness integrations are - recognized and normalized; or a resolvable package identity. The - open vocabulary is the likely landing: fragmented spellings - would undermine cross-agent queries, while no single package - ecosystem could serve as an authority given how heterogeneously - harnesses are distributed. - Should the set of files that constitutes a harness's configuration surface be defined by per-harness integrations (the harness integrations contemplated by the skill tracing From e8b4282b0e2ba1fad9f77270ce6e6f4aa0f2e1cf Mon Sep 17 00:00:00 2001 From: Bill Murdock Date: Wed, 9 Sep 2026 15:23:17 -0400 Subject: [PATCH 27/34] Make the agent the entity GenAI users create; deployment vocabulary Adopts the bolder positioning from review: for GenAI work an agent, not an experiment, is what users create and trace against, with experiments continuing for model training and, underneath, as the storage and permission unit. Mechanics stay additive. Multi-audience isolation is expressed as trace locations per deployment of the agent rather than as experiment overrides, with set_active_agent(..., deployment=...) resolving to a deployment's location. Renames the sketched destination type to MlflowAgentTraceLocation to avoid confusion with an agent's endpoint. Co-Authored-By: Claude Fable 5 --- .../0011-agent-registry.md | 112 ++++++++++-------- 1 file changed, 60 insertions(+), 52 deletions(-) diff --git a/rfcs/0011-agent-registry/0011-agent-registry.md b/rfcs/0011-agent-registry/0011-agent-registry.md index a047146f..fa942bdc 100644 --- a/rfcs/0011-agent-registry/0011-agent-registry.md +++ b/rfcs/0011-agent-registry/0011-agent-registry.md @@ -53,9 +53,12 @@ or framework that runs it) plus at least one definitional anchor configuration); A2A Agent Cards are fetched from the agent's endpoint, never stored; endpoints are mutable, protocol-typed access bindings rather than -version fields; and an agent is a trace destination with one -default experiment, with the version recorded on every trace and -evaluation run. +version fields; and for GenAI work an agent, not an experiment, is +the entity users create and trace against: it carries the +traces-and-evaluations experience, with a default trace location +plus one per deployment that needs its own, and the version +recorded on every trace and evaluation run, while experiments +continue for model training and underneath. **Relationship to other RFCs.** RFC-0004 establishes the access binding pattern this RFC reuses (its canonical-payload pattern is @@ -456,14 +459,16 @@ organized by agent and version, not by experiment. `mlflow.genai.get_default_experiment_id("acme/billing-agent")`, where `get_default_experiment_id` is a public lookup that takes only the agent: the version is never part of the destination. (An - `MlflowAgentLocation` naming the agent works anywhere MLflow + `MlflowAgentTraceLocation` naming the agent works anywhere MLflow accepts a trace destination.) Second, it records the agent and version as trace-level metadata, the way session and user metadata are recorded today; this is what per-version filtering and comparison use. A deployment that - overrides its destination (below) swaps only the first piece, - pointing `set_experiment` at its own experiment, and still - declares the agent and version so its traces stay labeled. + needs its own trace location (below) names it instead: + `set_active_agent("acme/billing-agent", version=3, + deployment="prod-eu")` resolves to that deployment's location + and still records the agent and version so the traces stay + labeled. Framework and harness autologgers respect the active destination and metadata, so instrumented applications need only state which agent they are. Agents that export traces through OpenTelemetry @@ -497,37 +502,37 @@ experiment-based workflows (including model training and fine-tuning) continue unchanged. The change is additive, not a data model rewrite. -The default experiment is a default, not a router. A running agent -logs traces to whatever destination its own deployment -configuration sets; the registry is not in the call path. When -nothing is set, traces land in the agent's default experiment, +The default trace location is a default, not a router. A running +agent logs traces to whatever location its own deployment +configuration names; the registry is not in the call path. When +nothing is named, traces land in the agent's default location, which is the right behavior for the development loop and registry-driven evaluations. Scale-out replicas of one deployment share its configuration, so their traces aggregate without further -arrangement. A deployment that needs its traces kept separate from -other deployments of the same agent (a different owner or user -base) overrides the destination in its own configuration; because -permissions are experiment-scoped, separation is done with -destinations, not trace tags. The same agent can have several such -audiences: two teams each running it for their own users, -per-tenant or per-customer deployments whose prompts and data must -not cross, production deployments whose traces carry stricter -access than non-production ones, or deployments split by region or -jurisdiction. Versions share the agent's experiment -for the same reason: a version is an analysis dimension recorded on -every trace, not an access boundary, and per-version experiments -would break the longitudinal view of an agent's behavior across -upgrades. Overrides do not make traces hard to find, because the -registry keeps the list: each agent has a set of registered trace -locations, consisting of its default experiment plus any experiment -a deployment registers when it overrides, and the agent's page -enumerates and searches across all of them. A deployment that -overrides its destination must register that location with the -agent; the default experiment itself is fixed when the agent is -created and is never re-pointed, since re-pointing it would be -redundant with overriding it, so locations are added rather than -moved. Automating the registration and upkeep of these locations at -deploy time belongs to the deferred registry synchronization glue. +arrangement. A deployment whose traces must be kept apart from +other deployments of the same agent gets its own trace location: +two teams each running the agent for their own users, per-tenant +or per-customer deployments whose prompts and data must not cross, +production deployments whose traces carry stricter access than +non-production ones, or deployments split by region or +jurisdiction. Each trace location is an experiment underneath, +which is what makes the separation enforceable, since permissions +are experiment-scoped; separation is therefore done with locations, +not trace tags. Versions share a location for the same reason: a +version is an analysis dimension recorded on every trace, not an +access boundary, and per-version locations would break the +longitudinal view of an agent's behavior across upgrades. Locations +never make traces hard to find, because the registry keeps the +list: an agent's registered trace locations are its default plus +one per registered deployment, presented on the agent's page as +deployments of the agent and searched across as one. A deployment +with its own location must be registered with the agent; the +default location is fixed when the agent is created and never +re-pointed, since re-pointing it would be redundant with +registering a deployment, so locations are added rather than moved. +Automatically registering deployments and their trace locations at +deploy time is part of the registry synchronization work deferred +in Out of scope. No endpoint is needed for any of this when the developer has the agent's code: the agent runs locally or in CI, autologging captures @@ -813,24 +818,27 @@ is always MCP, an agent binding declares its protocol: `a2a`, `mcp` accepts an optional endpoint as a convenience that creates a binding. -**Agents become the primary anchor for GenAI traces and -evaluations.** Today traces and evaluation runs attach to +**For GenAI work, an agent is the entity users create, not an +experiment.** Today traces and evaluation runs attach to experiments, an abstraction that fits model training but not the -agent development loop. This RFC makes an agent a trace -destination: a new `MlflowAgentLocation` joins the existing -`MlflowExperimentLocation`, usable wherever MLflow accepts a trace -destination today, and evaluation and trace-search APIs gain agent -identity alongside experiment identity. A destination identifies -the agent only and resolves to the agent's one default experiment; -the version is never part of the destination and is instead -recorded on every trace and evaluation run as metadata, which is -what per-version filtering and comparison use. The default -experiment is fixed when the agent is created; a deployment that -sends its traces elsewhere registers that experiment as an -additional trace location on the agent, so the agent's page always -knows where its traces are. Traces and eval results appear on the -agent's registry page, filterable by version. The change is -additive: the default experiment exists under the hood, and +agent development loop. This RFC makes the agent the thing a GenAI +user creates and traces against: the agent's page carries the +traces-and-evaluations experience, and experiments continue as the +entity for model training and, underneath, as the storage and +permission unit for agent traces. Mechanically the change is +additive: a new `MlflowAgentTraceLocation` joins the existing +`MlflowExperimentLocation` (named to avoid confusion with an +agent's endpoint; a shorter name is welcome), usable wherever +MLflow accepts a trace destination today, and evaluation and +trace-search APIs gain agent identity alongside experiment +identity. A destination identifies the agent and optionally one of +its deployments, never the version; the version is recorded on +every trace and evaluation run as metadata, which is what +per-version filtering and comparison use. An agent has one or more +trace locations: a default, fixed when the agent is created, plus +one for each deployment that needs its own, each an experiment +underneath, and the agent's page presents them as deployments of +the agent, enumerating and searching across all of them. Existing experiment-based workflows continue unchanged. # Drawbacks From 15fb2e4d60d9918c42caa88965191b32fb7f03a1 Mon Sep 17 00:00:00 2001 From: Bill Murdock Date: Wed, 9 Sep 2026 17:29:40 -0400 Subject: [PATCH 28/34] Add a prompts BOM axis backed by the Prompt Registry Co-Authored-By: Claude Fable 5.1 --- .../0011-agent-registry.md | 45 +++++++++++-------- 1 file changed, 27 insertions(+), 18 deletions(-) diff --git a/rfcs/0011-agent-registry/0011-agent-registry.md b/rfcs/0011-agent-registry/0011-agent-registry.md index fa942bdc..8f42c586 100644 --- a/rfcs/0011-agent-registry/0011-agent-registry.md +++ b/rfcs/0011-agent-registry/0011-agent-registry.md @@ -39,16 +39,18 @@ The Agent Registry is the third registry in a series, following the and the [Skill Registry (RFC-0008)](https://github.com/mlflow/rfcs/blob/main/rfcs/0008-mvp-skill-registry/0008-mvp-skill-registry.md). It completes a progression: MLflow can govern the tools an agent calls -(MCP servers), the expertise it carries (skills), and the models it -invokes (Model Registry), but the agent itself, the thing that acts, -has no registry entry. This RFC adds one. +(MCP servers), the expertise it carries (skills), the models it +invokes (Model Registry), and the prompts it sends them (Prompt +Registry), but the agent itself, the thing that acts, has no +registry entry. This RFC adds one. In brief, the design takes these positions, each stated in full in [Design positions](#design-positions): the registry is record-level, not runtime-aware; an agent's versions are immutable snapshots of its composition (a bill of materials of skills, agent -plugins, MCP servers, models, other agents it calls, and the harness -or framework that runs it) plus at least one definitional anchor +plugins, MCP servers, models, prompts, other agents it calls, and +the harness or framework that runs it) plus at least one +definitional anchor (typed source pointers, which for a harness-based agent point at its configuration); A2A Agent Cards are fetched from the agent's endpoint, never stored; @@ -102,6 +104,7 @@ mlflow.genai.register_agent( agents=["agents:/acme/records-agent/2"], mcp_servers=["mcp-servers:/acme.internal/payments-db/2.0.0"], models=["models:/acme-billing-llm/3", "gpt-4o"], + prompts=["prompts:/billing-system-prompt/4"], framework="langgraph", framework_version="0.3.1", ) @@ -208,11 +211,12 @@ ways. answered by asking around. 2. **Composition is untracked.** An agent is a composition of skills, - MCP servers, and models, each versioned independently. No record - captures which versions of which components a given build of the - agent used. When behavior changes, "something changed and the - agent broke; what was it?" requires reconstructing the composition - from memory, commit history, and luck. + MCP servers, models, and prompts, each versioned independently. + No record captures which versions of which components a given + build of the agent used. When behavior changes, "something + changed and the agent broke; what was it?" requires + reconstructing the composition from memory, commit history, and + luck. 3. **Experiments do not map to agents.** MLflow traces and evaluation runs attach to experiments. Experiments fit training workflows, @@ -626,7 +630,8 @@ every affected agent without inspecting deployments one by one. The same query works for the other BOM axes ("which agents use MCP server Y whose tool schema changed?", "which agents call model Z -being retired?") and for source entries: "which agents ship OCI +being retired?", "which agents send prompt W, whose latest edit +regressed?") and for source entries: "which agents ship OCI image X?" is the container-CVE variant. Agent references close the highest-impact case: "which agents call the compromised agent?" is a query over the same axis. Agent plugin references @@ -721,7 +726,9 @@ SDK namespace, following the pattern of RFC-0004 and RFC-0008: references (a plugin is referenced as a composed unit and expands through its registered members for queries), MCP server references, model references (registry models or external model - identifiers such as `gpt-4o`), references to other agents it calls + identifiers such as `gpt-4o`), prompt references (Prompt Registry + entries, which cover system prompts and any other prompt the agent + sends a model), references to other agents it calls (pinned to a version when the referencing team controls the callee's deployment, as with a set of agents versioned and deployed together as one application, and name-level when the @@ -766,16 +773,18 @@ and `semver`; `freeform` versions order by registration time. BOM entries are soft references, structured values rather than foreign keys. They resolve against the Skill Registry (which RFC-0008 defines for both skills and agent plugins), MCP Server -Registry, and Model Registry when matching entries exist, and they -remain valid when they do not. This makes cross-registry questions -("which agents use skill X?") answerable as registry queries -without constraining registration order. +Registry, Model Registry, and Prompt Registry when matching entries +exist, and they remain valid when they do not. This makes +cross-registry questions ("which agents use skill X?") answerable +as registry queries without constraining registration order. **The BOM is a component inventory, not a complete recipe.** Its structured axes exist because corresponding registries or identifier conventions exist, so it is bounded by MLflow's governance surface -rather than by agent anatomy: an agent's prompt strategy or memory -configuration has no axis because nothing governs one. Three layers +rather than by agent anatomy: an agent's prompts have an axis +because the Prompt Registry governs them, while its memory +configuration or context-compaction strategy has none because +nothing governs one. Three layers share the job of describing an agent. Structured BOM references are selective but queryable across agents. Definitional anchors (source pointers) are complete but opaque: they From 24a46b4f1bb10cd45ae7d6fe4be2d5ebe4fcf097 Mon Sep 17 00:00:00 2001 From: Bill Murdock Date: Thu, 10 Sep 2026 18:01:36 -0400 Subject: [PATCH 29/34] Shape the deferred trace-based composition discovery follow-on Co-Authored-By: Claude Fable 5.1 --- .../0011-agent-registry.md | 23 +++++++++++++++---- 1 file changed, 19 insertions(+), 4 deletions(-) diff --git a/rfcs/0011-agent-registry/0011-agent-registry.md b/rfcs/0011-agent-registry/0011-agent-registry.md index 8f42c586..4a6dba81 100644 --- a/rfcs/0011-agent-registry/0011-agent-registry.md +++ b/rfcs/0011-agent-registry/0011-agent-registry.md @@ -677,10 +677,25 @@ rejection. example a Kubernetes controller) pushing to the registry APIs this RFC defines. Deferred. - **Auto-discovery of composition.** BOMs are developer-asserted in - the MVP. Inferring actual composition from traces (for example, - from RFC-0009 `SKILL` spans) and notifying owners when assertion - and observation disagree is a follow-on that the span data from - this RFC and RFC-0009 is designed to enable. + the MVP. Inferring composition from traces is deliberately + deferred, including the user-initiated form: select traces, infer + the observed skills, MCP servers, models, and prompts, review the + proposed BOM, and register it as a draft version; and, once a + version exists, flag traces recorded against it that use + undeclared components or different component versions. That is + the intended shape of the follow-on, and this RFC lays its + groundwork: every trace carries the agent version it was recorded + against, which is what drift detection compares. The other half + is trace conventions that identify components by registry + reference, and coverage is uneven today. Prompts have one: MLflow + links a prompt loaded from the Prompt Registry to the active + trace by name and version. The skill tracing proposal supplies + one for skills (`SKILL` spans). LLM spans record the provider's + model name, which matches an external model identifier in the BOM + but not a Model Registry entry. Tool spans record a tool name and + no MCP server identity, so MCP server inference would rely on + matching tool names against registered servers' tool lists. The + follow-on should close those gaps before it is built. - **Detection of unregistered agents.** Surfacing "shadow" agents running without registry entries requires runtime scanning, which is platform work built on top of this registry. From e1820ca331f8ac168e6672230b82ec18ab7fd653 Mon Sep 17 00:00:00 2001 From: Bill Murdock Date: Fri, 11 Sep 2026 08:49:56 -0400 Subject: [PATCH 30/34] Binding description and platform_url; narrow the discovery exclusion Co-Authored-By: Claude Fable 5.1 --- .../0011-agent-registry.md | 36 ++++++++++++++----- 1 file changed, 27 insertions(+), 9 deletions(-) diff --git a/rfcs/0011-agent-registry/0011-agent-registry.md b/rfcs/0011-agent-registry/0011-agent-registry.md index 4a6dba81..4c7ed15e 100644 --- a/rfcs/0011-agent-registry/0011-agent-registry.md +++ b/rfcs/0011-agent-registry/0011-agent-registry.md @@ -377,16 +377,24 @@ disturbing the immutable version history. target_alias="production", endpoint_url="https://agents.acme.internal/billing", protocol="a2a", + platform_url="https://console.acme.internal/agents/billing-prod", ) ``` The endpoint accepted at registration time is sugar for creating a binding; the A2A registration path creates an `a2a` binding - automatically. + automatically. Two optional fields describe the deployment + without recording its state: `platform_url` links to wherever + the serving platform shows this deployment (a console page, a + Kubernetes resource), and a free-text `description` holds + connection notes. 3. The agent's detail page lists its bindings. Bindings whose protocol is self-describing (`a2a`, `mcp`) are actionable: they are the entry points for the endpoint-driven tracing and - evaluation in the trace-and-eval journey below. An `other` - binding is a documented pointer. + evaluation in the trace-and-eval journey below. They are also + how a developer, or another agent, goes from a registry search + to a live endpoint: the binding gives the URL, and the protocol + gives the rest. An `other` binding is a documented pointer, and + its `description` is where the operator says how to call it. 4. The deployment moves to a new URL. The operator updates the binding; no version record changes. 5. The deployment is retired. The operator deletes the binding; the @@ -702,9 +710,14 @@ rejection. - **Automated notifications.** The blast-radius journey ends with the registry naming owners; notifying them is left to the organization in the MVP. -- **Agent-to-agent runtime discovery.** A programmatic "find me an - agent that can do X and call it" surface for running agents is a - gateway concern, as is any request routing. +- **Agent-to-agent discovery beyond search.** The registry answers + the first half of "find me an agent that can do X and call it": a + developer or an agent can search it, and an `a2a` or `mcp` + binding leads to a live endpoint whose protocol describes the + rest, an Agent Card in one case and the MCP handshake in the + other. The second half is a gateway concern: routing requests, + choosing among live instances by health or load, and mediating + authentication are not registry functions. - **Cost attribution.** Per-agent token cost is an observability rollup over agent-linked traces, not registry metadata. - **Cross-workspace federation.** Discovery across registries is @@ -838,9 +851,14 @@ approved endpoints are separate mutable binding records that target a version or alias, created and deleted as connectivity changes without touching version history. Where an MCP binding's protocol is always MCP, an agent binding declares its protocol: `a2a`, `mcp` -(for agents exposed as MCP servers), or `other`. Registration -accepts an optional endpoint as a convenience that creates a -binding. +(for agents exposed as MCP servers), or `other`. A binding may +also carry a free-text `description` and a `platform_url`, both +optional: the description tells a caller how to use an endpoint +whose protocol does not say, and the platform URL points at the +serving platform's own view of the deployment, so that runtime +state stays with the platform while the registry records where to +find it. Registration accepts an optional endpoint as a +convenience that creates a binding. **For GenAI work, an agent is the entity users create, not an experiment.** Today traces and evaluation runs attach to From 3da1e2cedfec44d252c75e93dee8c11bfdf8d07e Mon Sep 17 00:00:00 2001 From: Bill Murdock Date: Fri, 11 Sep 2026 09:21:29 -0400 Subject: [PATCH 31/34] Skill-style coordinates, keep display_name, access endpoint vocabulary Co-Authored-By: Claude Fable 5.1 --- .../0011-agent-registry.md | 150 ++++++++++-------- 1 file changed, 85 insertions(+), 65 deletions(-) diff --git a/rfcs/0011-agent-registry/0011-agent-registry.md b/rfcs/0011-agent-registry/0011-agent-registry.md index 4c7ed15e..65d12533 100644 --- a/rfcs/0011-agent-registry/0011-agent-registry.md +++ b/rfcs/0011-agent-registry/0011-agent-registry.md @@ -54,16 +54,17 @@ definitional anchor (typed source pointers, which for a harness-based agent point at its configuration); A2A Agent Cards are fetched from the agent's endpoint, never stored; -endpoints are mutable, protocol-typed access bindings rather than -version fields; and for GenAI work an agent, not an experiment, is -the entity users create and trace against: it carries the +endpoints are mutable, protocol-typed access endpoint records +rather than version fields; and for GenAI work an agent, not an +experiment, is the entity users create and trace against: it +carries the traces-and-evaluations experience, with a default trace location plus one per deployment that needs its own, and the version recorded on every trace and evaluation run, while experiments continue for model training and underneath. **Relationship to other RFCs.** RFC-0004 establishes the access -binding pattern this RFC reuses (its canonical-payload pattern is +endpoint pattern this RFC reuses (its canonical-payload pattern is deliberately not applied to Agent Cards; see [Design positions](#design-positions)). RFC-0008 defines the skills and agent plugins that agent BOMs reference; note that an @@ -90,7 +91,8 @@ import mlflow from mlflow.genai import GitSource mlflow.genai.register_agent( - name="acme/billing-agent", + organization="acme", + name="billing-agent", description="Answers customer billing questions.", sources=[ GitSource( @@ -101,7 +103,7 @@ mlflow.genai.register_agent( ], skills=["skills:/billing-policy/1", "skills:/refund-rules/2"], agent_plugins=["agent-plugins:/billing-workflow/1.2.0"], - agents=["agents:/acme/records-agent/2"], + agents=["agents:/@acme/records-agent/2"], mcp_servers=["mcp-servers:/acme.internal/payments-db/2.0.0"], models=["models:/acme-billing-llm/3", "gpt-4o"], prompts=["prompts:/billing-system-prompt/4"], @@ -122,7 +124,8 @@ the rationale). ```python mlflow.genai.register_agent( - name="acme/travel-agent", + organization="acme", + name="travel-agent", a2a_endpoint="https://agents.acme.internal/travel", skills=["skills:/itinerary-planning/4"], ) @@ -132,9 +135,9 @@ Given an endpoint, the client SDK fetches the Agent Card from the endpoint's well-known path, imports its descriptive metadata (description, capabilities, and its free-form name, which seeds the mutable MLflow-managed `display_name`), and creates an `a2a` access -binding for the endpoint. The card content is not persisted: the +endpoint record for it. The card content is not persisted: the endpoint is the card's system of record, and the UI renders the -card read-only by fetching it through the binding at view time. The +card read-only by fetching it through that record at view time. The registry `name` is always chosen by the registrant, since a card's `name` is a display string, not an identity. Fetches happen in the client, never in the registry server, consistent with RFC-0008; a @@ -147,7 +150,8 @@ interface-only record (see the register journey). ```python mlflow.genai.register_agent( - name="acme/oncall-helper", + organization="acme", + name="oncall-helper", description="On-call assistant run in OpenCode.", harness="opencode", harness_version="0.5.3", @@ -174,7 +178,7 @@ stored. ## Trace and evaluate against the agent ```python -mlflow.genai.set_active_agent("acme/billing-agent", version=3) +mlflow.genai.set_active_agent("agents:/@acme/billing-agent", version=3) with mlflow.start_span(name="answer-question"): result = agent.run(question) @@ -182,7 +186,7 @@ with mlflow.start_span(name="answer-question"): mlflow.genai.evaluate( data=eval_dataset, scorers=[correctness_scorer], - agent_id="acme/billing-agent", + agent_id="agents:/@acme/billing-agent", agent_version=3, ) ``` @@ -252,7 +256,8 @@ the record. composition: ```python mlflow.genai.register_agent( - name="acme/billing-agent", + organization="acme", + name="billing-agent", description="Answers customer billing questions.", sources=[GitSource( url="https://github.com/acme/billing-agent.git", @@ -274,7 +279,7 @@ the record. version where the agent's version scheme takes one (the default `monotonic` scheme assigns versions automatically), and an endpoint (a URL plus protocol; the endpoint journey below covers - bindings). + endpoint records). 2. MLflow creates an `AgentVersion` record with initial status `draft`. 3. The agent appears in the registry listing for its workspace, with @@ -285,8 +290,8 @@ the record. from A2A card" and "manual"; the import mode pre-fills descriptive and capability fields from the card (its free-form name seeds the mutable `display_name`; the registry `name` is - supplied by the registrant) and creates an `a2a` access binding - for the endpoint. In the SDK and CLI, the client fetches the + supplied by the registrant) and creates an `a2a` access endpoint + record for it. In the SDK and CLI, the client fetches the card at import; in the UI, the browser fetches it when the endpoint permits, or the user pastes it, since the server never fetches user-supplied URLs. The BOM is supplied alongside, since @@ -303,7 +308,8 @@ the record. that defines it: ```python mlflow.genai.register_agent( - name="acme/oncall-helper", + organization="acme", + name="oncall-helper", description="On-call assistant run in OpenCode.", harness="opencode", harness_version="0.5.3", @@ -368,12 +374,12 @@ disturbing the immutable version history. 1. The agent is already registered (any path above) and a version has been promoted. The platform team deploys it. -2. The operator creates an access binding for the deployment, +2. The operator creates an access endpoint for the deployment, targeting a version or an alias and declaring the endpoint's protocol: ```python - mlflow.genai.create_agent_access_binding( - agent="acme/billing-agent", + mlflow.genai.create_agent_access_endpoint( + agent="agents:/@acme/billing-agent", target_alias="production", endpoint_url="https://agents.acme.internal/billing", protocol="a2a", @@ -381,34 +387,35 @@ disturbing the immutable version history. ) ``` The endpoint accepted at registration time is sugar for creating - a binding; the A2A registration path creates an `a2a` binding - automatically. Two optional fields describe the deployment + this record; the A2A registration path creates an `a2a` endpoint + record automatically. Two optional fields describe the deployment without recording its state: `platform_url` links to wherever the serving platform shows this deployment (a console page, a Kubernetes resource), and a free-text `description` holds connection notes. -3. The agent's detail page lists its bindings. Bindings whose +3. The agent's detail page lists its access endpoints. Those whose protocol is self-describing (`a2a`, `mcp`) are actionable: they are the entry points for the endpoint-driven tracing and evaluation in the trace-and-eval journey below. They are also how a developer, or another agent, goes from a registry search - to a live endpoint: the binding gives the URL, and the protocol - gives the rest. An `other` binding is a documented pointer, and + to a live endpoint: the record gives the URL, and the protocol + gives the rest. An `other` endpoint is a documented pointer, and its `description` is where the operator says how to call it. 4. The deployment moves to a new URL. The operator updates the - binding; no version record changes. -5. The deployment is retired. The operator deletes the binding; the + endpoint record; no version record changes. +5. The deployment is retired. The operator deletes the record; the agent, its versions, and its history remain untouched. -The protocol field is where agent bindings depart from RFC-0004, -whose bindings are always MCP and vary only by transport. The field +The protocol field is where agent access endpoints depart from the +MCP Server Registry's, which are always MCP and vary only by +transport. The field is limited to values that tell a caller something actionable: `a2a` and `mcp` are self-describing (an Agent Card at the well-known path; the MCP handshake), so URL plus protocol is enough to connect. Labels like REST or gRPC name a transport without telling anyone how to call the agent, so they are deliberately collapsed into `other`, which records where an agent lives without claiming -MLflow can invoke it. As in RFC-0004, a binding that targets an +MLflow can invoke it. As in RFC-0004, an endpoint that targets an alias such as `production` follows the alias as it moves between versions. @@ -459,7 +466,7 @@ organized by agent and version, not by experiment. 1. Log traces against the agent instead of an experiment: ```python - mlflow.genai.set_active_agent("acme/billing-agent", version=3) + mlflow.genai.set_active_agent("agents:/@acme/billing-agent", version=3) with mlflow.start_span(name="answer-question"): result = agent.run(question) @@ -468,7 +475,7 @@ organized by agent and version, not by experiment. First, it sets the trace destination to the agent's one default experiment, equivalent to calling `mlflow.set_experiment` on the result of - `mlflow.genai.get_default_experiment_id("acme/billing-agent")`, + `mlflow.genai.get_default_experiment_id("agents:/@acme/billing-agent")`, where `get_default_experiment_id` is a public lookup that takes only the agent: the version is never part of the destination. (An `MlflowAgentTraceLocation` naming the agent works anywhere MLflow @@ -477,7 +484,7 @@ organized by agent and version, not by experiment. session and user metadata are recorded today; this is what per-version filtering and comparison use. A deployment that needs its own trace location (below) names it instead: - `set_active_agent("acme/billing-agent", version=3, + `set_active_agent("agents:/@acme/billing-agent", version=3, deployment="prod-eu")` resolves to that deployment's location and still records the agent and version so the traces stay labeled. @@ -490,7 +497,7 @@ organized by agent and version, not by experiment. ```python mlflow.genai.evaluate(data=eval_dataset, scorers=[correctness_scorer], - agent_id="acme/billing-agent", + agent_id="agents:/@acme/billing-agent", agent_version=3) ``` As with tracing, `agent_id` determines where the results land @@ -552,11 +559,11 @@ traces during execution, and evaluation scores outputs against a test dataset. The exception is agents whose code the user cannot run: another team's A2A agent, a vendor agent, a partner service. For those, the endpoint is the only execution surface, and -tracing and evaluation work by invoking the agent's access binding -with test inputs and observing responses. This track requires a -binding whose protocol MLflow can speak: `a2a` (invoked through the +tracing and evaluation work by invoking the agent's access endpoint +with test inputs and observing responses. This track requires an +endpoint whose protocol MLflow can speak: `a2a` (invoked through the card's declared interface) or `mcp` (through the MCP handshake). An -`other` binding records where the agent lives but does not by +`other` endpoint records where the agent lives but does not by itself tell MLflow how to call it, so it does not enable endpoint-driven tracing or evaluation. The registry supports both tracks; the optional endpoint exists largely for the second. @@ -676,8 +683,8 @@ rejection. affected?" question is that join, with this registry supplying the consumer-and-owner half. - **Deployment and orchestration.** The registry does not deploy - agents. Access bindings record where an approved endpoint is; they - do not create it. + agents. Access endpoint records say where an approved endpoint + is; they do not create it. - **Registry synchronization from deployments.** Auto-registering agents when they deploy, keeping BOMs fresh when composition changes at deploy time, and maintaining deployment trace-location @@ -713,9 +720,10 @@ rejection. - **Agent-to-agent discovery beyond search.** The registry answers the first half of "find me an agent that can do X and call it": a developer or an agent can search it, and an `a2a` or `mcp` - binding leads to a live endpoint whose protocol describes the - rest, an Agent Card in one case and the MCP handshake in the - other. The second half is a gateway concern: routing requests, + access endpoint leads to a live endpoint whose protocol + describes the rest, an Agent Card in one case and the MCP + handshake in the other. The second half is a gateway concern: + routing requests, choosing among live instances by health or load, and mediating authentication are not registry functions. - **Cost attribution.** Per-agent token cost is an observability @@ -744,11 +752,15 @@ registry and a gateway. The registry manages two primary entities under the `mlflow.genai` SDK namespace, following the pattern of RFC-0004 and RFC-0008: -- **Agent**: a named, owned entity with DNS-style naming - (`org/agent-name`), in the spirit of the namespaced names the MCP - and Skill registries use; exact alignment with RFC-0008's - `{workspace, organization, name}` coordinates is a detailed-design - point. +- **Agent**: a named, owned entity addressed by the same + coordinates the Skill Registry uses, `(workspace, organization, + name)`, with `organization` an optional field rather than a + segment of the name, and referenced in the same grammar + (`agents:/@acme/billing-agent/2`). Nothing external constrains + agent naming (an A2A card's name is free-form), so consistency + with the sibling MLflow-native registry decides; MCP server names + differ only because the upstream MCP registry specification fixes + them. - **AgentVersion**: an immutable snapshot of the agent's composition, its **bill of materials (BOM)**: skill references, agent plugin references (a plugin is referenced as a composed unit and expands @@ -767,7 +779,7 @@ SDK namespace, following the pattern of RFC-0004 and RFC-0008: (LangGraph, CrewAI), each with a version. Harness and framework values come from a set of well-known identifiers shipped with MLflow, with `other` plus a free-text name as the escape hatch, - the same shape as the binding protocol field, so that spelling + the same shape as the endpoint protocol field, so that spelling variants of well-known names cannot fragment queries. Each version also carries at least one **definitional anchor**: source provenance, as one or more typed source pointers of the kinds the @@ -828,9 +840,14 @@ the live card from the endpoint's well-known path, and the registry follows suit. Registering from an endpoint imports the card's descriptive metadata (description, capabilities, and its free-form name, which seeds the mutable MLflow-managed `display_name`) into -ordinary registry fields and creates an `a2a` access binding; the -card content itself is not persisted. The UI renders the card -read-only by fetching it through the binding at view time, so what +ordinary registry fields and creates an `a2a` access endpoint; the +card content itself is not persisted. The display name stays, +although the Skill Registry dropped its own, because an agent's +registry name carries no human-readability guarantee: a skill name +is a slash command users type, readable by construction, while an +agent's identity comes from a protocol name or an endpoint path, +named for a purpose, an operation, or both. The UI renders the card +read-only by fetching it through the endpoint at view time, so what MLflow displays can never drift from what the agent serves. This deliberately departs from the canonical-payload pattern of RFC-0004 (`server_json`) and RFC-0008 (`plugin.json`): MLflow is the system @@ -841,24 +858,26 @@ alone, with no source, is an (identity, imported metadata, endpoint) and marks that it holds no definitional anchor. -**Endpoints are access bindings, not version fields.** Some agents +**Endpoints are separate records, not version fields.** Some agents are reachable at a URL (A2A agents inherently; deployed agents generally), and recording that URL lets the registry drive tracing and evaluation for agents whose code the user cannot run. But endpoints change independently of composition, and agent versions -are immutable. Following RFC-0004's `MCPAccessBinding` model, -approved endpoints are separate mutable binding records that target -a version or alias, created and deleted as connectivity changes -without touching version history. Where an MCP binding's protocol -is always MCP, an agent binding declares its protocol: `a2a`, `mcp` -(for agents exposed as MCP servers), or `other`. A binding may +are immutable. Following the MCP Server Registry's access endpoint +model (`MCPAccessEndpoint` in the implementation; the RFC text +still calls it `MCPAccessBinding`), approved endpoints are separate +mutable **access endpoint** records that target a version or +alias, created and deleted as connectivity changes without touching +version history. Where an MCP access endpoint's protocol is always +MCP, an agent's declares it: `a2a`, `mcp` (for agents exposed as +MCP servers), or `other`. The record may also carry a free-text `description` and a `platform_url`, both optional: the description tells a caller how to use an endpoint whose protocol does not say, and the platform URL points at the serving platform's own view of the deployment, so that runtime state stays with the platform while the registry records where to find it. Registration accepts an optional endpoint as a -convenience that creates a binding. +convenience that creates the record. **For GenAI work, an agent is the entity users create, not an experiment.** Today traces and evaluation runs attach to @@ -913,7 +932,7 @@ TBD. - **How thin may an interface-only record be?** A black-box A2A agent registers with a name, metadata imported from its card, an - access binding, undeclared composition, and no definitional + access endpoint, undeclared composition, and no definitional anchor. Is that enough of a record to be worth governing, and should the registry require anything more of it before such a record can be promoted to `active`? @@ -930,7 +949,7 @@ TBD. prefer a narrower registry RFC, the journey splits cleanly along the RFC-0008/0009 seam. -- **Do endpoint records belong in MLflow at all?** The access binding +- **Do endpoint records belong in MLflow at all?** The access endpoint model resolves the mechanical objections to endpoints (mutability against immutable versions, staleness on version records), and RFC-0004 sets the precedent. The remaining objection is @@ -942,7 +961,7 @@ TBD. the registry cannot trace or evaluate agents whose code the user cannot run. This boundary needs explicit review. A related sub-question: when an agent is exposed as an MCP server, should an - `mcp` binding cross-reference the MCP Registry entry for the same + `mcp` endpoint cross-reference the MCP Registry entry for the same endpoint instead of duplicating it? - **How should harness-based agents be described?** Agents that run @@ -979,8 +998,9 @@ TBD. - **What is the BOM reference format?** The journeys sketch URI-style references (`skills:/billing-policy/1`, `mcp-servers:/acme.internal/payments-db/2.0.0`, - `models:/acme-billing-llm/3`). The skill scheme follows RFC-0008's - member references. RFC-0004 defines no MCP URI scheme, so the MCP + `models:/acme-billing-llm/3`). The skill and agent schemes follow + the Skill Registry's member references, including the + `@organization` prefix. RFC-0004 defines no MCP URI scheme, so the MCP scheme adopts the `mcp-servers:/` proposal from RFC-0010; the refs here respect RFC-0004's reverse-DNS server names and semantic versions. Bare identifiers cover external models (`gpt-4o`). BOM From aa9e5ea73f7ccfdd0b1ea1d9b65a6c09e8b54c3e Mon Sep 17 00:00:00 2001 From: Bill Murdock Date: Fri, 11 Sep 2026 10:51:09 -0400 Subject: [PATCH 32/34] Editorial pass: registration rule, journey/design split, vetting journey, card fields Co-Authored-By: Claude Fable 5.1 --- .../0011-agent-registry.md | 150 ++++++++++-------- 1 file changed, 87 insertions(+), 63 deletions(-) diff --git a/rfcs/0011-agent-registry/0011-agent-registry.md b/rfcs/0011-agent-registry/0011-agent-registry.md index 65d12533..cb41ccfb 100644 --- a/rfcs/0011-agent-registry/0011-agent-registry.md +++ b/rfcs/0011-agent-registry/0011-agent-registry.md @@ -132,10 +132,10 @@ mlflow.genai.register_agent( ``` Given an endpoint, the client SDK fetches the Agent Card from the -endpoint's well-known path, imports its descriptive metadata -(description, capabilities, and its free-form name, which seeds the -mutable MLflow-managed `display_name`), and creates an `a2a` access -endpoint record for it. The card content is not persisted: the +endpoint's well-known path, stores its description, its free-form +name (which seeds the mutable MLflow-managed `display_name`), and +its skills list in registry fields, and creates an `a2a` access +endpoint record for it. The card as a document is not persisted: the endpoint is the card's system of record, and the UI renders the card read-only by fetching it through that record at view time. The registry `name` is always chosen by the registrant, since a card's @@ -268,18 +268,15 @@ the record. models=["models:/acme-billing-llm/3"], ) ``` - Required: name, description, and at least one of: a definitional - anchor (one or more typed source pointers) or an A2A - endpoint. These combine freely; a first-party A2A agent registers - with both source and endpoint. Only a registration with an - endpoint and no anchor produces an interface-only record (see - below). Composition (the BOM) is required when an anchor is - present and may be empty or partial for interface-only records - (see below). Optional: tags, an explicit - version where the agent's version scheme takes one (the default - `monotonic` scheme assigns versions automatically), and an - endpoint (a URL plus protocol; the endpoint journey below covers - endpoint records). + Required: a name, a description, and at least one of a + definitional anchor (one or more typed source pointers) or an + endpoint. A registration with an anchor must also declare + composition (the BOM). Optional: tags, an explicit version where + the agent's version scheme takes one (the default `monotonic` + scheme assigns versions), and, alongside an anchor, an endpoint. + A registration with an endpoint and no anchor produces an + interface-only record, whose composition may be partial or + undeclared. 2. MLflow creates an `AgentVersion` record with initial status `draft`. 3. The agent appears in the registry listing for its workspace, with @@ -287,9 +284,9 @@ the record. and model registry pages where matching entries exist. 4. **A2A path:** an agent that serves an Agent Card registers from its endpoint. The UI registration form offers two modes, "import - from A2A card" and "manual"; the import mode pre-fills - descriptive and capability fields from the card (its free-form - name seeds the mutable `display_name`; the registry `name` is + from A2A card" and "manual"; the import mode pre-fills the + description and skills list from the card (its free-form name + seeds the mutable `display_name`; the registry `name` is supplied by the registrant) and creates an `a2a` access endpoint record for it. In the SDK and CLI, the client fetches the card at import; in the UI, the browser fetches it when the @@ -341,20 +338,6 @@ the record. an embedded copy is invisible to cross-registry queries and can drift from the declared references. -Composition is required wherever it is knowable, because the BOM is -the value: a registry record without composition is just a name in -a list. A registrant anchoring on a source has the composition in -front of them. The registrant of a -black-box vendor or partner agent does not, and forcing a -declaration would invite invented BOMs that pollute cross-registry -queries; for interface-only records the BOM may therefore be empty -or partial, an absent BOM is recorded as *undeclared* composition -rather than an empty dependency list, and the record is marked as -holding no definitional anchor: the registry knows the agent's -claim surface, not its contents. Everything else is progressive -enrichment. The endpoint is optional; the trace-and-eval journey -below explains which agents need one. - The harness path is the newest part of this design and the least settled (see [Open questions](#open-questions)). For framework-built and custom agents, the registered source is the natural complete @@ -534,13 +517,8 @@ two teams each running the agent for their own users, per-tenant or per-customer deployments whose prompts and data must not cross, production deployments whose traces carry stricter access than non-production ones, or deployments split by region or -jurisdiction. Each trace location is an experiment underneath, -which is what makes the separation enforceable, since permissions -are experiment-scoped; separation is therefore done with locations, -not trace tags. Versions share a location for the same reason: a -version is an analysis dimension recorded on every trace, not an -access boundary, and per-version locations would break the -longitudinal view of an agent's behavior across upgrades. Locations +jurisdiction. Versions share a location: a version is a filter +recorded on every trace, not an audience. Locations never make traces hard to find, because the registry keeps the list: an agent's registered trace locations are its default plus one per registered deployment, presented on the agent's page as @@ -553,20 +531,11 @@ Automatically registering deployments and their trace locations at deploy time is part of the registry synchronization work deferred in Out of scope. -No endpoint is needed for any of this when the developer has the -agent's code: the agent runs locally or in CI, autologging captures -traces during execution, and evaluation scores outputs against a -test dataset. The exception is agents whose code the user -cannot run: another team's A2A agent, a vendor agent, a partner -service. For those, the endpoint is the only execution surface, and -tracing and evaluation work by invoking the agent's access endpoint -with test inputs and observing responses. This track requires an -endpoint whose protocol MLflow can speak: `a2a` (invoked through the -card's declared interface) or `mcp` (through the MCP handshake). An -`other` endpoint records where the agent lives but does not by -itself tell MLflow how to call it, so it does not enable -endpoint-driven tracing or evaluation. The registry supports both -tracks; the optional endpoint exists largely for the second. +None of this needs an endpoint when the developer has the agent's +code: the agent runs locally or in CI, autologging captures traces +during execution, and evaluation scores outputs against a test +dataset. Agents whose code the user cannot run are the next +journey. Where [RFC-0009](https://github.com/mlflow/rfcs/pull/37) annotates spans inside a trace with the skill that produced them, this journey @@ -586,6 +555,42 @@ produces its own trace in its own destination, linked to the caller's trace through propagated trace context, using the span links MLflow already supports for OpenTelemetry. +### Vet an agent you did not build + +A platform or governance team must decide whether an agent the +organization did not build, a vendor's or a partner's, may be used; +a consuming team wants the same assurance about another team's +deployed agent. Neither can run the agent's code, and neither has +any access to whoever built it beyond the ability to call its +endpoint. + +1. The agent is registered as an interface-only record with an + `a2a` endpoint (the A2A path of the register journey); its + composition is undeclared. +2. The team runs an evaluation from the registry against that + endpoint, using its own test datasets and scorers. MLflow invokes + the endpoint with each test input, records the request and + response as a trace in the team's own MLflow instance, and scores + the outputs. Nothing on the agent's side is touched or seen; this + is black-box evaluation. +3. The results appear on the agent's page like any other evaluation + run. The run records the agent version as usual and, for an A2A + agent, the card's own version string read at run time, so that a + later change in behavior can be matched to a change on the + provider's side. +4. The results gate the lifecycle decision (next journey): the + record is promoted to `active` once the agent meets the + organization's bar and stays `draft` otherwise. +5. The team reruns the same evaluation on a schedule, because a + third party can change the agent behind the endpoint without + notice, and the agent's page holds the history of runs. + +This journey needs an endpoint whose protocol MLflow can speak: +`a2a`, invoked through the card's declared interface, or `mcp`, +through the MCP handshake. An `other` endpoint records where the +agent lives but does not tell MLflow how to call it, so it does not +enable endpoint-driven evaluation. + ### Manage an agent's lifecycle An agent owner or platform team needs agents to carry an explicit, @@ -837,11 +842,15 @@ grows. [A2A protocol](https://a2a-protocol.org/) makes an agent's endpoint the authoritative home of its Agent Card: every A2A client reads the live card from the endpoint's well-known path, and the registry -follows suit. Registering from an endpoint imports the card's -descriptive metadata (description, capabilities, and its free-form -name, which seeds the mutable MLflow-managed `display_name`) into -ordinary registry fields and creates an `a2a` access endpoint; the -card content itself is not persisted. The display name stays, +follows suit. Registering from an endpoint stores three things +from the card in ordinary registry fields: its description, its +free-form name (which seeds the mutable MLflow-managed +`display_name`), and its skills list with each entry's tags, which +is what lets an agent be searched for by what it does. A2A's +"skills" are the card's own account of what the agent can do, not +Skill Registry entries; the two share a word and nothing else. +Registration also creates an `a2a` access endpoint. The card as a +document is not persisted. The display name stays, although the Skill Registry dropped its own, because an agent's registry name carries no human-readability guarantee: a skill name is a slash command users type, readable by construction, while an @@ -856,7 +865,16 @@ record is the agent itself. An agent registered from an endpoint alone, with no source, is an **interface-only record**: the registry captures the claim surface (identity, imported metadata, endpoint) and marks that it holds no -definitional anchor. +definitional anchor. Anchored records must declare their +composition, because the BOM is the registry's value and a +registrant with the source in front of them can supply it. +Interface-only records may declare composition as partial or +undeclared, because the registrant of a black-box agent cannot +supply it truthfully, and forcing a declaration would invite +invented BOMs that pollute cross-registry queries. An absent BOM is +recorded as *undeclared* composition rather than an empty +dependency list: the registry knows the agent's claim surface, not +its contents. **Endpoints are separate records, not version fields.** Some agents are reachable at a URL (A2A agents inherently; deployed agents @@ -897,9 +915,15 @@ its deployments, never the version; the version is recorded on every trace and evaluation run as metadata, which is what per-version filtering and comparison use. An agent has one or more trace locations: a default, fixed when the agent is created, plus -one for each deployment that needs its own, each an experiment -underneath, and the agent's page presents them as deployments of -the agent, enumerating and searching across all of them. Existing +one for each deployment that needs its own. Each location is an +experiment underneath, which is what makes the separation +enforceable: MLflow permissions are experiment-scoped, so audiences +are kept apart by location, not by trace tag. Versions never get +locations of their own; a version is an analysis dimension, not an +access boundary, and per-version locations would break the +longitudinal view of an agent's behavior across upgrades. The +agent's page presents the locations as deployments of the agent, +enumerating and searching across all of them. Existing experiment-based workflows continue unchanged. # Drawbacks From 0002c4b6e62c118d779c288fd5cff1268c611656 Mon Sep 17 00:00:00 2001 From: Bill Murdock Date: Sat, 12 Sep 2026 10:22:19 -0400 Subject: [PATCH 33/34] Scan-and-register path and resolved BOM export Co-Authored-By: Claude Fable 5.1 --- .../0011-agent-registry.md | 50 +++++++++++++++++++ 1 file changed, 50 insertions(+) diff --git a/rfcs/0011-agent-registry/0011-agent-registry.md b/rfcs/0011-agent-registry/0011-agent-registry.md index cb41ccfb..16cefff3 100644 --- a/rfcs/0011-agent-registry/0011-agent-registry.md +++ b/rfcs/0011-agent-registry/0011-agent-registry.md @@ -165,6 +165,18 @@ mlflow.genai.register_agent( ) ``` +Or, from a checkout of that configuration, let MLflow propose the +registration: + +```bash +uvx mlflow@latest agent register --organization acme --name oncall-helper +``` + +The command scans the tree, shows the skills, MCP servers, models, +and prompts it found and which registry entries they match, +registers the agent after confirmation, and writes a coordinates +file back into the tree (the scan path in the register journey). + There is no agent code of the user's own: the agent is the harness plus its configuration, so the source pointer points at the configuration and serves as the definitional anchor. Any of the @@ -337,6 +349,28 @@ the record. skill directories or MCP server definitions, stays out, because an embedded copy is invisible to cross-registry queries and can drift from the declared references. +7. **Scan path:** from the agent's source tree, `mlflow agent + register` (also `uvx mlflow@latest agent register`, alongside the + existing `agent setup`) scans the tree and proposes a BOM. The + scan is exact for MLflow's own footprints: the Skill Registry's + resolution lock file and any `skills:/`, `prompts:/`, `models:/`, + or `mcp-servers:/` references in code or configuration. It is + best-effort for harness-native configuration, such as MCP server + declarations in a harness's settings file, through per-harness + adapters scoped to the well-known harness list; a generic mode + finds only the exact layer. The registrant reviews the proposal, + with matched registry entries and unmatched findings shown + apart, and confirms or edits it. Unmatched skills and MCP servers + can be registered on the spot through the Skill Registry's + import adapters and the MCP Server Registry's create path, so + the references resolve; the agent and its version are then + registered with the source pointer set to the tree's Git remote + and commit. Finally the command writes a coordinates file into + the tree recording the agent's coordinates and its resolved BOM + (the form described in Design positions), so the next version + registration and any later scan are exact. The file format is + shared with the resolution lock the Skill Registry work defers + to a separate RFC and is specified there. The harness path is the newest part of this design and the least settled (see [Open questions](#open-questions)). For framework-built @@ -823,6 +857,22 @@ exist, and they remain valid when they do not. This makes cross-registry questions ("which agents use skill X?") answerable as registry queries without constraining registration order. +**A version's BOM is exported in resolved form.** The soft +references are what the registry stores; integrating systems +usually want the expansion. The registry therefore serves any +version's BOM as a JSON document in which each reference that +matches a registry entry is expanded to that entry's record (a +skill version, an MCP server version with its server definition, a +prompt version with its text, a registry model version) and each +that does not is kept as the pointer and marked unresolved. Aliases +are pinned to the concrete version at export time, and the document +records when it was resolved, since aliases move. Agent plugins are +expanded to their members, and agents the version calls are +resolved one level, with nested BOMs reachable by repeating the +call. The SDK and REST API expose this as an option on fetching a +version, and it is the same document the scan path writes back +into a source tree. + **The BOM is a component inventory, not a complete recipe.** Its structured axes exist because corresponding registries or identifier conventions exist, so it is bounded by MLflow's governance surface From 926565d7b1c49dbdd355d5a245a61aae90581c83 Mon Sep 17 00:00:00 2001 From: Bill Murdock Date: Sat, 12 Sep 2026 10:47:21 -0400 Subject: [PATCH 34/34] Close three open questions; note mcp cross-referencing as out of scope Co-Authored-By: Claude Fable 5.1 --- .../0011-agent-registry.md | 58 ++++++------------- 1 file changed, 19 insertions(+), 39 deletions(-) diff --git a/rfcs/0011-agent-registry/0011-agent-registry.md b/rfcs/0011-agent-registry/0011-agent-registry.md index 16cefff3..bc3c5f09 100644 --- a/rfcs/0011-agent-registry/0011-agent-registry.md +++ b/rfcs/0011-agent-registry/0011-agent-registry.md @@ -762,9 +762,14 @@ rejection. access endpoint leads to a live endpoint whose protocol describes the rest, an Agent Card in one case and the MCP handshake in the other. The second half is a gateway concern: - routing requests, - choosing among live instances by health or load, and mediating - authentication are not registry functions. + routing requests, choosing among live instances by health or + load, and mediating authentication are not registry functions. +- **Cross-referencing MCP Server Registry entries from `mcp` + endpoints.** An agent exposed as an MCP server records its `mcp` + endpoint on its own record. The same server may also be + registered in the MCP Server Registry, and nothing links the two + for now; linking them is a possible later addition if the + duplication turns out to matter. - **Cost attribution.** Per-agent token cost is an observability rollup over agent-linked traces, not registry metadata. - **Cross-workspace federation.** Discovery across registries is @@ -924,12 +929,21 @@ supply it truthfully, and forcing a declaration would invite invented BOMs that pollute cross-registry queries. An absent BOM is recorded as *undeclared* composition rather than an empty dependency list: the registry knows the agent's claim surface, not -its contents. +its contents. Such a record is as thin as it sounds, name, +description, and endpoint, and nothing more is required to promote +it to `active`: whether a black-box agent is fit for use is a +judgment the vetting journey's evaluations inform, not a schema +gate. **Endpoints are separate records, not version fields.** Some agents are reachable at a URL (A2A agents inherently; deployed agents generally), and recording that URL lets the registry drive tracing -and evaluation for agents whose code the user cannot run. But +and evaluation for agents whose code the user cannot run. It is +also what lets MLflow be the glue between the many systems that +define and serve agents: the registry is authoritative about which +agents exist and where each is reached, while those systems stay +authoritative about running them, and the registry persists no +runtime state. But endpoints change independently of composition, and agent versions are immutable. Following the MCP Server Registry's access endpoint model (`MCPAccessEndpoint` in the implementation; the RFC text @@ -1004,40 +1018,6 @@ TBD. per deployment, so the question is which behavior makes the better default, not which is possible. -- **How thin may an interface-only record be?** A black-box A2A - agent registers with a name, metadata imported from its card, an - access endpoint, undeclared composition, and no definitional - anchor. Is that enough of a record to be worth governing, and - should the registry require anything more of it before such a - record can be promoted to `active`? - -- **Should agent-centric traces and evaluations be a separate RFC?** - The experiments bridge (`agent_id` resolving to a default - experiment) touches tracing APIs, evaluation APIs, and UI surface - area well beyond the registry itself, and there is precedent for - splitting: RFC-0008 defined the Skill Registry and RFC-0009 - followed with skill tracing. The counter-argument is that - agent-anchored traces and evals are the registry's core value; a - registry without them is a list of names. This RFC keeps the - journey in scope on the additive framing above, but if reviewers - prefer a narrower registry RFC, the journey splits cleanly along - the RFC-0008/0009 seam. - -- **Do endpoint records belong in MLflow at all?** The access endpoint - model resolves the mechanical objections to endpoints (mutability - against immutable versions, staleness on version records), and - RFC-0004 sets the precedent. The remaining objection is - architectural: an endpoint could be considered runtime metadata, and - the record/runtime boundary could place all of it on the platform - side. The position taken here is that "where an approved endpoint - for this agent is" belongs to the record, while "whether anything - answers there" belongs to the platform, and that without endpoints - the registry cannot trace or evaluate agents whose code the user - cannot run. This boundary needs explicit review. A related - sub-question: when an agent is exposed as an MCP server, should an - `mcp` endpoint cross-reference the MCP Registry entry for the same - endpoint instead of duplicating it? - - **How should harness-based agents be described?** Agents that run as configurations of a packaged harness (Claude Code, OpenCode, Goose) have no user source repository; the agent is the harness