From 522597555fa896a06290b6587cb90dd4b9037ebf Mon Sep 17 00:00:00 2001 From: lucarlig Date: Wed, 12 Aug 2026 10:52:30 +0100 Subject: [PATCH 1/3] docs: add MCP capability allocation Signed-off-by: lucarlig --- _context/wiki/SUMMARY.md | 1 + _context/wiki/index.md | 1 + _context/wiki/mcp-capability-allocation.md | 42 ++++++++++++++++++++++ 3 files changed, 44 insertions(+) create mode 100644 _context/wiki/mcp-capability-allocation.md diff --git a/_context/wiki/SUMMARY.md b/_context/wiki/SUMMARY.md index 852527c6..d9627750 100644 --- a/_context/wiki/SUMMARY.md +++ b/_context/wiki/SUMMARY.md @@ -11,6 +11,7 @@ - [Architecture](architecture.md) - [MCP Routing Semantics](routing.md) +- [MCP Capability Allocation](mcp-capability-allocation.md) - [Security Model](security.md) - [Failure Modes](failure-modes.md) diff --git a/_context/wiki/index.md b/_context/wiki/index.md index bf43ccb5..bd48ce63 100644 --- a/_context/wiki/index.md +++ b/_context/wiki/index.md @@ -13,6 +13,7 @@ then follow only the links that are relevant. | [preferences.md](preferences.md) | Working standards, code style, logging rules, branch naming, AI interaction preferences | | [architecture.md](architecture.md) | Middleware stack order, pipeline shape, module boundaries, state ownership, executor shapes | | [routing.md](routing.md) | Backend prefix contract, list/routed ops, federated pagination, session state, capability merge | +| [mcp-capability-allocation.md](mcp-capability-allocation.md) | Ownership, initial scope, and recommended allocation of MCP protocol surfaces | | [failure-modes.md](failure-modes.md) | HTTP/MCP/routing/backend/plugin failure table — exact HTTP codes and JSON-RPC errors | | [config.md](config.md) | Key CLI flags, JWT claims, UserConfig shape, plugin config, telemetry debugging, startup validation, local observability stack | | [deployment.md](deployment.md) | Deployment checklist, health endpoint caveat, nginx routing, TLS choices, session affinity, Redis availability, image pinning | diff --git a/_context/wiki/mcp-capability-allocation.md b/_context/wiki/mcp-capability-allocation.md new file mode 100644 index 00000000..85b556d0 --- /dev/null +++ b/_context/wiki/mcp-capability-allocation.md @@ -0,0 +1,42 @@ +# MCP Capability Allocation + +This table assigns the +[MCP 2026-07-28](https://modelcontextprotocol.io/specification/2026-07-28) +protocol surfaces across the ContextForge control plane, dataplane, upstream +backends, and MCP host. + +The control plane owns descriptive and durable control state. The dataplane owns +targeted execution, validation, and streaming. Upstream backends remain +authoritative for backend-created durable state, while the MCP host owns user +interaction and MCP App rendering. + +`Essential` identifies the initial scope. Advanced capabilities remain `TBD`; +their recommended allocation records the intended direction without committing +them to the initial implementation. + +| MCP surface | Primary owner | Status | Recommended behavior | +| --- | --- | --- | --- | +| `server/discover` | Control plane | Essential | Produce the merged logical-server identity, instructions, capabilities, extensions, TTL, and cache scope. | +| `tools/list` | Control plane | Essential | Serve a materialized, policy-filtered catalog with stable names and composite pagination. Avoid synchronous fanout on every request. | +| `tools/call` | Dataplane | Essential | Route one target, validate input and output, execute plugins, stream progress, and propagate cancellation and multi-round-trip requests (MRTR). | +| `resources/list` | Control plane | Essential | Catalog resources, filter them by authorization, and merge them deterministically. | +| `resources/templates/list` | Control plane | Essential | Catalog URI templates and their metadata. | +| `resources/read` | Dataplane | Essential | Route and stream potentially large, dynamic, or private content on the hot path. | +| `prompts/list` | Control plane | Essential | Catalog and policy-filter prompt definitions. | +| `prompts/get` | Dataplane | Essential | Render prompts that may be dynamic, expensive, private, or require MRTR. | +| `completion/complete` | Dataplane | Essential | Route interactive, latency-sensitive completion requests using control-plane mappings. | +| `subscriptions/listen` | Dataplane | TBD | Maintain long-lived streams and multiplex upstream notifications. | +| List-change events | Control plane to dataplane | TBD | Generate normalized catalog invalidations in the control plane and deliver them through active dataplane subscriptions. | +| Resource subscriptions | Dataplane | TBD | Subscribe upstream and relay content-change notifications without durable subscription state. | +| Progress and cancellation | Dataplane | TBD | Keep progress on the originating response stream and immediately propagate cancellation upstream. | +| MRTR | Dataplane and MCP host | TBD | Transparently carry `InputRequiredResult`, `requestState`, and retry responses in the dataplane; perform user or model interaction in the host. | +| Tasks | Dataplane routing; backend state | TBD | Route `tasks/get`, `tasks/update`, and `tasks/cancel`; keep durable task state in the originating backend. The control plane owns enablement and policy. | +| OAuth and OIDC endpoints | Control plane | TBD | Own authorization-server discovery, login, consent, registration, token issuance, refresh, and step-up. | +| Access-token enforcement | Dataplane | TBD | Verify signature, expiry, audience, scope, and resource policy on every request. Never forward a downstream bearer token upstream. | +| MCP Apps metadata and assets | Control plane | TBD | Validate and version app manifests, UI assets, CSP, permissions, tool visibility, and origin-server associations. | +| MCP Apps asset delivery and calls | Dataplane | TBD | Serve `ui://` resources and route app-originated tool calls without executing UI HTML. | +| MCP Apps iframe and UI | MCP host | TBD | Own iframe sandboxing, CSP enforcement, `postMessage`, `ui/initialize`, permission prompts, and user consent. | +| Pagination | Control plane for list methods | TBD | Own opaque composite cursors and stable catalog snapshots. | +| Caching | Split | TBD | Cache discovery and lists in the control plane. Cache targeted resource reads in the dataplane only when authorization and `cacheScope` permit it. | +| Stdio adapters | Connector or control-plane tier | TBD | Keep process lifecycle and local adapter management out of the Rust dataplane. | +| Deprecated and legacy MCP | Control plane only | TBD | Do not add legacy sessions, SSE, Roots, Sampling, or Logging to the modern dataplane. | From 952cfb43857877a4dd825d1f9edb71aaf928c98c Mon Sep 17 00:00:00 2001 From: lucarlig Date: Thu, 13 Aug 2026 11:37:40 +0100 Subject: [PATCH 2/3] docs: add tentative MCP architecture flows Signed-off-by: lucarlig --- _context/wiki/mcp-capability-allocation.md | 188 +++++++++++++++++++-- 1 file changed, 176 insertions(+), 12 deletions(-) diff --git a/_context/wiki/mcp-capability-allocation.md b/_context/wiki/mcp-capability-allocation.md index 85b556d0..87087dcd 100644 --- a/_context/wiki/mcp-capability-allocation.md +++ b/_context/wiki/mcp-capability-allocation.md @@ -5,24 +5,26 @@ This table assigns the protocol surfaces across the ContextForge control plane, dataplane, upstream backends, and MCP host. -The control plane owns descriptive and durable control state. The dataplane owns -targeted execution, validation, and streaming. Upstream backends remain -authoritative for backend-created durable state, while the MCP host owns user -interaction and MCP App rendering. +The control plane owns descriptive and durable control state, upstream catalog +reconciliation, policy, and runtime snapshot compilation. The dataplane handles +all live modern MCP client requests. It serves discovery and list methods from +the published snapshot and routes targeted operations to one upstream backend. +Upstream backends remain authoritative for backend-created durable state, while +the MCP host owns user interaction and MCP App rendering. `Essential` identifies the initial scope. Advanced capabilities remain `TBD`; their recommended allocation records the intended direction without committing them to the initial implementation. -| MCP surface | Primary owner | Status | Recommended behavior | +| MCP surface | Authority and delivery | Status | Recommended behavior | | --- | --- | --- | --- | -| `server/discover` | Control plane | Essential | Produce the merged logical-server identity, instructions, capabilities, extensions, TTL, and cache scope. | -| `tools/list` | Control plane | Essential | Serve a materialized, policy-filtered catalog with stable names and composite pagination. Avoid synchronous fanout on every request. | +| `server/discover` | Control plane content; dataplane delivery | Essential | The control plane compiles the merged logical-server identity, instructions, capabilities, extensions, TTL, and cache scope. The dataplane serves the live request from the published snapshot. | +| `tools/list` | Control plane catalog; dataplane delivery | Essential | The control plane materializes and policy-filters the catalog. The dataplane serves stable names and pagination from the snapshot without synchronous upstream fanout. | | `tools/call` | Dataplane | Essential | Route one target, validate input and output, execute plugins, stream progress, and propagate cancellation and multi-round-trip requests (MRTR). | -| `resources/list` | Control plane | Essential | Catalog resources, filter them by authorization, and merge them deterministically. | -| `resources/templates/list` | Control plane | Essential | Catalog URI templates and their metadata. | +| `resources/list` | Control plane catalog; dataplane delivery | Essential | The control plane catalogs and policy-filters resources. The dataplane serves the deterministic snapshot. | +| `resources/templates/list` | Control plane catalog; dataplane delivery | Essential | The control plane catalogs URI templates and metadata. The dataplane serves the live request from the snapshot. | | `resources/read` | Dataplane | Essential | Route and stream potentially large, dynamic, or private content on the hot path. | -| `prompts/list` | Control plane | Essential | Catalog and policy-filter prompt definitions. | +| `prompts/list` | Control plane catalog; dataplane delivery | Essential | The control plane catalogs and policy-filters prompt definitions. The dataplane serves the live request from the snapshot. | | `prompts/get` | Dataplane | Essential | Render prompts that may be dynamic, expensive, private, or require MRTR. | | `completion/complete` | Dataplane | Essential | Route interactive, latency-sensitive completion requests using control-plane mappings. | | `subscriptions/listen` | Dataplane | TBD | Maintain long-lived streams and multiplex upstream notifications. | @@ -36,7 +38,169 @@ them to the initial implementation. | MCP Apps metadata and assets | Control plane | TBD | Validate and version app manifests, UI assets, CSP, permissions, tool visibility, and origin-server associations. | | MCP Apps asset delivery and calls | Dataplane | TBD | Serve `ui://` resources and route app-originated tool calls without executing UI HTML. | | MCP Apps iframe and UI | MCP host | TBD | Own iframe sandboxing, CSP enforcement, `postMessage`, `ui/initialize`, permission prompts, and user consent. | -| Pagination | Control plane for list methods | TBD | Own opaque composite cursors and stable catalog snapshots. | -| Caching | Split | TBD | Cache discovery and lists in the control plane. Cache targeted resource reads in the dataplane only when authorization and `cacheScope` permit it. | +| Pagination | Control plane snapshot; dataplane delivery | TBD | The control plane produces a stable ordered snapshot. The dataplane owns the client-facing cursor over that snapshot. | +| Caching | Split | TBD | Materialize discovery and lists in the control plane and cache the published snapshot locally in the dataplane. Cache targeted resource reads only when authorization and `cacheScope` permit it. | | Stdio adapters | Connector or control-plane tier | TBD | Keep process lifecycle and local adapter management out of the Rust dataplane. | | Deprecated and legacy MCP | Control plane only | TBD | Do not add legacy sessions, SSE, Roots, Sampling, or Logging to the modern dataplane. | + +## Upcoming Architecture (Tentative) + +> **Proposed direction, not an approved final design.** These sequences are +> intended to make the control-plane/dataplane boundary concrete enough for +> review. Details such as snapshot shape, invalidation transport, failure +> policy, cache lifetime, and notification behavior may change. + +The central proposal is: + +- the control plane performs management-time upstream discovery, persists the + catalog and policy, and publishes versioned runtime snapshots; +- the dataplane terminates every live modern MCP client request; +- discovery and list requests are answered from the published snapshot; +- targeted requests such as `tools/call` go directly from the dataplane to one + upstream MCP server; and +- Redis distributes state and invalidations. It is not an RPC or result bus. + +### 1. Create a Virtual Server and Select Capabilities + +```mermaid +sequenceDiagram + autonumber + actor User + participant UI as Admin UI or API + participant CP as Control Plane + participant DB as Control Plane DB + participant MCP1 as MCP Server 1 + participant MCP2 as MCP Server 2 + participant Redis + participant DP as Rust Dataplane + + User->>UI: Create virtual server + UI->>CP: Submit virtual server + CP->>DB: Store virtual server + + User->>UI: Assign MCP Server 1 and MCP Server 2 + UI->>CP: Update backend associations + CP->>DB: Store backend associations + + par Discover MCP Server 1 + CP->>MCP1: server/discover and list methods + MCP1-->>CP: Capabilities and catalog + and Discover MCP Server 2 + CP->>MCP2: server/discover and list methods + MCP2-->>CP: Capabilities and catalog + end + + CP->>DB: Reconcile normalized catalog + User->>UI: View available tools + UI->>CP: Request reconciled catalog + CP->>DB: Read catalog + DB-->>CP: inc, sum, dec, diff + CP-->>UI: Display available tools + + User->>UI: Allow inc and sum + UI->>CP: Update virtual server policy + CP->>DB: Store selected tools and policy + + CP->>CP: Compile user runtime snapshot + CP->>Redis: Atomically publish revision N + Redis-->>DP: Revision notification + DP->>Redis: Load revision N + DP->>DP: Replace local cache atomically +``` + +### 2. Discover the Server and List Tools + +```mermaid +sequenceDiagram + autonumber + participant Client as Modern MCP Client + participant Ingress + participant DP as Rust Dataplane + participant Cache as Local Cache + participant Redis + + Client->>Ingress: server/discover + Ingress->>DP: Forward modern MCP request + DP->>DP: Validate authentication and metadata + DP->>Cache: Get virtual server snapshot + + alt Snapshot available + Cache-->>DP: Snapshot revision N + else Snapshot missing or expired + Cache-->>DP: Cache miss + DP->>Redis: Read compiled snapshot + Redis-->>DP: Snapshot revision N + DP->>Cache: Store revision N + end + + DP-->>Client: Server identity and capabilities + Client->>Ingress: tools/list + Ingress->>DP: Forward modern MCP request + DP->>Cache: Read visible tools + Cache-->>DP: inc and sum + DP-->>Client: tools/list result + + Note over DP,Redis: Redis distributes compiled state + Note over DP: No live upstream call for discovery or lists +``` + +### 3. Call a Tool + +```mermaid +sequenceDiagram + autonumber + participant Client as Modern MCP Client + participant Ingress + participant DP as Rust Dataplane + participant Cache as Local Cache + participant CPEX as Policy and CPEX + participant MCP as Selected MCP Server + + Client->>Ingress: tools/call name inc + Ingress->>DP: Forward modern MCP request + DP->>DP: Validate authentication and metadata + DP->>Cache: Resolve exposed tool inc + Cache-->>DP: Backend MCP, upstream name inc, allowed + + DP->>CPEX: Run pre-call policy + CPEX-->>DP: Allow or modify request + DP->>MCP: tools/call name inc + MCP-->>DP: Tool result + DP->>CPEX: Run post-call policy + CPEX-->>DP: Allow or modify result + DP-->>Client: Return tool result directly + + Note over DP,MCP: Exactly one upstream is called + Note over DP: Control Plane, DB and Redis are not on this result path +``` + +### 4. Reconcile an Upstream Catalog Change + +```mermaid +sequenceDiagram + autonumber + participant MCP as MCP Server + participant CP as Control Plane Reconciler + participant DB as Control Plane DB + participant Redis + participant DP as Rust Dataplane + participant Client as Subscribed MCP Client + + MCP-->>CP: Tools list changed + CP->>MCP: Refresh discovery and tools list + MCP-->>CP: Updated catalog + CP->>DB: Reconcile catalog changes + CP->>CP: Recompile affected snapshots + CP->>Redis: Atomically publish revision N plus 1 + + Redis-->>DP: Revision notification + DP->>Redis: Load revision N plus 1 + DP->>DP: Replace local cache atomically + + opt Client subscribed to tool list changes + DP-->>Client: Tools list changed notification + end + + Client->>DP: tools/list + DP-->>Client: Updated list from local snapshot +``` From 2c3a0cf991987433d584360f5a74be54d4bda0ad Mon Sep 17 00:00:00 2001 From: lucarlig Date: Thu, 13 Aug 2026 11:43:09 +0100 Subject: [PATCH 3/3] docs: focus Phase 3 architecture on flows Signed-off-by: lucarlig --- _context/wiki/SUMMARY.md | 5 +- _context/wiki/index.md | 2 +- _context/wiki/mcp-capability-allocation.md | 65 ++-------------------- 3 files changed, 9 insertions(+), 63 deletions(-) diff --git a/_context/wiki/SUMMARY.md b/_context/wiki/SUMMARY.md index d9627750..b42c69ea 100644 --- a/_context/wiki/SUMMARY.md +++ b/_context/wiki/SUMMARY.md @@ -11,10 +11,13 @@ - [Architecture](architecture.md) - [MCP Routing Semantics](routing.md) -- [MCP Capability Allocation](mcp-capability-allocation.md) - [Security Model](security.md) - [Failure Modes](failure-modes.md) +# Upcoming + +- [Phase 3 Final Architecture](mcp-capability-allocation.md) + # Operations - [Configuration Reference](config.md) diff --git a/_context/wiki/index.md b/_context/wiki/index.md index bd48ce63..c46a7c70 100644 --- a/_context/wiki/index.md +++ b/_context/wiki/index.md @@ -13,7 +13,7 @@ then follow only the links that are relevant. | [preferences.md](preferences.md) | Working standards, code style, logging rules, branch naming, AI interaction preferences | | [architecture.md](architecture.md) | Middleware stack order, pipeline shape, module boundaries, state ownership, executor shapes | | [routing.md](routing.md) | Backend prefix contract, list/routed ops, federated pagination, session state, capability merge | -| [mcp-capability-allocation.md](mcp-capability-allocation.md) | Ownership, initial scope, and recommended allocation of MCP protocol surfaces | +| [mcp-capability-allocation.md](mcp-capability-allocation.md) | Tentative Phase 3 final architecture shown through four Mermaid sequences | | [failure-modes.md](failure-modes.md) | HTTP/MCP/routing/backend/plugin failure table — exact HTTP codes and JSON-RPC errors | | [config.md](config.md) | Key CLI flags, JWT claims, UserConfig shape, plugin config, telemetry debugging, startup validation, local observability stack | | [deployment.md](deployment.md) | Deployment checklist, health endpoint caveat, nginx routing, TLS choices, session affinity, Redis availability, image pinning | diff --git a/_context/wiki/mcp-capability-allocation.md b/_context/wiki/mcp-capability-allocation.md index 87087dcd..a4f7734c 100644 --- a/_context/wiki/mcp-capability-allocation.md +++ b/_context/wiki/mcp-capability-allocation.md @@ -1,64 +1,7 @@ -# MCP Capability Allocation - -This table assigns the -[MCP 2026-07-28](https://modelcontextprotocol.io/specification/2026-07-28) -protocol surfaces across the ContextForge control plane, dataplane, upstream -backends, and MCP host. - -The control plane owns descriptive and durable control state, upstream catalog -reconciliation, policy, and runtime snapshot compilation. The dataplane handles -all live modern MCP client requests. It serves discovery and list methods from -the published snapshot and routes targeted operations to one upstream backend. -Upstream backends remain authoritative for backend-created durable state, while -the MCP host owns user interaction and MCP App rendering. - -`Essential` identifies the initial scope. Advanced capabilities remain `TBD`; -their recommended allocation records the intended direction without committing -them to the initial implementation. - -| MCP surface | Authority and delivery | Status | Recommended behavior | -| --- | --- | --- | --- | -| `server/discover` | Control plane content; dataplane delivery | Essential | The control plane compiles the merged logical-server identity, instructions, capabilities, extensions, TTL, and cache scope. The dataplane serves the live request from the published snapshot. | -| `tools/list` | Control plane catalog; dataplane delivery | Essential | The control plane materializes and policy-filters the catalog. The dataplane serves stable names and pagination from the snapshot without synchronous upstream fanout. | -| `tools/call` | Dataplane | Essential | Route one target, validate input and output, execute plugins, stream progress, and propagate cancellation and multi-round-trip requests (MRTR). | -| `resources/list` | Control plane catalog; dataplane delivery | Essential | The control plane catalogs and policy-filters resources. The dataplane serves the deterministic snapshot. | -| `resources/templates/list` | Control plane catalog; dataplane delivery | Essential | The control plane catalogs URI templates and metadata. The dataplane serves the live request from the snapshot. | -| `resources/read` | Dataplane | Essential | Route and stream potentially large, dynamic, or private content on the hot path. | -| `prompts/list` | Control plane catalog; dataplane delivery | Essential | The control plane catalogs and policy-filters prompt definitions. The dataplane serves the live request from the snapshot. | -| `prompts/get` | Dataplane | Essential | Render prompts that may be dynamic, expensive, private, or require MRTR. | -| `completion/complete` | Dataplane | Essential | Route interactive, latency-sensitive completion requests using control-plane mappings. | -| `subscriptions/listen` | Dataplane | TBD | Maintain long-lived streams and multiplex upstream notifications. | -| List-change events | Control plane to dataplane | TBD | Generate normalized catalog invalidations in the control plane and deliver them through active dataplane subscriptions. | -| Resource subscriptions | Dataplane | TBD | Subscribe upstream and relay content-change notifications without durable subscription state. | -| Progress and cancellation | Dataplane | TBD | Keep progress on the originating response stream and immediately propagate cancellation upstream. | -| MRTR | Dataplane and MCP host | TBD | Transparently carry `InputRequiredResult`, `requestState`, and retry responses in the dataplane; perform user or model interaction in the host. | -| Tasks | Dataplane routing; backend state | TBD | Route `tasks/get`, `tasks/update`, and `tasks/cancel`; keep durable task state in the originating backend. The control plane owns enablement and policy. | -| OAuth and OIDC endpoints | Control plane | TBD | Own authorization-server discovery, login, consent, registration, token issuance, refresh, and step-up. | -| Access-token enforcement | Dataplane | TBD | Verify signature, expiry, audience, scope, and resource policy on every request. Never forward a downstream bearer token upstream. | -| MCP Apps metadata and assets | Control plane | TBD | Validate and version app manifests, UI assets, CSP, permissions, tool visibility, and origin-server associations. | -| MCP Apps asset delivery and calls | Dataplane | TBD | Serve `ui://` resources and route app-originated tool calls without executing UI HTML. | -| MCP Apps iframe and UI | MCP host | TBD | Own iframe sandboxing, CSP enforcement, `postMessage`, `ui/initialize`, permission prompts, and user consent. | -| Pagination | Control plane snapshot; dataplane delivery | TBD | The control plane produces a stable ordered snapshot. The dataplane owns the client-facing cursor over that snapshot. | -| Caching | Split | TBD | Materialize discovery and lists in the control plane and cache the published snapshot locally in the dataplane. Cache targeted resource reads only when authorization and `cacheScope` permit it. | -| Stdio adapters | Connector or control-plane tier | TBD | Keep process lifecycle and local adapter management out of the Rust dataplane. | -| Deprecated and legacy MCP | Control plane only | TBD | Do not add legacy sessions, SSE, Roots, Sampling, or Logging to the modern dataplane. | - -## Upcoming Architecture (Tentative) - -> **Proposed direction, not an approved final design.** These sequences are -> intended to make the control-plane/dataplane boundary concrete enough for -> review. Details such as snapshot shape, invalidation transport, failure -> policy, cache lifetime, and notification behavior may change. - -The central proposal is: - -- the control plane performs management-time upstream discovery, persists the - catalog and policy, and publishes versioned runtime snapshots; -- the dataplane terminates every live modern MCP client request; -- discovery and list requests are answered from the published snapshot; -- targeted requests such as `tools/call` go directly from the dataplane to one - upstream MCP server; and -- Redis distributes state and invalidations. It is not an RPC or result bus. +# Phase 3 Final Architecture + +> **Tentative:** this is the proposed Phase 3 final architecture and remains +> subject to review. ### 1. Create a Virtual Server and Select Capabilities