From 4ea68b45e8d5f555873c0e6d947cf3a4f31c8cdb Mon Sep 17 00:00:00 2001 From: jedigalt <201954051+jedigalt@users.noreply.github.com> Date: Mon, 2 Nov 2026 13:37:44 +0000 Subject: [PATCH 1/7] [needs-review] docs(release-notes): week of 2026-10-26 --- src/pages/docs/release-notes.mdx | 50 ++++++++++++++++++++++++++++++++ 1 file changed, 50 insertions(+) diff --git a/src/pages/docs/release-notes.mdx b/src/pages/docs/release-notes.mdx index ca9500c1e..3eab42f78 100644 --- a/src/pages/docs/release-notes.mdx +++ b/src/pages/docs/release-notes.mdx @@ -5,6 +5,56 @@ description: "Latest Future AGI release notes covering new features, improvement {/* release-notes:insert-below — automation inserts new releases here; do not remove */} +## Week of 2026-10-26 + +
+ +
Features
+ +- **Share and Export Error Feed Issues:** Share an Error Feed issue through a link anyone can open with no sign-in, or restrict it to people you add by email, who then need to sign in to view it. From an issue page you can also export its traces as a CSV, the latest 200, ready to upload into a dataset, and copying the cluster ID now shows a confirmation instead of a toast. + +- **MCP Tools for Error Feed:** The Future AGI MCP server now offers read-only tools to list Error Feed issues, pull the data for one, and debug a cluster, so an AI assistant or script can work with Error Feed without opening the app. + +- **OTLP Ingestion Accepts Gzip and a Project Header:** The trace ingestion endpoint now accepts gzip-compressed OTLP/HTTP uploads, which many OpenTelemetry SDKs and collectors send by default; a compressed body is decompressed under the same size limit as an uncompressed one, with a nesting check that rejects a malformed payload before it is fully decoded. For destinations that can only set request headers, not resource attributes, the project can now be named with an `X-Project-Name` header (alias `FI-Project-Name`), so they can send straight into Observe without a collector in front. + +- **Observe Reads OpenRouter Broadcast Traces in Full:** Traces sent from OpenRouter Broadcast now show their prompt and completion content instead of leaving the Input and Output panels empty. Token counts and cost are summed across a trace's spans so the trace list and session list are no longer blank, small costs under a cent show with enough precision instead of rounding to zero, and OpenRouter's own pricing is used when a model has no cost reported directly. + +- **Choose Where a Rerun Lands:** Run again now lets you choose whether selected calls are rerun inside the current run or start a new run. Rerunning in place replaces only the rows you reran, while the earlier attempts stay in the backend, and a second rerun is refused while one is already in progress. + +- **Reliability Reporting for Repeated Trials:** A simulation run with more than one trial per scenario now reports pass^1 (the average, across scenarios, of each scenario's share of trials that passed) and pass^k (the share of scenarios that passed every trial) with a 95% interval on the run header, and each scenario row shows how many of its trials passed. + +- **More Realistic Simulated Callers:** A simulated caller's voice now matches the persona's age and speaks in the persona's own language across many languages and accents, and a caller written as frustrated actually sounds frustrated instead of only using frustrated wording. Sub-goal grading now reads the call recording directly instead of only the transcript, and checks that a step produced the correct values for that caller rather than only that the step happened, so a wrong value can now fail a sub-goal that previously passed on activity alone. + +- **Environments Check Themselves Against the Agent:** When an environment finishes building, the builder now runs one call per distinct solution path against the live agent and corrects a wrong reference answer on its own instead of leaving it for your first real run to catch. + +- **Personas Tab in Environments:** Personas now has its own tab inside Environments. + +- **Perplexity Preset and Named Custom Providers:** Add Provider now offers Perplexity as a built-in preset, and an organization can add more than one custom provider, each with its own name. + +
Bugs/Improvements
+ +- **Simulated Caller Openings:** A simulated caller no longer reads the scenario's own instruction out loud as its opening line. When no usable opening is available it greets normally instead of ending the call, and it retries in the caller's own language. + +- **Scenario Renames Keep Their History:** Renaming a scenario in the environment builder chat keeps its existing identity, so its run history stays linked to it instead of being treated as a new scenario. + +- **Invites Succeed Even When the Email Fails to Send:** Inviting a team member now creates the member and workspace invitation even if the invitation email fails to send, with a delivery warning shown instead of the whole invite failing. A batch of invites keeps the members it already created when a later entry in the batch fails. + +- **Invite Resend Now Extends Expiry Everywhere:** Resending an invite from the members screen or through a bulk resend now resets its expiry, matching the single-invite resend path. A resent invite no longer shows as expired right after you resend it. + +- **Custom Google Provider Discovery Keeps Your Host:** A custom Google-format provider's model discovery now calls the host you configured instead of Google's public endpoint, so your API key is never sent anywhere else, and custom Google-compatible providers can discover their own models. + +- **Gateway Streams Report Errors and Cost:** A streamed request cut short by a timeout or an upstream failure now ends with a proper error event instead of stopping silently, and is recorded as a failed request. Every route now reports a cost, including a cut stream, priced from the output it produced before it stopped. + +- **One Request Log Row Per Request:** Request Logs no longer show the same request more than once. Tool calls on the Messages route now work for models that need the Responses API underneath, and a malformed metadata header is flagged instead of being silently dropped. + +- **Budgets, Guardrails and Alerts Cover More Routes:** Budgets now price usage against the full pricing catalogue, and input guardrails now see the request on the native Messages, Responses and GenAI routes. A response an output guardrail blocks now returns 403 with its usage still priced and is kept out of the cache, and a budget-exceeded webhook fires right away while the matching email alert goes out once per cooldown instead of on every request. + +- **Pricing Catalogue Refresh:** The bundled model pricing and capability catalogue has been refreshed, so current models are priced instead of showing zero cost, and older model IDs that had lost their price in an earlier refresh are priced again. A model with no capability flag recorded for tools, vision or structured output is no longer refused that capability outright, a resold model is now priced under the reseller's own catalogue entry instead of at zero, and Fireworks and Together models are now looked up under the price table's own naming instead of being missed by a mismatched provider prefix. + +
+ +{/* SKIPPED: internal-only changes with no customer-visible effect (API contract and schema-validation fixes, OpenAPI/MCP manifest regeneration, internal observability dashboards and alerting, end-to-end and build-sandbox test infrastructure, a connected-agents list filter that shipped and was reverted in the same window, and release bookkeeping). Held for confirmation: restricting part of Simulate's navigation to a single unified workspace view for newly created organizations (a staged rollout decision, not on its own a customer-configurable capability), and a trace-ingestion hardening item that was still being scoped and not confirmed shipped. */} + ## Week of 2026-10-19
From c8d48860237d5b2357e729e04972f56bad61319a Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 4 Nov 2026 05:18:15 +0000 Subject: [PATCH 2/7] docs(release-notes): week of 2026-10-26, apply review findings Grounds every bullet in this week's PRs, corrects the MCP, OpenRouter pricing, rename and builder-check descriptions, adds the Observe sorting and filtering changes and the shared-link and navigation changes, and adds Breaking Changes for the run analytics interval shape and trace ingestion Content-Encoding handling. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01CFSMeVkmL3svj9cTBumVXR --- src/pages/docs/release-notes.mdx | 48 +++++++++++++++++++------------- 1 file changed, 28 insertions(+), 20 deletions(-) diff --git a/src/pages/docs/release-notes.mdx b/src/pages/docs/release-notes.mdx index 3eab42f78..f2415c6e7 100644 --- a/src/pages/docs/release-notes.mdx +++ b/src/pages/docs/release-notes.mdx @@ -11,49 +11,57 @@ description: "Latest Future AGI release notes covering new features, improvement
Features
-- **Share and Export Error Feed Issues:** Share an Error Feed issue through a link anyone can open with no sign-in, or restrict it to people you add by email, who then need to sign in to view it. From an issue page you can also export its traces as a CSV, the latest 200, ready to upload into a dataset, and copying the cluster ID now shows a confirmation instead of a toast. +- **Share and Export Error Feed Issues:** Share an Error Feed issue through the same Share dialog as traces, with anyone who has the link or only people you add by email. A shared link opens the issue read-only, with its Overview, Traces, Trends and Fix tabs, and shared traces open in the read-only trace drawer. From an issue page you can also export its traces, the latest 200, as a CSV ready to upload into a dataset, and Copy cluster ID briefly shows a check once the ID is copied. Shared links of every type now say Sign in required, Access denied, Link not found or Link expired when they cannot open, and keep a signed-out viewer on the link. -- **MCP Tools for Error Feed:** The Future AGI MCP server now offers read-only tools to list Error Feed issues, pull the data for one, and debug a cluster, so an AI assistant or script can work with Error Feed without opening the app. +- **MCP Tools for Error Feed:** The Future AGI MCP server's read-only `error_feed` tool group adds `get_error_cluster_overview`, which returns a cluster's pattern summary, daily error counts and representative traces, and `list_error_cluster_traces`, which lists a cluster's traces with score, latency, tokens and cost. `get_error_cluster` now also returns a cluster's stored root-cause analysis, so an AI assistant can go from the issue list to its analysis, pattern and traces. A connection created before `error_feed` became a default group needs that group turned on to see the new tools. -- **OTLP Ingestion Accepts Gzip and a Project Header:** The trace ingestion endpoint now accepts gzip-compressed OTLP/HTTP uploads, which many OpenTelemetry SDKs and collectors send by default; a compressed body is decompressed under the same size limit as an uncompressed one, with a nesting check that rejects a malformed payload before it is fully decoded. For destinations that can only set request headers, not resource attributes, the project can now be named with an `X-Project-Name` header (alias `FI-Project-Name`), so they can send straight into Observe without a collector in front. +- **OTLP Ingestion Accepts Gzip and a Project Header:** The trace ingestion endpoint now accepts gzip-compressed OTLP/HTTP uploads, protobuf or JSON, with the 16 MiB limit applied both to the body as sent and after decompression. OTLP over gRPC still accepts uncompressed requests only, so a collector exporting over gRPC should keep compression off. The project can also be named with an `X-Project-Name` header (alias `FI-Project-Name`; lowercase `x-project-name` metadata over gRPC), carrying one name of up to 255 characters, which fills in the project only where a resource does not already set its `project_name` attribute. -- **Observe Reads OpenRouter Broadcast Traces in Full:** Traces sent from OpenRouter Broadcast now show their prompt and completion content instead of leaving the Input and Output panels empty. Token counts and cost are summed across a trace's spans so the trace list and session list are no longer blank, small costs under a cent show with enough precision instead of rounding to zero, and OpenRouter's own pricing is used when a model has no cost reported directly. +- **Fuller Traces in Observe, Including OpenRouter Broadcast:** Spans that carry their prompt and completion in the OpenTelemetry GenAI content keys, as OpenRouter Broadcast sends them, now show that content in the Input and Output panels, and the older `gen_ai.usage.prompt_tokens` and `completion_tokens` names are read for token counts. Trace and session lists use each trace's whole usage, so OpenRouter Broadcast traces show their tokens and cost, and token and cost filters match the totals shown. Sorting a single project's trace list by cost or tokens, with date or usage filters, orders every matching trace before paging, and small costs under a cent keep their precision. A cost the sender reports is stored as sent, and a span with no reported cost is priced from the model's entry in the price table. -- **Choose Where a Rerun Lands:** Run again now lets you choose whether selected calls are rerun inside the current run or start a new run. Rerunning in place replaces only the rows you reran, while the earlier attempts stay in the backend, and a second rerun is refused while one is already in progress. +- **Choose Where a Re-run Lands:** The Re-run menu on an Environments run now offers **Re-run in this run** alongside **Run as a new simulation**. Re-running in this run replaces only the selected calls' results, and a new re-run is refused until the current one and its grading have finished, with the API answering 409 `rerun_in_progress`. -- **Reliability Reporting for Repeated Trials:** A simulation run with more than one trial per scenario now reports pass^1 (the average, across scenarios, of each scenario's share of trials that passed) and pass^k (the share of scenarios that passed every trial) with a 95% interval on the run header, and each scenario row shows how many of its trials passed. +- **Reliability Reporting for Repeated Trials:** A simulation run with more than one trial per scenario now reports pass^1 with a 95% range and, when every scenario ran the same number of trials, pass^k (the share of scenarios that passed every trial) on the run header, and each scenario row shows how many of its trials passed. Runs with fewer than 10 scenarios show counts instead of rates and no range. -- **More Realistic Simulated Callers:** A simulated caller's voice now matches the persona's age and speaks in the persona's own language across many languages and accents, and a caller written as frustrated actually sounds frustrated instead of only using frustrated wording. Sub-goal grading now reads the call recording directly instead of only the transcript, and checks that a step produced the correct values for that caller rather than only that the step happened, so a wrong value can now fail a sub-goal that previously passed on activity alone. +- **More Realistic Simulated Callers:** A simulated caller's voice now matches the persona's age and speaks in the persona's own language across many languages and accents, and a caller written as frustrated sounds frustrated. Sub-goal grading reads the call recording directly, not only the transcript, and checks that a step produced the correct values for that caller, not only that the step happened. {/* REVIEW: not found in the platform repo's PRs this week; kept from the CI draft, which also scanned agent-learning-kit. Confirm the source. */} -- **Environments Check Themselves Against the Agent:** When an environment finishes building, the builder now runs one call per distinct solution path against the live agent and corrects a wrong reference answer on its own instead of leaving it for your first real run to catch. +- **Environments Check Themselves Against Your Agent:** For a repository agent built in the environment's builder chat, the builder now plays one call per distinct solution path against the live agent once the build completes, corrects a wrong reference answer on its own, and reports each correction and any fault it found in the agent in the chat. -- **Personas Tab in Environments:** Personas now has its own tab inside Environments. +- **Simulate Navigation Centers on Environments:** Personas now has its own tab inside Environments, and the old Personas page opens it. The Agent Definition list opens Connected agents, and Simulate's home and the old Harness and Simulator Agent links open Environments. -- **Perplexity Preset and Named Custom Providers:** Add Provider now offers Perplexity as a built-in preset, and an organization can add more than one custom provider, each with its own name. +- **Perplexity Preset and Named Custom Providers:** Add Provider now offers Perplexity as a built-in preset, custom providers can be given their own names, and a model written as `/` is routed to the provider it names.
Bugs/Improvements
-- **Simulated Caller Openings:** A simulated caller no longer reads the scenario's own instruction out loud as its opening line. When no usable opening is available it greets normally instead of ending the call, and it retries in the caller's own language. +- **Simulated Caller Openings:** A simulated caller opens with a natural line rather than the scenario's own instruction. When no usable opening is available it greets normally and keeps the call going, in the caller's own language. {/* REVIEW: not found in the platform repo's PRs this week; kept from the CI draft, which also scanned agent-learning-kit. Confirm the source. */} -- **Scenario Renames Keep Their History:** Renaming a scenario in the environment builder chat keeps its existing identity, so its run history stays linked to it instead of being treated as a new scenario. +- **Scenarios Keep Their Rows Through Renames:** When an environment's suite is published, each scenario is matched to its row by its key first. A renamed scenario keeps its row and history, a new scenario that reuses a renamed scenario's old name gets a row of its own, and a suite with a repeated key saves. -- **Invites Succeed Even When the Email Fails to Send:** Inviting a team member now creates the member and workspace invitation even if the invitation email fails to send, with a delivery warning shown instead of the whole invite failing. A batch of invites keeps the members it already created when a later entry in the batch fails. +- **Invites Go Through When the Email Fails to Send:** Inviting a team member creates the member and the workspace invitation even when the invitation email fails to send, and shows a delivery warning. A batch of invites keeps the members it already created when a later entry fails. -- **Invite Resend Now Extends Expiry Everywhere:** Resending an invite from the members screen or through a bulk resend now resets its expiry, matching the single-invite resend path. A resent invite no longer shows as expired right after you resend it. +- **Resending an Invite Resets Its Expiry Everywhere:** Resending an invite from the members screen or through a bulk resend resets its expiry, the same as resending a single invite, so a resent invite shows as Pending with a fresh expiry. -- **Custom Google Provider Discovery Keeps Your Host:** A custom Google-format provider's model discovery now calls the host you configured instead of Google's public endpoint, so your API key is never sent anywhere else, and custom Google-compatible providers can discover their own models. +- **Custom Google Provider Discovery Uses Your Host:** Model discovery for a custom Google-format provider calls the host you configured, so custom Google-compatible providers can discover their own models. -- **Gateway Streams Report Errors and Cost:** A streamed request cut short by a timeout or an upstream failure now ends with a proper error event instead of stopping silently, and is recorded as a failed request. Every route now reports a cost, including a cut stream, priced from the output it produced before it stopped. +- **Gateway Streams Report Errors and Cost:** A streamed request cut short by a timeout or an upstream failure ends with an error event and is recorded as a failed request. Every route reports a cost, including a cut stream, priced from the output it produced before it stopped. -- **One Request Log Row Per Request:** Request Logs no longer show the same request more than once. Tool calls on the Messages route now work for models that need the Responses API underneath, and a malformed metadata header is flagged instead of being silently dropped. +- **One Request Log Row Per Request:** Request Logs show each request once. Upgrading a self-hosted deployment removes rows already saved twice and then builds an index without blocking writes, which can take a while on a large request-log table. -- **Budgets, Guardrails and Alerts Cover More Routes:** Budgets now price usage against the full pricing catalogue, and input guardrails now see the request on the native Messages, Responses and GenAI routes. A response an output guardrail blocks now returns 403 with its usage still priced and is kept out of the cache, and a budget-exceeded webhook fires right away while the matching email alert goes out once per cooldown instead of on every request. +- **Messages Route Tool Calls and Metadata Header:** gpt-5.6 tool calls on the Messages route are served through the Responses API, and a malformed `x-agentcc-metadata` header is flagged. -- **Pricing Catalogue Refresh:** The bundled model pricing and capability catalogue has been refreshed, so current models are priced instead of showing zero cost, and older model IDs that had lost their price in an earlier refresh are priced again. A model with no capability flag recorded for tools, vision or structured output is no longer refused that capability outright, a resold model is now priced under the reseller's own catalogue entry instead of at zero, and Fireworks and Together models are now looked up under the price table's own naming instead of being missed by a mismatched provider prefix. +- **Budgets, Guardrails and Alerts Cover More Routes:** Budgets price usage against the full pricing catalogue, and input guardrails see the request on the native Messages, Responses and GenAI routes. A non-streamed response an output guardrail blocks returns 403 with its usage priced and is not cached, and a budget-exceeded webhook fires while the matching email alert goes out once per cooldown. + +- **Pricing Catalogue Refresh:** The bundled model pricing and capability catalogue has been refreshed, so current models are priced, and pinned older model IDs such as the Claude 3.5 Sonnet and Haiku IDs are priced again. A capability such as tools, vision or structured output is refused only when the catalogue states a model lacks it. A resold model is priced under the reseller's own entry, and a Gemini model the catalogue lists only under `gemini/` or `vertex_ai/` is priced on a Gemini provider of any name. If your organization set a custom price for a model the refresh newly lists without a price, check that model's cost. + +
Breaking Changes
+ +- **Run Analytics Interval Shape:** In the run analytics API, `pass_rate_interval` is now `{low, high, scenarios, evaluated, resamples}`, and `effective_n` and `clusters` have been removed. Migration: read the new fields and drop any use of `effective_n` or `clusters`. + +- **Trace Ingestion Checks Content-Encoding:** The trace ingestion endpoint now honors `Content-Encoding`. A body labelled `gzip` that is not valid gzip returns 400, an encoding other than gzip or identity, or more than one gzip layer, returns 415, and a gzip body over 16 MiB after decompression returns 413. Migration: send OTLP/HTTP uncompressed or gzip-compressed, and label it accurately.
-{/* SKIPPED: internal-only changes with no customer-visible effect (API contract and schema-validation fixes, OpenAPI/MCP manifest regeneration, internal observability dashboards and alerting, end-to-end and build-sandbox test infrastructure, a connected-agents list filter that shipped and was reverted in the same window, and release bookkeeping). Held for confirmation: restricting part of Simulate's navigation to a single unified workspace view for newly created organizations (a staged rollout decision, not on its own a customer-configurable capability), and a trace-ingestion hardening item that was still being scoped and not confirmed shipped. */} +{/* SKIPPED: internal-only changes with no customer-visible effect (API contract and schema fixes, OpenAPI and MCP manifest regeneration, internal observability dashboards, end-to-end and build-sandbox test infrastructure, a connected-agents list filter that shipped and was undone in the same window, and release bookkeeping). Held for confirmation: limiting Simulate to Environments for newly created organizations, and a further trace-ingestion hardening item not yet shipped. */} ## Week of 2026-10-19 From d0c269dd8ffc3d346dbbc303ad02809d34390a38 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 4 Nov 2026 06:27:10 +0000 Subject: [PATCH 3/7] docs(release-notes): week of 2026-10-26, tighten to house length Cuts each bullet to one or two sentences (average 36 words, longest 54), matching the page's earlier entries. No facts added or removed beyond trimming secondary detail. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01CFSMeVkmL3svj9cTBumVXR --- src/pages/docs/release-notes.mdx | 46 ++++++++++++++++---------------- 1 file changed, 23 insertions(+), 23 deletions(-) diff --git a/src/pages/docs/release-notes.mdx b/src/pages/docs/release-notes.mdx index f2415c6e7..d4d90fe7f 100644 --- a/src/pages/docs/release-notes.mdx +++ b/src/pages/docs/release-notes.mdx @@ -11,57 +11,57 @@ description: "Latest Future AGI release notes covering new features, improvement
Features
-- **Share and Export Error Feed Issues:** Share an Error Feed issue through the same Share dialog as traces, with anyone who has the link or only people you add by email. A shared link opens the issue read-only, with its Overview, Traces, Trends and Fix tabs, and shared traces open in the read-only trace drawer. From an issue page you can also export its traces, the latest 200, as a CSV ready to upload into a dataset, and Copy cluster ID briefly shows a check once the ID is copied. Shared links of every type now say Sign in required, Access denied, Link not found or Link expired when they cannot open, and keep a signed-out viewer on the link. +- **Share and Export Error Feed Issues:** Share an Error Feed issue by link, with anyone or only people you add by email, and it opens read-only with its Overview, Traces, Trends and Fix tabs. You can also export an issue's latest 200 traces as a CSV ready to upload into a dataset. -- **MCP Tools for Error Feed:** The Future AGI MCP server's read-only `error_feed` tool group adds `get_error_cluster_overview`, which returns a cluster's pattern summary, daily error counts and representative traces, and `list_error_cluster_traces`, which lists a cluster's traces with score, latency, tokens and cost. `get_error_cluster` now also returns a cluster's stored root-cause analysis, so an AI assistant can go from the issue list to its analysis, pattern and traces. A connection created before `error_feed` became a default group needs that group turned on to see the new tools. +- **MCP Tools for Error Feed:** The MCP server's read-only `error_feed` group adds `get_error_cluster_overview` and `list_error_cluster_traces`, and `get_error_cluster` now returns the stored root-cause analysis. Connections created before `error_feed` became a default group need to turn it on. -- **OTLP Ingestion Accepts Gzip and a Project Header:** The trace ingestion endpoint now accepts gzip-compressed OTLP/HTTP uploads, protobuf or JSON, with the 16 MiB limit applied both to the body as sent and after decompression. OTLP over gRPC still accepts uncompressed requests only, so a collector exporting over gRPC should keep compression off. The project can also be named with an `X-Project-Name` header (alias `FI-Project-Name`; lowercase `x-project-name` metadata over gRPC), carrying one name of up to 255 characters, which fills in the project only where a resource does not already set its `project_name` attribute. +- **Gzip and a Project Header for OTLP Ingestion:** Trace ingestion accepts gzip-compressed OTLP/HTTP, with the 16 MiB limit applied before and after decompression; gRPC stays uncompressed. An `X-Project-Name` header (alias `FI-Project-Name`) sets the project when a resource has no `project_name` attribute. -- **Fuller Traces in Observe, Including OpenRouter Broadcast:** Spans that carry their prompt and completion in the OpenTelemetry GenAI content keys, as OpenRouter Broadcast sends them, now show that content in the Input and Output panels, and the older `gen_ai.usage.prompt_tokens` and `completion_tokens` names are read for token counts. Trace and session lists use each trace's whole usage, so OpenRouter Broadcast traces show their tokens and cost, and token and cost filters match the totals shown. Sorting a single project's trace list by cost or tokens, with date or usage filters, orders every matching trace before paging, and small costs under a cent keep their precision. A cost the sender reports is stored as sent, and a span with no reported cost is priced from the model's entry in the price table. +- **Fuller Traces in Observe:** Spans that carry content in the OpenTelemetry GenAI keys, as OpenRouter Broadcast sends them, now fill the Input and Output panels, and trace lists show each trace's whole token count and cost. Sorting by cost or tokens orders every matching trace, and costs under a cent keep their precision. -- **Choose Where a Re-run Lands:** The Re-run menu on an Environments run now offers **Re-run in this run** alongside **Run as a new simulation**. Re-running in this run replaces only the selected calls' results, and a new re-run is refused until the current one and its grading have finished, with the API answering 409 `rerun_in_progress`. +- **Choose Where a Re-run Lands:** The Re-run menu on an Environments run offers **Re-run in this run** alongside **Run as a new simulation**, replacing only the selected calls' results. Another re-run is refused until the current one and its grading finish. -- **Reliability Reporting for Repeated Trials:** A simulation run with more than one trial per scenario now reports pass^1 with a 95% range and, when every scenario ran the same number of trials, pass^k (the share of scenarios that passed every trial) on the run header, and each scenario row shows how many of its trials passed. Runs with fewer than 10 scenarios show counts instead of rates and no range. +- **Reliability Reporting for Repeated Trials:** Runs with several trials per scenario report pass^1 with a 95% range and, when every scenario ran the same number of trials, pass^k, the share of scenarios that passed every trial. Each scenario row shows how many of its trials passed. -- **More Realistic Simulated Callers:** A simulated caller's voice now matches the persona's age and speaks in the persona's own language across many languages and accents, and a caller written as frustrated sounds frustrated. Sub-goal grading reads the call recording directly, not only the transcript, and checks that a step produced the correct values for that caller, not only that the step happened. {/* REVIEW: not found in the platform repo's PRs this week; kept from the CI draft, which also scanned agent-learning-kit. Confirm the source. */} +- **More Realistic Simulated Callers:** Simulated callers match the persona's age, language and accent, and sound frustrated when written that way. Sub-goal grading reads the call recording and checks that each step produced the right values for that caller. {/* REVIEW: source likely agent-learning-kit, not scanned in this pass. Confirm. */} -- **Environments Check Themselves Against Your Agent:** For a repository agent built in the environment's builder chat, the builder now plays one call per distinct solution path against the live agent once the build completes, corrects a wrong reference answer on its own, and reports each correction and any fault it found in the agent in the chat. +- **Environments Check Themselves Against Your Agent:** After building an environment for a repository agent in the builder chat, the builder plays one call per solution path against the live agent, corrects wrong reference answers, and reports what it changed in the chat. -- **Simulate Navigation Centers on Environments:** Personas now has its own tab inside Environments, and the old Personas page opens it. The Agent Definition list opens Connected agents, and Simulate's home and the old Harness and Simulator Agent links open Environments. +- **Simulate Navigation Centers on Environments:** Personas is now a tab inside Environments. Old links to Personas, Agent Definitions and Simulate open their new homes in Environments or Connected agents. -- **Perplexity Preset and Named Custom Providers:** Add Provider now offers Perplexity as a built-in preset, custom providers can be given their own names, and a model written as `/` is routed to the provider it names. +- **Perplexity Preset and Named Custom Providers:** Add Provider offers a Perplexity preset, custom providers can have their own names, and a `/` model routes to the provider it names.
Bugs/Improvements
-- **Simulated Caller Openings:** A simulated caller opens with a natural line rather than the scenario's own instruction. When no usable opening is available it greets normally and keeps the call going, in the caller's own language. {/* REVIEW: not found in the platform repo's PRs this week; kept from the CI draft, which also scanned agent-learning-kit. Confirm the source. */} +- **Simulated Caller Openings:** Simulated callers open with a natural greeting in their own language and keep the call going when no scripted opening fits. {/* REVIEW: source likely agent-learning-kit, not scanned in this pass. Confirm. */} -- **Scenarios Keep Their Rows Through Renames:** When an environment's suite is published, each scenario is matched to its row by its key first. A renamed scenario keeps its row and history, a new scenario that reuses a renamed scenario's old name gets a row of its own, and a suite with a repeated key saves. +- **Scenarios Keep Their Rows Through Renames:** Published scenarios are matched to their rows by key first, so a renamed scenario keeps its history and a new scenario reusing the old name gets its own row. -- **Invites Go Through When the Email Fails to Send:** Inviting a team member creates the member and the workspace invitation even when the invitation email fails to send, and shows a delivery warning. A batch of invites keeps the members it already created when a later entry fails. +- **Invites Go Through When the Email Fails:** Inviting a member creates the member and invitation even if the email fails to send, with a delivery warning, and a batch keeps the invites it already created. -- **Resending an Invite Resets Its Expiry Everywhere:** Resending an invite from the members screen or through a bulk resend resets its expiry, the same as resending a single invite, so a resent invite shows as Pending with a fresh expiry. +- **Resent Invites Get a Fresh Expiry:** Resending an invite from the members screen or in bulk resets its expiry and shows it as Pending. -- **Custom Google Provider Discovery Uses Your Host:** Model discovery for a custom Google-format provider calls the host you configured, so custom Google-compatible providers can discover their own models. +- **Custom Google Providers Discover Their Own Models:** Model discovery for a custom Google-format provider calls the host you configured. -- **Gateway Streams Report Errors and Cost:** A streamed request cut short by a timeout or an upstream failure ends with an error event and is recorded as a failed request. Every route reports a cost, including a cut stream, priced from the output it produced before it stopped. +- **Gateway Streams Report Errors and Cost:** A stream cut short by a timeout or upstream failure ends with an error event and is logged as failed, and every route reports a cost, including cut streams. -- **One Request Log Row Per Request:** Request Logs show each request once. Upgrading a self-hosted deployment removes rows already saved twice and then builds an index without blocking writes, which can take a while on a large request-log table. +- **One Request Log Row Per Request:** Request Logs show each request once. On self-hosted deployments, the upgrade removes existing duplicate rows, which can take a while on a large table. -- **Messages Route Tool Calls and Metadata Header:** gpt-5.6 tool calls on the Messages route are served through the Responses API, and a malformed `x-agentcc-metadata` header is flagged. +- **Gateway Messages Route and Metadata:** gpt-5.6 tool calls on the Messages route are served through the Responses API, and a malformed `x-agentcc-metadata` header is flagged. -- **Budgets, Guardrails and Alerts Cover More Routes:** Budgets price usage against the full pricing catalogue, and input guardrails see the request on the native Messages, Responses and GenAI routes. A non-streamed response an output guardrail blocks returns 403 with its usage priced and is not cached, and a budget-exceeded webhook fires while the matching email alert goes out once per cooldown. +- **Budgets, Guardrails and Alerts Cover More Routes:** Budgets use the full pricing catalogue, input guardrails cover the native Messages, Responses and GenAI routes, and a response blocked by an output guardrail returns 403 with its usage priced. Budget email alerts go out once per cooldown. -- **Pricing Catalogue Refresh:** The bundled model pricing and capability catalogue has been refreshed, so current models are priced, and pinned older model IDs such as the Claude 3.5 Sonnet and Haiku IDs are priced again. A capability such as tools, vision or structured output is refused only when the catalogue states a model lacks it. A resold model is priced under the reseller's own entry, and a Gemini model the catalogue lists only under `gemini/` or `vertex_ai/` is priced on a Gemini provider of any name. If your organization set a custom price for a model the refresh newly lists without a price, check that model's cost. +- **Pricing Catalogue Refresh:** The model pricing and capability catalogue is refreshed, so current and pinned older models are priced, and a capability is refused only when the catalogue says a model lacks it. If you set a custom price for a model the refresh adds without a price, check that model's cost.
Breaking Changes
-- **Run Analytics Interval Shape:** In the run analytics API, `pass_rate_interval` is now `{low, high, scenarios, evaluated, resamples}`, and `effective_n` and `clusters` have been removed. Migration: read the new fields and drop any use of `effective_n` or `clusters`. +- **Run Analytics Interval Shape:** `pass_rate_interval` is now `{low, high, scenarios, evaluated, resamples}`, and `effective_n` and `clusters` are removed. Migration: read the new fields. -- **Trace Ingestion Checks Content-Encoding:** The trace ingestion endpoint now honors `Content-Encoding`. A body labelled `gzip` that is not valid gzip returns 400, an encoding other than gzip or identity, or more than one gzip layer, returns 415, and a gzip body over 16 MiB after decompression returns 413. Migration: send OTLP/HTTP uncompressed or gzip-compressed, and label it accurately. +- **Trace Ingestion Checks Content-Encoding:** A body labelled `gzip` that isn't valid gzip returns 400, and any other encoding returns 415. Migration: send OTLP/HTTP uncompressed or gzip, labelled accurately. -{/* SKIPPED: internal-only changes with no customer-visible effect (API contract and schema fixes, OpenAPI and MCP manifest regeneration, internal observability dashboards, end-to-end and build-sandbox test infrastructure, a connected-agents list filter that shipped and was undone in the same window, and release bookkeeping). Held for confirmation: limiting Simulate to Environments for newly created organizations, and a further trace-ingestion hardening item not yet shipped. */} +{/* SKIPPED: internal-only changes (API contract and schema fixes, manifest regeneration, internal dashboards, test infrastructure, a list filter shipped and undone in the same window, release bookkeeping). Held for confirmation: limiting Simulate to Environments for newly created organizations, and further ingestion hardening not yet shipped. */} ## Week of 2026-10-19 From c5dc04c10acd119ca95916bd2f56458e2a408dbd Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 11 Nov 2026 10:35:51 +0000 Subject: [PATCH 4/7] docs(release-notes): week of 2026-10-26, address review Moves every former Features item into Bugs/Improvements as enhancements to existing features, drops the sub-goal grading sentence, removes the gateway items (streams, request log, Messages route and metadata, budgets and guardrails, pricing catalogue, Perplexity and named providers) that the week of 2026-11-02 section already covers, and removes the SKIPPED and REVIEW notes. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01CFSMeVkmL3svj9cTBumVXR --- src/pages/docs/release-notes.mdx | 22 +++------------------- 1 file changed, 3 insertions(+), 19 deletions(-) diff --git a/src/pages/docs/release-notes.mdx b/src/pages/docs/release-notes.mdx index d4d90fe7f..8ab22f06f 100644 --- a/src/pages/docs/release-notes.mdx +++ b/src/pages/docs/release-notes.mdx @@ -9,7 +9,7 @@ description: "Latest Future AGI release notes covering new features, improvement
-
Features
+
Bugs/Improvements
- **Share and Export Error Feed Issues:** Share an Error Feed issue by link, with anyone or only people you add by email, and it opens read-only with its Overview, Traces, Trends and Fix tabs. You can also export an issue's latest 200 traces as a CSV ready to upload into a dataset. @@ -23,17 +23,13 @@ description: "Latest Future AGI release notes covering new features, improvement - **Reliability Reporting for Repeated Trials:** Runs with several trials per scenario report pass^1 with a 95% range and, when every scenario ran the same number of trials, pass^k, the share of scenarios that passed every trial. Each scenario row shows how many of its trials passed. -- **More Realistic Simulated Callers:** Simulated callers match the persona's age, language and accent, and sound frustrated when written that way. Sub-goal grading reads the call recording and checks that each step produced the right values for that caller. {/* REVIEW: source likely agent-learning-kit, not scanned in this pass. Confirm. */} +- **More Realistic Simulated Callers:** Simulated callers match the persona's age, language and accent, and sound frustrated when written that way. - **Environments Check Themselves Against Your Agent:** After building an environment for a repository agent in the builder chat, the builder plays one call per solution path against the live agent, corrects wrong reference answers, and reports what it changed in the chat. - **Simulate Navigation Centers on Environments:** Personas is now a tab inside Environments. Old links to Personas, Agent Definitions and Simulate open their new homes in Environments or Connected agents. -- **Perplexity Preset and Named Custom Providers:** Add Provider offers a Perplexity preset, custom providers can have their own names, and a `/` model routes to the provider it names. - -
Bugs/Improvements
- -- **Simulated Caller Openings:** Simulated callers open with a natural greeting in their own language and keep the call going when no scripted opening fits. {/* REVIEW: source likely agent-learning-kit, not scanned in this pass. Confirm. */} +- **Simulated Caller Openings:** Simulated callers open with a natural greeting in their own language and keep the call going when no scripted opening fits. - **Scenarios Keep Their Rows Through Renames:** Published scenarios are matched to their rows by key first, so a renamed scenario keeps its history and a new scenario reusing the old name gets its own row. @@ -43,16 +39,6 @@ description: "Latest Future AGI release notes covering new features, improvement - **Custom Google Providers Discover Their Own Models:** Model discovery for a custom Google-format provider calls the host you configured. -- **Gateway Streams Report Errors and Cost:** A stream cut short by a timeout or upstream failure ends with an error event and is logged as failed, and every route reports a cost, including cut streams. - -- **One Request Log Row Per Request:** Request Logs show each request once. On self-hosted deployments, the upgrade removes existing duplicate rows, which can take a while on a large table. - -- **Gateway Messages Route and Metadata:** gpt-5.6 tool calls on the Messages route are served through the Responses API, and a malformed `x-agentcc-metadata` header is flagged. - -- **Budgets, Guardrails and Alerts Cover More Routes:** Budgets use the full pricing catalogue, input guardrails cover the native Messages, Responses and GenAI routes, and a response blocked by an output guardrail returns 403 with its usage priced. Budget email alerts go out once per cooldown. - -- **Pricing Catalogue Refresh:** The model pricing and capability catalogue is refreshed, so current and pinned older models are priced, and a capability is refused only when the catalogue says a model lacks it. If you set a custom price for a model the refresh adds without a price, check that model's cost. -
Breaking Changes
- **Run Analytics Interval Shape:** `pass_rate_interval` is now `{low, high, scenarios, evaluated, resamples}`, and `effective_n` and `clusters` are removed. Migration: read the new fields. @@ -61,8 +47,6 @@ description: "Latest Future AGI release notes covering new features, improvement
-{/* SKIPPED: internal-only changes (API contract and schema fixes, manifest regeneration, internal dashboards, test infrastructure, a list filter shipped and undone in the same window, release bookkeeping). Held for confirmation: limiting Simulate to Environments for newly created organizations, and further ingestion hardening not yet shipped. */} - ## Week of 2026-10-19
From cacc9159ca70d6e045fcb7e914382dcb6d6caaa0 Mon Sep 17 00:00:00 2001 From: jedigalt Date: Wed, 11 Nov 2026 16:09:20 +0530 Subject: [PATCH 5/7] docs(release-notes): restore Pricing Catalogue Refresh for week of 2026-10-26 The catalogue refresh (v1.56.0) was removed with the gateway items moved to the week of 2026-11-02, but that week does not cover it. Co-Authored-By: Claude Opus 5.5 --- src/pages/docs/release-notes.mdx | 2 ++ 1 file changed, 2 insertions(+) diff --git a/src/pages/docs/release-notes.mdx b/src/pages/docs/release-notes.mdx index 8ab22f06f..a2354b8a1 100644 --- a/src/pages/docs/release-notes.mdx +++ b/src/pages/docs/release-notes.mdx @@ -39,6 +39,8 @@ description: "Latest Future AGI release notes covering new features, improvement - **Custom Google Providers Discover Their Own Models:** Model discovery for a custom Google-format provider calls the host you configured. +- **Pricing Catalogue Refresh:** The model pricing and capability catalogue is refreshed, so current and pinned older models are priced, and a capability is refused only when the catalogue says a model lacks it. If you set a custom price for a model the refresh adds without a price, check that model's cost. +
Breaking Changes
- **Run Analytics Interval Shape:** `pass_rate_interval` is now `{low, high, scenarios, evaluated, resamples}`, and `effective_n` and `clusters` are removed. Migration: read the new fields. From 012d3c7cdd412dc4ddeaf9c547070fdfadf51e42 Mon Sep 17 00:00:00 2001 From: jedigalt Date: Wed, 11 Nov 2026 16:44:15 +0530 Subject: [PATCH 6/7] docs(release-notes): add the v1.56.1 gateway fixes to week of 2026-10-26 The stream, Request Log, provider, cost, budget, guardrail and alert fixes first shipped in v1.56.1 (28 Oct, hotfix future-agi/future-agi#3643), so they belong to this week. Combined into two gateway bullets and one guardrail bullet, with the Google discovery and catalogue refresh items folded in. Co-Authored-By: Claude Opus 5.5 --- src/pages/docs/release-notes.mdx | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/src/pages/docs/release-notes.mdx b/src/pages/docs/release-notes.mdx index a2354b8a1..377054da0 100644 --- a/src/pages/docs/release-notes.mdx +++ b/src/pages/docs/release-notes.mdx @@ -37,9 +37,11 @@ description: "Latest Future AGI release notes covering new features, improvement - **Resent Invites Get a Fresh Expiry:** Resending an invite from the members screen or in bulk resets its expiry and shows it as Pending. -- **Custom Google Providers Discover Their Own Models:** Model discovery for a custom Google-format provider calls the host you configured. +- **Gateway Streams, Request Log, and Providers:** Streams on chat, Messages, Responses, Completions, GenAI, speech, and Assistants runs now end with a clear error event when a request times out or the provider fails, are logged as failed, and carry cost and latency metadata on every route. Messages requests report the timeout that was applied. Each request appears once in the Request Log, and a malformed metadata header is flagged there instead of disappearing without a trace. On self-hosted deployments, the upgrade removes existing duplicate Request Log rows, which can take a while on a large table. gpt-5.6 tool calls on the Messages route are served through the Responses API, Add Provider offers a Perplexity preset, custom providers can have their own names, a `/` model routes to the provider it names, and model discovery for a custom Google-format provider calls the host you configured. -- **Pricing Catalogue Refresh:** The model pricing and capability catalogue is refreshed, so current and pinned older models are priced, and a capability is refused only when the catalogue says a model lacks it. If you set a custom price for a model the refresh adds without a price, check that model's cost. +- **Gateway Cost and Budgets:** Cost is now tracked for streamed Responses calls, image, rerank, OCR, and GenAI embedding requests, and streams that end before usage arrives, which are priced from an estimate. Anything without a known price is clearly marked as unpriced, and models offered under more than one provider name, or through a reseller, are billed at their real rate. The model pricing and capability catalogue is refreshed, so current and pinned older models are priced, and a capability is refused only when the catalogue says a model lacks it. If you set a custom price for a model the refresh adds without a price, check that model's cost. Budgets now use the same prices you see in the Request Log, so a budget stops spend at the limit you set, and budget-exceeded webhooks and email alerts are now sent, at most once per cooldown. + +- **Guardrails on Every Route:** Input guardrails now check request text on the native Messages, Responses, and GenAI routes, and output guardrails block a non-streamed response on every route with that route's own error, with its usage still priced. A blocked response is never cached.
Breaking Changes
From 85295e206cbb7bb0779544918a87b0715277dee1 Mon Sep 17 00:00:00 2001 From: jedigalt Date: Wed, 11 Nov 2026 18:02:51 +0530 Subject: [PATCH 7/7] docs(release-notes): fold Breaking Changes into improvements, week of 2026-10-26 The page has no Breaking Changes class. The run analytics interval shape moves into Reliability Reporting for Repeated Trials, and the Content-Encoding check moves into the OTLP gzip bullet, each with its one-clause migration note. Co-Authored-By: Claude Opus 5.5 --- src/pages/docs/release-notes.mdx | 10 ++-------- 1 file changed, 2 insertions(+), 8 deletions(-) diff --git a/src/pages/docs/release-notes.mdx b/src/pages/docs/release-notes.mdx index 377054da0..c26b52254 100644 --- a/src/pages/docs/release-notes.mdx +++ b/src/pages/docs/release-notes.mdx @@ -15,13 +15,13 @@ description: "Latest Future AGI release notes covering new features, improvement - **MCP Tools for Error Feed:** The MCP server's read-only `error_feed` group adds `get_error_cluster_overview` and `list_error_cluster_traces`, and `get_error_cluster` now returns the stored root-cause analysis. Connections created before `error_feed` became a default group need to turn it on. -- **Gzip and a Project Header for OTLP Ingestion:** Trace ingestion accepts gzip-compressed OTLP/HTTP, with the 16 MiB limit applied before and after decompression; gRPC stays uncompressed. An `X-Project-Name` header (alias `FI-Project-Name`) sets the project when a resource has no `project_name` attribute. +- **Gzip and a Project Header for OTLP Ingestion:** Trace ingestion accepts gzip-compressed OTLP/HTTP, with the 16 MiB limit applied before and after decompression; gRPC stays uncompressed. An `X-Project-Name` header (alias `FI-Project-Name`) sets the project when a resource has no `project_name` attribute. Label each body's encoding accurately: a body marked `gzip` must be valid gzip (otherwise it returns 400), and any other encoding returns 415. - **Fuller Traces in Observe:** Spans that carry content in the OpenTelemetry GenAI keys, as OpenRouter Broadcast sends them, now fill the Input and Output panels, and trace lists show each trace's whole token count and cost. Sorting by cost or tokens orders every matching trace, and costs under a cent keep their precision. - **Choose Where a Re-run Lands:** The Re-run menu on an Environments run offers **Re-run in this run** alongside **Run as a new simulation**, replacing only the selected calls' results. Another re-run is refused until the current one and its grading finish. -- **Reliability Reporting for Repeated Trials:** Runs with several trials per scenario report pass^1 with a 95% range and, when every scenario ran the same number of trials, pass^k, the share of scenarios that passed every trial. Each scenario row shows how many of its trials passed. +- **Reliability Reporting for Repeated Trials:** Runs with several trials per scenario report pass^1 with a 95% range and, when every scenario ran the same number of trials, pass^k, the share of scenarios that passed every trial. Each scenario row shows how many of its trials passed. In the run analytics API, `pass_rate_interval` now returns `{low, high, scenarios, evaluated, resamples}`; read these fields in place of `effective_n` and `clusters`. - **More Realistic Simulated Callers:** Simulated callers match the persona's age, language and accent, and sound frustrated when written that way. @@ -43,12 +43,6 @@ description: "Latest Future AGI release notes covering new features, improvement - **Guardrails on Every Route:** Input guardrails now check request text on the native Messages, Responses, and GenAI routes, and output guardrails block a non-streamed response on every route with that route's own error, with its usage still priced. A blocked response is never cached. -
Breaking Changes
- -- **Run Analytics Interval Shape:** `pass_rate_interval` is now `{low, high, scenarios, evaluated, resamples}`, and `effective_n` and `clusters` are removed. Migration: read the new fields. - -- **Trace Ingestion Checks Content-Encoding:** A body labelled `gzip` that isn't valid gzip returns 400, and any other encoding returns 415. Migration: send OTLP/HTTP uncompressed or gzip, labelled accurately. -
## Week of 2026-10-19