From fef1ab28e21a83785db7bdd125757a6f01b7138d Mon Sep 17 00:00:00 2001 From: jedigalt <201954051+jedigalt@users.noreply.github.com> Date: Mon, 9 Nov 2026 12:21:17 +0000 Subject: [PATCH 1/7] [needs-review] docs(release-notes): week of 2026-11-02 --- src/pages/docs/release-notes.mdx | 56 ++++++++++++++++++++++++++++++++ 1 file changed, 56 insertions(+) diff --git a/src/pages/docs/release-notes.mdx b/src/pages/docs/release-notes.mdx index ca9500c1e..c888b9e3a 100644 --- a/src/pages/docs/release-notes.mdx +++ b/src/pages/docs/release-notes.mdx @@ -5,6 +5,62 @@ description: "Latest Future AGI release notes covering new features, improvement {/* release-notes:insert-below — automation inserts new releases here; do not remove */} +## Week of 2026-11-02 + +
+ +
Features
+ +- **Slack Alerts for Continuous Eval Tasks:** A continuous evaluation task now posts a Slack alert when it fails or stalls, so a silently stuck task no longer goes unnoticed. + +- **Open the Source Trace From Eval Task Usage:** From any run on an eval task's Usage tab, Open trace now opens the exact trace with the evaluated span selected in a drawer, without leaving Usage, and previous/next steps through the task's traces in order. A voice project opens the voice call drawer instead. + +
Bugs/Improvements
+ +- **Environment Workspace Fixes:** Hover tooltips were removed from the environment workspace tabs, and an environment's scenario count now updates live instead of freezing at the first value it loaded. The environment builder chat also now offers to approve every change in a request at once, instead of confirming each edit separately. + +- **Faster, More Reliable Environment Builds:** An agent that reads its assigned network port now builds and tests across several workers at once instead of one at a time, and an agent backed by its own embedded SQLite database now builds correctly instead of failing. Environments built on an earlier platform version also keep working after an upgrade, and a build or check run under heavy load no longer fails from a lost results file. + +- **Debug Failures Groups Duplicate Findings:** Debug failures investigations now group accepted findings together instead of listing the same failure more than once, and revisit findings that were set aside instead of dropping them. A goal with no findings no longer shows as an unexplained row. + +- **Error Feed Issues Reflect Only Live Eval Results:** An Error Feed issue no longer keeps listing a trace after its evaluation result has been deleted, and an issue's score, turns, and Trends statistics now reflect only live results instead of reading zero when nothing remains to measure. + +- **Eval Task List Shows Accurate Last-Run Times:** An eval task in the Tasks list now shows when it last actually produced work, instead of continuing to show its creation date no matter how many times it had run since. + +- **More Reliable Few-Shot Evaluation Inputs:** Few-shot evaluation inputs that are zero or false are no longer dropped as if they were empty, and structured (JSON) inputs are now matched and sent to the embedding model correctly instead of failing outright. + +- **Prevent Silent Blank Dataset Rows:** Writing dataset rows through the platform's AI assistant or its MCP tools with an unrecognized column, or in a malformed batch, is now rejected with a clear error instead of silently saved as blank rows. + +- **Gateway Streaming and Cost Accuracy:** A request that times out or hits a provider error mid-stream now ends with a proper error event instead of cutting off silently, across chat, Messages, Responses, Completions, GenAI, speech, and Assistants run streams. Streamed Responses calls, plus image, rerank, search, OCR, and embedding requests, are now priced or clearly flagged as unpriced instead of appearing free, a stream cut off before usage arrives is now priced from an estimate instead of showing no cost at all, and pricing catalogue gaps are now filled from additional public sources. Cost and latency metadata now reach every streaming route instead of chat alone, and Messages requests now report the timeout that was actually applied. + +- **Gateway Billing Now Covers Every Request Type:** Assistants runs and Vector Store calls now go through the gateway's usual budget and request-logging checks instead of running unbilled and unlogged. Several provider pricing lookups are also fixed so models offered under more than one provider name, or through a reseller, are billed at their real price instead of silently at zero. + +- **Gateway Budgets, Guardrails, and Alerts on Every Route:** Budgets now price spend from the live model catalogue instead of an outdated fallback table, input guardrails see request text on the native Messages and Responses routes, a blocked response returns the correct error on every route including streaming, and budget alerts are now actually sent. + +- **Gateway Request Log and Provider Fixes:** A request is now logged exactly once instead of several times, gpt-5.6 models support tool calls over the Responses API, a malformed metadata header is now rejected instead of silently dropped, and an organization can add Perplexity and more than one custom provider. Gemini tool calls also get a unique id per call instead of ids that repeat across responses, which previously broke agents that track calls by id across multiple rounds. + +- **Gateway Model List Matches Key Access:** The gateway's model list no longer advertises provider models an API key is not entitled to call; it now lists only the models the key can actually use. + +- **Guardrail Credentials and Request Logs Stay Scoped to Their Organization:** A gateway configuration could previously decrypt another organization's stored guardrail credentials, and gateway traffic spanning multiple organizations in one batch could file requests, webhook events, and usage under the wrong one. Both are now resolved per organization before anything is stored or decrypted. + +- **More Reliable Guardrail and Gateway Config Saves:** Activating a Guardrails or Fallbacks change no longer resets an organization's other settings, such as budgets, usage limits, and IP restrictions, back to their defaults, and toggling or editing a guardrail no longer fails for organizations managed through the Settings editor. The UI also now warns clearly when a save or rollback did not reach the gateway, instead of reporting success. + +- **Guardrail Testing Shows the Real Outcome:** The Guardrails Test tab now shows a warning for rules in Warn or Log mode instead of reporting a plain OK, along with the real HTTP status code. A guardrail whose authentication fails is now treated as a failure instead of silently calling the check with no credentials. + +- **Future AGI Eval Guardrail Works on Self-Hosted Installs:** The Future AGI Eval guardrail no longer defaults its Base URL to our cloud endpoint, so self-hosted organizations get their own credentials injected instead of the check running with no, or the wrong, credentials. + +- **Trace Ingestion Limits:** The trace collector now enforces per-request limits on OTLP payload size, element count, and nesting depth, and authenticates gRPC connections before decoding them, so an oversized or malformed request now gets a clear error instead of exhausting memory. + +- **SQS Export Reliability:** The SQS export integration now retries individual failed messages and only advances past a batch once every record in it is confirmed delivered, instead of silently skipping undelivered records. + +- **Annotation Labels Filter Order:** The Labels filter toolbar now matches the Queues toolbar's order (search, then Active/Archived, then type), for consistent navigation between the two. + +- **Self-Hosted Windows Startup Fix:** Self-hosted deployments checked out on Windows now start correctly; a line-ending mismatch in shell scripts and container build files previously stopped the stack from starting. + +
+ +{/* SKIPPED: internal-only churn (version-bump and branch-sync merges, generated-contract and tool-manifest regeneration, test-only changes); two local-development-only contract-validation fixes confirmed to have no effect in production; a short-lived SDK issue introduced and fixed within this same window with no net customer impact; an internal operational incident whose only customer-facing outcome is the Slack-alert item above. Held for confirmation: environment, agent and scenario-suite versioning with run comparison and rollback (merged but still behind platform flags that were off in production as of this release); a capability for repairing a failing agent from a failed simulation run (its screens shipped, but the backend run pipeline still depends on a production template update that was not yet complete); multi-agent framework reference fixtures (partial hosted coverage per the author); runtime sync of the pricing catalogue from a public source (only the static half merged); nesting of sub-agent traces under their parent call (the fix lives in a dependency outside this week's reviewed repositories); comparing an agent's turns against its own prompt and tool calls as part of a Debug Failures investigation (the data is now sent, but the comparison logic itself lives in a dependency outside this week's reviewed repositories, so it cannot be confirmed as live). */} + ## Week of 2026-10-19
From 8c481aff0fb96d154b184424d65f87bfa88c0904 Mon Sep 17 00:00:00 2001 From: jedigalt Date: Tue, 10 Nov 2026 18:07:12 +0530 Subject: [PATCH 2/7] docs(release-notes): apply review fixes for week of 2026-11-02 - Add Fix My Agent for Environments (shipped in v1.58.0; beta, per organization) - Drop Trace Ingestion Limits (source work not accepted for release) - Malformed metadata header is flagged, not rejected - Output guardrails: non-streamed responses on every route; streaming not claimed - Input guardrails: add native GenAI route - Pricing: drop search (/v1/search not available yet), GenAI embedding only, one additional catalogue source, reseller models were mispriced not only zero - Vector Store: writes, searches and deletes only; deletes never refused for budget Co-Authored-By: Claude Opus 5.5 --- src/pages/docs/release-notes.mdx | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/src/pages/docs/release-notes.mdx b/src/pages/docs/release-notes.mdx index c888b9e3a..6fcc0e96b 100644 --- a/src/pages/docs/release-notes.mdx +++ b/src/pages/docs/release-notes.mdx @@ -13,6 +13,8 @@ description: "Latest Future AGI release notes covering new features, improvement - **Slack Alerts for Continuous Eval Tasks:** A continuous evaluation task now posts a Slack alert when it fails or stalls, so a silently stuck task no longer goes unnoticed. +- **Fix My Agent for Environments:** From a failed environment run, choose **Improve** in the Debug Failures drawer to have Future AGI repair the agent for you. Pick the failing cases to fix and the passing cases the fix must not break, the parts of the agent it may change, files to leave untouched, and a budget and time limit. Each candidate fix is tested as a new agent version against your cases, and the best one is then verified on a held-back set of cases it never saw while searching, so a fix ends with a verdict and an explanation of how it was decided. From there you can make the winning version your default, open a pull request for an agent connected from GitHub, or download the patch. Fixes are listed on the environment's new **Improvements** tab, where a running fix shows its steps and trials live and can be stopped at any time. Your default agent version never changes unless you choose to ship a fix. Fix My Agent works with agents uploaded as an archive or connected from GitHub, and needs a run with enough failing cases to set some aside for verification. It is in beta and is turned on per organization. + - **Open the Source Trace From Eval Task Usage:** From any run on an eval task's Usage tab, Open trace now opens the exact trace with the evaluated span selected in a drawer, without leaving Usage, and previous/next steps through the task's traces in order. A voice project opens the voice call drawer instead.
Bugs/Improvements
@@ -31,13 +33,13 @@ description: "Latest Future AGI release notes covering new features, improvement - **Prevent Silent Blank Dataset Rows:** Writing dataset rows through the platform's AI assistant or its MCP tools with an unrecognized column, or in a malformed batch, is now rejected with a clear error instead of silently saved as blank rows. -- **Gateway Streaming and Cost Accuracy:** A request that times out or hits a provider error mid-stream now ends with a proper error event instead of cutting off silently, across chat, Messages, Responses, Completions, GenAI, speech, and Assistants run streams. Streamed Responses calls, plus image, rerank, search, OCR, and embedding requests, are now priced or clearly flagged as unpriced instead of appearing free, a stream cut off before usage arrives is now priced from an estimate instead of showing no cost at all, and pricing catalogue gaps are now filled from additional public sources. Cost and latency metadata now reach every streaming route instead of chat alone, and Messages requests now report the timeout that was actually applied. +- **Gateway Streaming and Cost Accuracy:** A request that times out or hits a provider error mid-stream now ends with a proper error event instead of cutting off silently, across chat, Messages, Responses, Completions, GenAI, speech, and Assistants run streams. Streamed Responses calls, plus image, rerank, OCR, and GenAI embedding requests, are now priced or clearly flagged as unpriced instead of appearing free, a stream cut off before usage arrives is now priced from an estimate instead of showing no cost at all, and pricing catalogue gaps are now filled from an additional public pricing source. Cost and latency metadata now reach every streaming route instead of chat alone, and Messages requests now report the timeout that was actually applied. -- **Gateway Billing Now Covers Every Request Type:** Assistants runs and Vector Store calls now go through the gateway's usual budget and request-logging checks instead of running unbilled and unlogged. Several provider pricing lookups are also fixed so models offered under more than one provider name, or through a reseller, are billed at their real price instead of silently at zero. +- **Gateway Billing Now Covers Every Request Type:** Assistants runs and Vector Store writes, searches, and deletes now go through the gateway's usual budget and request-logging checks instead of running unbilled and unlogged. A Vector Store delete is never refused for budget, so a key that is out of budget can still clean up. Several provider pricing lookups are also fixed so models offered under more than one provider name, or through a reseller, are billed at their real price instead of at zero or at another provider's rate. -- **Gateway Budgets, Guardrails, and Alerts on Every Route:** Budgets now price spend from the live model catalogue instead of an outdated fallback table, input guardrails see request text on the native Messages and Responses routes, a blocked response returns the correct error on every route including streaming, and budget alerts are now actually sent. +- **Gateway Budgets, Guardrails, and Alerts on Every Route:** Budgets now price spend from the live model catalogue instead of an outdated fallback table, input guardrails now see request text on the native Messages, Responses, and GenAI routes, output guardrails now block a non-streamed response on every route with that route's own error instead of a server error or the blocked text, and a blocked response is no longer cached, and budget alerts are now actually sent. -- **Gateway Request Log and Provider Fixes:** A request is now logged exactly once instead of several times, gpt-5.6 models support tool calls over the Responses API, a malformed metadata header is now rejected instead of silently dropped, and an organization can add Perplexity and more than one custom provider. Gemini tool calls also get a unique id per call instead of ids that repeat across responses, which previously broke agents that track calls by id across multiple rounds. +- **Gateway Request Log and Provider Fixes:** A request is now logged exactly once instead of several times, gpt-5.6 models support tool calls over the Responses API, a malformed metadata header is now flagged in the Request Log and in an `x-agentcc-metadata-error` response header instead of being silently dropped (the request itself still succeeds, so existing callers keep working), and an organization can add Perplexity and more than one custom provider. Gemini tool calls also get a unique id per call instead of ids that repeat across responses, which previously broke agents that track calls by id across multiple rounds. - **Gateway Model List Matches Key Access:** The gateway's model list no longer advertises provider models an API key is not entitled to call; it now lists only the models the key can actually use. @@ -49,8 +51,6 @@ description: "Latest Future AGI release notes covering new features, improvement - **Future AGI Eval Guardrail Works on Self-Hosted Installs:** The Future AGI Eval guardrail no longer defaults its Base URL to our cloud endpoint, so self-hosted organizations get their own credentials injected instead of the check running with no, or the wrong, credentials. -- **Trace Ingestion Limits:** The trace collector now enforces per-request limits on OTLP payload size, element count, and nesting depth, and authenticates gRPC connections before decoding them, so an oversized or malformed request now gets a clear error instead of exhausting memory. - - **SQS Export Reliability:** The SQS export integration now retries individual failed messages and only advances past a batch once every record in it is confirmed delivered, instead of silently skipping undelivered records. - **Annotation Labels Filter Order:** The Labels filter toolbar now matches the Queues toolbar's order (search, then Active/Archived, then type), for consistent navigation between the two. @@ -59,7 +59,7 @@ description: "Latest Future AGI release notes covering new features, improvement
-{/* SKIPPED: internal-only churn (version-bump and branch-sync merges, generated-contract and tool-manifest regeneration, test-only changes); two local-development-only contract-validation fixes confirmed to have no effect in production; a short-lived SDK issue introduced and fixed within this same window with no net customer impact; an internal operational incident whose only customer-facing outcome is the Slack-alert item above. Held for confirmation: environment, agent and scenario-suite versioning with run comparison and rollback (merged but still behind platform flags that were off in production as of this release); a capability for repairing a failing agent from a failed simulation run (its screens shipped, but the backend run pipeline still depends on a production template update that was not yet complete); multi-agent framework reference fixtures (partial hosted coverage per the author); runtime sync of the pricing catalogue from a public source (only the static half merged); nesting of sub-agent traces under their parent call (the fix lives in a dependency outside this week's reviewed repositories); comparing an agent's turns against its own prompt and tool calls as part of a Debug Failures investigation (the data is now sent, but the comparison logic itself lives in a dependency outside this week's reviewed repositories, so it cannot be confirmed as live). */} +{/* SKIPPED: internal-only churn (version-bump and branch-sync merges, generated-contract and tool-manifest regeneration, test-only changes); two local-development-only contract-validation fixes confirmed to have no effect in production; a short-lived SDK issue introduced and fixed within this same window with no net customer impact; an internal operational incident whose only customer-facing outcome is the Slack-alert item above. Held for confirmation: environment, agent and scenario-suite versioning with run comparison and rollback (merged but still behind platform flags that were off in production as of this release); multi-agent framework reference fixtures (partial hosted coverage per the author); runtime sync of the pricing catalogue from a public source (only the static half merged); nesting of sub-agent traces under their parent call (the fix lives in a dependency outside this week's reviewed repositories); comparing an agent's turns against its own prompt and tool calls as part of a Debug Failures investigation (the data is now sent, but the comparison logic itself lives in a dependency outside this week's reviewed repositories, so it cannot be confirmed as live). */} ## Week of 2026-10-19 From 801909227c14b1810196ed242dc967bbca430ae8 Mon Sep 17 00:00:00 2001 From: jedigalt Date: Tue, 10 Nov 2026 18:22:40 +0530 Subject: [PATCH 3/7] docs(release-notes): tighten week of 2026-11-02 per review - Shorten Faster, More Reliable Environment Builds - Drop the Assistants runs / Vector Store budget and logging sentence; retitle the remaining bullet to Gateway Pricing Lookup Fixes - Metadata header: flagged in the Request Log instead of disappearing without a trace Co-Authored-By: Claude Opus 5.5 --- src/pages/docs/release-notes.mdx | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/src/pages/docs/release-notes.mdx b/src/pages/docs/release-notes.mdx index 6fcc0e96b..8e552d487 100644 --- a/src/pages/docs/release-notes.mdx +++ b/src/pages/docs/release-notes.mdx @@ -21,7 +21,7 @@ description: "Latest Future AGI release notes covering new features, improvement - **Environment Workspace Fixes:** Hover tooltips were removed from the environment workspace tabs, and an environment's scenario count now updates live instead of freezing at the first value it loaded. The environment builder chat also now offers to approve every change in a request at once, instead of confirming each edit separately. -- **Faster, More Reliable Environment Builds:** An agent that reads its assigned network port now builds and tests across several workers at once instead of one at a time, and an agent backed by its own embedded SQLite database now builds correctly instead of failing. Environments built on an earlier platform version also keep working after an upgrade, and a build or check run under heavy load no longer fails from a lost results file. +- **Faster, More Reliable Environment Builds:** Agents that read their assigned port now build and test in parallel, agents with an embedded SQLite database now build correctly, environments from an earlier platform version keep working after an upgrade, and builds under heavy load no longer fail. - **Debug Failures Groups Duplicate Findings:** Debug failures investigations now group accepted findings together instead of listing the same failure more than once, and revisit findings that were set aside instead of dropping them. A goal with no findings no longer shows as an unexplained row. @@ -35,11 +35,11 @@ description: "Latest Future AGI release notes covering new features, improvement - **Gateway Streaming and Cost Accuracy:** A request that times out or hits a provider error mid-stream now ends with a proper error event instead of cutting off silently, across chat, Messages, Responses, Completions, GenAI, speech, and Assistants run streams. Streamed Responses calls, plus image, rerank, OCR, and GenAI embedding requests, are now priced or clearly flagged as unpriced instead of appearing free, a stream cut off before usage arrives is now priced from an estimate instead of showing no cost at all, and pricing catalogue gaps are now filled from an additional public pricing source. Cost and latency metadata now reach every streaming route instead of chat alone, and Messages requests now report the timeout that was actually applied. -- **Gateway Billing Now Covers Every Request Type:** Assistants runs and Vector Store writes, searches, and deletes now go through the gateway's usual budget and request-logging checks instead of running unbilled and unlogged. A Vector Store delete is never refused for budget, so a key that is out of budget can still clean up. Several provider pricing lookups are also fixed so models offered under more than one provider name, or through a reseller, are billed at their real price instead of at zero or at another provider's rate. +- **Gateway Pricing Lookup Fixes:** Several provider pricing lookups are fixed so models offered under more than one provider name, or through a reseller, are billed at their real price instead of at zero or at another provider's rate. - **Gateway Budgets, Guardrails, and Alerts on Every Route:** Budgets now price spend from the live model catalogue instead of an outdated fallback table, input guardrails now see request text on the native Messages, Responses, and GenAI routes, output guardrails now block a non-streamed response on every route with that route's own error instead of a server error or the blocked text, and a blocked response is no longer cached, and budget alerts are now actually sent. -- **Gateway Request Log and Provider Fixes:** A request is now logged exactly once instead of several times, gpt-5.6 models support tool calls over the Responses API, a malformed metadata header is now flagged in the Request Log and in an `x-agentcc-metadata-error` response header instead of being silently dropped (the request itself still succeeds, so existing callers keep working), and an organization can add Perplexity and more than one custom provider. Gemini tool calls also get a unique id per call instead of ids that repeat across responses, which previously broke agents that track calls by id across multiple rounds. +- **Gateway Request Log and Provider Fixes:** A request is now logged exactly once instead of several times, gpt-5.6 models support tool calls over the Responses API, a malformed metadata header is now flagged in the Request Log instead of disappearing without a trace, and an organization can add Perplexity and more than one custom provider. Gemini tool calls also get a unique id per call instead of ids that repeat across responses, which previously broke agents that track calls by id across multiple rounds. - **Gateway Model List Matches Key Access:** The gateway's model list no longer advertises provider models an API key is not entitled to call; it now lists only the models the key can actually use. From b62d0adef10c91762b630fc889b4fd8033be7750 Mon Sep 17 00:00:00 2001 From: jedigalt Date: Wed, 11 Nov 2026 15:50:05 +0530 Subject: [PATCH 4/7] docs(release-notes): address review comments for week of 2026-11-02 - Remove the internal Slack alerts item and the SKIPPED notes - Rename Fix My Agent to Epoch, reword its opening, drop the beta caveat - Move Open the Source Trace to Bugs/Improvements - Reword Environment Builds and Debug Failures as improvements - Combine gateway changes into two bullets and guardrail changes into two, with plain-English budget wording and no repeated claims Co-Authored-By: Claude Opus 5.5 --- src/pages/docs/release-notes.mdx | 32 +++++++++----------------------- 1 file changed, 9 insertions(+), 23 deletions(-) diff --git a/src/pages/docs/release-notes.mdx b/src/pages/docs/release-notes.mdx index 8e552d487..74cbbbd21 100644 --- a/src/pages/docs/release-notes.mdx +++ b/src/pages/docs/release-notes.mdx @@ -11,19 +11,17 @@ description: "Latest Future AGI release notes covering new features, improvement
Features
-- **Slack Alerts for Continuous Eval Tasks:** A continuous evaluation task now posts a Slack alert when it fails or stalls, so a silently stuck task no longer goes unnoticed. - -- **Fix My Agent for Environments:** From a failed environment run, choose **Improve** in the Debug Failures drawer to have Future AGI repair the agent for you. Pick the failing cases to fix and the passing cases the fix must not break, the parts of the agent it may change, files to leave untouched, and a budget and time limit. Each candidate fix is tested as a new agent version against your cases, and the best one is then verified on a held-back set of cases it never saw while searching, so a fix ends with a verdict and an explanation of how it was decided. From there you can make the winning version your default, open a pull request for an agent connected from GitHub, or download the patch. Fixes are listed on the environment's new **Improvements** tab, where a running fix shows its steps and trials live and can be stopped at any time. Your default agent version never changes unless you choose to ship a fix. Fix My Agent works with agents uploaded as an archive or connected from GitHub, and needs a run with enough failing cases to set some aside for verification. It is in beta and is turned on per organization. - -- **Open the Source Trace From Eval Task Usage:** From any run on an eval task's Usage tab, Open trace now opens the exact trace with the evaluated span selected in a drawer, without leaving Usage, and previous/next steps through the task's traces in order. A voice project opens the voice call drawer instead. +- **Epoch, Repair Your Agent From Failed Runs:** From the runs that failed in a simulation environment, choose **Improve** in the Debug Failures drawer and Epoch repairs the agent for you. Pick the failing cases to fix and the passing cases the fix must not break, the parts of the agent it may change, files to leave untouched, and a budget and time limit. Each candidate fix is tested as a new agent version against your cases, and the best one is then verified on a held-back set of cases it never saw while searching, so every fix ends with a verdict and an explanation of how it was decided. From there you can make the winning version your default, open a pull request for an agent connected from GitHub, or download the patch. Fixes are listed on the environment's new **Improvements** tab, where a running fix shows its steps and trials live and can be stopped at any time. Your default agent version never changes unless you choose to ship a fix. Epoch works with agents uploaded as an archive or connected from GitHub, and needs a run with enough failing cases to set some aside for verification.
Bugs/Improvements
- **Environment Workspace Fixes:** Hover tooltips were removed from the environment workspace tabs, and an environment's scenario count now updates live instead of freezing at the first value it loaded. The environment builder chat also now offers to approve every change in a request at once, instead of confirming each edit separately. -- **Faster, More Reliable Environment Builds:** Agents that read their assigned port now build and test in parallel, agents with an embedded SQLite database now build correctly, environments from an earlier platform version keep working after an upgrade, and builds under heavy load no longer fail. +- **Faster, More Reliable Environment Builds:** Agents that read their assigned port now build and test in parallel, agents that store their data in an embedded SQLite database are now supported, and environments created on an earlier platform version keep working after an upgrade. Builds and checks also no longer fail more often when many environments run at the same time. -- **Debug Failures Groups Duplicate Findings:** Debug failures investigations now group accepted findings together instead of listing the same failure more than once, and revisit findings that were set aside instead of dropping them. A goal with no findings no longer shows as an unexplained row. +- **Clearer Debug Failures Results:** Debug Failures investigations now combine matching findings into a single entry, come back to findings that were set aside, and clearly label a goal that has no findings. + +- **Open the Source Trace From Eval Task Usage:** From any run on an eval task's Usage tab, Open trace now opens the exact trace with the evaluated span selected in a drawer, without leaving Usage, and previous/next steps through the task's traces in order. A voice project opens the voice call drawer instead. - **Error Feed Issues Reflect Only Live Eval Results:** An Error Feed issue no longer keeps listing a trace after its evaluation result has been deleted, and an issue's score, turns, and Trends statistics now reflect only live results instead of reading zero when nothing remains to measure. @@ -33,23 +31,13 @@ description: "Latest Future AGI release notes covering new features, improvement - **Prevent Silent Blank Dataset Rows:** Writing dataset rows through the platform's AI assistant or its MCP tools with an unrecognized column, or in a malformed batch, is now rejected with a clear error instead of silently saved as blank rows. -- **Gateway Streaming and Cost Accuracy:** A request that times out or hits a provider error mid-stream now ends with a proper error event instead of cutting off silently, across chat, Messages, Responses, Completions, GenAI, speech, and Assistants run streams. Streamed Responses calls, plus image, rerank, OCR, and GenAI embedding requests, are now priced or clearly flagged as unpriced instead of appearing free, a stream cut off before usage arrives is now priced from an estimate instead of showing no cost at all, and pricing catalogue gaps are now filled from an additional public pricing source. Cost and latency metadata now reach every streaming route instead of chat alone, and Messages requests now report the timeout that was actually applied. - -- **Gateway Pricing Lookup Fixes:** Several provider pricing lookups are fixed so models offered under more than one provider name, or through a reseller, are billed at their real price instead of at zero or at another provider's rate. +- **Gateway Cost, Budgets, and Alerts:** Cost is now tracked for streamed Responses calls, image, rerank, OCR, and GenAI embedding requests, and streams that end before usage arrives, which are priced from an estimate. Anything without a known price is clearly marked as unpriced. Models offered under more than one provider name, or through a reseller, are billed at their real rate, and an additional public pricing source fills gaps in the model catalogue. Budgets now use the same prices you see in the Request Log, so a budget stops spend at the limit you set, and budget-exceeded webhooks and email alerts are now sent. -- **Gateway Budgets, Guardrails, and Alerts on Every Route:** Budgets now price spend from the live model catalogue instead of an outdated fallback table, input guardrails now see request text on the native Messages, Responses, and GenAI routes, output guardrails now block a non-streamed response on every route with that route's own error instead of a server error or the blocked text, and a blocked response is no longer cached, and budget alerts are now actually sent. +- **Gateway Streaming, Request Log, and Providers:** Streams on chat, Messages, Responses, Completions, GenAI, speech, and Assistants runs now end with a clear error event when a request times out or the provider fails, and carry cost and latency metadata on every route. Messages requests report the timeout that was applied. Each request appears once in the Request Log, and a malformed metadata header is flagged there instead of disappearing without a trace. gpt-5.6 models support tool calls over the Responses API, Gemini tool calls each get their own id, an organization can add Perplexity and more than one custom provider, and the model list shows only the models an API key can use. -- **Gateway Request Log and Provider Fixes:** A request is now logged exactly once instead of several times, gpt-5.6 models support tool calls over the Responses API, a malformed metadata header is now flagged in the Request Log instead of disappearing without a trace, and an organization can add Perplexity and more than one custom provider. Gemini tool calls also get a unique id per call instead of ids that repeat across responses, which previously broke agents that track calls by id across multiple rounds. +- **Guardrails on Every Route:** Input guardrails now check request text on the native Messages, Responses, and GenAI routes, and output guardrails block a non-streamed response on every route with that route's own error. A blocked response is never cached. The Guardrails Test tab shows a warning for rules in Warn or Log mode, along with the real HTTP status code, and a guardrail whose authentication fails is treated as a failure. On self-hosted installs, the Future AGI Eval guardrail uses your own deployment and credentials. -- **Gateway Model List Matches Key Access:** The gateway's model list no longer advertises provider models an API key is not entitled to call; it now lists only the models the key can actually use. - -- **Guardrail Credentials and Request Logs Stay Scoped to Their Organization:** A gateway configuration could previously decrypt another organization's stored guardrail credentials, and gateway traffic spanning multiple organizations in one batch could file requests, webhook events, and usage under the wrong one. Both are now resolved per organization before anything is stored or decrypted. - -- **More Reliable Guardrail and Gateway Config Saves:** Activating a Guardrails or Fallbacks change no longer resets an organization's other settings, such as budgets, usage limits, and IP restrictions, back to their defaults, and toggling or editing a guardrail no longer fails for organizations managed through the Settings editor. The UI also now warns clearly when a save or rollback did not reach the gateway, instead of reporting success. - -- **Guardrail Testing Shows the Real Outcome:** The Guardrails Test tab now shows a warning for rules in Warn or Log mode instead of reporting a plain OK, along with the real HTTP status code. A guardrail whose authentication fails is now treated as a failure instead of silently calling the check with no credentials. - -- **Future AGI Eval Guardrail Works on Self-Hosted Installs:** The Future AGI Eval guardrail no longer defaults its Base URL to our cloud endpoint, so self-hosted organizations get their own credentials injected instead of the check running with no, or the wrong, credentials. +- **Guardrail and Gateway Settings Stay Scoped and Intact:** Guardrail credentials, requests, webhook events, and usage are now always kept within their own organization, including when one batch spans several organizations. Saving a Guardrails or Fallbacks change keeps your other settings, such as budgets, usage limits, and IP restrictions, and guardrails can be toggled and edited for organizations managed through the Settings editor. The UI also tells you when a save or rollback did not reach the gateway. - **SQS Export Reliability:** The SQS export integration now retries individual failed messages and only advances past a batch once every record in it is confirmed delivered, instead of silently skipping undelivered records. @@ -59,8 +47,6 @@ description: "Latest Future AGI release notes covering new features, improvement -{/* SKIPPED: internal-only churn (version-bump and branch-sync merges, generated-contract and tool-manifest regeneration, test-only changes); two local-development-only contract-validation fixes confirmed to have no effect in production; a short-lived SDK issue introduced and fixed within this same window with no net customer impact; an internal operational incident whose only customer-facing outcome is the Slack-alert item above. Held for confirmation: environment, agent and scenario-suite versioning with run comparison and rollback (merged but still behind platform flags that were off in production as of this release); multi-agent framework reference fixtures (partial hosted coverage per the author); runtime sync of the pricing catalogue from a public source (only the static half merged); nesting of sub-agent traces under their parent call (the fix lives in a dependency outside this week's reviewed repositories); comparing an agent's turns against its own prompt and tool calls as part of a Debug Failures investigation (the data is now sent, but the comparison logic itself lives in a dependency outside this week's reviewed repositories, so it cannot be confirmed as live). */} - ## Week of 2026-10-19
From d52c8e367c037b855a6872e4cba45a91e2faf7a3 Mon Sep 17 00:00:00 2001 From: jedigalt Date: Wed, 11 Nov 2026 16:09:23 +0530 Subject: [PATCH 5/7] docs(release-notes): keep the self-hosted Request Log upgrade note Carried over from the week of 2026-10-26, where the duplicate gateway items were removed. Co-Authored-By: Claude Opus 5.5 --- src/pages/docs/release-notes.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/pages/docs/release-notes.mdx b/src/pages/docs/release-notes.mdx index 74cbbbd21..17acd7a53 100644 --- a/src/pages/docs/release-notes.mdx +++ b/src/pages/docs/release-notes.mdx @@ -33,7 +33,7 @@ description: "Latest Future AGI release notes covering new features, improvement - **Gateway Cost, Budgets, and Alerts:** Cost is now tracked for streamed Responses calls, image, rerank, OCR, and GenAI embedding requests, and streams that end before usage arrives, which are priced from an estimate. Anything without a known price is clearly marked as unpriced. Models offered under more than one provider name, or through a reseller, are billed at their real rate, and an additional public pricing source fills gaps in the model catalogue. Budgets now use the same prices you see in the Request Log, so a budget stops spend at the limit you set, and budget-exceeded webhooks and email alerts are now sent. -- **Gateway Streaming, Request Log, and Providers:** Streams on chat, Messages, Responses, Completions, GenAI, speech, and Assistants runs now end with a clear error event when a request times out or the provider fails, and carry cost and latency metadata on every route. Messages requests report the timeout that was applied. Each request appears once in the Request Log, and a malformed metadata header is flagged there instead of disappearing without a trace. gpt-5.6 models support tool calls over the Responses API, Gemini tool calls each get their own id, an organization can add Perplexity and more than one custom provider, and the model list shows only the models an API key can use. +- **Gateway Streaming, Request Log, and Providers:** Streams on chat, Messages, Responses, Completions, GenAI, speech, and Assistants runs now end with a clear error event when a request times out or the provider fails, and carry cost and latency metadata on every route. Messages requests report the timeout that was applied. Each request appears once in the Request Log, and a malformed metadata header is flagged there instead of disappearing without a trace. On self-hosted deployments, the upgrade removes existing duplicate Request Log rows, which can take a while on a large table. gpt-5.6 models support tool calls over the Responses API, Gemini tool calls each get their own id, an organization can add Perplexity and more than one custom provider, and the model list shows only the models an API key can use. - **Guardrails on Every Route:** Input guardrails now check request text on the native Messages, Responses, and GenAI routes, and output guardrails block a non-streamed response on every route with that route's own error. A blocked response is never cached. The Guardrails Test tab shows a warning for rules in Warn or Log mode, along with the real HTTP status code, and a guardrail whose authentication fails is treated as a failure. On self-hosted installs, the Future AGI Eval guardrail uses your own deployment and credentials. From 898068cbee3e86419dc2c2994d39145108d1b6eb Mon Sep 17 00:00:00 2001 From: jedigalt Date: Wed, 11 Nov 2026 16:44:19 +0530 Subject: [PATCH 6/7] docs(release-notes): keep only v1.57/v1.58 gateway items in week of 2026-11-02 The stream, Request Log, provider, cost, budget and guardrail-route fixes first shipped in v1.56.1 and now live in the week of 2026-10-26. What remains: models.dev pricing gaps, key-scoped model list, Gemini tool call ids, Guardrails Test tab, auth failures and self-hosted Eval guardrail. Co-Authored-By: Claude Opus 5.5 --- src/pages/docs/release-notes.mdx | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/src/pages/docs/release-notes.mdx b/src/pages/docs/release-notes.mdx index 17acd7a53..c50a606c2 100644 --- a/src/pages/docs/release-notes.mdx +++ b/src/pages/docs/release-notes.mdx @@ -31,11 +31,9 @@ description: "Latest Future AGI release notes covering new features, improvement - **Prevent Silent Blank Dataset Rows:** Writing dataset rows through the platform's AI assistant or its MCP tools with an unrecognized column, or in a malformed batch, is now rejected with a clear error instead of silently saved as blank rows. -- **Gateway Cost, Budgets, and Alerts:** Cost is now tracked for streamed Responses calls, image, rerank, OCR, and GenAI embedding requests, and streams that end before usage arrives, which are priced from an estimate. Anything without a known price is clearly marked as unpriced. Models offered under more than one provider name, or through a reseller, are billed at their real rate, and an additional public pricing source fills gaps in the model catalogue. Budgets now use the same prices you see in the Request Log, so a budget stops spend at the limit you set, and budget-exceeded webhooks and email alerts are now sent. +- **Gateway Pricing and Model Access:** An additional public pricing source now fills gaps in the model catalogue, the model list shows only the models an API key can use, and Gemini tool calls each get their own id. -- **Gateway Streaming, Request Log, and Providers:** Streams on chat, Messages, Responses, Completions, GenAI, speech, and Assistants runs now end with a clear error event when a request times out or the provider fails, and carry cost and latency metadata on every route. Messages requests report the timeout that was applied. Each request appears once in the Request Log, and a malformed metadata header is flagged there instead of disappearing without a trace. On self-hosted deployments, the upgrade removes existing duplicate Request Log rows, which can take a while on a large table. gpt-5.6 models support tool calls over the Responses API, Gemini tool calls each get their own id, an organization can add Perplexity and more than one custom provider, and the model list shows only the models an API key can use. - -- **Guardrails on Every Route:** Input guardrails now check request text on the native Messages, Responses, and GenAI routes, and output guardrails block a non-streamed response on every route with that route's own error. A blocked response is never cached. The Guardrails Test tab shows a warning for rules in Warn or Log mode, along with the real HTTP status code, and a guardrail whose authentication fails is treated as a failure. On self-hosted installs, the Future AGI Eval guardrail uses your own deployment and credentials. +- **Guardrail Testing and Self-Hosted Eval:** The Guardrails Test tab shows a warning for rules in Warn or Log mode, along with the real HTTP status code, and a guardrail whose authentication fails is treated as a failure. On self-hosted installs, the Future AGI Eval guardrail uses your own deployment and credentials. - **Guardrail and Gateway Settings Stay Scoped and Intact:** Guardrail credentials, requests, webhook events, and usage are now always kept within their own organization, including when one batch spans several organizations. Saving a Guardrails or Fallbacks change keeps your other settings, such as budgets, usage limits, and IP restrictions, and guardrails can be toggled and edited for organizations managed through the Settings editor. The UI also tells you when a save or rollback did not reach the gateway. From 969ae73be0375c85b6ddd6610b47992c43ea5bdf Mon Sep 17 00:00:00 2001 From: jedigalt Date: Wed, 11 Nov 2026 18:02:47 +0530 Subject: [PATCH 7/7] docs(release-notes): correct four claims in week of 2026-11-02 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - The auth-failure fix covers the CrowdStrike guardrail only. - Parallelism applies to chat runs (24-scenario suite: 31–38 min to 6.8 min), not builds; retitled accordingly. - Approve-all in the builder chat keeps separate approval for deletions and structural changes. - The REST API also rejects malformed dataset row writes. Co-Authored-By: Claude Opus 5.5 --- src/pages/docs/release-notes.mdx | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/src/pages/docs/release-notes.mdx b/src/pages/docs/release-notes.mdx index c50a606c2..f04270dbe 100644 --- a/src/pages/docs/release-notes.mdx +++ b/src/pages/docs/release-notes.mdx @@ -15,9 +15,9 @@ description: "Latest Future AGI release notes covering new features, improvement
Bugs/Improvements
-- **Environment Workspace Fixes:** Hover tooltips were removed from the environment workspace tabs, and an environment's scenario count now updates live instead of freezing at the first value it loaded. The environment builder chat also now offers to approve every change in a request at once, instead of confirming each edit separately. +- **Environment Workspace Fixes:** Hover tooltips were removed from the environment workspace tabs, and an environment's scenario count now updates live instead of freezing at the first value it loaded. The environment builder chat also lets you approve all the changes in a request with one answer, while deletions and structural changes still ask for their own approval. -- **Faster, More Reliable Environment Builds:** Agents that read their assigned port now build and test in parallel, agents that store their data in an embedded SQLite database are now supported, and environments created on an earlier platform version keep working after an upgrade. Builds and checks also no longer fail more often when many environments run at the same time. +- **Faster Environment Runs, More Reliable Builds:** Chat runs for agents that read their assigned port now run in parallel, cutting a 24-scenario suite from 31 to 38 minutes to 6.8 minutes. Agents that store their data in an embedded SQLite database are now supported, and environments created on an earlier platform version keep working after an upgrade. Builds and checks also no longer fail more often when many environments run at the same time. - **Clearer Debug Failures Results:** Debug Failures investigations now combine matching findings into a single entry, come back to findings that were set aside, and clearly label a goal that has no findings. @@ -29,11 +29,11 @@ description: "Latest Future AGI release notes covering new features, improvement - **More Reliable Few-Shot Evaluation Inputs:** Few-shot evaluation inputs that are zero or false are no longer dropped as if they were empty, and structured (JSON) inputs are now matched and sent to the embedding model correctly instead of failing outright. -- **Prevent Silent Blank Dataset Rows:** Writing dataset rows through the platform's AI assistant or its MCP tools with an unrecognized column, or in a malformed batch, is now rejected with a clear error instead of silently saved as blank rows. +- **Prevent Silent Blank Dataset Rows:** Writing dataset rows through the REST API, the platform's AI assistant or its MCP tools with an unrecognized column, or in a malformed batch, is now rejected with a clear error instead of silently saved as blank rows. - **Gateway Pricing and Model Access:** An additional public pricing source now fills gaps in the model catalogue, the model list shows only the models an API key can use, and Gemini tool calls each get their own id. -- **Guardrail Testing and Self-Hosted Eval:** The Guardrails Test tab shows a warning for rules in Warn or Log mode, along with the real HTTP status code, and a guardrail whose authentication fails is treated as a failure. On self-hosted installs, the Future AGI Eval guardrail uses your own deployment and credentials. +- **Guardrail Testing and Self-Hosted Eval:** The Guardrails Test tab shows a warning for rules in Warn or Log mode, along with the real HTTP status code, and a CrowdStrike guardrail whose authentication fails is treated as a failure. On self-hosted installs, the Future AGI Eval guardrail uses your own deployment and credentials. - **Guardrail and Gateway Settings Stay Scoped and Intact:** Guardrail credentials, requests, webhook events, and usage are now always kept within their own organization, including when one batch spans several organizations. Saving a Guardrails or Fallbacks change keeps your other settings, such as budgets, usage limits, and IP restrictions, and guardrails can be toggled and edited for organizations managed through the Settings editor. The UI also tells you when a save or rollback did not reach the gateway.