From 48c4c6ae7115a0780e2096dffcfa8c6b8271d3c6 Mon Sep 17 00:00:00 2001 From: chrikrah <48338417+chrikrah@users.noreply.github.com> Date: Sun, 11 Oct 2026 14:32:18 +0200 Subject: [PATCH 1/2] Add wiretruth to LLM & MCP Testing --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index b83b3db..e7f046c 100644 --- a/README.md +++ b/README.md @@ -86,6 +86,7 @@ Collection of awesome Python resources for testing and generating test data. - [OrcaReplay](https://github.com/Continuum-AI-Corp/OrcaReplay) - Records a Python agent's calls to its model provider from outside the process, then serves the recording back so the same run happens again with the provider unreachable and no key. Turns a failed session into a regression test. Apache-2.0. - [Sabot](https://github.com/Jott2121/sabot) - Plants controlled faults inside running multi-agent pipelines (LangGraph, CrewAI, AutoGen) and scores whether each pipeline's own reviewer and guardrail stages detect them. Pre-registered spec, deterministic adjudication, published raw traces. - [Tenro](https://github.com/tenro-ai/tenro-python) - An open-source, provider-agnostic testing framework for AI agents that integrates with pytest. It simulates LLM and tool calls to test edge cases, failure paths, and agent logic without live API calls. +- [wiretruth](https://github.com/Plexito-de/wiretruth) - Test suite for Python LLM telemetry libraries such as OpenLLMetry and OpenInference. A local fake provider serves OpenAI and Anthropic responses, including streams, prompt-cache hits, 429s and dropped connections, and wiretruth checks the token counts, models and errors on the resulting spans, and whether each span ends, against what was sent. Apache-2.0. ## Load Testing From 7fb065f93c3d4a2c34871c3d2f289edd7b744b56 Mon Sep 17 00:00:00 2001 From: Christopher Krah Date: Mon, 12 Oct 2026 23:09:39 +0200 Subject: [PATCH 2/2] Rename wiretruth to spantruth --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index e7f046c..c357289 100644 --- a/README.md +++ b/README.md @@ -85,8 +85,8 @@ Collection of awesome Python resources for testing and generating test data. - [mcp-server-fuzzer](https://github.com/Agent-Hellboy/mcp-server-fuzzer) - A comprehensive fuzzing tool designed specifically for testing Model Context Protocol (MCP) servers. It supports both tool argument fuzzing and protocol type fuzzing across multiple transport protocols. - [OrcaReplay](https://github.com/Continuum-AI-Corp/OrcaReplay) - Records a Python agent's calls to its model provider from outside the process, then serves the recording back so the same run happens again with the provider unreachable and no key. Turns a failed session into a regression test. Apache-2.0. - [Sabot](https://github.com/Jott2121/sabot) - Plants controlled faults inside running multi-agent pipelines (LangGraph, CrewAI, AutoGen) and scores whether each pipeline's own reviewer and guardrail stages detect them. Pre-registered spec, deterministic adjudication, published raw traces. +- [spantruth](https://github.com/Plexito-de/spantruth) - Test suite for Python LLM telemetry libraries such as OpenLLMetry and OpenInference. A local fake provider serves OpenAI and Anthropic responses, including streams, prompt-cache hits, 429s and dropped connections, and spantruth checks the token counts, models and errors on the resulting spans, and whether each span ends, against what was sent. Apache-2.0. - [Tenro](https://github.com/tenro-ai/tenro-python) - An open-source, provider-agnostic testing framework for AI agents that integrates with pytest. It simulates LLM and tool calls to test edge cases, failure paths, and agent logic without live API calls. -- [wiretruth](https://github.com/Plexito-de/wiretruth) - Test suite for Python LLM telemetry libraries such as OpenLLMetry and OpenInference. A local fake provider serves OpenAI and Anthropic responses, including streams, prompt-cache hits, 429s and dropped connections, and wiretruth checks the token counts, models and errors on the resulting spans, and whether each span ends, against what was sent. Apache-2.0. ## Load Testing