Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
f91e16c
feat: add MiniMax LLM provider instrumentation
Mar 15, 2026
20ef0bf
feat: upgrade MiniMax default model to M2.7
octo-patch Mar 18, 2026
7150178
Address code review feedback from @NVJKKartik
octo-patch Mar 19, 2026
8d296ff
Merge pull request #179 from future-agi/dev
JayaSurya-27 May 15, 2026
7558c6b
fix(otel): exporter shutdown data loss + thread safety in span proces…
pmady Jun 3, 2026
3ef6e5d
Merge pull request #183 from future-agi/dev
NVJKKartik Jun 3, 2026
c624d4b
Merge pull request #140 from octo-patch/feature/add-minimax-provider
NVJKKartik Jun 15, 2026
bdfc41b
Merge pull request #190 from future-agi/dev
JayaSurya-27 Jul 30, 2026
54421a6
Merge pull request #196 from future-agi/dev
JayaSurya-27 Aug 27, 2026
410b5df
fix(livekit): keep the Future AGI exporter when LiveKit adds its priv…
khushalsonawat Oct 19, 2026
3bfa974
chore(livekit): run the tests on livekit-agents 1.8 in the package's …
khushalsonawat Oct 19, 2026
db2c1c2
ci: run each touched Python package's tests on every PR, and fix fi_i…
khushalsonawat Oct 19, 2026
278c170
fix(livekit): send the conversation content LiveKit 1.7+ records, and…
khushalsonawat Oct 19, 2026
3399575
fix(livekit): map a copy of each span, never the span other exporters…
khushalsonawat Oct 19, 2026
d87fd79
ci: a visible known-broken list, and make 23 packages installable (TH…
khushalsonawat Oct 19, 2026
aa6c3fe
ci: list the 15 packages that now install but fail their tests, and t…
khushalsonawat Oct 19, 2026
c1225f1
ci: list four packages whose locked fi-instrumentation-otel predates …
khushalsonawat Oct 19, 2026
6db150e
ci: test packages against the fi-instrumentation-otel customers get, …
khushalsonawat Oct 19, 2026
ffde393
ci: take anthropic, crewai, google-adk and openai off the known-broke…
khushalsonawat Oct 19, 2026
2ddc7b9
ci: test the traceai SDK too, and correct list reasons and comments (…
khushalsonawat Oct 19, 2026
4da156d
ci: run on PRs into dev too, and make the traceai SDK installable (TH…
khushalsonawat Oct 19, 2026
5838b23
ci: judge a listed package by its test outcome, and say when dev gets…
khushalsonawat Oct 19, 2026
272bc6c
ci: separate the failed steps in the known-broken verdict, and name a…
khushalsonawat Oct 19, 2026
57c9e00
ci: plan moved files and core changes in full, and test the planner (…
khushalsonawat Oct 19, 2026
5f8e90a
ci: never pass on an empty plan or on package jobs that did not run (…
khushalsonawat Oct 19, 2026
b529fd8
Merge pull request #274 from future-agi/ci/TH-8673-python-tests
khushalsonawat Oct 19, 2026
22a21df
Merge main into fix/TH-8665-livekit-keep-exporter (CI from TH-8673)
khushalsonawat Oct 19, 2026
708a623
Merge pull request #273 from future-agi/fix/TH-8665-livekit-keep-expo…
khushalsonawat Oct 19, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 37 additions & 0 deletions .github/python-tests-known-broken.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
# Python packages whose tests are known to fail today (see .github/workflows/python-tests.yml).
# A listed package still runs in CI and shows a warning, but its failures do not fail the required "Python tests" check.
# If a listed package's tests all pass, CI fails until its line is removed from this file.
# So a PR that makes a listed package pass must remove its line in the same PR, or CI fails
# (for example, a LangChain change that fixes its tests). A listed package with no tests at all also fails: remove its line.
# Each entry should name its follow-up (a ticket or PR) in the reason once one exists.
# Format: <package name: its dir under python/frameworks, or traceai for python/traceai> | <reason in plain words>
agno | 1 test expects gen_ai.system where the code now writes gen_ai.provider.name
autogen | 17 tests (12 own, 5 core): the instrumentor no longer has _original_generate or _safe_json_dumps, traceai_autogen no longer exports _is_v04_available, and a dependency name
bedrock | 1 test expects image.url where the code now writes message_content.image
cerebras | 3 tests expect the reply text where the code now records the whole response
cohere | 5 tests: llm.system and mock constants that no longer exist, and generated text that is no longer recorded
deepseek | 4 tests: mock constants that no longer exist, llm.system, and generated text that is no longer recorded
dspy | 1 test expects raw.output on embedding spans
fireworks | 3 tests expect the reply text where the code now records the whole response
groq | 5 tests: pre-GenAI attribute names (fi.span.kind, eval.input) and missing output checks in 4, and a string instead of a dependency tuple in 1
guardrails | its core test file does not load (SpanAttributes.FI_SPAN_KIND no longer exists)
haystack | 5 tests expect pre-GenAI attribute names (llm.token_count.*, fi.span.kind) and a list of dependencies
huggingface | 3 tests expect llm.system; 1 expects generated text that is no longer recorded
instructor | 6 tests: instructor.core no longer exists in current instructor (5), and a dependency string check (1)
langchain | 2 tests: dependency count and a set_attributes call no longer made
litellm | 3 tests use SpanAttributes.FI_SPAN_KIND, which no longer exists
llama_index | 14 tests: llama_index.core.base.agent no longer exists; 1 expects SpanAttributes.LLM_MODEL_NAME; 1 a dependency count
mcp | 1 test expects two instrumentation dependencies, the package declares one
mistralai | 1 test expects raw.input where the code now writes input.value
ollama | 2 tests expect llm.system; 2 expect generated text and embedding size that are no longer recorded
openai-agents | its own tests do not load (conftest imports traceai_openai, not installed); 2 core tests expect raw.output and a trace-start call
pinecone | 5 tests error: pinecone-client is now a stub that raises "renamed to pinecone"; the package has to move to the pinecone package
portkey | 10 tests expect pre-GenAI attribute names (llm.model_name, llm.invocation_parameters)
pydantic-ai | 1 test builds an agent with model "test-model", which pydantic-ai no longer accepts
qdrant | 2 tests error: QdrantClient has no query method in current qdrant-client
smolagents | 6 tests expect pre-GenAI attribute names; its own test file does not load (smolagents.models no longer has ChatMessageToolCallDefinition)
together | 5 tests: llm.system and mock constants that no longer exist, and generated text that is no longer recorded
traceai | 1 test expects f"{SemanticConvention.FI}" to give "fi"; on Python 3.11 and later it gives "SemanticConvention.FI"
vertexai | 2 tests expect pre-GenAI attribute names (llm.output_messages, llm.token_count.prompt)
vllm | 3 tests expect the reply text where the code now records the whole response
xai | 3 tests expect the reply text where the code now records the whole response
39 changes: 39 additions & 0 deletions .github/scripts/known_broken_verdict.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
#!/usr/bin/env bash
# The "Known-broken verdict" step of .github/workflows/python-tests.yml (tested in test_python_test_plan.py).
# Reads PKG_NAME, REASON, INSTALL_POETRY, INSTALL_HATCH, CORE_INSTALL, TESTS, OWN_TESTS and CORE_TESTS from the environment.
set -eo pipefail

echo "outcomes: install (poetry)=$INSTALL_POETRY install (hatch)=$INSTALL_HATCH fi_instrumentation=$CORE_INSTALL tests=$TESTS"
install_failed=""
[ "$INSTALL_POETRY" = "failure" ] && install_failed="${install_failed:+$install_failed, }install (poetry)"
[ "$INSTALL_HATCH" = "failure" ] && install_failed="${install_failed:+$install_failed, }install (hatch)"
[ "$CORE_INSTALL" = "failure" ] && install_failed="${install_failed:+$install_failed, }fi_instrumentation install"
# Any other Tests outcome (empty, or a renamed step id) is a wiring bug: fail rather than warn.
case "$TESTS" in
success|failure|cancelled|skipped) ;;
*)
echo "::error::$PKG_NAME: unexpected Tests outcome '$TESTS'; check the verdict step's env in python-tests.yml"
exit 1
;;
esac
# The Tests step is skipped only when an earlier step (checkout or setup) failed outright, or
# the job was cancelled: that is not a pass, and the job fails with it.
if [ "$TESTS" = "skipped" ]; then
echo "::error::$PKG_NAME is known broken ($REASON), but its tests did not run (an earlier step failed, or the job was cancelled)"
exit 1
fi
# The tests passed (whatever an install step did): the line has to come off the list.
if [ "$TESTS" = "success" ]; then
if [ "$OWN_TESTS" != "true" ] && [ -z "$CORE_TESTS" ]; then
echo "::error::$PKG_NAME is on the known-broken list but has no tests; remove its line from .github/python-tests-known-broken.txt"
else
echo "::error::$PKG_NAME is on the known-broken list but its tests pass; remove its line from .github/python-tests-known-broken.txt${install_failed:+ (failed in this run: $install_failed)}"
fi
exit 1
fi
# The tests failed. After a failed install that may not be the listed reason: say so.
if [ -n "$install_failed" ]; then
echo "::warning::$PKG_NAME is known broken, but in this run these steps failed: $install_failed; its test results may not reflect the listed reason ($REASON)"
exit 0
fi
echo "::warning::$PKG_NAME is known broken ($REASON): its failures do not fail the check"
Loading
Loading