Skip to content

feat(examples): EverOS native OTel tracing recipe and contract test (TH-8329) - #232

Merged
nik13 merged 12 commits into
devfrom
feat/th-8329-everos
Oct 26, 2026
Merged

nik13 merged 12 commits into
devfrom
feat/th-8329-everos

Conversation

@nik13

@nik13 nik13 commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

Status 2026-10-21: merged dev into the branch (4880866, no conflicts). CI: Plan, fi_instrumentation and Python tests are green. This is a recipe under python/examples, not a package, so no package job runs. Local recipe suite with the README command (Python 3.12, everos 1.4.2): 37 passed. Nothing to publish. Note: everos 1.5.0 was released on 2026-10-21 and is untested. The recipe and docs#1004 pin 1.4.2, and the live tests accept only 1.4.1/1.4.2.


Summary

This PR adds an EverOS → Future AGI tracing recipe at python/examples/everos/ (Linear TH-8329, decision D-8329). There is no traceai-everos package. EverOS (EverMind-AI/EverOS, PyPI everos, Apache-2.0) has exported its own OpenTelemetry spans since 1.2.0, off by default. This recipe only turns that export on and points it at fi-collector. It adds no instrumentor, no span processor and no HTTP wrapper.

Recipe. The settings go in EverOS's own [observability] keys, as env vars or in ~/.everos/everos.toml (config/settings.py:624-668):

  • EVEROS_OBSERVABILITY__ENABLED=true
  • EVEROS_OBSERVABILITY__ENDPOINT=https://api.futureagi.com/tracer/v1/traces. The full path is required because EverOS uses the value verbatim (provider.py:134-138); a bare origin posts to / and gets a 404 (tested).
  • EVEROS_OBSERVABILITY__HEADERS as JSON with X-Api-Key/X-Secret-Key. Values containing ,, = or spaces need no encoding (tested).
  • OTEL_RESOURCE_ATTRIBUTES=project_name=…,project_type=observe. EverOS has no resource setting, so this is the only way to set the project (provider.py:127-129).
  • Content capture stays at EverOS's default, which is off.

Tests and fixtures. The fixtures are a live recording of EverOS 1.4.1's own spans, not a hand-built body. tests/everos_session.py runs everos init, then EverOS's own create_app() (what everos server start serves) through Starlette's in-process TestClient:

  • no port is bound and nothing is wrapped;
  • the app's real lifespan installs EverOS's tracer from EVEROS_OBSERVABILITY__*;
  • the model and embedding endpoints are a loopback OpenAI fake;
  • a loopback guard, with a positive control, refuses and logs any other connection.

The recordings, one with capture off and one with capture on, are posted with the harness post_otlp() and checked with compare().

What arrives, and what fi-collector does with it (source reading at future-agi main 4af5338)

EverOS span (1.4.1) Source Stored kind
everos.memory.add, .flush, everos.memcell.boundary, everos.extract, everos.persist.markdown service/memorize.py:211-260, user_memory.py:117-178 unknown
everos.memory.search, everos.search.recall, everos.search.rank memory/search/manager.py:192-676 unknown
everos.embedding, everos.ome.<strategy> openai_provider.py:121-123, ome/_dispatch/runner.py:229-238 unknown
  • Kind is unknown for every span. fi-collector reads fi.span.kind, gen_ai.span.kind, llm.request.type and openinference.span.kind, then falls back to gen_ai.operation.name (exporter/clickhouse25exporter/converter.go:79-131). EverOS sets none of them; the tests assert this. See D2 below.
  • Model and tokens fill:
    • gen_ai.request.model and gen_ai.usage.input_tokens/output_tokens are on the alias lists (pkg/adapter/adapter.go:279-310).
    • EverOS writes them only when tracing is on (component/llm/client.py:111-114).
    • They also appear on the agent-typed everos.ome.extract_atomic_facts span, because the usage wrapper writes to whatever span is current (_usage_client.py:53-57).
  • No cost attribute is exported (tested). fi-collector may still price the tokens itself when the model is in its price table (converter.go:285-294). That was not tested against a collector.

Privacy

  • Capture off (the default): in a run where nothing fails, no query text and no extracted memory text (tested); error text is the exception, below. The content switch gates only langfuse.observation.input/output (core/observability/tracing/attributes.py:68-77).
  • Still exported with capture off (tested, documented in the README):
    • session, user, owner, app, project, memcell, request and run ids;
    • model names and the search's top score;
    • error text, in every capture setting (verify-r1 V1, fixed in docs at d602c40). OTel records any exception as an ERROR status description and an exception event (message plus chained stack trace) on the failing span and every EverOS span it propagates out of. capture_content, set_redactor and the 4096-character cap don't apply to those channels. Two cases: provider error bodies (OpenAI's client puts the whole body in the message and EverOS keeps it, component/embedding/openai_provider.py:39-49, 121-134, component/llm/openai_provider.py:106-109; tested for embeddings: a 400 that quotes the query plus a marker 4,200 characters in lands in status and events of everos.embedding, everos.search.recall and everos.memory.search, in no attribute; the chat path is source reading), and a model reply quoted in an extraction error (tested). The README advises keeping tracing off for inputs that must not leave the process.
  • EverOS's "redaction" is an identity hook unless the app calls set_redactor (attributes.py:46, 55-58). The recipe cannot attach a span processor: EverOS builds a module-local provider and processor, not the OTel global (provider.py:130-139, tracing_lifespan.py:42). So the README documents these exceptions instead of filtering them.

Deviations from the spec (approved: Nikhil 2026-10-03 blanket)

  • D1, live recording as the fixture. The architecture asked for a hand-built body and "no live EverOS server". This recording is in-process, with no server and no port, which the brief preferred when feasible. post_otlp() + compare() are still used.
  • D2, kinds (PRD AC-03) not delivered → SF-1.
    • Why no processor: PRD r2 §0 makes the shared processors (SF-1) the home for kind mapping, and EverOS's own provider leaves nowhere to attach one.
    • Input for SF-1: EverOS already stamps langfuse.observation.type (attributes.py:126). Mapped the way the backend Langfuse adapter maps it (futureagi/tracer/utils/adapters/langfuse.py:40-51), it would give the architecture's kinds. The exception is everos.search.rank, which EverOS types span (CHAIN), not RETRIEVER.
  • D3, cost: the export has none, but fi-collector may compute one (above).
  • D4, span set: no names were renamed in 1.4.1.
    • Extra spans: everos.memory.flush, everos.persist.markdown and everos.ome.<strategy>.
    • everos.memory.buffer and everos.reflect.consolidate exist in source only.
  • D5, PRD O-2: EverOS has its own endpoint and headers keys. OTel's OTEL_EXPORTER_OTLP_* also works; it posts to <base>/v1/traces and merges per header (tested). The docs draft is corrected in company-brain Dev #63.
  • D6, versions: the pin is ==1.4.1, the only version tested; the 1.2.0 floor is stated as untested. Python 3.12 and 3.13 only, because EverOS requires >=3.12 (METADATA:22).
  • D7, capture off still exports identifiers and error text (above).

Review round 1 (pr-reviewer t_8be03933 APPROVE; pr-verifier t_049d8fb2 CHANGES_REQUESTED at 62b2d18)

  • V1 (P2), fixed at d602c40 (D-8329a, approved: Nikhil 2026-10-03 blanket): the capture-off caveat named only extraction failures; provider error bodies are a wider channel (see Privacy). Decision: document and pin, no filter. Rejected: a recipe-local export filter around EverOS's own provider (EverOS builds a module-local provider/processor, so it would be a new instrumentation component beyond D-8329, untested against EverOS internals) — recorded as a follow-up option.

  • R1 (P3), fixed: the README said tokens are on generation and embedding spans only; the agent-typed everos.ome.extract_atomic_facts also carries the call's model and tokens. A fixture-driven README test now checks the Model/Tokens bullets name every span that carries them.

  • RED (tests before docs): on the 62b2d18 README the three new README tests fail (Model, Tokens, error-text caveat); 34 passed. The new embedding_error live run passes before and after by design: it pins existing EverOS behaviour, which the README now documents.

  • Round 2: pr-verifier t_32c56149 VERIFIED at d602c40, no blocking findings; V1 and R1 resolved. V2 (P3, this body's capture-off wording) fixed here. Optional follow-ups: check the two error runs for the secret/message markers too (source tracing shows neither fake error path quotes them); the recipe-local status/event filter stays a rejected follow-up option.

Review round 2 (pr-verifier t_32c56149 at d602c40: VERIFIED)

V1 resolved: the README states that error text (provider error bodies, extraction replies) is exported in ERROR status, exception.message and stack traces in every capture setting and past the 4,096-character cap; tests pin it on EverOS 1.4.1 / OTel 1.45.0, and the README test fails 17/17 on 62b2d18. R1 resolved. No blocking findings. V2 (P3, wording) fixed in this body's Privacy section. Optional follow-ups retained: the two error runs are not separately scanned for the placeholder keys or message marker (source tracing shows neither fake error path quotes them), and a recipe-local status/event filter stays a rejected option under D-8329a. Companion docs and architecture corrected to this head in company-brain #63 (bdfcea9, open).

Tests

Head d602c40. Coordinator rerun from the repo root on a clean tree, in a scrubbed environment, with real process exit codes and JUnit (tickets/TH-8329/exact-head-d602c40-checked/results.json):

env -u PYTHONPATH PYTHONPATH="python:python/tests" uv run --no-project --python <py> \
  --with pytest --with 'everos[otel]==1.4.1' \
  --with 'opentelemetry-sdk==1.45.0' --with 'opentelemetry-exporter-otlp-proto-http==1.45.0' \
  --with requests --with jsonschema --with wrapt \
  pytest python/examples/everos/tests -q -p no:cacheprovider --noconftest -o addopts= -rfEs
Python 3.12   37 passed
Python 3.13   37 passed
Python 3.12 without everos installed (fixture half): 20 passed, 17 skipped
Python 3.10 / 3.11: not run (EverOS requires >=3.12)
  • 12 files, all under python/examples/everos/ (the fix round touched README, tests and the OpenAI fake). 0 secret-like added lines, and 0 host paths in the tracked fixtures.
  • The scenario uses an offline stand-in for tiktoken's o200k_base. The guard caught tiktoken's download from openaipublic.blob.core.windows.net, and token counts only feed everalgo's boundary limits, never a span.

Limits

  • There is no fi-collector or backend run. The unknown kind, pricing, 401, project stamping and storage are source reading.
  • The model is a loopback fake, and only one hybrid search was recorded. Background everos.ome.* spans are timing-dependent, so the live-vs-fixture check tolerates missing ones. Request-path spans must match exactly.
  • The app runs in-process via TestClient, not uvicorn.
  • Not tested:
    • the /api/v1 alias;
    • OTEL_EXPORTER_OTLP_TRACES_ENDPOINT;
    • the langfuse_* settings and sample_rate;
    • library use without the server;
    • EverOS without the otel extra.
  • The guard covers Python-level sockets only; native code such as LanceDB is not intercepted.
  • No CI job runs this example.

Video demo

Recorded build: head d602c400b46dc3d2fe08b2b10a247c606240856f (this PR); 10 real terminal chapters of the demo driver from a clean read-only worktree plus intro and review cards; 6:54, 12 chapters, narrated with captions.

The video (SHA-256 838bf2a73fac2f62715835156abf523ef7fbba41b9d9ebc826ca42cb4b9814be), captions (SRT), transcript, chapter list, preview and both media-verification notes are private attachments on Linear issue TH-8329 (Future AGI workspace; sign-in required). They are not linked here as raw upload URLs.

Time Chapter
00:00 Problem: EverOS spans to Future AGI
00:48 head: PR 232 at d602c40
01:20 run: the real recipe scenario
02:20 fixture: compare vs the committed recording
02:49 kinds: observation types and SF-1
03:21 content: capture off vs capture on
03:48 error-text: what capture off does not cover
04:48 config: probe cases
05:14 collector: source reading of fi-collector 4af5338
05:44 tests: live suite and logged matrix
06:11 limits
06:33 Review status

What it shows: real EverOS 1.4.1 in-process against the recipe's loopback OpenAI fake, exporting its own spans to /tracer/v1/traces with header names only; everos.ome.extract_atomic_facts carries model and tokens (R1); the fixture compare passes and a one-value change fails; every span's kind is unknown until SF-1 (source reading); capture off exports no content in a run where nothing fails, capture on adds exactly the four documented attributes; the main D-8329a proof: an embedding 400 that echoes the query puts the query marker and a tail marker at character 4200 (past EverOS's 4,096 cap) in the ERROR status, exception.message and stack trace of everos.embedding, everos.search.recall and everos.memory.search, and in no attribute or resource; the rejected extraction reply lands on extract_atomic_facts; the README section is present at d602c40 and its test fails 17/17 on 62b2d18. Marker text and header values are never printed, only letters, counts and key names.

Independent media checks (Rick): full audio/video decode exit 0; 12 embedded chapters (one sidecar time corrected 03:47 → 03:48 to match the MP4); no audio gap of 1.5 s or more anywhere; every chapter-end frame and the preview inspected (legible, nothing under the captions, no key or header values, local paths or ports); 7 delivered files scanned with the driver's 19 fixture needles plus path, port, header and credential checks: zero hits. Disclosed edits: only static waiting shortened and final real frames held; long output paged from the same live run's saved output (labelled). The recorder started on Claude Opus 5.5 and finished on the configured fallback (GPT-6.1 Sol via Codex). The closing card says the company-brain #63 docs correction is in progress; it has since been verified and pushed (bdfcea9), and no product code changed after recording. Not shown: a live model or embedding provider, EverMind cloud, a running fi-collector, auth, storage or the trace view.

Stacked on #203 (shared harness, 3eaadc8), base dev. Not merged.

Linear: TH-8329. approved: Nikhil 2026-10-03 blanket

Verification sweep (2026-10-12)

Re-verification (2026-10-16)

Head a817a6d (new commit on top of 730afe9; nothing rewritten). Docs page: future-agi/docs#1004 (draft, base main, no navigation.ts; suggested nav: Integrations > traceAI > "EverOS").

Versions. Latest stable everos on PyPI is 1.4.2 (released 2026-10-14); previously pinned 1.4.1. OpenTelemetry 1.45.0. Both versions run on Python 3.12; 1.4.2 also on 3.13.

What was wrong. Only the pin: 1.4.2 existed, and on it test_recipe_resource_is_an_observe_project failed (it hard-coded service.version == 1.4.1). The wheels' core/observability/** and tracing_lifespan.py sources are identical between 1.4.1 and 1.4.2; 1.4.2 changed persistence (LanceDB/index) and the embedding provider. No span renamed.

What changed (commit a817a6d). requirements.txt pins everos[otel]==1.4.2; live tests accept exactly 1.4.1 and 1.4.2 and assert the resource's service.version equals the installed version (any other version fails loudly); README updated (versions, a "Real Future AGI run" note). No change to recipe settings.

Tests. pytest python/examples/everos/tests: 37 passed on 3.12 with 1.4.2; 37 passed on 3.12 with 1.4.1; 37 passed on 3.13 with 1.4.2; without everos installed 20 passed, 17 skipped. Before the test change: 1.4.2 36 passed, 1 failed (above).

Real E2E against local Future AGI (recipe exactly as the README: EVEROS_OBSERVABILITY__* env, endpoint http://localhost:14338/tracer/v1/traces, project via OTEL_RESOURCE_ATTRIBUTES; everos server start; real chat model gpt-6-luna via a local OpenAI-compatible proxy; real embeddings gemini-embedding-001; one add, flush, search over HTTP):

run project result
1.4.2 capture on th8103-rick-everos-on142 15 spans / 3 traces
1.4.2 capture off th8103-rick-everos-off142 15 spans / 3 traces
1.4.1 capture off th8103-rick-everos-off141 15 spans / 3 traces
1.4.2 failure (embedding key rejected, HTTP 400) th8103-rick-everos-err142 6 spans / 3 traces, 3 ERROR

1.4.2 capture-off trace ids: add 1ed90493-51dd-2a53-6dc9-d643ccb6a779, flush 7527966d-e14c-26da-966b-61010f125aed, search cadc9e02-5d76-a275-01c4-49c1fed4a8d3. Failure search trace 8a6cd544-5115-2cf3-f342-24b7e1009475.

fi_verify excerpt (1.4.2, off): everos.memory.flush > everos.memcell.boundary model=gpt-6-luna tok=1509/114, everos.extract model=gpt-6-luna tok=2051/295, everos.persist.markdown, everos.ome.extract_atomic_facts tok=1531/230, everos.ome.trigger_profile_clustering > everos.embedding model=gemini-embedding-001; search: everos.memory.search > everos.search.recall > everos.embedding, everos.search.rank. Hierarchy correct, model and tokens filled where EverOS records them. Failure: everos.embedding, everos.search.recall, everos.memory.search stored ERROR with EmbeddingInputError: embedding input rejected (400).... fi_verify PASS on all four. All spans stored with kind unknown (as documented; SF-1).

Privacy proof (unique marker in the message and the query; every ClickHouse spans column scanned, incl. resource_attrs, attrs_string, span_events): capture off on 1.4.2 and 1.4.1: 0 hits for the marker and for the message text. Capture on (1.4.2): marker in attrs_string of everos.memory.search and everos.extract only (the four documented attributes). No FI keys, model keys, home paths or username in any stored row (all four runs); the error run's marker count is 0 (its rejected input was not quoted by that provider). The error-text caveat in the README (error status/event text is exported in every capture setting) is unchanged and still true.

UI. The Observe UI trace page never finished rendering on the shared stack (trace-detail API answered 503 "temporarily unavailable" under host load average ~34; ClickHouse hit its 1.46 GiB cap on a wide scan); screenshots show only loading states, so the UI check is NOT proven. ClickHouse rows are the evidence.

Not covered. /api/v1 alias, OTEL_EXPORTER_OTLP_TRACES_ENDPOINT, library use without the server, 1.2.0 to 1.3.x, a Python 3.10/3.11 run (EverOS needs 3.12+), a hosted Future AGI endpoint.

Verdict: Rick's recipe was correct; fixes were the version pin and the version-pinned test.

nik13 and others added 9 commits October 3, 2026 18:58
The Node OTLP exporter streams with Transfer-Encoding: chunked and sends
no Content-Length. The Receiver read Content-Length only, so every Node
export came back 400 with zero spans; two TH-8103 children had to put a
de-chunking relay in front of it. Receiver now accepts both framings.

It also records one entry per accepted export (path, lower-cased
headers, flattened resource attributes) via requests(), so a contract
test can assert X-Api-Key/X-Secret-Key and project_name through the
shared harness instead of a private recorder.

Verified: 6 harness tests pass, and the TanStack example's real Node
exporter delivered 4 chunked exports (4 spans, collector path, both
auth headers, project_name/project_type) with no relay.

Refs: TH-8339, TH-8103
tests/loopback_guard/sitecustomize.py is the Cognee recipe's guard: loaded
through PYTHONPATH in every process a scenario starts (the scenario and the
`everos init` it runs), it refuses and logs any connect/connect_ex/
getaddrinfo to a host other than 127.0.0.1, ::1 or localhost.
tests/_guard_probe.py is its positive control (TEST-NET-1, 2001:db8::1,
example.invalid).

tests/_fake_openai.py answers the chat and embeddings calls EverOS 1.4.1
makes through everalgo in one add/flush/search run: memcell boundary
detection, episode extraction and atomic-fact extraction, with 1024-wide
embeddings (EverOS's LanceDB width). One switch makes the atomic-fact reply
malformed, for the error-text test.

approved: Nikhil 2026-10-03 blanket
Refs: TH-8329
tests/everos_session.py runs `everos init`, then drives EverOS's own app
(create_app(), what `everos server start` serves) in-process with
Starlette's TestClient: no port is bound, no server or HTTP wrapper is
started, and the app's own lifespan installs EverOS's tracer from the
[observability] settings. It adds three messages, flushes, waits for the
episode to be indexed through /api/v2/memory/get (which opens no span) and
runs one hybrid search.

everalgo counts tokens with tiktoken's o200k_base, which tiktoken
downloads from openaipublic.blob.core.windows.net on first use (the guard
caught it); the script registers a byte-level stand-in under that name.

tests/everos_config_probe.py exports one everos.memory.search span through
everos.config.load_settings and init_tracing/shutdown_tracing, to test the
endpoint, header and resource settings without the whole app.

approved: Nikhil 2026-10-03 blanket
Refs: TH-8329
EverOS (EverMind-AI/EverOS, PyPI everos; not EverMemOS Cloud or the legacy
Claude Code plugin) exports its own OTLP spans. The recipe pins
everos[otel]==1.4.1 (latest on PyPI 2026-10-05) and sets EverOS's own
[observability] keys (settings.py:624-668): enabled, endpoint (used as
given, so the full /tracer/v1/traces path) and headers (a toml table or a
JSON EVEROS_OBSERVABILITY__HEADERS), plus OTEL_RESOURCE_ATTRIBUTES for
project_name/project_type, since EverOS has no resource setting. Content
capture stays off.

The README records the spans 1.4.1 emits (no architecture name renamed),
that fi-collector derives kind "unknown" for all of them
(converter.go:79-131) so the architecture's kind table is an SF-1 gap, why
no processor is written, model/token/cost behaviour, and what content
capture does and does not cover. requirements.txt pins the tested
OpenTelemetry 1.45.0.

approved: Nikhil 2026-10-03 blanket
Refs: TH-8329
The fixtures are EverOS 1.4.1's own spans from the live scenario, as the
harness Receiver decoded them: capture off, capture on, and the capture-off
run's resource. Fixture tests (no EverOS needed) post each with
post_otlp(), check it with compare(), and assert: span names and tree,
langfuse.observation.type per span, no fi-collector kind key and no cost
key, gen_ai model/token keys only on generation and embedding spans, no
query or other content with capture off, the query on
everos.memory.search with capture on, project_name and
project_type=observe on the resource, and no keys or host paths.

Live tests (when everos is installed) run the recipe, capture-on and a
rejected-model-reply scenario under the loopback guard and match them to
the fixtures, check the export path, both auth headers and the resource,
and probe the toml block, the default-off switch, the OTel header and
endpoint variables, an endpoint without its path and a missing
OTEL_RESOURCE_ATTRIBUTES. Re-record with EVEROS_RECORD_FIXTURE=1.

33 passed on Python 3.12.10 and 3.13.15; without everos, 17 passed and 16
skipped (3.12).

approved: Nikhil 2026-10-03 blanket
Refs: TH-8329
…ummary

verify-r1 V1: with capture off, EverOS 1.4.1 exports a provider's error
body as span status and exception text. The loopback OpenAI fake can now
answer an embeddings request that contains a given text with HTTP 400 and
an OpenAI-style error body quoting the rejected input, followed by a
marker at character 4200 of the message. A fourth live scenario,
embedding_error, runs the recipe's add/flush/hybrid search with the fake
rejecting the query's embedding. The test pins what EverOS does today:
the search answers 422; everos.embedding, everos.search.recall and
everos.memory.search carry ERROR status descriptions and exception events
(message and chained stack trace) holding the query and the late marker,
uncut past 4096 characters; neither marker is in any span attribute or the
resource, and no span has a content key. It passes on the current code by
design.

README tests, failing until the README changes:
- the error-text caveat must name the channels, those spans and
  everos.ome.extract_atomic_facts, state that capture_content = false,
  set_redactor and the 4096 cap do not apply (attributes.py:61-77), cite
  the embedding and LLM provider error paths and both error tests, and
  recommend keeping tracing off for sensitive inputs; the capture-off and
  never-exported bullets must link to it.
- review R1: the collector summary's Model and Tokens bullets must cover
  every span the capture-off fixture gives a model and tokens, naming the
  agent-typed everos.ome.extract_atomic_facts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
approved: Nikhil 2026-10-03 blanket
Refs: TH-8329
…ge summary

verify-r1 V1: the Privacy section named only an extraction error quoting a
model reply. It now has an "Error text is not covered by the switch"
subsection: OpenTelemetry records an exception's message as the ERROR
status description and in an exception event (message and chained stack
trace, with installation paths) on the failing span and every EverOS span
it propagates out of; capture_content = false, set_redactor and the
4096-character cap act only on the two content attributes
(attributes.py:61-77), and the server's own provider has no filter
setting. Provider error bodies kept by EverOS (embedding
openai_provider.py:39-49, 121-134; LLM openai_provider.py:106-109) are
the main case: the embedding_error test shows a quoted query and a marker
4200 characters in on everos.embedding, everos.search.recall and
everos.memory.search with capture off. The section recommends keeping
tracing off for sensitive inputs until EverOS or a controlled collector
strips exception events and status, and says this recipe adds no filter.
The capture-off and never-exported bullets are limited to the tested runs
and link to it; the setup text no longer implies the switch is enough.

review R1: the Model and Tokens bullets said generation and embedding
spans only. They now cover the span current at each recorded call,
including the agent-typed everos.ome.extract_atomic_facts
(extract_atomic_facts.py:38-50, runner.py:229-251,
_usage_client.py:44-57).

The Tests section lists the embedding_error run and the new totals: 37
tests, 20 fixture and 17 live.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
approved: Nikhil 2026-10-03 blanket
Refs: TH-8329
@nik13
nik13 requested a review from NVJKKartik October 10, 2026 08:22
nik13 and others added 3 commits October 11, 2026 20:35
…pans

The README left the collector side as source reading ("whether that
adapter runs ... was not established") and said nothing about Input,
Output, sessions or users. A live run (everos server start configured as
in Set up, real model and embeddings, fi-collector from future-agi dev)
settles them:

- every span is stored as unknown; the backend Langfuse adapter does not
  run on fi-collector-ingested spans;
- Input/Output stay empty in both capture settings (captured content is
  only in langfuse.observation.* attributes, which fi-collector does not
  read);
- langfuse.session.id / langfuse.user.id do not reach Sessions/Users;
- a failed chat call's span carries no model or tokens, and embedding
  spans carry tokens only when the provider reports usage;
- unknown-model cost is stored as 0; no project_name -> 400, bad keys
  -> 401, endpoint without its path -> 404.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ture AGI run

Re-verification 2026-10-16: 1.4.2 (latest on PyPI) exports the same spans as
1.4.1; live tests now accept 1.4.1 and 1.4.2 and fail on anything else.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@nik13
nik13 merged commit 75c7af6 into dev Oct 26, 2026
8 checks passed
@nik13
nik13 deleted the feat/th-8329-everos branch October 26, 2026 10:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant