Repository navigation
feat(examples): Cognee tracing recipe and conformance test on Cognee's own spans (TH-8328) - #226
Merged
Merged
Conversation
The Node OTLP exporter streams with Transfer-Encoding: chunked and sends no Content-Length. The Receiver read Content-Length only, so every Node export came back 400 with zero spans; two TH-8103 children had to put a de-chunking relay in front of it. Receiver now accepts both framings. It also records one entry per accepted export (path, lower-cased headers, flattened resource attributes) via requests(), so a contract test can assert X-Api-Key/X-Secret-Key and project_name through the shared harness instead of a private recorder. Verified: 6 harness tests pass, and the TanStack example's real Node exporter delivered 4 chunked exports (4 spans, collector path, both auth headers, project_name/project_type) with no relay. Refs: TH-8339, TH-8103
Test helpers for the TH-8328 Cognee recipe, used by the contract test in the next commits. - _guarded_run.py runs a script with every non-127.0.0.1 connection and DNS lookup refused and logged, so a run proves Cognee, LiteLLM and the example reached nothing else. - _fake_openai.py serves /v1/chat/completions and /v1/embeddings on loopback. It answers each request cognee 1.6.2 makes in one add/cognify/search run with a value that validates first time (KnowledgeGraph and SessionTurnAnalysis by name, other schemas from the JSON schema), so each Cognee LLM call is exactly one HTTP request. A schema-derived SessionTurnAnalysis fails Cognee's validation and makes the native adapter send a second, json_object request inside the same span. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> approved: Nikhil 2026-10-03 blanket Refs: TH-8328
TH-8328 (D-8328): no traceai-cognee package and no instrumentor. Cognee 1.6.2 creates its own OpenTelemetry spans when COGNEE_TRACING_ENABLED is true; the recipe only routes them to Future AGI. Option A (src/app.py): register(project_type=OBSERVE, set_global_tracer_provider=True) before Cognee's first traced call, so Cognee's setup_tracing attaches to that provider (tracing.py 396-403) instead of creating one. One extra line, provider.add_span_processor(BatchSpanProcessor()), is required: register() sets _default_processor (otel.py 243), and the provider's add_span_processor() shuts down and removes a default processor (otel.py 336-339). Cognee's attach call (tracing.py 403) would otherwise replace the Future AGI exporter and nothing would be exported. A parent span around add/cognify/search makes one trace; add.py 263-267 closes memory.store before ingestion runs. Option B: Cognee's own exporter via OTEL_EXPORTER_OTLP_ENDPOINT (full /tracer/v1/traces path, passed through unchanged, tracing.py 335), OTEL_EXPORTER_OTLP_HEADERS and OTEL_RESOURCE_ATTRIBUTES. The URL keeps :443 because Cognee uses gRPC for any URL its classifier does not mark HTTP-only (tracing.py 285-302, 358-366) and cognee[tracing] installs the gRPC exporter. README documents the span names and kinds, what fi-collector derives from them (only embeddings get a type; search is stored as unknown, not RETRIEVER), that gen_ai.request.model is on every LLM span so no copy processor is written, that no token usage is recorded, and the content Cognee exports by default with no off switch short of leaving tracing off (prompts with document text, questions, answers, search query, graph query text, stack traces). The license is not stated. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> approved: Nikhil 2026-10-03 blanket Refs: TH-8328
Runs real cognee 1.6.2 in four loopback-guarded subprocesses side by side (importing cognee alone takes about a minute), against the OpenAI fake and the shared harness Receiver, with placeholder keys only: - recipe: src/app.py as written. Asserts every export goes to /tracer/v1/traces with X-Api-Key and X-Secret-Key, the resource is project_type=observe from one provider, the exact span name/kind set the README lists, one trace under the app span, one LLM span per chat request and one embedding span per embeddings request (no duplicates), the keys fi-collector maps from (no span-kind key anywhere, gen_ai.operation.name=embeddings only on embed_text, gen_ai.request.model on every LLM span, no token keys), that document text, question and answer appear under exactly the documented keys, and that no key reaches a span or resource. - tracing_off: same app, COGNEE_TRACING_ENABLED unset; only the app's own span is exported. - no_readd: register() without the re-add line; Cognee buffers spans but nothing is exported. - otlp_env: option B's variables; Cognee posts to exactly /tracer/v1/traces with both headers and project_name/project_type from OTEL_RESOURCE_ATTRIBUTES, logs "Failed to export logs batch code: 404" for /tracer/v1/logs, and its classifier marks the :443 URL HTTP-only and the bare URL not. fixtures/cognee-1.6.2-recipe-spans.json is the recipe run's spans as the Receiver decoded them, with stack-trace paths replaced. The fixture tests post it with post_otlp(), check it with compare() and rerun the mapping, content and secret assertions without Cognee installed; the live run must match its shape (names, kinds, keys, statuses, counts). COGNEE_RECORD_FIXTURE=1 re-records it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> approved: Nikhil 2026-10-03 blanket Refs: TH-8328
Review R1: the guard patched sockets only in the scenario's main interpreter, but Cognee 1.6.2 runs LanceDB and Ladybug in multiprocessing "spawn" workers (lancedb/subprocess/proxy.py:47, graph/kuzu/subprocess/proxy.py:40-45) that never ran the patch. _guarded_run.py becomes loopback_guard/sitecustomize.py, put first on each scenario's PYTHONPATH. Every interpreter a scenario starts, including Cognee's workers (they inherit the environment), now installs the guard at startup and logs an "installed" record; the file then runs the interpreter's own sitecustomize it shadows (Homebrew's Python ships one). Scenarios run their script directly. New test: every scenario's log shows the guard installed in at least one --multiprocessing-fork worker; assert_ran_offline still requires zero refusals. README scopes the claim: Python-level connections and DNS in the scenario processes and Cognee's workers; native code that opens sockets itself is not intercepted. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> approved: Nikhil 2026-10-03 blanket Refs: TH-8328
Review R6: scrub() only rewrote `File ".../lib/python3.x/` frames and the repository root, and the leak check looked for /Users/ only, so a fixture re-recorded on Linux (or anywhere the data dirs show up in an exception message) could carry /home/<user> or pytest tmp paths. scrub() now also replaces pytest's tmp base dir (and its realpath) with <tmp> and the home directory with <home>, before the fixture is written. The fixture check asserts that none of str(Path.home()), /Users/, /home/, /tmp/pytest-of-, /var/folders and /private/ appears, and a new test plants those paths and checks scrub() removes them. The committed fixture already passes; it is not re-recorded. The substring checks collect matches into a list before asserting: pytest's explanation for a failing `x not in <200 KB JSON>` runs a character-level diff that does not finish in useful time. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> approved: Nikhil 2026-10-03 blanket Refs: TH-8328
Decision D-8328a (amends D-8328 for this recipe): Cognee 1.6.2 exports
document text, prompts, answers and the search question with no switch
(PRD 8/J4), and sets no span-kind key, so search is stored as unknown
(AC-03). Option A now wraps its re-added exporter in one recipe-local
export filter, src/cognee_filter.py:
exporter = CogneeExportFilter(HTTPSpanExporter())
provider.add_span_processor(BatchSpanProcessor(exporter))
- Content off by default: drops langfuse.observation.input/output
(get_observe.py:43-47, 86-88), memory.query.text and
cognee.search.query (api/v1/search/search.py:291, 294;
modules/search/methods/search.py:166; api/v1/recall/recall.py:534)
and cognee.db.query (graph/ladybug/adapter.py:637,
neo4j_driver/adapter.py:268). The same text (and the string fields of
the JSON prompt) is replaced with __REDACTED__ in that span's status
description and event attributes. capture_content=True or
COGNEE_FI_CAPTURE_CONTENT=true keeps everything.
- fi.span.kind by span name, only where none of fi.span.kind,
gen_ai.span.kind, llm.request.type, openinference.span.kind is set:
RETRIEVER for memory.retrieve and cognee.search*, LLM for
cognee.observe.acreate_structured_output (get_observe.py:176; every
@observe(as_type="generation") method in 1.6.2 is
acreate_structured_output). Embedding spans are left to the
collector's gen_ai.operation.name mapping.
- Exports ReadableSpan copies, so Cognee's buffer and other processors
see the original spans. On an internal error a span is exported
without content attributes, events or status description; if that
fails too it is dropped. Nothing is raised into the export path.
Tests: a new `capture` scenario (COGNEE_FI_CAPTURE_CONTENT=true); the
recipe run carries no planted marker anywhere and no content key, and
the right fi.span.kind; the capture run carries the markers under the
documented keys. The recorded fixture stays Cognee's own output and is
rebuilt as SDK spans to drive the filter offline (content, kinds,
status/event redaction, capture switch, no mutation of spans other
processors see, fail-closed and drop paths); live runs must match the
filtered fixture's shape. COGNEE_RECORD_FIXTURE=1 now records the
capture run without the filter's fi.span.kind.
README: the filter, option A's new behaviour, and that option B
exports content and has no kind mapping (option A is the
privacy-safe path).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
approved: Nikhil 2026-10-03 blanket
Refs: TH-8328
Review R2: the recipe's re-add line depends on register() marking its exporter as a replaceable default (otel.py:243, 336-339). If a later release stops doing that, the line would add a second Future AGI exporter and every span would be exported twice. requirements.txt pins fi-instrumentation-otel>=1.1.0,<1.2. The published 1.1.0's register(), TracerProvider.add_span_processor, BatchSpanProcessor and _build_default_exporter are identical to this repository's source; HTTPSpanExporter differs only in shutdown(). README and app.py cite follow-up TH-8394 (register()'s exporter is replaced by any add_span_processor) and say the re-add line goes away once it is fixed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> approved: Nikhil 2026-10-03 blanket Refs: TH-8328
- R3: register()'s processor is discarded by the re-add line, so any exporter or batch options given to register() must be given to the re-added exporter and processor too (README bullet, app.py comment). - R4: most cognee.db.vector.search spans end in ERROR: 60 of 66 in the recorded run (60 of its 117 spans), which shows in Future AGI's error filters and error rates. A fixture test checks the count the README states. - R5: option B needs pip install "cognee[tracing]==1.6.2"; plain cognee has no OpenTelemetry packages and then silently uses no-op spans (trace_context.py:91-96). Troubleshooting says so. - R7: with option A, do not call cognee.disable_tracing() or shutdown_tracing(): they shut down the provider Cognee attached to, the global Future AGI provider (tracing.py:459-465, trace_context.py:43-46). Unset COGNEE_TRACING_ENABLED instead. - D9: quote the wheel METADATA field License-Expression: Apache-2.0 (with License-File LICENSE and NOTICE.md); nothing more is claimed. - D8: the suite now also runs on Python 3.10 (cognee allows >=3.10); the README lists 3.10, 3.11 and 3.13. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> approved: Nikhil 2026-10-03 blanket Refs: TH-8328
A content-policy rejection in cognee 1.6.2 (litellm_native/native_adapter.py: 512-518) raises an error whose message is the whole prompt. OpenTelemetry records it on the LLM span and again on each pipeline-task span it passes through (run_tasks_base.py:243-245, new_span). The filter only replaced exact copies of a removed attribute on the same span, so the task spans (no content keys) and LLM spans with a prompt over the 8000-character cap (cut-off JSON) exported the text in messages, stack traces and statuses. By default the filter now removes free-text error detail on every span: exception.type and exception.escaped are kept, exception.message is replaced, stack traces keep only their headers, frame lines, chain separators and the exception type (an allowlist of lines), status descriptions name only the exception type, and other event values are kept only if they are numbers or booleans. Content capture still exports everything. The exact-match redaction is removed: default mode no longer needs it and capture mode never used it. The README states the debugging trade-off and the opt-in. approved: Nikhil 2026-10-03 blanket Refs: TH-8328 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…utes With content capture on, the README said content arrives under the documented keys "and nowhere else", but the test checks span attributes only, and with capture on error text (exception events, status) is exported as Cognee recorded it, so it carries content whenever an error quotes it. Say "among span attributes" and name the error-text case; the test helper's docstring says the same. approved: Nikhil 2026-10-03 blanket Refs: TH-8328 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
cognee.api.recall (api/v1/recall/recall.py:533) and cognee.agent_memory.retrieve (modules/agent_memory/runtime.py:405) are retrieval entry points the recipe's add/cognify/search path does not reach. The filter leaves them unknown on purpose; the README now says so and where their query text goes: cognee.search.query on recall, and the memory.retrieve and cognee.search.* spans of the search that agent_memory.retrieve runs (that span itself records only the query length). The filter removes that query text. approved: Nikhil 2026-10-03 blanket Refs: TH-8328 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ames The filter tests plant errors in recorded spans; this checks the live runs too. Cognee's vector-search probes fail with CollectionNotFoundError (naming the collection) in every run. In the recipe run (capture off) each exception event and status now carries only the exception type and the frame lines; in the capture run the error arrives as Cognee recorded it, as the README says. approved: Nikhil 2026-10-03 blanket Refs: TH-8328 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A serialized traceback has no trustworthy boundary between real frames and traceback-shaped exception-message content, so the content-off filter now omits exception.stacktrace entirely instead of reducing it to frame lines. Exception type, escaped flag and the fixed-marker message/status remain; capture_content=True still exports the original error detail. README states the narrower tested scope and the debugging trade-off. RED: the final tests on 12e0bd0 package code give 11 behavioral failures (incl. the forged chained-traceback marker leak), 48 passed. GREEN working tree: 59 passed on Python 3.10, 3.11, 3.13. approved: Nikhil 2026-10-03 blanket Refs: TH-8328
With COGNEE_FI_CAPTURE_CONTENT=true the export filter kept Cognee's prompts, questions and answers only under langfuse.observation.*, memory.query.text and cognee.search.query. fi-collector fills a span's Input and Output from input.value/output.value only, so a live run against a local Future AGI stack showed every LLM and retriever span with empty Input and Output and the content buried in raw attributes (and nothing for evals to read). In capture mode the filter now copies the LLM call's arguments (or a search span's question) to input.value and the response to output.value, with input/output.mime_type, never overwriting keys a span already has. Default (content-off) mode is unchanged and adds none of these keys. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
In every search Cognee probes vector collections that do not exist yet, and each probe's cognee.db.vector.search span ends in ERROR with CollectionNotFoundError: 60 of 66 vector searches in the recorded run. So every successful trace showed dozens of errors in Future AGI and inflated error rates and error filters. By default the export filter now sends a cognee.db.vector.search span whose only error is CollectionNotFoundError as OK, without its exception event, and with cognee.vector_search.collection_missing=true (the collection name stays in cognee.vector.collection). Every other error stays ERROR: another exception type, a second exception on the same span, the same error on any other span, or an ERROR status without an exception event. collection_missing_as_error=True or COGNEE_FI_COLLECTION_MISSING_AS_ERROR=true keeps the probes as errors. Tests first (fixture and switch tests, plus a sixth live loopback scenario with the switch on); 71 pass. The README no longer says the recipe was tested against loopback fakes only: it was also checked live against a local Future AGI stack with a real OpenAI-compatible LLM endpoint and real embeddings, where a successful add/cognify/search trace now has no error spans and a failing model call still marks its LLM span and parents ERROR. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nee 1.6.5 With content capture on, the export filter passed exception.stacktrace through unchanged. A serialized traceback names local files (home directory, user name, virtualenv layout), which the TH-8103 privacy rule forbids in every mode. The filter now drops exception.stacktrace with capture on as well and keeps the exception type, message and status description that the user opted into. Default mode is unchanged. Tests: capture-mode expectations no longer include the stack trace, and a new parametrized test sends the recorded run's real error spans (stack traces naming the Python install) through the filter in both modes. On the previous filter, 9 of the updated filter tests fail. Cognee 1.6.5 builds its embedding tokenizer with tiktoken before LiteLLM sets TIKTOKEN_CACHE_DIR, so the guarded scenarios tried to download cl100k_base and every live scenario failed. The scenarios now point TIKTOKEN_CACHE_DIR at LiteLLM's bundled tokenizer files (a test setting only). The live-vs-fixture shape check treats the cognee.db.graph.query count as version-dependent (14 on 1.6.2, 17 on 1.6.5); names, kinds, keys and statuses still have to match exactly. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…the re-check cognee 1.6.5 (2026-10-16) is the latest release; 1.6.2 was stale. Both versions emit the same span names, kinds, attribute keys and statuses, and the suite passes on both. README: the new pin, the live re-verification (2026-10-16, real OpenAI models against a local Future AGI stack), the stack-trace rule, and option B's self-hosted case: a URL without :443 or :4318 makes Cognee use gRPC, so point it at the collector's gRPC port with lower-case header names (with X-Api-Key every export fails with "Invalid metadata"). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Status 2026-10-21: merged
devinto the branch (0f8073c, no conflicts). CI: Plan, fi_instrumentation and Python tests are green. This is a recipe under python/examples, not a package, so no package job runs, and the recipe suite runs only locally. Local recipe suite with the README command (Python 3.11, cognee 1.6.5): 75 passed. A first run under load average around 50 lost 4 tests to Cognee's 30 s LLM connection check timing out against the loopback fake. That is environmental, and a rerun passed. Nothing to publish. Merge this before docs#1017, whose links point attree/dev/python/examples/cognee.Summary
This PR adds a recipe and conformance test for Cognee in
python/examples/cognee. Following decision D-8328 it traces Cognee through Cognee's own OpenTelemetry spans, so there is no package and no second instrumentor. Decision D-8328a (below) adds one small export filter that lives in the recipe.src/app.py:register(project_type=OBSERVE, set_global_tracer_provider=True).provider.add_span_processor(BatchSpanProcessor(CogneeExportFilter(HTTPSpanExporter()))). See "register() needs one extra line" below.cognee.add/cognify/searchrun inside one app span, so the whole flow is one trace.<origin>/tracer/v1/tracesplusOTEL_RESOURCE_ATTRIBUTES=project_name=...,project_type=observe.cognee[tracing]==1.6.2; without it, tracing silently does nothing.tracing.py:285-302, 358-366). The README therefore publisheshttps://api.futureagi.com:443/tracer/v1/traces, checked through Cognee's own classifier.cognee==1.6.2andfi-instrumentation-otel>=1.1.0,<1.2. The second pin exists because the re-add line depends on howregister()behaves today; follow-up TH-8394 covers that. The wheel METADATA'sLicense-Expression: Apache-2.0is quoted as a fact.D-8328a: recipe-local export filter (approved: Nikhil 2026-10-03 blanket)
D-8328 said "no processor". It was written before measurement showed that Cognee 1.6.2 does two things:
unknown(AC-03 not met).Both are product requirements, so option A wraps the exporter it already re-adds with
src/cognee_filter.py. The file lives in the recipe; users copy it next to their app.Content (off by default).
langfuse.observation.inputand.output(get_observe.py)memory.query.textandcognee.search.query(api/v1/search/search.py,modules/search/methods/search.py)cognee.db.query(ladybug and neo4j adapters)exception.stacktraceis omitted entirely (D-8328c, below), and status descriptions keep the exception type without its message. Non-exception event string values are dropped.CogneeExportFilter(capture_content=True)orCOGNEE_FI_CAPTURE_CONTENT=trueexports everything.Kinds.
fi.span.kind=RETRIEVERonmemory.retrieveandcognee.search*.LLMoncognee.observe.acreate_structured_output, the only@observe(as_type="generation")method in 1.6.2.fi.span.kind,gen_ai.span.kind,llm.request.typeoropeninference.span.kindis already set.gen_ai.operation.name=embeddings.Safety.
Rejected alternatives:
Other findings that change the spec (with vendor evidence)
register()plus Cognee's attach exports nothing without the extra line.add_span_processor(SimpleSpanProcessor(exporter))(tracing.py:396-403).register()marks its Future AGI exporter as a replaceable default (otel.py:243), andadd_span_processorremoves that default (:336-339).no_readd: 0 exports.register(), pass the same ones to the re-added exporter.cognee.disable_tracing()/shutdown_tracing(): they shut down the global Future AGI provider. UnsetCOGNEE_TRACING_ENABLEDinstead./var/folderspaths), checked withpost_otlp()andcompare().Review round 1 (pr-reviewer t_6538e669 APPROVE, pr-verifier t_34f2f8e3 VERIFIED at
e144df1)There were no blocking findings. Everything was fixed test-first. Against e144df1's recipe code the new tests gave 11 failed and 16 errored, all of them new tests. The two decision points the reviewer raised (D3 AC-03, D4 §8) are decided in D-8328a above.
ba65e6fa6b8f91sitecustomize.pyin Cognee's spawned LanceDB/Ladybug workers too (4-5 guarded processes per scenario, 0 refusals)c13525fPath.home(),/home/,/tmp/pytest-of-,/var/foldersfi-instrumentation-otel26bf146>=1.1.0,<1.2; checked against the published 1.1.0 (register,add_span_processor,BatchSpanProcessoridentical)28cce74Review round 2 → D-8328b fixes
pr-verifier t_37d7b8c8 requested changes at
28cce74: N1 (P2) was a content-policy error containing the full chunk/prompt on both the LLM and its pipeline parent. Parent spans have no content keys, and Cognee truncates LLM input JSON at 8000 characters, defeating the old same-span exact-match redaction.Fixes at
12e0bd0:Review round 3 → D-8328c fix
pr-verifier t_c1531ea4 requested changes at
12e0bd0: V2 (P2), a forged chained-traceback frame inside an exception message (a separator, a blank line, aTracebackheader andFile "...", line N, in <name>lines whose path or name hold prompt text) survived the structure-based reducer and exported content by default. Rick reproduced it independently (fix-r3/red-probe.py, rc1).Decision D-8328c (approved: Nikhil 2026-10-03 blanket): a serialized traceback has no trustworthy boundary between real frames and traceback-shaped message content, so in content-off mode the filter now omits
exception.stacktracealtogether. Exception type, escaped flag, the fixed-marker message and the type-only status remain;capture_content=Truestill exports the original error detail unchanged. Rejected: another parser/regex round (the same class of bypass), and dropping the whole exception event (loses the type users need).Fix at
c5fa217(README,src/app.pydocstring,src/cognee_filter.py, tests):12e0bd0package code give 11 behavioral failures, 48 passed, including the forged-frame marker leak (fix-r3/red-final-tests-on-12e0bd0.log).Review round 4 (pr-verifier t_3cbf3e34 at
c5fa217: VERIFIED)V2 resolved: in content-off mode
exception.stacktraceis omitted, and the forged chained-traceback marker cannot reach the export through the message, stack trace or status. No new or blocking findings. Observations, not findings: event names, links, resources andexception.typepass through unchanged (Cognee 1.6.2 writes no free text there); two capture-mode tests compare Event/Status objects by identity, which is brittle but correct. Companion docs and architecture corrected to this head in company-brain #63 (bdfcea9, open).Tests
Head
c5fa217. Coordinator rerun from the repo root with real process exit codes and JUnit, clean tree and sanitized environment (tickets/TH-8328/exact-head-c5fa217-checked/results.json):Nothing changed outside
python/examples/cognee. No secret-like lines were added, and no host paths are in the fixtures.Setup: real cognee 1.6.2 runs against a loopback OpenAI chat and embeddings fake, with placeholder keys and the shared harness
Receiver.Checks:
/tracer/v1/traces, both auth headers,project_nameandproject_type=observeon the resource.RETRIEVERand LLM spans carryLLM.tracing_off,no_readd,otlp_env(option B).post_otlp()+compare().Limits
cognee.api.recallandcognee.agent_memory.retrieveget no kind; their query attribute is still dropped.fi_instrumentationdeclares<3.14). Option B over gRPC is covered only through Cognee's classifier.Video demo
Recorded build: head
c5fa21725c8ff548bd41e4262ea8413ee357e422(this PR); 11 real terminal chapters of the demo driver from a clean read-only worktree plus intro and review cards; 7:56, 13 chapters, narrated with captions.The video (SHA-256
4ec87e3b9d9f42d1be3424dc4e247c54dcf616eceb7b59736c8bb8eedec4cfb1), captions (SRT), transcript, chapter list, preview and both media-verification notes are private attachments on Linear issue TH-8328 (Future AGI workspace; sign-in required). They are not linked here as raw upload URLs.register()plus the re-add line4af5338: SOURCE READING only12e0bd0What it shows: unfiltered Cognee 1.6.2 exports 117 spans carrying the planted document, question and answer, no span kinds and 60 error spans with full stack traces;
register()alone exports 0 spans and the app's re-add line exports all 117 (TH-8394 tracks the root fix); the default filter exports 0 marker occurrences, adds LLM/RETRIEVER kinds and leaves the original spans untouched; the D-8328c proof, with the old filter extracted from12e0bd0by blob hash: 6 vs 0 marker occurrences on a recorded LLM/task pair and 2 vs 0 on a real raise; capture on keeps everything and adds onlyfi.span.kind; 60 of 66 vector-search spans end in ERROR in a successful run; option B is unfiltered. Marker text and header values are never printed, only counts and key names.Independent media checks (Rick): full audio/video decode exit 0; 13 embedded chapters match the list; the only audio gaps are the 12 chapter transitions; all chapter-end frames and the preview inspected (legible, nothing under the captions, no key or header values, local paths or ports); 7 delivered files scanned with zero fixture, path, port, header or credential hits. Disclosed edits: waiting time trimmed in 7 chapters and settled frames held for narration; takes hit by a temp-directory clash between parallel lanes were discarded and re-recorded one at a time. The narration says the company-brain #63 docs were still pending; they have since been verified and pushed (
bdfcea9), and no product code changed after recording. Not shown: a live model, running fi-collector, auth, tenant, storage or the trace view.Stacked on #203 (shared harness,
3eaadc8), basedev. Not merged.Linear: TH-8328. approved: Nikhil 2026-10-03 blanket
Verification sweep (2026-10-12)
dev, and the spans were checked in ClickHouse and in the Observe UI (hierarchy, span types, input/output, model and tokens, sessions, error status, and a secret scan of everything stored).Re-verification (2026-10-16)
Re-checked end to end at head
c3a6ca6against a local Future AGI stack built fromdev, with real OpenAI models (gpt-4o-mini,text-embedding-3-small) and the recipe run exactly as the README says (pip install -r requirements.txt, environment variables,python src/app.py).Versions. cognee 1.6.5 (latest on PyPI, released 2026-10-16) as the primary and 1.6.2 (the previous pin);
fi-instrumentation-otel1.1.0 from PyPI; OpenTelemetry SDK 1.45.1; live runs on Python 3.12. Cognee 1.6.5 changed no span name, attribute key or tracing hook (cognee/modules/observabilityis byte-identical to 1.6.2); a live run issues 17 graph queries instead of 14.Commits added
9f174e3fix(cognee): never export stack traces; keep the suite offline on cognee 1.6.5exception.stacktracethrough unchanged. On a real failing run that put 10 stack traces with 422 occurrences of the local home path and user name on the wire. Stack traces name local files, so the filter now dropsexception.stacktracein both modes and keeps the type, message and status that capture opts into. Default (content-off) behaviour is unchanged. The updated tests fail 9 times against the previous filter.TIKTOKEN_CACHE_DIR, so the loopback-guarded scenarios tried to downloadcl100k_baseand all live scenarios failed (15 failures). The scenarios now pointTIKTOKEN_CACHE_DIRat LiteLLM's bundled files (a test setting only). The live-vs-fixture shape check compares thecognee.db.graph.querycount as "at least one"; names, kinds, keys and statuses still match exactly.c3a6ca6build(examples): pin cognee 1.6.5 and document the re-checkrequirements.txtpinscognee==1.6.5; README updated for the pin, the stack-trace rule, and option B on a self-hosted collector (a URL without:443/:4318makes Cognee use gRPC; point it at the gRPC port with lower-casex-api-key/x-secret-key, sinceX-Api-Keyfails withInvalid metadata).Tests (from the repo root, README command): 75 passed on Python 3.10, 3.11 and 3.13 with cognee 1.6.5, and 75 passed on Python 3.11 with cognee 1.6.2. At the previous head, cognee 1.6.2 gave 71 passed and cognee 1.6.5 gave 15 failed.
Live E2E (local Future AGI, verified from its trace store)
7befb18a-2497-4590-b5c8-f9a2c5c5bb19openai/gpt-4o-mini), retriever 3, embedding 10 (openai/text-embedding-3-small), 0 errorsgpt-th8328-no-such-model)ce1159ea-c50f-4b15-8399-6aa7bbcc1e81memory.processand the root; statuslitellm.exceptions.NotFoundError (detail removed: content capture off)17fb9d86-8dc8-4c93-9185-3b99962ced91c23c4360-9d41-4f8e-b781-41a8153ff34b/ failure30da4db6-39fc-4922-bc56-7650b3ab6f17https://api.futureagi.com:443/tracer/v1/traces(Future AGI cloud):443Verifier excerpt (content-off project,
--expect-type llm --expect-type retriever --expect-type embedding):traces=4 spans=262,note: 14 span(s) with status ERROR,PASS. Tokens and cost stay 0: Cognee records no usage.Privacy. Default mode: a unique marker in the document and the question, the document text, the answer, local paths, user name, host name and every key (Future AGI and model): 0 hits across all stored columns (span attributes, resource attributes, events, input/output, status, trace and session rows) of 262 spans. Capture on: the marker appears only in the content keys and Input/Output; a capture-mode failing run sent 0
exception.stacktracekeys and 0 local paths on the wire (checked on the raw OTLP bodies).Not covered. No live content-policy rejection (Cognee retries for minutes); Cognee Cloud; non-default graph/vector backends;
cognee.recalland agent-memory entry points stay unmapped. Docs page: future-agi/docs#1017 (draft).🤖 Generated with Claude Code