Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 57 additions & 0 deletions .github/workflows/python-voice-attachments.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
name: Python Voice Attachments

on:
pull_request:
paths:
- 'python/fi_instrumentation/attachments/**'
- 'python/frameworks/livekit/**'
- 'python/frameworks/pipecat/**'
- 'python/voice-tests/**'
- 'python/traceai/**'
- 'python/tests/attachments/**'
- '.github/workflows/python-voice-attachments.yml'
workflow_dispatch:

permissions:
contents: read

jobs:
components:
runs-on: ubuntu-latest
timeout-minutes: 20
strategy:
fail-fast: false
matrix:
# Pipecat 1.15.0 (the qualified recorder surface) requires Python >=3.11.
python: ['3.11', '3.12']
adapter:
- {name: pipecat, upstream: pipecat-ai, version: '1.15.0'}
- {name: livekit, upstream: livekit-agents, version: '1.8.10'}
- {name: livekit, upstream: livekit-agents, version: '1.8.11'}
include:
- python: '3.10'
adapter: {name: livekit, upstream: livekit-agents, version: '1.8.10'}
- python: '3.10'
adapter: {name: livekit, upstream: livekit-agents, version: '1.8.11'}
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python }}
- name: Install exact upstream with this checkout
run: |
python -m pip install -e python -e python/frameworks/${{ matrix.adapter.name }} \
'${{ matrix.adapter.upstream }}==${{ matrix.adapter.version }}' \
opentelemetry-exporter-otlp-proto-grpc pytest pytest-asyncio jsonschema
python -m pip check
python -m pip freeze > voice-versions.txt
- name: Run voice components and negative controls
run: python -m pytest python/voice-tests/test_${{ matrix.adapter.name }}_attachments.py -q --tb=short
- name: Run existing LiveKit regression tests
if: matrix.adapter.name == 'livekit'
run: python -m pytest python/frameworks/livekit/tests -q --tb=short
- uses: actions/upload-artifact@v4
if: always()
with:
name: voice-versions-${{ matrix.python }}-${{ matrix.adapter.name }}-${{ matrix.adapter.version }}
path: voice-versions.txt
71 changes: 69 additions & 2 deletions python/frameworks/livekit/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,39 @@ drops the Future AGI exporter. LiveKit installs its privacy filter once per prov
ignored. To keep conversation content out of your traces, set `LIVEKIT_TELEMETRY_ALLOW_PII=false` before calling `enable_*`
(livekit-agents 1.8 and later).

### Grouping a call into a session

After enabling attribute mapping, use `using_session` inside each job entrypoint. Wrap the call's
`AgentSession` startup and work, not just tracing setup at worker import time:

```python
from fi_instrumentation import using_session
from livekit.agents import Agent, AgentSession

# Inside entrypoint(ctx), after register() and enable_http_attribute_mapping():
with using_session(session_id=ctx.room.name):
await ctx.connect()
session = AgentSession()
await session.start(agent=Agent(instructions="Answer briefly."), room=ctx.room)
# Run your conversation here.
```

The native spans capture `session.id` when they start, so concurrent calls can use different IDs
on the same provider. An explicit `session.id` takes precedence; an existing `gen_ai.conversation.id`
(including LiveKit's own conversation ID) is preserved. Grouping does not enable attachments or
recording. With no session context, no `session.id` is added.

On livekit-agents 1.8, context-derived IDs follow `LIVEKIT_TELEMETRY_ALLOW_PII` and project
redaction. Set your privacy choice before enabling mapping; changing it afterward does not
replace LiveKit's installed filter. The IDs are mapped only on the Future AGI export copy, so
they cannot bypass LiveKit Cloud redaction.

A child can inherit from its in-process parent span even after leaving the context block. A
bare `SpanContext` or `NonRecordingSpan` carries only tracing IDs, not session metadata: watchdog
diagnostics such as `event_loop_blocked` may lack a session when emitted with only those IDs. Likewise,
a span started before entering the block is not retroactively grouped. Pass the context through
your own asynchronous work instead of assigning one session globally to the worker.

### Flushing at the end of a job

On Linux a LiveKit job process ends without running `atexit`, and LiveKit flushes only its own processors, so the spans
Expand All @@ -57,8 +90,9 @@ them twice over:
trace, `agent_session` included, as soon as the call ends.
- **When the process exits**, just before it stops: this sends what LiveKit ends after the shutdown callbacks, its own
`job_shutdown` span, and the job's root `job_entrypoint` span when tracing is set up before the job starts. With
set-up inside the entrypoint, LiveKit starts `job_entrypoint` before tracing is on, so it is never recorded and the
job trace arrives rooted at `job_shutdown`.
set-up inside the entrypoint, LiveKit starts `job_entrypoint` before tracing is on, so it is never recorded. The call
then arrives as separate traces: the session trace rooted at `agent_session`, and `job_shutdown` as a root of its own
(see "How Future AGI Voice finds a LiveKit call").

Each waits at most 3 seconds for the collector, well inside the 10 seconds LiveKit gives a job process to exit, so its own
clean-up still runs when the collector is slow.
Expand Down Expand Up @@ -188,3 +222,36 @@ The pinned development lock contains LiveKit Agents 1.8.6, API 1.2.1, protocol 1
Check the `examples/` directory for agent examples (validate recording SDK compatibility before deployment):
* `simple_agent.py`: A basic voice assistant agent.
* `egress_agent.py`: An agent with opt-in Egress metadata capture, an explicit session envelope and bounded stop/close cleanup. The modern `LiveKitAPI().egress` request shape and native-session linkage require the deployment's SDK/mode validation.

## Call recordings as trace attachments (opt-in)

Attachments are off by default. Both paths below need the attachment-enabled `fi-instrumentation-otel` and `livekit-agents` 1.8.10 or 1.8.11. Neither starts, stops or enables recording.

**Local session recording.** If your session already records audio (`record=True`), bind it after `await session.start(...)`:

```python
binding = traceai.bind_livekit_session(session, ctx, owned_recording_files=True)
```

When the session ends, the SDK's `audio.ogg` is uploaded as one `mixed` file. This happens before LiveKit deletes its temporary directory, with a deadline of up to 30 s. Hard links, symlinks, foreign owners and replaced files are refused. LiveKit redaction, `LIVEKIT_TELEMETRY_ALLOW_PII` and FI hide flags all deny. The session span is marked `fi.span.kind=CONVERSATION` only if it has no parent, which depends on where tracing is set up (below).

**How Future AGI Voice finds a LiveKit call.** The SDK never invents a conversation root: it marks the session span only when that span is already the root of its trace. Which span is the root depends on where tracing is set up (checked with a real livekit-agents 1.8.10 and 1.8.11 worker):

- **Tracing set up inside the entrypoint** (the Quick Start above). LiveKit starts `job_entrypoint` before tracing is on, so it is not recorded and `agent_session` is the root of the call's trace (it carries `lk.job_id`). An admitted `bind_livekit_session(..., owned_recording_files=True)` marks `agent_session` as `CONVERSATION`, the recording is attached to it, and the call is listed in Voice. Without an admitted binding (attachments off, a denial, or `owned_recording_files=False`) nothing is marked, and the call is not listed. `job_shutdown` (with the session-end upload span) and `room_connect` arrive as separate root traces.
- **Tracing set up before the job starts** (for example in `setup_fnc`, or at worker start-up). LiveKit's `job_entrypoint` span is the trace root and carries `lk.job_id`; `agent_session` and `job_shutdown` are its children, so the binding does not mark `agent_session`. When attachments are enabled on the Future AGI server, ingest treats an untyped, parentless `job_entrypoint` span with a non-empty `lk.job_id` as the call's conversation root, so the call is listed in Voice and its recording is found on the child `agent_session` span.
- Traces ingested before attachments were enabled on the server are not re-classified.
- Do not set `fi.span.kind` on `job_entrypoint` or `agent_session` yourself to force a root; that changes how the whole trace is classified.

**Completed Egress.** Pass a completed audio-only Egress that your app is authorized to use:

```python
handle = await traceai.import_livekit_completed_egress(
egress_info, session_context=session_span, egress_id=egress_info.egress_id,
room_name=room, connection_id=CONNECTION_ID, registry=registry, existing_source_authorized=True)
```

- `egress_info` is an object you already have. Alternatively, pass `egress_client=lkapi.egress`, which makes one read by that ID.
- `connection_id` and `registry` identify the output that Future AGI has registered for your LiveKit connection.
- Only `EGRESS_COMPLETE` jobs from audio-only room or track composites with exactly one file are accepted. Anything else returns `source_unavailable`.
- No location, filename, URL or credential is ever sent.
- `handle.reason == "queued"` means the server accepted the import. It does not mean the recording is playable yet.
58 changes: 57 additions & 1 deletion python/frameworks/livekit/tests/test_integration.py
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,10 @@ def test_enabling_http_mapping_keeps_the_exporter_alongside_livekits_privacy_fil
assert enable_http_attribute_mapping() is True

processors = _processors(provider)
assert len(processors) == 2
from traceai_livekit._session_context import _SessionContextProcessor

assert len(processors) == 3
assert isinstance(processors[2], _SessionContextProcessor)
assert isinstance(processors[0], pii._PIIFilteringSpanProcessor)
assert isinstance(processors[1], BatchSpanProcessor)
assert isinstance(processors[1].span_exporter, MappedHTTPSpanExporter)
Expand Down Expand Up @@ -406,6 +409,59 @@ def test_the_allow_pii_opt_out_keeps_content_out_of_the_export(
assert [k for k in attributes if k.startswith("lk.pii.")] == []


@pytest.mark.parametrize("allow_pii", [True, False])
def test_overlapping_native_sessions_reach_collector_with_isolated_grouping(
monkeypatch: pytest.MonkeyPatch, collector: _Collector, make_provider: Any, allow_pii: bool,
) -> None:
from fi_instrumentation import using_session
from livekit.agents.telemetry import tracer

monkeypatch.setenv("LIVEKIT_TELEMETRY_ALLOW_PII", str(allow_pii).lower())
provider = make_provider()
enable_http_attribute_mapping()
roots = {}

async def run():
entered = [asyncio.Event(), asyncio.Event()]

async def call(index):
with using_session("native-call-" + str(index)):
async with AgentSession(llm=_FakeLLM()) as session:
await session.start(Agent(instructions="Synthetic session isolation test."))
roots[session._session_span.get_span_context().trace_id] = "native-call-" + str(index)
entered[index].set()
await entered[1 - index].wait()
await session.run(user_input=USER_TEXT)

await asyncio.gather(call(0), call(1))
tracer.start_span("outside-session").end()

asyncio.run(run())
provider.force_flush()
assert len(roots) == 2
names = {trace_id: set() for trace_id in roots}
outside = []
for body in collector.bodies:
request = ExportTraceServiceRequest.FromString(body)
for resource in request.resource_spans:
for scope in resource.scope_spans:
for span in scope.spans:
attrs = {a.key: _any_value(a.value) for a in span.attributes}
trace_id = int.from_bytes(span.trace_id, "big")
if trace_id in roots:
names[trace_id].add(span.name)
if allow_pii:
assert attrs[SpanAttributes.SESSION_ID] == roots[trace_id]
else:
assert SpanAttributes.SESSION_ID not in attrs
elif span.name == "outside-session":
outside.append(attrs)
for seen in names.values():
assert {"agent_session", "agent_turn", "llm_node", "session_close"} <= seen
assert len(outside) == 1
assert SpanAttributes.SESSION_ID not in outside[0]


# Recording rows use real LiveKit telemetry, but never start an RPC/collector.
@pytest.mark.parametrize('allow_pii', [True, False])
def test_recording_metadata_respects_livekit_pii_opt_out_and_cloud_redaction(
Expand Down
Loading
Loading