Skip to content

fix(voyage): trace multimodal and contextualized embeddings - #462

Merged
nik13 merged 1 commit into
devfrom
fix/voyage-embedding-coverage
Nov 1, 2026
Merged

nik13 merged 1 commit into
devfrom
fix/voyage-embedding-coverage

Conversation

@nik13

@nik13 nik13 commented Oct 31, 2026

Copy link
Copy Markdown
Contributor

Summary

Retain traceAI-voyage and close its missing Python SDK coverage: Client and AsyncClient now trace multimodal_embed and contextualized_embed, alongside the existing embed and rerank entry points.

  • One EMBEDDING span per public call, preserving the active parent, SDK host choice, return values, errors and batch boundaries. Existing VoyageInstrumentor().instrument() / .uninstrument() APIs remain unchanged.
  • Record the requested model/options, SDK-reported total input tokens and unpacked dimensions. Contextualized document counts are distinct from returned chunk-vector counts, including client/server chunking. Multimodal pixel usage is kept separate from token usage.
  • Keep embedding inputs opt-in and bounded. Nested capture preserves groups with a global 64-segment limit. Media URLs/payloads/object representations and embedding vectors are never captured. New-method error spans omit the original message because SDK validation errors can echo whole media/document inputs; the application's exception is unchanged.
  • Update the package README with the supported methods and privacy behavior; remove obsolete public verification-diary prose.

No dependency/version/CHANGELOG bump. No copied vendor instrumentation and no new native telemetry/export path.

Verification

Reproduced on unchanged dev d6a9966dd4d5d5fb344f7f5ab11b7483c2c2a6bc: all four sync/async coverage cases failed because the SDK calls succeeded but emitted no Voyage span. The tests call real voyageai classes against a loopback HTTP server, not mocked public methods.

After the change, Python 3.12.10, both editable installs and the built wheel:

SDK / core Package suite
voyageai 0.5.0 / fi-instrumentation-otel 1.2.2 132 passed
voyageai 0.3.7 / fi-instrumentation-otel 1.1.0 129 passed, 3 skipped

The floor skips the existing local-model test and the two new server-auto-chunking cases because those SDK APIs do not exist there. Both SDK versions cover all four newly wrapped methods.

Coverage includes nested batches, positional options, packed binary dimensions, chunk callbacks called only by the SDK, empty batches, default/opt-in/hidden inputs, API-key and PII redaction, PIL images, media dictionaries, validation errors, exact usage, parentage and restoration of all eight methods. OTLP/HTTP protobuf receiver checks verify span shape and absence of media, vectors and hidden inputs, including the validation-error wire path.

All tests and example execution ran with outbound network denied except localhost using a macOS sandbox. No vendor API or live credentials were used. The README and staged docs setup snippets were compiled and executed through the real SDK and Future AGI register/export path with local endpoints. Wheel/sdist build, Ruff check and git diff --check passed.

Documentation / release order

Existing docs PR future-agi/docs#995 does not fix this gap; it still excludes these methods. No duplicate docs PR is opened: the updated page depends on this package release and is staged for the release owner to apply after registry publication. The docs-site MDX build/render has not been run for that staged page.

Review / rollout

Self-reviewed against the internal minimal-code, typing, error-handling, testing and PR-review standards. No new public API removal, database migration, frontend, enterprise boundary, or provider/network override. The existing embed/rerank contract remains covered by the full package suite. Reverting this commit removes the additional coverage without changing application behavior. Human review and package release remain required; nothing is merged or published by this PR.

E2E coverage

not applicable (no e2e/ harness) · flows hit: n/a · marker: n/a · verdict: not applicable

🤖 Generated with Claude Code

Preserve the instrumentor API while tracing both embedding families on sync and async clients. Capture nested text only by opt-in, omit media and vectors, and report contextualized chunk counts and multimodal usage without double counting.

Co-Authored-By: Claude Code <noreply@anthropic.com>
@nik13
nik13 merged commit ed22eaf into dev Nov 1, 2026
21 checks passed
@nik13
nik13 deleted the fix/voyage-embedding-coverage branch November 1, 2026 01:28
This was referenced Nov 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant