Skip to content

fix(ollama): preserve native Python streaming contracts - #426

Open
nik13 wants to merge 10 commits into
devfrom
fix/ollama-python-stream-lifecycle
Open

nik13 wants to merge 10 commits into
devfrom
fix/ollama-python-stream-lifecycle

Conversation

@nik13

@nik13 nik13 commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Retain the native Ollama Python package and repair its tracing contract. OpenAI-compatible tracing does not cover applications using ollama.Client or ollama.AsyncClient.

  • Return an async iterator after a single await for streaming chat and generate calls, including positional stream arguments.
  • Replace four duplicated streaming generators with shared lifecycle adapters. Finish spans on exhaustion, explicit close (including before iteration), stream errors, and cancellation; restore caller context between chunks.
  • Aggregate text, thinking, complete tool calls, and final usage without buffering every raw chunk. Keep a 65,536-character telemetry preview and at most 32 tools, mark truncation, and leave application responses unchanged.
  • Normalize tools and thinking in streaming and non-streaming output. Apply output privacy to serialized payloads as well as normalized fields.
  • Restore all four module-level helpers and the optional Protect hook on uninstrument; re-instrumentation exports one span per call.
  • Correct native embedding examples and document stream closure and preview limits.

Verification

  • Baseline d6a9966dd4d5d5fb344f7f5ab11b7483c2c2a6bc: the original 22-case regression set had 18 failures and 4 passes, independently with Ollama 0.3.0 and 0.6.3. Failures include the extra-coroutine async contract, lost thinking/tools, missing alias restoration, and unclosed spans.
  • Fixed package: 129 tests passed on each SDK version, using Python 3.12, pytest 8.4.2, and pytest-asyncio 0.24.0.
  • Tests use real SDK clients with synthetic HTTP transports and serialize/decode real OTLP protobufs. They cover normal/partial/empty streams, cancellation, HTTP stream cleanup, context restoration, privacy, bounded accumulation, suppression, module helper calls, and re-instrumentation.
  • Local tests ran inside a macOS sandbox denying outbound networking. No vendor/model calls or real credentials were used.
  • Added Ollama Python contracts CI for both SDK versions, with test execution inside a network-disabled Linux namespace.

Run locally after installing the package and test tools:

python -m pytest python/frameworks/ollama/tests -q -p no:cacheprovider

Review and rollout

  • Self-review: no introduced blocking findings. E2E coverage: not applicable; this repository has no product e2e/ coverage harness. No migrations, frontend contracts, or configured Ruff gate apply to this package.
  • The public OllamaInstrumentor entry point and supported SDK floor are unchanged; no version or changelog sections were edited.
  • This does not modify TypeScript ESM work or the separate OpenAI-compatible provider fix (fix(openai): migrate vLLM and attribute Ollama clients #424).
  • A native-client customer guide is prepared separately and held until this package fix is released. Its setup, sync, and async snippets were executed offline against real SDK clients; no live Cloud/UI validation is claimed.
  • Release through the normal release-please process after merge; no tags, registry publication, or merge performed here.

Privacy settings: what now holds

Ollama typed tool inputs retain adapter-side sanitization, while exported opaque aggregates follow the core whole-value __REDACTED__ contract. The tests now check both layers independently without dropping marker-absence, request integrity, or existing parameterized cases. The RAG example also defines its context before use.

Validation Ollama 0.3.0 Ollama 0.6.3
Complete package suite 190 passed, 0 failed 190 passed, 0 failed
Local provider + HTTP OTLP receiver 72 passed, 0 failed 72 passed, 0 failed

The RAG regression fails on the previous round-one head with undefined context, then passes on the final head. Historical PR-base and current-dev comparisons were recorded separately; their failures were not used to waive final-head failures.

Limits: validation uses Python 3.12 and the two SDK versions in the package CI matrix, synthetic data, offline SDK transports, and a VM-local provider/OTLP receiver. The 72 receiver cases per version cover typed chat inputs with input-text, input-message, and whole-input hiding across synchronous/asynchronous and streaming/non-streaming calls. Output/image/embedding behavior is covered by the package serialization suite, not an equivalent expanded HTTP receiver matrix. No real provider or production workload was contacted. Core privacy policy is unchanged.

🤖 Generated with Claude Code

nik13 and others added 10 commits October 31, 2026 09:44
Restore single-await async streams, aggregate bounded thinking and tools, preserve output privacy, and restore module helpers on uninstrument. Cover real SDK 0.3.0 and 0.6.3 with offline OTLP contracts.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
Normalize typed tool calls and functions in detached telemetry copies. Keep spans for SDK message mappings with optional images set to null. Add sync and async streaming privacy regressions using the real SDK.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
Keep detached-copy structure checks and require whole-value privacy placeholders on the wire.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Exercise the RAG example with a real offline SDK client.

Co-Authored-By: Claude Code <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant