Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions scripts/integrations.test.mjs
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
import './ollama-java.test.mjs';
import './azure-openai-java.test.mjs';
import './langsmith-page.test.mjs';
import './vercel-platform-page.test.mjs';
Expand Down
34 changes: 34 additions & 0 deletions scripts/ollama-java.test.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
import test from 'node:test';
import assert from 'node:assert/strict';
import { readFileSync } from 'node:fs';

const page = readFileSync(new URL('../src/pages/docs/integrations/traceai/java/llm-providers.mdx', import.meta.url), 'utf8');
const section = page.split('## Ollama\n')[1]?.split('\n## IBM Watsonx')[0];

test('Ollama Java installs the released wrapper and its explicit legacy SDK dependency', () => {
assert.ok(section);
assert.match(section, /<groupId>com\.futureagi<\/groupId>\s*<artifactId>traceai-java-ollama<\/artifactId>\s*<version>1\.0\.0<\/version>/);
assert.match(section, /<groupId>io\.github\.ollama4j<\/groupId>\s*<artifactId>ollama4j<\/artifactId>\s*<version>1\.0\.79<\/version>/);
assert.match(section, /com\.futureagi:traceai-java-ollama:1\.0\.0/);
assert.match(section, /io\.github\.ollama4j:ollama4j:1\.0\.79/);
assert.doesNotMatch(section, /main-SNAPSHOT|com\.github\.future-agi/);
});

test('Ollama Java states the SDK boundary and uses working released chat with shutdown', () => {
assert.match(section, /1\.1\.7/);
assert.match(section, /not compatible/);
assert.match(section, /provided/);
assert.match(section, /traced\.chat\(/);
assert.match(section, /finally\s*\{\s*TraceAI\.shutdown\(\)/);
assert.doesNotMatch(section, /traced\.generate\(/);
assert.match(section, /`generate\(\)`.*1\.0\.0/);
});

test('Ollama Java does not promise unsupported streaming, tools or usage', () => {
assert.match(section, /[Ss]treaming callbacks/);
assert.match(section, /tool-call attributes/);
assert.match(section, /token usage/);
assert.match(section, /unwrap\(\)/);
assert.match(section, /client-side/);
assert.doesNotMatch(section, /server's own timing/);
});
67 changes: 42 additions & 25 deletions src/pages/docs/integrations/traceai/java/llm-providers.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -183,24 +183,40 @@ Streaming, `OpenAIAsyncClient`, Responses, audio, images, and files are not trac

## Ollama

Wraps `io.github.ollama4j.OllamaAPI` for local Ollama models.
Trace local Ollama chat and embedding requests with `TracedOllamaAPI`, the wrapper for the legacy `io.github.ollama4j.OllamaAPI` client.

Use **Ollama4j 1.0.79** with this wrapper. Ollama4j 1.1.7 uses `io.github.ollama4j.Ollama` instead and is not compatible. The SDK is a `provided` dependency, so add both packages explicitly from Maven Central; no JitPack repository is needed.

<CodeGroup titles={["Maven", "Gradle"]}>

```xml Maven
<dependency>
<groupId>com.github.future-agi.traceAI</groupId>
<groupId>com.futureagi</groupId>
<artifactId>traceai-java-ollama</artifactId>
<version>main-SNAPSHOT</version>
<version>1.0.0</version>
</dependency>
<dependency>
<groupId>io.github.ollama4j</groupId>
<artifactId>ollama4j</artifactId>
<version>1.0.79</version>
</dependency>
```

```groovy Gradle
implementation 'com.github.future-agi.traceAI:traceai-java-ollama:main-SNAPSHOT'
repositories {
mavenCentral()
}

dependencies {
implementation 'com.futureagi:traceai-java-ollama:1.0.0'
implementation 'io.github.ollama4j:ollama4j:1.0.79'
}
```

</CodeGroup>

Start your Ollama server at `http://localhost:11434` and make sure the `llama3` model is installed, or replace the model name below with one you already have. Configure the `FI_*` variables from the [Java SDK setup](/docs/integrations/traceai/java) before running this example. Local model inference stays on your Ollama server; captured trace content is sent to Future AGI.

```java
import ai.traceai.TraceAI;
import ai.traceai.ollama.TracedOllamaAPI;
Expand All @@ -210,33 +226,34 @@ import io.github.ollama4j.models.chat.OllamaChatMessageRole;
import java.util.List;

TraceAI.initFromEnvironment();
try {
OllamaAPI api = new OllamaAPI("http://localhost:11434");
TracedOllamaAPI traced = new TracedOllamaAPI(api);

OllamaAPI api = new OllamaAPI("http://localhost:11434");
TracedOllamaAPI traced = new TracedOllamaAPI(api);

// Generate
var result = traced.generate("llama3", "What is the capital of France?");
System.out.println(result.getResponse());
var response = traced.chat("llama3", List.of(
new OllamaChatMessage(OllamaChatMessageRole.USER,
"What is the capital of France?")
));
System.out.println(response.getResponse());
} finally {
TraceAI.shutdown();
}
```

// Chat
var chatResult = traced.chat("llama3", List.of(
new OllamaChatMessage(OllamaChatMessageRole.USER, "Hello!")
));
For a long-running application, initialize once at startup and call `TraceAI.shutdown()` when the application exits. Open your project in Future AGI to inspect the **Ollama Chat** span.

// Embeddings
var embedding = traced.embed("llama3", "Hello world");
### Supported operations

// List models
var models = traced.listModels();
```
| Wrapper method | Span |
| --- | --- |
| `chat(model, messages)` | Ollama Chat (LLM) |
| `embed(model, text)` | Ollama Embed (EMBEDDING) |
| `embedBatch(model, texts)` | Ollama Embed Batch (EMBEDDING) |
| `listModels()` | Ollama List Models (LLM) |

**Spans created:**
- `generate()` - "Ollama Generate" (LLM)
- `chat()` - "Ollama Chat" (LLM)
- `embed()` - "Ollama Embed" (EMBEDDING)
- `listModels()` - "Ollama List Models" (LLM)
Use `chat()` for text generation: `generate()` in traceAI 1.0.0 throws before sending the request. Choose an embedding-capable model when using `embed()` or `embedBatch()`.

Ollama spans include `ollama.response_time_ms` from the Ollama server's own timing.
Only calls through the wrapper are traced. Streaming callbacks, structured tool-call attributes and token usage are not supported; calls through `unwrap()` bypass tracing. `ollama.response_time_ms` records the SDK's client-side elapsed time for chat, not the server's model-evaluation duration.

---

Expand Down
Loading