From ac9515b2e57adc54db8bea34c41e9dd2bbb78df2 Mon Sep 17 00:00:00 2001 From: Nikhil Pareek Date: Fri, 16 Oct 2026 23:07:18 -0700 Subject: [PATCH 1/4] docs(integrations): Google Discovery Engine tracing (TH-8332) Page for traceAI-discoveryengine (future-agi/traceAI#224): install, credentials vs Future AGI project, register/instrument, what is captured, privacy default (query via TraceConfig, FI_HIDE_INPUTS hides it), tested versions, troubleshooting, limitations, difference from Google GenAI / Vertex AI and gRPC instrumentation. navigation.ts is not edited. Co-Authored-By: Claude Opus 5.5 --- .../traceai/google-discovery-engine.mdx | 143 ++++++++++++++++++ 1 file changed, 143 insertions(+) create mode 100644 src/pages/docs/integrations/traceai/google-discovery-engine.mdx diff --git a/src/pages/docs/integrations/traceai/google-discovery-engine.mdx b/src/pages/docs/integrations/traceai/google-discovery-engine.mdx new file mode 100644 index 000000000..82d56cd45 --- /dev/null +++ b/src/pages/docs/integrations/traceai/google-discovery-engine.mdx @@ -0,0 +1,143 @@ +--- +title: "Google Discovery Engine (Vertex AI Search) Integration with Future AGI Tracing" +description: "Trace Google Discovery Engine (Vertex AI Search) search, search_lite and answer_query calls in Future AGI with the traceAI-discoveryengine instrumentor." +--- + +## About + +`traceAI-discoveryengine` wraps the official [`google-cloud-discoveryengine`](https://pypi.org/project/google-cloud-discoveryengine/) Python client (Google Discovery Engine, sold as Vertex AI Search) so every `search`, `search_lite` and `answer_query` call becomes a `RETRIEVER` span in Future AGI. + +The client does not trace itself. At `google-cloud-discoveryengine` 0.20.5 an uninstrumented call emits no span, even with `GOOGLE_SDK_EXPERIMENTAL_PYTHON_TRACING_ENABLED=true`, and a gRPC instrumentation you add yourself only creates transport spans without the query, result count or retriever kind. The package fills that gap with one span per call. + +The package is Alpha. It is released from [future-agi/traceAI](https://github.com/future-agi/traceAI) (`python/frameworks/discoveryengine`). Until a release is on PyPI, install it from that directory of the repository. + +## 1. Installation + +```bash +pip install traceAI-discoveryengine +``` + +Until the first release reaches PyPI, install from a checkout of the repository: + +```bash +pip install "fi-instrumentation-otel>=1.1.0" "google-cloud-discoveryengine>=0.20.5,<1" \ + ./python/frameworks/discoveryengine +``` + +Tested with `google-cloud-discoveryengine` 0.20.5 (the latest release when this page was written, 2026-10-16) on Python 3.10, 3.11, 3.12 and 3.13. The package accepts `google-cloud-discoveryengine>=0.20.5,<1`; outside that range `instrument()` logs an error and wraps nothing. + +## 2. Set environment variables + +```python +import os + +os.environ["FI_API_KEY"] = "your-futureagi-api-key" +os.environ["FI_SECRET_KEY"] = "your-futureagi-secret-key" +``` + +Google credentials stay where the Google client already reads them: Application Default Credentials (`gcloud auth application-default login` or `GOOGLE_APPLICATION_CREDENTIALS`), credentials you pass to the client, or `client_options={"api_key": ...}` for `search_lite`. The package never reads ADC and never records a credential. + +The Future AGI project (`project_name` below) is not your GCP project. The GCP project only appears inside resource names such as the serving config. + +## 3. Register and instrument + +```python +from fi_instrumentation import register +from fi_instrumentation.fi_types import ProjectType +from google.cloud import discoveryengine_v1 as discoveryengine +from traceai_discoveryengine import DiscoveryEngineInstrumentor + +trace_provider = register( + project_type=ProjectType.OBSERVE, + project_name="support-search", +) +DiscoveryEngineInstrumentor().instrument(tracer_provider=trace_provider) + +serving_config = ( + "projects/my-gcp-project/locations/global/collections/default_collection/" + "engines/my-engine/servingConfigs/default_search" +) +search = discoveryengine.SearchServiceClient() # Application Default Credentials +pager = search.search(request={"serving_config": serving_config, "query": "rotate an API key"}) + +answers = discoveryengine.ConversationalSearchServiceClient() +answers.answer_query( + request={"serving_config": serving_config, "query": {"text": "How do I rotate an API key?"}} +) +``` + +Import the clients from `google.cloud.discoveryengine_v1`. **`from google.cloud import discoveryengine` produces no spans**: at 0.20.5 that unversioned module re-exports the v1beta clients, which are not traced. + +An engine in a multi-region location needs its regional endpoint, for example `client_options={"api_endpoint": "eu-discoveryengine.googleapis.com"}`. The gRPC (default), gRPC asyncio and REST (`transport="rest"`) transports are traced the same way. + +Call `register()` first so the instrumentor has a tracer provider. Under an active span (an agent or a RAG pipeline step) the Discovery Engine span is its child, so a retrieval followed by an LLM call shows as one trace. + +## What is captured + +| Call | Span name | Span kind | +|---|---|---| +| `SearchServiceClient.search`, `SearchServiceAsyncClient.search` | `discoveryengine.search` | `RETRIEVER` | +| `SearchServiceClient.search_lite`, `SearchServiceAsyncClient.search_lite` | `discoveryengine.search_lite` | `RETRIEVER` | +| `ConversationalSearchServiceClient.answer_query`, `ConversationalSearchServiceAsyncClient.answer_query` | `discoveryengine.answer_query` | `RETRIEVER` | + +| Attribute | Value | +|---|---| +| `gen_ai.retrieval.query`, `input.value` | The query (`SearchRequest.query` or `AnswerQueryRequest.query.text`), credentials removed, cut to 1 KB. Shown as **Query** in the span view. | +| `discoveryengine.serving_config` | The serving config resource name from the request. | +| `discoveryengine.result_count` | Results on the returned (first) page for search; number of `references` for answers. | +| `discoveryengine.answer.length`, `discoveryengine.answer.state` | Length of the answer text in characters, and the answer state (`SUCCEEDED`, `FAILED` ...). | +| `discoveryengine.session` | The Discovery Engine session resource name, from the request and then the response. | +| `discoveryengine.error.status`, `discoveryengine.error.code` | On failure: the status name (`PERMISSION_DENIED`) and code (`403`). | + +Iterating a search pager past the first page adds no span: later pages are fetched by the pager, and `result_count` counts the first page only. + +An error raised by the client (`PermissionDenied`, `InvalidArgument`, `ServiceUnavailable`, `Forbidden` on REST, or `RetryError`) ends the span with status `ERROR` and a message such as `PermissionDenied: 403 ...`; your code still receives the exception unchanged. An answer returned with state `FAILED` is marked `ERROR` (`answer state FAILED`) without an exception. + +Not captured: search results (titles, snippets, documents, summaries), answer text, citations and reference content, request metadata, model name, token counts and cost. The answer response carries no model field, and Discovery Engine is billed in GCP rather than per token. Google credentials, stack traces and local file paths are never exported. + +Not traced: conversation and session CRUD (`create_conversation`, `converse_conversation` ...), `stream_answer_query`, document ingestion and data store or engine administration, and the v1beta and v1alpha clients. + +## Privacy + +traceAI instrumentors follow the standard `TraceConfig`: the query is recorded by default. To turn it off, set `FI_HIDE_INPUTS=true` or pass a config: + +```python +from fi_instrumentation import TraceConfig + +DiscoveryEngineInstrumentor().instrument( + tracer_provider=trace_provider, config=TraceConfig(hide_inputs=True) +) +``` + +With hidden inputs, `input.value` is `__REDACTED__`, `gen_ai.retrieval.query` is dropped, and the query is removed from server error text too (verbatim and in the escaped copies the Google client libraries write). Counts, the state and resource names stay. `FI_PII_REDACTION=true` replaces emails, phone numbers and similar patterns in every recorded text; it also matches the last ten digits of a numeric GCP project number in a resource name. `using_session`, `using_user`, `using_metadata` and `using_tags` apply. + +## Tested versions + +| Component | Version | +|---|---| +| `google-cloud-discoveryengine` | 0.20.5 | +| `traceAI-discoveryengine` | 0.1.0 | +| `fi-instrumentation-otel` | 1.1.0 | +| Python | 3.10, 3.11, 3.12, 3.13 | + +Verified on 2026-10-16 against a local Future AGI deployment: the real `google-cloud-discoveryengine` clients (REST, gRPC and async gRPC transports) called local stand-ins of the documented v1 API, inside an agent span next to an LLM call. The hierarchy, span kinds, an `ERROR` span with `PERMISSION_DENIED`, and the `FI_HIDE_INPUTS` switch were checked in the stored spans. `discoveryengine.googleapis.com` itself was not called (no GCP project was available for the check). + +## Troubleshooting + +- **No spans appear.** Import the clients from `google.cloud.discoveryengine_v1` (not `google.cloud.discoveryengine`), call `instrument()` before the first call, and check `FI_API_KEY` and `FI_SECRET_KEY`. +- **`instrument()` logs a version error.** The installed `google-cloud-discoveryengine` is outside `>=0.20.5,<1`; upgrade it. +- **Only gRPC transport spans, no query.** Those come from a gRPC instrumentation, not from this package. Install and instrument `traceAI-discoveryengine`; the gRPC spans then nest under its spans. +- **`PermissionDenied` or `Forbidden` on every call.** That is the GCP side: check the credentials, the serving config name and the regional endpoint. The span records the status and code, not the credential. +- **Duplicate spans.** Do not combine this package with another instrumentor that wraps the same client methods. + +## Limitations + +- Python only; the Java and Node clients are not traced. +- Only v1 clients. `stream_answer_query` and `converse_conversation` are not traced. +- Query removal from error text matches the verbatim query and the escaped copies the client libraries write; a reworded or truncated server copy is not matched, and other request fields a server error may quote (a `filter`, `user_pseudo_id`) are not removed. +- The async REST transport is not exercised by the tests. + +## Difference from related integrations + +- [Google GenAI](/docs/integrations/traceai/google_genai) and [Vertex AI](/docs/integrations/traceai/vertexai) trace Gemini model calls (`LLM` spans with model and tokens). `traceAI-discoveryengine` traces the search and grounded-answer calls of Vertex AI Search (`RETRIEVER` spans); it does not instrument `google.genai`, and the two can be used together in one RAG trace. +- A gRPC OpenTelemetry instrumentation records the transport call only. This package adds the retrieval span with the query, serving config, result count and error status. From b13738afe58334006dd7a7675b2d514ce5dbbe21 Mon Sep 17 00:00:00 2001 From: Nikhil Pareek Date: Sun, 18 Oct 2026 15:32:11 -0700 Subject: [PATCH 2/4] docs(integrations): Google Discovery Engine live verification (TH-8332) Replace the 2026-10-16 stand-in note with the live run against discoveryengine.googleapis.com on 2026-10-18: versions, the calls and errors traced, the stored RAG trace and the privacy check. Explain the server-side requirements the live run hit (search_lite needs a public website data store and an API key; answer_query needs the LLM add-on) and add troubleshooting entries for them, for NOT_FOUND on a missing engine and for the indexing delay. Keep stand-in wording only for the paths not verified live. Co-Authored-By: Claude Opus 5.5 --- .../traceai/google-discovery-engine.mdx | 39 +++++++++++++++++-- 1 file changed, 35 insertions(+), 4 deletions(-) diff --git a/src/pages/docs/integrations/traceai/google-discovery-engine.mdx b/src/pages/docs/integrations/traceai/google-discovery-engine.mdx index 82d56cd45..b4dc129b0 100644 --- a/src/pages/docs/integrations/traceai/google-discovery-engine.mdx +++ b/src/pages/docs/integrations/traceai/google-discovery-engine.mdx @@ -24,7 +24,7 @@ pip install "fi-instrumentation-otel>=1.1.0" "google-cloud-discoveryengine>=0.20 ./python/frameworks/discoveryengine ``` -Tested with `google-cloud-discoveryengine` 0.20.5 (the latest release when this page was written, 2026-10-16) on Python 3.10, 3.11, 3.12 and 3.13. The package accepts `google-cloud-discoveryengine>=0.20.5,<1`; outside that range `instrument()` logs an error and wraps nothing. +Tested with `google-cloud-discoveryengine` 0.20.5 (still the latest release on PyPI on 2026-10-18), against the live API on Python 3.12 and in the test suite on Python 3.10, 3.11, 3.12 and 3.13. The package accepts `google-cloud-discoveryengine>=0.20.5,<1`; outside that range `instrument()` logs an error and wraps nothing. ## 2. Set environment variables @@ -35,7 +35,7 @@ os.environ["FI_API_KEY"] = "your-futureagi-api-key" os.environ["FI_SECRET_KEY"] = "your-futureagi-secret-key" ``` -Google credentials stay where the Google client already reads them: Application Default Credentials (`gcloud auth application-default login` or `GOOGLE_APPLICATION_CREDENTIALS`), credentials you pass to the client, or `client_options={"api_key": ...}` for `search_lite`. The package never reads ADC and never records a credential. +Google credentials stay where the Google client already reads them: Application Default Credentials (`gcloud auth application-default login` or `GOOGLE_APPLICATION_CREDENTIALS`), credentials you pass to the client, or `client_options={"api_key": ...}` for `search_lite` (public website search only). The package never reads ADC and never records a credential. The Future AGI project (`project_name` below) is not your GCP project. The GCP project only appears inside resource names such as the serving config. @@ -70,6 +70,8 @@ Import the clients from `google.cloud.discoveryengine_v1`. **`from google.cloud An engine in a multi-region location needs its regional endpoint, for example `client_options={"api_endpoint": "eu-discoveryengine.googleapis.com"}`. The gRPC (default), gRPC asyncio and REST (`transport="rest"`) transports are traced the same way. +What the server accepts depends on your app, not on this package. `search_lite` serves public website search with an API key; against any other data store Discovery Engine answers `FAILED_PRECONDITION` ("API key is only supported for public website search."). `answer_query` needs an app with the Large Language Model add-on (Enterprise edition); a Standard edition app answers `FAILED_PRECONDITION` ("This feature is only available when Large Language Model add-on is enabled."). Both show up as `ERROR` spans with `discoveryengine.error.status` = `FAILED_PRECONDITION`. + Call `register()` first so the instrumentor has a tracer provider. Under an active span (an agent or a RAG pipeline step) the Discovery Engine span is its child, so a retrieval followed by an LLM call shows as one trace. ## What is captured @@ -118,15 +120,44 @@ With hidden inputs, `input.value` is `__REDACTED__`, `gen_ai.retrieval.query` is | `google-cloud-discoveryengine` | 0.20.5 | | `traceAI-discoveryengine` | 0.1.0 | | `fi-instrumentation-otel` | 1.1.0 | -| Python | 3.10, 3.11, 3.12, 3.13 | +| `google-api-core` / `google-auth` | 2.42.0 / 2.62.0 | +| `grpcio` / `protobuf` | 1.84.0 / 7.36.2 | +| Python | 3.10, 3.11, 3.12, 3.13 (live run on 3.12) | + +### Live vendor verification (2026-10-18) + +The package was run as this page describes against the real `discoveryengine.googleapis.com` (v1, `global`), exporting to a local Future AGI deployment. The app was the smallest Standard edition setup: one unstructured data store with three short inline documents and one generic search engine without the LLM add-on, deleted after the run. The only change from the code above: ADC was not set up on the test machine, so each client got `credentials=google.oauth2.credentials.Credentials(token=..., quota_project_id=...)`. + +| Call | Transport | Result from Google | Span in Future AGI | +|---|---|---|---| +| `search` | gRPC, REST, gRPC asyncio | 2 results | `OK`, `discoveryengine.result_count` = 2, query, serving config | +| `search` on a serving config that does not exist | gRPC, REST | `NotFound` 404 | `ERROR`, `NOT_FOUND` / 404 | +| `search_lite` with OAuth credentials | gRPC | `FailedPrecondition` 400 | `ERROR`, `FAILED_PRECONDITION` / 400 | +| `answer_query` on the Standard edition app | gRPC and REST | `FailedPrecondition` 400 (`BadRequest` on REST) | `ERROR`, `FAILED_PRECONDITION` / 400 | + +The RAG trace (agent span, three Discovery Engine spans, then an LLM call that used the returned snippets) as stored, trace `276682b6-0d8f-43de-bf3a-ab48b507cf22`: + +```text +- [agent] orbit-support-agent + - [retriever] discoveryengine.search + - [retriever] discoveryengine.search_lite status=ERROR: FailedPrecondition: 400 API key is only supported for public website search. + - [retriever] discoveryengine.answer_query status=ERROR: FailedPrecondition: 400 This feature is only available when Large Language Model add-on is enabled. + - [llm] ChatCompletion model=gpt-5.6-terra tokens=453/31 +``` + +Privacy was checked in every stored column. With the defaults the query was on each Discovery Engine span; with `FI_HIDE_INPUTS=true` it was in no column (`input.value` = `__REDACTED__`, no `gen_ai.retrieval.query`). In both modes the Discovery Engine spans held no result content, and nothing stored contained a credential, an access token, a local path, a user name, a host name or a stack trace. -Verified on 2026-10-16 against a local Future AGI deployment: the real `google-cloud-discoveryengine` clients (REST, gRPC and async gRPC transports) called local stand-ins of the documented v1 API, inside an agent span next to an LLM call. The hierarchy, span kinds, an `ERROR` span with `PERMISSION_DENIED`, and the `FI_HIDE_INPUTS` switch were checked in the stored spans. `discoveryengine.googleapis.com` itself was not called (no GCP project was available for the check). +Not verified against Google yet, only against local stand-ins of the documented API: a successful `search_lite` (needs a public website data store and an API key), a successful `answer_query` with its answer state, length and session (needs the LLM add-on), pagination, a regional endpoint, the async REST transport, and removing a hidden query from a server error that quotes it (the real errors seen did not quote the query). ## Troubleshooting - **No spans appear.** Import the clients from `google.cloud.discoveryengine_v1` (not `google.cloud.discoveryengine`), call `instrument()` before the first call, and check `FI_API_KEY` and `FI_SECRET_KEY`. - **`instrument()` logs a version error.** The installed `google-cloud-discoveryengine` is outside `>=0.20.5,<1`; upgrade it. - **Only gRPC transport spans, no query.** Those come from a gRPC instrumentation, not from this package. Install and instrument `traceAI-discoveryengine`; the gRPC spans then nest under its spans. +- **`FailedPrecondition` on `answer_query`: "This feature is only available when Large Language Model add-on is enabled."** The app is Standard edition. Enable the LLM add-on (Enterprise edition) or use `search`. +- **`FailedPrecondition` on `search_lite`: "API key is only supported for public website search."** `search_lite` only serves public website data stores with an API key. Use `search` with OAuth or service account credentials for other data stores. +- **`NotFound`: "We were unable to find an engine to service the request."** The engine ID in the serving config is wrong, or the engine was created moments ago. Check the resource name and the location. +- **`search` returns no results right after you add documents.** Indexing takes a few minutes (about six in our test for three small documents), and `search` returns an empty page until then. - **`PermissionDenied` or `Forbidden` on every call.** That is the GCP side: check the credentials, the serving config name and the regional endpoint. The span records the status and code, not the credential. - **Duplicate spans.** Do not combine this package with another instrumentor that wraps the same client methods. From 824d9a42a4407c2592e86ccd78a8a388edf5adc4 Mon Sep 17 00:00:00 2001 From: Nikhil Pareek Date: Wed, 21 Oct 2026 16:20:05 -0700 Subject: [PATCH 3/4] docs(integrations): Discovery Engine page reads as released (TH-8332) The page goes live only after traceAI-discoveryengine 0.1.0 is on PyPI, so the two 'until the first release' install notes are dropped. Co-Authored-By: Claude Opus 5.5 --- .../integrations/traceai/google-discovery-engine.mdx | 9 +-------- 1 file changed, 1 insertion(+), 8 deletions(-) diff --git a/src/pages/docs/integrations/traceai/google-discovery-engine.mdx b/src/pages/docs/integrations/traceai/google-discovery-engine.mdx index b4dc129b0..2b2fb1c94 100644 --- a/src/pages/docs/integrations/traceai/google-discovery-engine.mdx +++ b/src/pages/docs/integrations/traceai/google-discovery-engine.mdx @@ -9,7 +9,7 @@ description: "Trace Google Discovery Engine (Vertex AI Search) search, search_li The client does not trace itself. At `google-cloud-discoveryengine` 0.20.5 an uninstrumented call emits no span, even with `GOOGLE_SDK_EXPERIMENTAL_PYTHON_TRACING_ENABLED=true`, and a gRPC instrumentation you add yourself only creates transport spans without the query, result count or retriever kind. The package fills that gap with one span per call. -The package is Alpha. It is released from [future-agi/traceAI](https://github.com/future-agi/traceAI) (`python/frameworks/discoveryengine`). Until a release is on PyPI, install it from that directory of the repository. +The package is Alpha. It is released from [future-agi/traceAI](https://github.com/future-agi/traceAI) (`python/frameworks/discoveryengine`). ## 1. Installation @@ -17,13 +17,6 @@ The package is Alpha. It is released from [future-agi/traceAI](https://github.co pip install traceAI-discoveryengine ``` -Until the first release reaches PyPI, install from a checkout of the repository: - -```bash -pip install "fi-instrumentation-otel>=1.1.0" "google-cloud-discoveryengine>=0.20.5,<1" \ - ./python/frameworks/discoveryengine -``` - Tested with `google-cloud-discoveryengine` 0.20.5 (still the latest release on PyPI on 2026-10-18), against the live API on Python 3.12 and in the test suite on Python 3.10, 3.11, 3.12 and 3.13. The package accepts `google-cloud-discoveryengine>=0.20.5,<1`; outside that range `instrument()` logs an error and wraps nothing. ## 2. Set environment variables From d080d2693b32e3268d740a0049b7240bc3fef4f6 Mon Sep 17 00:00:00 2001 From: Nikhil Pareek Date: Thu, 5 Nov 2026 22:04:33 -0800 Subject: [PATCH 4/4] docs(discoveryengine): publish task-focused tracing guide Replace vendor verification notes with the released package setup, a standalone search example, and actionable Google API and privacy prerequisites. Register the guide in the shared catalog and protect its customer-facing contract in CI. Co-Authored-By: Claude Code --- scripts/integrations.test.mjs | 50 +++++ src/lib/integrations.mjs | 1 + .../traceai/google-discovery-engine.mdx | 189 ++++++++---------- 3 files changed, 131 insertions(+), 109 deletions(-) diff --git a/scripts/integrations.test.mjs b/scripts/integrations.test.mjs index 30ff909ed..f9a0fa61e 100644 --- a/scripts/integrations.test.mjs +++ b/scripts/integrations.test.mjs @@ -419,6 +419,56 @@ test('Java pgvector qualifies successful-operation privacy and hides database er assert.ok(section.indexOf('FI_HIDE_ERROR_DETAILS=true') < section.indexOf('new TracedPgVectorStore(connection)')); }); +test('Discovery Engine uses released packages and a standalone v1 search example', () => { + const page = read('src/pages/docs/integrations/traceai/google-discovery-engine.mdx'); + const catalog = integrations.find(i => i.id === 'google-discovery-engine'); + assert.equal(catalog?.href, '/docs/integrations/traceai/google-discovery-engine'); + assert.equal(catalog?.category, 'frameworks'); + assert.equal(catalog?.method, 'traceAI SDK'); + assert.ok(catalog?.aliases?.includes('Vertex AI Search')); + for (const dependency of [ + 'traceAI-discoveryengine==0.1.1', + 'fi-instrumentation-otel==1.2.3', + 'google-cloud-discoveryengine==0.20.5', + ]) assert.ok(page.includes(dependency), dependency); + assert.match(page, /Python 3\.10–3\.13/); + assert.match(page, /google-cloud-discoveryengine>=0\.20\.5,<1/); + assert.match(page, /from google\.cloud import discoveryengine_v1 as discoveryengine/); + const example = page.split('## Run a traced search\n')[1].split('## Trace answers')[0]; + assert.match(example, /DiscoveryEngineInstrumentor\(\)\.instrument\(tracer_provider=trace_provider\)/); + assert.ok(example.indexOf('register(') < example.indexOf('.instrument(')); + assert.ok(example.indexOf('.instrument(') < example.indexOf('client.search(')); + assert.match(example, /first_page = next\(pager\.pages\)/); + assert.match(example, /finally:\n trace_provider\.force_flush\(\)/); + assert.match(example, /python discoveryengine_example\.py/); + assert.match(example, /Observe → Traces/); + assert.doesNotMatch(example, /answers\.answer_query\(/); +}); + +test('Discovery Engine preserves API prerequisites and privacy limits without a QA diary', () => { + const page = read('src/pages/docs/integrations/traceai/google-discovery-engine.mdx'); + const setup = page.split('## Run a traced search\n')[0]; + assert.match(setup, /Application Default Credentials/); + assert.match(setup, /FI_API_KEY/); + assert.match(setup, /FI_SECRET_KEY/); + assert.match(setup, /export FI_HIDE_INPUTS=true/); + assert.match(setup, /export FI_HIDE_ERROR_DETAILS=true/); + assert.match(page, /Public website search with a Google API key/); + assert.match(page, /Enterprise edition features and Generative responses/); + assert.match(page, /Large Language Model add-on/); + assert.match(page, /Advanced website indexing/); + assert.match(page, /eu-discoveryengine\.googleapis\.com/); + assert.match(page, /first page creates no additional spans/); + assert.match(page, /Only the Python v1 clients/); + assert.match(page, /reworded or truncated copies/); + assert.match(page, /custom gRPC channel/); + assert.match(page, /Serving config and session names remain/); + assert.doesNotMatch(page, /Live vendor verification|Tested versions|stand-ins|fi_verify|our test|276682b6|2026-10-18/i); + for (const [, route] of page.matchAll(/\]\((\/docs\/[^)#]+)(?:#[^)]*)?\)/g)) { + assert.ok(exists(route), `Missing guide link: ${route}`); + } +}); + test('public prose and catalog labels contain no internal QA label or Alpha release status', () => { for (const file of routeFiles.filter(f=>f.endsWith('.mdx'))) { const prose=publicProse(fs.readFileSync(file,'utf8')); diff --git a/src/lib/integrations.mjs b/src/lib/integrations.mjs index df0f390a2..90e8287b4 100644 --- a/src/lib/integrations.mjs +++ b/src/lib/integrations.mjs @@ -79,6 +79,7 @@ export const integrations = [ { id: 'llamaindex-workflows', name: 'LlamaIndex Workflows', href: '/docs/integrations/traceai/llamaindex-workflows', category: 'frameworks', summary: 'Capture steps in event-driven agent workflows.', method: 'traceAI SDK', logo: 'llamaindex' }, { id: 'openai-agents', name: 'OpenAI Agents SDK', href: '/docs/integrations/traceai/openai_agents', category: 'frameworks', summary: 'Trace agent runs, handoffs and tools.', method: 'traceAI SDK', logo: 'openai' }, { id: 'google-adk', name: 'Google ADK', href: '/docs/integrations/traceai/google_adk', category: 'frameworks', summary: 'Instrument agents built with Google ADK for Python.', method: 'traceAI SDK', logo: 'google', aliases: ['Agent Development Kit'] }, + { id: 'google-discovery-engine', name: 'Google Discovery Engine', href: '/docs/integrations/traceai/google-discovery-engine', category: 'frameworks', summary: 'Trace Vertex AI Search queries and grounded-answer requests.', method: 'traceAI SDK', logo: 'google', aliases: ['Vertex AI Search', 'Agent Search'] }, { id: 'crewai', name: 'CrewAI', href: '/docs/integrations/traceai/crewai', category: 'frameworks', summary: 'Trace crews, standalone agents, Flows and model calls.', method: 'Native OTLP', logo: 'crewai' }, { id: 'autogen', name: 'AutoGen', href: '/docs/integrations/traceai/autogen', category: 'frameworks', summary: 'Trace conversations between AutoGen agents.', method: 'traceAI SDK', logo: 'autogen' }, { id: 'haystack', name: 'Haystack', href: '/docs/integrations/traceai/haystack', category: 'frameworks', summary: 'Trace component execution in Haystack pipelines.', method: 'traceAI SDK', logo: 'haystack' }, diff --git a/src/pages/docs/integrations/traceai/google-discovery-engine.mdx b/src/pages/docs/integrations/traceai/google-discovery-engine.mdx index 2b2fb1c94..f6b7950a2 100644 --- a/src/pages/docs/integrations/traceai/google-discovery-engine.mdx +++ b/src/pages/docs/integrations/traceai/google-discovery-engine.mdx @@ -1,38 +1,46 @@ --- -title: "Google Discovery Engine (Vertex AI Search) Integration with Future AGI Tracing" -description: "Trace Google Discovery Engine (Vertex AI Search) search, search_lite and answer_query calls in Future AGI with the traceAI-discoveryengine instrumentor." +title: "Trace Google Discovery Engine requests" +description: "Trace Vertex AI Search search, search_lite, and answer_query calls in Future AGI with the maintained traceAI-discoveryengine Python package." --- -## About +Trace Google Discovery Engine (Vertex AI Search) requests as retrieval spans in Future AGI with `traceAI-discoveryengine`. -`traceAI-discoveryengine` wraps the official [`google-cloud-discoveryengine`](https://pypi.org/project/google-cloud-discoveryengine/) Python client (Google Discovery Engine, sold as Vertex AI Search) so every `search`, `search_lite` and `answer_query` call becomes a `RETRIEVER` span in Future AGI. +## Before you start -The client does not trace itself. At `google-cloud-discoveryengine` 0.20.5 an uninstrumented call emits no span, even with `GOOGLE_SDK_EXPERIMENTAL_PYTHON_TRACING_ENABLED=true`, and a gRPC instrumentation you add yourself only creates transport spans without the query, result count or retriever kind. The package fills that gap with one span per call. +You need Python 3.10–3.13, [Future AGI API keys](/docs/admin-settings/api-keys), and a Google Cloud project with the Discovery Engine API enabled. Create a search app connected to an indexed data store, and give your Google credentials permission to query its serving config. -The package is Alpha. It is released from [future-agi/traceAI](https://github.com/future-agi/traceAI) (`python/frameworks/discoveryengine`). - -## 1. Installation +## Install ```bash -pip install traceAI-discoveryengine +pip install 'traceAI-discoveryengine==0.1.1' 'fi-instrumentation-otel==1.2.3' 'google-cloud-discoveryengine==0.20.5' ``` -Tested with `google-cloud-discoveryengine` 0.20.5 (still the latest release on PyPI on 2026-10-18), against the live API on Python 3.12 and in the test suite on Python 3.10, 3.11, 3.12 and 3.13. The package accepts `google-cloud-discoveryengine>=0.20.5,<1`; outside that range `instrument()` logs an error and wraps nothing. +The instrumentor accepts `google-cloud-discoveryengine>=0.20.5,<1`. Use the versioned `google.cloud.discoveryengine_v1` clients: the unversioned `google.cloud.discoveryengine` module re-exports v1beta clients in 0.20.5 and is not instrumented. -## 2. Set environment variables +## Configure credentials and privacy -```python -import os +Use [Application Default Credentials](https://cloud.google.com/docs/authentication/provide-credentials-adc) for Google Cloud. For local development: + +```bash +gcloud auth application-default login +export FI_API_KEY="your-future-agi-api-key" +export FI_SECRET_KEY="your-future-agi-secret-key" +``` + +In a deployed application, use its service account or another supported ADC configuration instead of interactive login. Google credentials authenticate the search request; the Future AGI keys authenticate trace export. The Future AGI `project_name` is separate from your Google Cloud project ID. -os.environ["FI_API_KEY"] = "your-futureagi-api-key" -os.environ["FI_SECRET_KEY"] = "your-futureagi-secret-key" +Queries and sanitized server error messages are recorded by default. To hide them, set these before initializing tracing: + +```bash +export FI_HIDE_INPUTS=true +export FI_HIDE_ERROR_DETAILS=true ``` -Google credentials stay where the Google client already reads them: Application Default Credentials (`gcloud auth application-default login` or `GOOGLE_APPLICATION_CREDENTIALS`), credentials you pass to the client, or `client_options={"api_key": ...}` for `search_lite` (public website search only). The package never reads ADC and never records a credential. +Counts, answer states, and resource names remain visible. These settings do not remove arbitrary custom attributes; review the [privacy limits](#privacy-limits) before sending sensitive data. -The Future AGI project (`project_name` below) is not your GCP project. The GCP project only appears inside resource names such as the serving config. +## Run a traced search -## 3. Register and instrument +Save this as `discoveryengine_example.py`. Replace the project and engine IDs in `serving_config` with your search app's values. This example uses the `global` location. ```python from fi_instrumentation import register @@ -50,118 +58,81 @@ serving_config = ( "projects/my-gcp-project/locations/global/collections/default_collection/" "engines/my-engine/servingConfigs/default_search" ) -search = discoveryengine.SearchServiceClient() # Application Default Credentials -pager = search.search(request={"serving_config": serving_config, "query": "rotate an API key"}) - -answers = discoveryengine.ConversationalSearchServiceClient() -answers.answer_query( - request={"serving_config": serving_config, "query": {"text": "How do I rotate an API key?"}} -) +try: + client = discoveryengine.SearchServiceClient() + pager = client.search( + request={"serving_config": serving_config, "query": "rotate an API key"} + ) + first_page = next(pager.pages) + print(f"Results on the first page: {len(first_page.results)}") +finally: + trace_provider.force_flush() ``` -Import the clients from `google.cloud.discoveryengine_v1`. **`from google.cloud import discoveryengine` produces no spans**: at 0.20.5 that unversioned module re-exports the v1beta clients, which are not traced. +```bash +python discoveryengine_example.py +``` -An engine in a multi-region location needs its regional endpoint, for example `client_options={"api_endpoint": "eu-discoveryengine.googleapis.com"}`. The gRPC (default), gRPC asyncio and REST (`transport="rest"`) transports are traced the same way. +For a multi-region app, change the location in the resource name and set the matching endpoint on each client, for example `client_options={"api_endpoint": "eu-discoveryengine.googleapis.com"}` for `eu`. See Google's [location configuration](https://cloud.google.com/generative-ai-app-builder/docs/locations#specify_a_multi-region_for_your_data_store). -What the server accepts depends on your app, not on this package. `search_lite` serves public website search with an API key; against any other data store Discovery Engine answers `FAILED_PRECONDITION` ("API key is only supported for public website search."). `answer_query` needs an app with the Large Language Model add-on (Enterprise edition); a Standard edition app answers `FAILED_PRECONDITION` ("This feature is only available when Large Language Model add-on is enabled."). Both show up as `ERROR` spans with `discoveryengine.error.status` = `FAILED_PRECONDITION`. +Open **Observe → Traces** in Future AGI and select the `support-search` project. Open the `discoveryengine.search` span to inspect its retrieval kind, latency, serving config, and first-page result count. The query appears only when input capture is enabled. Calls inside an existing agent or pipeline span become children of that span. -Call `register()` first so the instrumentor has a tracer provider. Under an active span (an agent or a RAG pipeline step) the Discovery Engine span is its child, so a retrieval followed by an LLM call shows as one trace. +## Trace answers and public website search -## What is captured +The same instrumentor also traces these v1 methods: -| Call | Span name | Span kind | +| Client method | Span name | Prerequisite | |---|---|---| -| `SearchServiceClient.search`, `SearchServiceAsyncClient.search` | `discoveryengine.search` | `RETRIEVER` | -| `SearchServiceClient.search_lite`, `SearchServiceAsyncClient.search_lite` | `discoveryengine.search_lite` | `RETRIEVER` | -| `ConversationalSearchServiceClient.answer_query`, `ConversationalSearchServiceAsyncClient.answer_query` | `discoveryengine.answer_query` | `RETRIEVER` | +| `SearchServiceClient.search_lite` | `discoveryengine.search_lite` | Public website search with a Google API key, passed as `client_options={"api_key": ...}`. Use `search` with ADC for other data stores. | +| `ConversationalSearchServiceClient.answer_query` | `discoveryengine.answer_query` | Enable Enterprise edition features and Generative responses (the Large Language Model add-on). Website data stores also need Advanced website indexing. See Google's [answer prerequisites](https://cloud.google.com/generative-ai-app-builder/docs/answer#before_you_begin). | -| Attribute | Value | -|---|---| -| `gen_ai.retrieval.query`, `input.value` | The query (`SearchRequest.query` or `AnswerQueryRequest.query.text`), credentials removed, cut to 1 KB. Shown as **Query** in the span view. | -| `discoveryengine.serving_config` | The serving config resource name from the request. | -| `discoveryengine.result_count` | Results on the returned (first) page for search; number of `references` for answers. | -| `discoveryengine.answer.length`, `discoveryengine.answer.state` | Length of the answer text in characters, and the answer state (`SUCCEEDED`, `FAILED` ...). | -| `discoveryengine.session` | The Discovery Engine session resource name, from the request and then the response. | -| `discoveryengine.error.status`, `discoveryengine.error.code` | On failure: the status name (`PERMISSION_DENIED`) and code (`403`). | - -Iterating a search pager past the first page adds no span: later pages are fetched by the pager, and `result_count` counts the first page only. - -An error raised by the client (`PermissionDenied`, `InvalidArgument`, `ServiceUnavailable`, `Forbidden` on REST, or `RetryError`) ends the span with status `ERROR` and a message such as `PermissionDenied: 403 ...`; your code still receives the exception unchanged. An answer returned with state `FAILED` is marked `ERROR` (`answer state FAILED`) without an exception. - -Not captured: search results (titles, snippets, documents, summaries), answer text, citations and reference content, request metadata, model name, token counts and cost. The answer response carries no model field, and Discovery Engine is billed in GCP rather than per token. Google credentials, stack traces and local file paths are never exported. - -Not traced: conversation and session CRUD (`create_conversation`, `converse_conversation` ...), `stream_answer_query`, document ingestion and data store or engine administration, and the v1beta and v1alpha clients. - -## Privacy - -traceAI instrumentors follow the standard `TraceConfig`: the query is recorded by default. To turn it off, set `FI_HIDE_INPUTS=true` or pass a config: +For an app configured for answers, replace the contents of the example's `try` block with the following, indented inside that block. Keep the `finally` block to flush the trace: ```python -from fi_instrumentation import TraceConfig - -DiscoveryEngineInstrumentor().instrument( - tracer_provider=trace_provider, config=TraceConfig(hide_inputs=True) +answers = discoveryengine.ConversationalSearchServiceClient() +response = answers.answer_query( + request={ + "serving_config": serving_config, + "query": {"text": "How do I rotate an API key?"}, + } ) +print(response.answer.state.name) ``` -With hidden inputs, `input.value` is `__REDACTED__`, `gen_ai.retrieval.query` is dropped, and the query is removed from server error text too (verbatim and in the escaped copies the Google client libraries write). Counts, the state and resource names stay. `FI_PII_REDACTION=true` replaces emails, phone numbers and similar patterns in every recorded text; it also matches the last ten digits of a numeric GCP project number in a resource name. `using_session`, `using_user`, `using_metadata` and `using_tags` apply. +The corresponding methods on `SearchServiceAsyncClient` and `ConversationalSearchServiceAsyncClient` are also instrumented. Use the gRPC transport for async clients; synchronous clients can use gRPC (the default) or `transport="rest"`. -## Tested versions +## What is captured -| Component | Version | -|---|---| -| `google-cloud-discoveryengine` | 0.20.5 | -| `traceAI-discoveryengine` | 0.1.0 | -| `fi-instrumentation-otel` | 1.1.0 | -| `google-api-core` / `google-auth` | 2.42.0 / 2.62.0 | -| `grpcio` / `protobuf` | 1.84.0 / 7.36.2 | -| Python | 3.10, 3.11, 3.12, 3.13 (live run on 3.12) | - -### Live vendor verification (2026-10-18) - -The package was run as this page describes against the real `discoveryengine.googleapis.com` (v1, `global`), exporting to a local Future AGI deployment. The app was the smallest Standard edition setup: one unstructured data store with three short inline documents and one generic search engine without the LLM add-on, deleted after the run. The only change from the code above: ADC was not set up on the test machine, so each client got `credentials=google.oauth2.credentials.Credentials(token=..., quota_project_id=...)`. - -| Call | Transport | Result from Google | Span in Future AGI | -|---|---|---|---| -| `search` | gRPC, REST, gRPC asyncio | 2 results | `OK`, `discoveryengine.result_count` = 2, query, serving config | -| `search` on a serving config that does not exist | gRPC, REST | `NotFound` 404 | `ERROR`, `NOT_FOUND` / 404 | -| `search_lite` with OAuth credentials | gRPC | `FailedPrecondition` 400 | `ERROR`, `FAILED_PRECONDITION` / 400 | -| `answer_query` on the Standard edition app | gRPC and REST | `FailedPrecondition` 400 (`BadRequest` on REST) | `ERROR`, `FAILED_PRECONDITION` / 400 | - -The RAG trace (agent span, three Discovery Engine spans, then an LLM call that used the returned snippets) as stored, trace `276682b6-0d8f-43de-bf3a-ab48b507cf22`: - -```text -- [agent] orbit-support-agent - - [retriever] discoveryengine.search - - [retriever] discoveryengine.search_lite status=ERROR: FailedPrecondition: 400 API key is only supported for public website search. - - [retriever] discoveryengine.answer_query status=ERROR: FailedPrecondition: 400 This feature is only available when Large Language Model add-on is enabled. - - [llm] ChatCompletion model=gpt-5.6-terra tokens=453/31 -``` +Each instrumented call creates one `RETRIEVER` span. Search results and generated answer text are not added as response attributes. -Privacy was checked in every stored column. With the defaults the query was on each Discovery Engine span; with `FI_HIDE_INPUTS=true` it was in no column (`input.value` = `__REDACTED__`, no `gen_ai.retrieval.query`). In both modes the Discovery Engine spans held no result content, and nothing stored contained a credential, an access token, a local path, a user name, a host name or a stack trace. +| Attribute | Value | +|---|---| +| `gen_ai.retrieval.query`, `input.value` | Query text, with recognized credentials removed and truncated to 1 KB; hidden when `FI_HIDE_INPUTS=true`. | +| `discoveryengine.serving_config` | Serving config resource name. | +| `discoveryengine.result_count` | Results on the first page for search; number of answer references for `answer_query`. | +| `discoveryengine.answer.length`, `discoveryengine.answer.state` | Answer length in characters and state, such as `SUCCEEDED` or `FAILED`; not the answer text. | +| `discoveryengine.session` | Discovery Engine session resource name, when supplied or returned. | +| `discoveryengine.error.status`, `discoveryengine.error.code` | Error status and code, such as `PERMISSION_DENIED` and `403`. | -Not verified against Google yet, only against local stand-ins of the documented API: a successful `search_lite` (needs a public website data store and an API key), a successful `answer_query` with its answer state, length and session (needs the LLM add-on), pagination, a regional endpoint, the async REST transport, and removing a hidden query from a server error that quotes it (the real errors seen did not quote the query). +Iterating a search pager past the first page creates no additional spans; the result count covers only the first page. Client exceptions set the span status to `ERROR` and are re-raised unchanged. An answer with state `FAILED` also marks the span as `ERROR`, without raising an exception. -## Troubleshooting +Model names, token usage, and cost are not captured. Conversation/session management, `converse_conversation`, `stream_answer_query`, ingestion, and administration calls are not instrumented. Only the Python v1 clients are covered, not `discoveryengine_v1beta`, `discoveryengine_v1alpha`, Java, or Node clients. -- **No spans appear.** Import the clients from `google.cloud.discoveryengine_v1` (not `google.cloud.discoveryengine`), call `instrument()` before the first call, and check `FI_API_KEY` and `FI_SECRET_KEY`. -- **`instrument()` logs a version error.** The installed `google-cloud-discoveryengine` is outside `>=0.20.5,<1`; upgrade it. -- **Only gRPC transport spans, no query.** Those come from a gRPC instrumentation, not from this package. Install and instrument `traceAI-discoveryengine`; the gRPC spans then nest under its spans. -- **`FailedPrecondition` on `answer_query`: "This feature is only available when Large Language Model add-on is enabled."** The app is Standard edition. Enable the LLM add-on (Enterprise edition) or use `search`. -- **`FailedPrecondition` on `search_lite`: "API key is only supported for public website search."** `search_lite` only serves public website data stores with an API key. Use `search` with OAuth or service account credentials for other data stores. -- **`NotFound`: "We were unable to find an engine to service the request."** The engine ID in the serving config is wrong, or the engine was created moments ago. Check the resource name and the location. -- **`search` returns no results right after you add documents.** Indexing takes a few minutes (about six in our test for three small documents), and `search` returns an empty page until then. -- **`PermissionDenied` or `Forbidden` on every call.** That is the GCP side: check the credentials, the serving config name and the regional endpoint. The span records the status and code, not the credential. -- **Duplicate spans.** Do not combine this package with another instrumentor that wraps the same client methods. +## Privacy limits -## Limitations +- `FI_HIDE_INPUTS=true` replaces `input.value` with `__REDACTED__` and omits `gen_ai.retrieval.query`. With the core version installed above, hiding inputs also hides error details unless you explicitly disable that behavior. `FI_HIDE_ERROR_DETAILS=true` removes error messages and status descriptions while preserving error status and exception types. +- If you explicitly keep error details enabled, query redaction matches the original query and known escaped forms, not reworded or truncated copies. Other fields quoted by the server, such as `filter` or `user_pseudo_id`, can remain in error text. Keep error details hidden when requests contain sensitive data. +- Serving config and session names remain in traces. `FI_PII_REDACTION=true` applies pattern-based redaction to recorded text; it can also redact part of a numeric Google Cloud project ID. It is not a guarantee that all sensitive text is removed. +- The instrumentor removes credentials visible on the client or authentication metadata, plus recognized Google token patterns. Tokens supplied only through a custom gRPC channel may not be visible to it; do not put secrets in query text or custom attributes. The instrumentor does not add stack traces, result documents, or answer content as span fields. -- Python only; the Java and Node clients are not traced. -- Only v1 clients. `stream_answer_query` and `converse_conversation` are not traced. -- Query removal from error text matches the verbatim query and the escaped copies the client libraries write; a reworded or truncated server copy is not matched, and other request fields a server error may quote (a `filter`, `user_pseudo_id`) are not removed. -- The async REST transport is not exercised by the tests. +## Troubleshooting -## Difference from related integrations +- **No spans:** use `google.cloud.discoveryengine_v1`, call `register()` and `instrument()` before requests, and check your Future AGI keys. Flush the provider before a short-lived script exits. +- **Version error during instrumentation:** install `google-cloud-discoveryengine` within `>=0.20.5,<1`. +- **`FailedPrecondition` on `answer_query`:** enable the app's LLM add-on and required Enterprise features, or use `search` instead. +- **`FailedPrecondition` on `search_lite`:** use a public website data store and API key. For private data, use `search` with OAuth or service account credentials. +- **`NotFound` or permission errors:** check the serving config's project, engine ID, location, matching endpoint, and Google credentials. Wait for a newly created engine to become available. +- **Empty results after adding documents:** wait for indexing to finish before searching. +- **Duplicate spans:** do not enable two instrumentors that wrap the same Discovery Engine methods. Separate gRPC transport spans can nest under the retrieval span. -- [Google GenAI](/docs/integrations/traceai/google_genai) and [Vertex AI](/docs/integrations/traceai/vertexai) trace Gemini model calls (`LLM` spans with model and tokens). `traceAI-discoveryengine` traces the search and grounded-answer calls of Vertex AI Search (`RETRIEVER` spans); it does not instrument `google.genai`, and the two can be used together in one RAG trace. -- A gRPC OpenTelemetry instrumentation records the transport call only. This package adds the retrieval span with the query, serving config, result count and error status. +For Gemini model calls rather than search, use the [Google GenAI](/docs/integrations/traceai/google_genai) or [Vertex AI](/docs/integrations/traceai/vertexai) guide. You can use those integrations alongside Discovery Engine tracing in a RAG application.