Skip to content

[Bug] gemini-enterprise: usage-audit logging half-applies and nothing notices — the engine is left observabilityEnabled without sensitiveLoggingEnabled, so every prompt and response logs as <elided> #182

Description

@JohnHales

Bug Description

deploy.sh offers to "enable Gemini Enterprise Usage Audit logs (conversation logging)" and warns that doing so "will write user queries, model thinking, and model responses to Cloud Logging". When the operator accepts, the post-apply step PATCHes the engine with {"observabilityConfig": {"observabilityEnabled": true, "sensitiveLoggingEnabled": true}}.

On a completed FedRAMP-High deployment the engine is instead left in a state the script never sends: observabilityEnabled: true with sensitiveLoggingEnabled absent (i.e. false). Per the Discovery Engine schema, sensitiveLoggingEnabled "includes customer core content (e.g. prompts, responses). If false, will sanitize all sensitive fields." The result is audit logging that looks enabled and captures none of the content it was enabled for.

Because the PATCH is issued as curl -s -o /dev/null with no status or body check (deploy.sh :2524-2540 — the same pattern is used for the assistant-compliance PATCH immediately after), a rejected, ignored or partially-applied field produces no output, no warning and no non-zero exit. Nothing later in the blueprint or the tooling reads observabilityConfig back.

Environment and Deployment Context

  • Stellar Engine Version/Commit: main at commit f64ce6cd (re-verified 2026-08-10)
  • Deployment Type:
    • US Region Restricted (e.g., Access Policy constraint)
    • FedRAMP Medium
    • FedRAMP High
    • FedRAMP Moderate
    • DoD IL4
    • DoD IL5
    • Stand-alone / Custom
  • FAST Stage (if applicable): N/A — this is a blueprint, not a FAST stage
    • Stage 0 (Bootstrap)
    • Stage 1 (Resource Management)
    • Stage 2 (Network Creation)
    • Stage 3 (Security and Audit)
  • Affected Component: blueprints/fedramp-high/gemini-enterprise/deploy.sh — post-apply observability PATCH
  • Terraform Version: 1.12.2 (pinned by deploy.sh via tfenv; the stage declares required_version >= 1.7.4)
  • GCP Provider Version: hashicorp/google >= 6.21.0 (stage-0 declared constraint)

Steps to Reproduce

  1. Run deploy.sh, answer y to the Usage Audit logs prompt, and complete the apply.
  2. GET .../engines/{ENGINE_ID} and read observabilityConfig.
  3. Run assistant and search queries against the app, then read the discoveryengine.googleapis.com/gemini_enterprise_user_activity, gen_ai.user.message and gen_ai.choice logs.

Expected Behavior

Either the engine reads {observabilityEnabled: true, sensitiveLoggingEnabled: true} and prompts and responses appear in the logs, or the deployment reports that the setting could not be applied.

Actual Behavior

observabilityConfig reads {"observabilityEnabled": true} only. In the logs (44 user-activity entries, 164 gen_ai.user.message, 200+ gen_ai.choice sampled):

  • sanitized to <elided>gen_ai.user.message.content (assistant prompt, 164/164), gen_ai.choice.content (model answer, 200/200), serviceTextReply, grounding-reference content / title / mimeType / domain, and userIamPrincipal (44/44);
  • retained in the clearrequest.query on SearchService.Search (search-box text), request.userEvent.searchInfo.searchQuery, grounding-reference uri, method names, timestamps and resource names;
  • StreamAssist entries carry request.query: {} — the assistant prompt is not present at all.

Re-issuing exactly the same PATCH by hand succeeds (HTTP 200, both flags echoed, and a follow-up GET confirms sensitiveLoggingEnabled: true persisted). So the API accepts the field; the deployment nevertheless ended up without it, and no code path in the blueprint produces the mixed state that was observed (deploy.sh sends both flags true or both false; gem4gov update-compliance patches the engine with updateMask=features only; gemini-stage-0 Terraform sets no observability config). Root cause is therefore undetermined from source alone — which is itself the point of the report: the silent write makes it undiagnosable after the fact.

Relevant Logs and Errors

Observed live 2026-08-06. The engine read back after a completed deployment:

{ "observabilityConfig": { "observabilityEnabled": true } }

sensitiveLoggingEnabled is absent. The PATCH that should have set it is issued as curl -s -o /dev/null with no status or body check, so nothing reported a failure:

# deploy.sh:2526-2531
curl -s -o /dev/null -X PATCH \
    -H "Authorization: Bearer ${ACCESS_TOKEN}" \
    ... \
    "https://us-discoveryengine.googleapis.com/v1alpha/projects/${PROJECT_ID}/locations/us/collections/default_collection/engines/${ENG_ID}?updateMask=observabilityConfig" \
    -d '{"observabilityConfig": {"observabilityEnabled": true, "sensitiveLoggingEnabled": true}}'

Additional Context

Impact is compliance-facing, not cosmetic: an operator who accepted the prompt will tell an auditor that conversation logging is on. What is actually retained is search-box query text with the acting principal removed — while the assistant conversation, every response, and the identity of the user are all withheld.

Minimal fix: capture the PATCH status/body (drop -o /dev/null, check the HTTP code and the echoed observabilityConfig), fail or warn loudly on a mismatch, and read the config back once after the apply. The same treatment is warranted for the assistant-compliance PATCH that follows it.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions