Skip to content

feat(google-api-core): extend ErrorInfo dict accessors, centralize status code extraction, and update StructuredLogFormatter - #18884

Merged
chalmerlowe merged 1 commit into
mainfrom
feat/otel-logging-m1-foundation
Nov 4, 2026
Merged

chalmerlowe merged 1 commit into
mainfrom
feat/otel-logging-m1-foundation

Conversation

@chalmerlowe

@chalmerlowe chalmerlowe commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Context

Internal issue: b/479827845

To support uniform structured error logging and OpenTelemetry telemetry across both gRPC and REST transports in google-api-core, several foundational utilities need to handle both transport formats consistently:

  1. REST ErrorInfo Accessors: GoogleAPICallError.reason, .domain, and .metadata previously assumed self._error_info was always a protobuf ErrorInfo message (as returned by gRPC google.rpc.status_pb2.Status). For REST JSON error payloads, ErrorInfo entries in error["details"] are parsed as standard Python dict objects, causing attribute access (self._error_info.reason) to fail or return None.
  2. Duplicate Status Code Extraction: Status code normalization across GoogleAPICallError and raw grpc.RpcError instances (_extract_status_code) lived inside google.api_core.gapic_v1.method, preventing reuse by transport-level observability helpers.
  3. Structured Log Field Allowlist: StructuredLogFormatter only recognized legacy camelCase fields and matched fields via exact set membership (in _recognized_logging_fields). It did not unpack record.json_fields or support OpenTelemetry semantic conventions (such as rpc.*, http.*, url.*, server.*, error.type, exception.*, and dynamic gcp.errors.metadata.* keys).

Solution

  • google/api_core/exceptions.py:
    • Updated GoogleAPICallError.reason, .domain, and .metadata to inspect isinstance(self._error_info, dict) first before falling back to protobuf attribute access.
    • Moved _extract_status_code(exc, default=None) to google.api_core.exceptions as the single source of truth so both gapic_v1.method and transport observability helpers share one status code normalization implementation.
  • google/api_core/gapic_v1/method.py:
    • Updated callers to invoke exceptions._extract_status_code directly and removed duplicate status code extraction logic and unit tests from method.py / test_method.py.
  • google/api_core/client_logging.py:
    • Extended _recognized_logging_fields with OpenTelemetry semantic convention attributes, Google Cloud error domain/metadata fields (gcp.errors.domain, gcp.errors.metadata, gcp.errors.metadata.*), and trace correlation keys (logging.googleapis.com/trace, logging.googleapis.com/spanId, logging.googleapis.com/trace_sampled).
    • Pre-compiled _EXACT_FIELDS (frozenset) and _PREFIX_FIELDS (tuple) at module import time and updated StructuredLogFormatter.format to extract exact fields via getattr() (supporting @property and class-level attributes), extract dynamic wildcard prefix fields via vars(record), and unpack record.json_fields last so explicit user-provided structured fields take precedence.
  • Unit Tests:
    • Added parametrized unit tests in tests/unit/test_exceptions.py and tests/unit/test_client_logging.py achieving 100% line and branch coverage on modified modules.
    • See CI results for test verification.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request enhances client logging and exception handling in google-api-core. It expands recognized logging fields, optimizes field extraction using pre-compiled allowlists, and updates StructuredLogFormatter to unpack record.json_fields and extract allowed fields from vars(record). Additionally, it updates exception properties to support dictionary-based error info and centralizes the _extract_status_code utility. The review feedback suggests reversing the processing order in StructuredLogFormatter so that custom fields in json_fields take precedence over automatically extracted attributes from vars(record).

@chalmerlowe
chalmerlowe force-pushed the feat/otel-logging-m1-foundation branch from e4c70cf to f4d9d6a Compare October 28, 2026 21:03
@chalmerlowe
chalmerlowe force-pushed the fix/otel-default-host-endpoint-attributes branch from 7568958 to 781a900 Compare October 28, 2026 21:03
@chalmerlowe
chalmerlowe force-pushed the feat/otel-logging-m1-foundation branch from f4d9d6a to 3240601 Compare October 29, 2026 10:13
@chalmerlowe
chalmerlowe force-pushed the fix/otel-default-host-endpoint-attributes branch from 781a900 to aeacfb7 Compare October 29, 2026 10:13
@chalmerlowe
chalmerlowe force-pushed the feat/otel-logging-m1-foundation branch from 3240601 to 7fad61b Compare October 29, 2026 11:29
@chalmerlowe
chalmerlowe force-pushed the fix/otel-default-host-endpoint-attributes branch from aeacfb7 to 813aa30 Compare October 29, 2026 11:29
@chalmerlowe
chalmerlowe force-pushed the feat/otel-logging-m1-foundation branch from 7fad61b to 858b5dc Compare October 29, 2026 13:37
@chalmerlowe
chalmerlowe force-pushed the fix/otel-default-host-endpoint-attributes branch from 813aa30 to 91cde6b Compare October 29, 2026 13:37
@chalmerlowe
chalmerlowe force-pushed the feat/otel-logging-m1-foundation branch from 858b5dc to f6ab8ea Compare October 29, 2026 14:01
@chalmerlowe
chalmerlowe force-pushed the fix/otel-default-host-endpoint-attributes branch from 91cde6b to d6f0e97 Compare October 29, 2026 14:01
@chalmerlowe
chalmerlowe force-pushed the feat/otel-logging-m1-foundation branch from f6ab8ea to 024c8b8 Compare October 29, 2026 14:19
@chalmerlowe
chalmerlowe force-pushed the fix/otel-default-host-endpoint-attributes branch from d6f0e97 to 9bca1d0 Compare October 29, 2026 14:19
@chalmerlowe
chalmerlowe force-pushed the feat/otel-logging-m1-foundation branch from 024c8b8 to b1c49d0 Compare October 29, 2026 14:59
@chalmerlowe
chalmerlowe force-pushed the fix/otel-default-host-endpoint-attributes branch from 9bca1d0 to 07e6fa4 Compare October 29, 2026 14:59
@chalmerlowe
chalmerlowe force-pushed the feat/otel-logging-m1-foundation branch from b1c49d0 to 95e34e8 Compare October 30, 2026 14:38
@chalmerlowe
chalmerlowe force-pushed the fix/otel-default-host-endpoint-attributes branch from 07e6fa4 to 81e79e2 Compare October 30, 2026 14:38
@chalmerlowe
chalmerlowe force-pushed the feat/otel-logging-m1-foundation branch 3 times, most recently from 0709089 to c64a257 Compare November 2, 2026 15:16
return method_str, service, method


def _extract_status_code(exc: Optional[BaseException]) -> str:

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NOTE: To reviewer: we are moving this function out of method.py and into exceptions.py because it is primarily focused on error code extraction.

@chalmerlowe
chalmerlowe force-pushed the feat/otel-logging-m1-foundation branch 2 times, most recently from 5a89260 to 0fdb7d0 Compare November 3, 2026 09:31
@chalmerlowe
chalmerlowe force-pushed the fix/otel-default-host-endpoint-attributes branch from a44b814 to ee5b3cd Compare November 3, 2026 11:24
@chalmerlowe
chalmerlowe force-pushed the feat/otel-logging-m1-foundation branch from 0fdb7d0 to 6062724 Compare November 3, 2026 11:49
@chalmerlowe
chalmerlowe force-pushed the fix/otel-default-host-endpoint-attributes branch from ee5b3cd to dd50ca8 Compare November 3, 2026 11:50
@chalmerlowe
chalmerlowe marked this pull request as ready for review November 3, 2026 11:58
@chalmerlowe
chalmerlowe requested a review from a team as a code owner November 3, 2026 11:58


def test_extract_status_code_variations():
"""Proves that _extract_status_code handles grpc status, callable/non-callable codes, ints, and exceptions."""

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test stack has been improved and moved to test_exceptions.py to align with the movement of the _extract_status_code() method to exceptions.py.

Base automatically changed from fix/otel-default-host-endpoint-attributes to main November 3, 2026 17:50
…de extraction, and update StructuredLogFormatter
@chalmerlowe
chalmerlowe force-pushed the feat/otel-logging-m1-foundation branch from 6062724 to d573011 Compare November 3, 2026 18:07
@chalmerlowe chalmerlowe closed this Nov 4, 2026
@chalmerlowe chalmerlowe reopened this Nov 4, 2026
@chalmerlowe chalmerlowe closed this Nov 4, 2026
@chalmerlowe chalmerlowe reopened this Nov 4, 2026
@chalmerlowe chalmerlowe closed this Nov 4, 2026
@chalmerlowe chalmerlowe reopened this Nov 4, 2026
@chalmerlowe
chalmerlowe merged commit 0c3fa2b into main Nov 4, 2026
342 of 610 checks passed
@chalmerlowe
chalmerlowe deleted the feat/otel-logging-m1-foundation branch November 4, 2026 18:50
@release-please release-please Bot mentioned this pull request Nov 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants