Skip to content

fix: include statement IDs in Thrift failure telemetry - #987

Open
AliAbouelazm wants to merge 1 commit into
databricks:mainfrom
AliAbouelazm:fix/thrift-error-statement-id
Open

AliAbouelazm wants to merge 1 commit into
databricks:mainfrom
AliAbouelazm:fix/thrift-error-statement-id

Conversation

@AliAbouelazm

Copy link
Copy Markdown

What type of PR is this?

  • Bug Fix

Description

Thrift failure events could not be correlated with their statements because Error and export_failure_log did not accept a SQL statement ID. The operation handle was already available when most of these errors were raised.

This change adds an optional statement ID to the failure-telemetry interface and forwards the current response or request handle through statement, polling, direct-result, and request errors. Result-format and fetch-offset validation errors also carry the available ID. The current session ID is supplied explicitly so a shared telemetry client does not attribute the error to another connection.

The error-message payload is unchanged. No SQL text, diagnostic context, or operation secret is added to the event. Failures before a statement handle is available continue to omit the statement ID, and disabled telemetry accepts the optional argument.

How is this tested?

  • Unit tests

The 22 new regression cases exercise real Thrift error paths with mocked RPCs and inspect serialized telemetry events. They cover direct and polled failures, both server-message forms, every direct-result error field, synchronous and asynchronous submission, request failures, unavailable handles, connection failures, closed statements, result validation, and disabled telemetry. They also verify that SQL and operation secrets are absent from exported events.

On unchanged main, these tests produce 20 failures and 2 passes. With the fix, all 22 pass with network sockets disabled.

Full unit suite on Python 3.12 with PyArrow: 1,268 passed, 5 skipped, 1 realkernel test deselected, and 351 subtests passed. The run used --disable-socket with proxy variables removed from the test process so the existing HTTP fixtures select their mocked pool.

black --check src, Black on the new test file, mypy src, and git diff --check pass. Live Databricks E2E tests and the real-kernel tier were not run.

Related Tickets & Documents

Addresses the Thrift TELEMETRY-002 finding in #927. The other findings remain outside this change.

Signed-off-by: Ali Abouelazm <aliazm419@gmail.com>
@AliAbouelazm
AliAbouelazm marked this pull request as ready for review October 16, 2026 07:51

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant