Problem
For a view, GlueMetadataClient.table_metadata (pyathena/glue.py) no longer returns the parameters that Athena reports, so the Glue metadata path is not equivalent to the Athena path for views.
Since about 2026-09-29 00:00 UTC in us-west-2, Athena GetTableMetadata and ListTableMetadata return a view's parameters without the null-valued storage keys:
Athena: {'comment': 'Presto View', 'location': '', 'presto_view': 'true'}
Glue: {'presto_view': 'true', 'comment': 'Presto View', 'location': '', 'inputformat': None, 'outputformat': None, 'serde.serialization.lib': None}
table_metadata always sets inputformat and outputformat from the storage descriptor, and sets serde.serialization.lib whenever the descriptor has SerdeInfo. A view's descriptor has SerdeInfo: {} and no formats. When the Glue path was designed (#803), Athena reported these keys with None values, and the two paths matched for every measured table kind, including views.
The following tests now fail against AWS:
tests/pyathena/test_glue.py::TestGlueMetadataClient::test_reads_what_athena_reportstests/pyathena/test_cursor.py::TestCursor::test_throttled_metadata_reads_gluetests/pyathena/aio/test_cursor.py::TestAioCursor::test_throttled_metadata_reads_glue
The Athena behavior looked like a gradual rollout. The same test at the same commit passed and then failed within about 30 minutes, locally and in CI (run 36499867658, both attempts). It then failed consistently on unrelated branches. It affects every branch that runs the AWS suite.
User impact: after a throttled Athena metadata read, Cursor.get_table_metadata and list_table_metadata fall back to Glue. For views, they then return three extra None-valued parameters that the Athena path does not return. Tables are unaffected in the failing runs. The Glue path is unreleased (merged in #803 after v3.36.0).
Reproduction
uv run --env-file .env pytest -n 1 tests/pyathena/test_glue.py::TestGlueMetadataClient::test_reads_what_athena_reports
At index 5 diff: {'presto_view': 'true', 'comment': 'Presto View', 'location': '', 'inputformat': None, 'outputformat': None, 'serde.serialization.lib': None} != {'comment': 'Presto View', 'location': '', 'presto_view': 'true'}
Environment
- master (
a86a180) and branches based on it; Python 3.13 locally and 3.14 in CI; boto3/botocore from uv.lock.
- Athena and Glue in us-west-2,
AwsDataCatalog.
Proposed fix (optional)
First, measure the current Athena shapes again for each table kind covered in #803: Hive tables, Iceberg, views, and S3 Tables. If Athena now omits every null-valued storage key, make table_metadata do the same: add location, inputformat, outputformat and serde.serialization.lib only when the Glue value is not None. Empty strings, such as a view's location: '', are kept.
If the rollout leaves both shapes in place, the tests need to compare in a way that accepts either shape without hiding other differences. Validate with the three tests above and the SQLAlchemy reflection tests that read table options.
Problem
For a view,
GlueMetadataClient.table_metadata(pyathena/glue.py) no longer returns the parameters that Athena reports, so the Glue metadata path is not equivalent to the Athena path for views.Since about 2026-09-29 00:00 UTC in us-west-2, Athena
GetTableMetadataandListTableMetadatareturn a view's parameters without the null-valued storage keys:table_metadataalways setsinputformatandoutputformatfrom the storage descriptor, and setsserde.serialization.libwhenever the descriptor hasSerdeInfo. A view's descriptor hasSerdeInfo: {}and no formats. When the Glue path was designed (#803), Athena reported these keys withNonevalues, and the two paths matched for every measured table kind, including views.The following tests now fail against AWS:
tests/pyathena/test_glue.py::TestGlueMetadataClient::test_reads_what_athena_reportstests/pyathena/test_cursor.py::TestCursor::test_throttled_metadata_reads_gluetests/pyathena/aio/test_cursor.py::TestAioCursor::test_throttled_metadata_reads_glueThe Athena behavior looked like a gradual rollout. The same test at the same commit passed and then failed within about 30 minutes, locally and in CI (run 36499867658, both attempts). It then failed consistently on unrelated branches. It affects every branch that runs the AWS suite.
User impact: after a throttled Athena metadata read,
Cursor.get_table_metadataandlist_table_metadatafall back to Glue. For views, they then return three extraNone-valued parameters that the Athena path does not return. Tables are unaffected in the failing runs. The Glue path is unreleased (merged in #803 after v3.36.0).Reproduction
Environment
a86a180) and branches based on it; Python 3.13 locally and 3.14 in CI; boto3/botocore fromuv.lock.AwsDataCatalog.Proposed fix (optional)
First, measure the current Athena shapes again for each table kind covered in #803: Hive tables, Iceberg, views, and S3 Tables. If Athena now omits every null-valued storage key, make
table_metadatado the same: addlocation,inputformat,outputformatandserde.serialization.libonly when the Glue value is notNone. Empty strings, such as a view'slocation: '', are kept.If the rollout leaves both shapes in place, the tests need to compare in a way that accepts either shape without hiding other differences. Validate with the three tests above and the SQLAlchemy reflection tests that read table options.