Backport bug fixes to 3.x for v3.38.0

[laughingman7743]

WHAT

This PR backports bug fixes from master to the 3.x maintenance branch for v3.38.0. It includes the fixes for the externally reported issues #854 and #857.

Each master PR is cherry-picked with -x in master merge order, one commit per PR. When a conflict came from a master PR that is not backported, only the backported PR's own changes are applied. Each commit message records how its conflict was resolved.

PR	Fix	3.x handling
#800	to_sql finds an existing table whatever the case of its name (#799)	clean
#817	Cache-hit tests use ENV.work_group instead of primary (#816)	clean
#831	Spark session readiness polling stops on TERMINATED, DEGRADED, or FAILED (#792)	docstring context from #803
#830	AsyncSparkCursor.close() shuts down its executor when session termination fails (#795)	clean
#832	A Spark session that a cursor starts is terminated when cursor setup fails (#794)	docstring and annotation context from #803
#899	Synchronous iteration of the asyncio cursors and result sets raises TypeError instead of looping forever (#898)	applied to 3.x's pre-#883 WithAsyncFetch
#900	Arrow decimal256 maps to Athena decimal instead of string (#888)	decimal256 only (see below)
#928	Stale S3FileSystem listing and object caches (#922)	clean
#929	Appending to an object keeps its data; this fixes data loss, truncation by an empty append, and duplicated bytes (#921)	docstring context from #919
#947	A single large write() no longer leaves a sub-5 MiB non-last part, which made the upload fail with EntityTooSmall (#942)	clean
#955	Multipart copy parts stay within the S3 part size limits (#951)	test import from #948
#959	The client-side result cache is skipped for qmark queries with parameters (#941)	applied to 3.x's _execute()
#950	as_pandas() returns the whole result when auto_optimize_chunksize chose a chunk size (#924)	docstring and docs context from #919, #940
#995	Top-level MAP and STRUCT/ROW columns reflect as AthenaMap / AthenaStruct (#857)	visit_null kept (see below)
#1023	ArrowCursor fallback values are not converted twice (#1021)	clean
#1029	A closed PolarsDataFrameIterator stops for every reader kind (#1019)	test file created on 3.x
#1031	ArrowCursor keeps the NULL rows of single-column CSV results (#1026)	clean
#1033	A numeric UTC offset in TIMESTAMP WITH TIME ZONE gives an aware datetime (#854)	#854 part only (see below)
#1045	ArrowCursor reads multi-line CSV values that cross a read block (#1040)	clean
#1075	PandasCursor uses the C engine for DDL .txt results with engine="pyarrow" (#1073)	docs context from #902 / #1057, test context from #1001

Ported in part, by maintainer decision:

• Map Arrow decimal32/64/256 to Athena decimal #900: 3.x allows pyarrow>=10.0.0, but Type_DECIMAL32 / Type_DECIMAL64 first appear in pyarrow 19.0.0.
  • Referencing them would raise AttributeError in get_athena_type() on older pyarrow.
  • Only the decimal256 fix is ported: Type_DECIMAL256 replaces the Decimal256Type class, which never matched a type id.
• Reflect the field and value types of top-level STRUCT and MAP columns #995: the reflection fix is ported unchanged. The visit_null removal, which makes NullType raise CompileError in DDL, is left out.
  • That removal relies on the DDL/DML type compiler split (Split SQLAlchemy type rendering into Hive DDL and Trino DML compilers #961, master only).
  • 3.x still renders CAST types through AthenaTypeCompiler, so removing visit_null would make CAST(x AS NULL) raise.
  • test_null_type_cast_is_unchanged is ported and guards this.
• Convert time zone values to aware times and empty TIME/JSON text to NULL #1033: only _parse_utc_offset() and its use in _to_datetime_with_tz() are ported, together with the converter test.
  • The rest of Convert time zone values to aware times and empty TIME/JSON text to NULL #1033 consists of 4.0.0 behavior changes that build on Keep the whole pandas and Polars results reusable #1005 / Return Athena JSON values from SQLAlchemy JSON columns and the pandas, Arrow, and S3FS cursors #1010 (not backported): TIME WITH TIME ZONE conversion, empty TIME/JSON text as NULL, and time zone converters for the pandas, Arrow, and Polars cursors.
  • 3.x keeps its strptime() parsing (Support SQLAlchemy datetime literals and microsecond timestamps #818 is not backported).

Not backported:

• Filesystem fixes other than Fix stale S3FileSystem listing and object caches #928, Keep the existing object when appending within a larger block size #929, Merge a short trailing block into the previous multipart part #947 and Keep multipart copy parts within the S3 part size limits #955. The rest form one long dependency chain (Accept version_id in S3FileSystem.open() and cat_file() #958, Accept max_workers in S3FileSystem.open() and pipe_file() #948, Wait for running parts before aborting multipart uploads, and validate S3File before init #992, Answer info() and exists() from cached parent listings and fix bucket lookups #1006, Delete only the copied source objects in mv() #1013, …) that includes 4.0.0 breaking changes and S3Core.
• 4.0.0 behavior or breaking changes, for example Re-raise and stop interrupted queries in the SQL cursors, sharing the Spark handling #853, Define best-effort Spark calculation cancellation #833, Terminate only the Spark sessions a cursor started on close #844, Cancel a Spark calculation interrupted while it is being started #861, Apply dict_type only to the dict cursor it was given to #930, Split SQLAlchemy type rendering into Hive DDL and Trino DML compilers #961, Keep result-set filesystems out of the fsspec instance cache and read pandas results through one filesystem #1001, Keep the whole pandas and Polars results reusable #1005, Return Athena JSON values from SQLAlchemy JSON columns and the pandas, Arrow, and S3FS cursors #1010, Keep integer and JSON values exact in PandasCursor results with NULL #1044, Share one S3 client per connection across result sets and filesystems #1058, and the S3Path/S3Core work.
• Metadata work (Restore SQLAlchemy reflection and identifier coverage #771 Fix metadata reflection throttling and reuse listed metadata #777 Reflect an empty information_schema comment as no comment #782 Preserve Hive string types in metadata fallback reflection #784 Read the dialect's own queries through an API cursor and resolve federated absence from information_schema #798 Report a missing federated table from table comment and option reflection #801 Answer throttled metadata requests from Glue in the cursor #803), as decided for v3.37.0.
• Dependency floors, refactors, license work, CI, benchmarks, and docs-only PRs.

WHY

master is 4.0.0 development. Many bugs fixed there since v3.36.0 also exist on 3.x.

v3.38.0 ships the fixes that are safe for the 3.x line, without its breaking changes. It includes the fixes for the issues @aminghadersohi reported: #854 (TIMESTAMP WITH TIME ZONE with a numeric offset came back naive) and #857 (top-level MAP columns reflected as String).

Release notes for v3.38.0 (in addition to the unreleased #839, #905, #906, #913 already on 3.x):

• SQLAlchemy reflection: reflected top-level map<...> columns are AthenaMap(key_type, value_type) instead of String. Top-level struct<...> / row(...) columns are AthenaStruct with their field types (Top-level map columns reflect as String #857, Reflect the field and value types of top-level STRUCT and MAP columns #995).
• TIMESTAMP WITH TIME ZONE: values with a numeric UTC offset such as +05:30 convert to aware datetimes with a fixed offset (TIMESTAMP WITH TIME ZONE values with a numeric UTC offset come back naive #854, Convert time zone values to aware times and empty TIME/JSON text to NULL #1033).
• Asyncio cursors: a plain for loop over an asyncio cursor or result set raises TypeError instead of never ending (Reject synchronous iteration of the asyncio cursors and result sets #899).
• Result cache: cache_size / cache_expiration_time have no effect on qmark queries with parameters, because Athena does not return the parameters of earlier executions (Skip the result cache for qmark queries with parameters #959). A repeated qmark query with the same parameters is therefore also executed again.
• PandasCursor:
  • as_pandas() returns the whole result when auto_optimize_chunksize chose a chunk size; it used to return only the first chunk (Return the whole result from as_pandas() when auto chunking applies #950).
  • Without chunksize, a second as_pandas() call, or one after the rows were fetched, returns an empty DataFrame instead of raising StopIteration (Return the whole result from as_pandas() when auto chunking applies #950).
  • PandasDataFrameIterator.as_pandas(), as returned by iter_chunks() or by as_pandas() with an explicit chunksize, keeps the chunks' index and categorical dtypes. With index_col, the joined DataFrame is indexed by that column instead of a new RangeIndex (Return the whole result from as_pandas() when auto chunking applies #950).
• S3 filesystem:
  • appends no longer lose or duplicate data (Keep the existing object when appending within a larger block size #929);
  • large single writes no longer fail with EntityTooSmall (Merge a short trailing block into the previous multipart part #947);
  • multipart copies of objects over 5 GiB stay within the part size limits (Keep multipart copy parts within the S3 part size limits #955);
  • multipart part boundaries change when the last part is short (Merge a short trailing block into the previous multipart part #947, Keep multipart copy parts within the S3 part size limits #955), so the multipart ETag of identical content can differ from before;
  • listing caches are invalidated correctly (Fix stale S3FileSystem listing and object caches #928).
• Spark: session readiness polling no longer runs forever on a failed session (Stop Spark session readiness polling on failure states #831). Failed cursor setup terminates the new session (Terminate a newly started Spark session when cursor setup fails #832).
• ArrowCursor with managed result storage (no output location): the fetch methods return the values that as_arrow() holds and no longer convert them a second time. TIME, VARBINARY and JSON values used to raise TypeError (Do not convert ArrowCursor fallback values twice #1023). A custom Arrow converter mapping is therefore not applied on this path, the same as for as_arrow().
• Smaller fixes: Arrow decimal256 (Map Arrow decimal32/64/256 to Athena decimal #900), NULL rows (Keep NULL rows of single-column CSV results in ArrowCursor #1031) and multi-line CSV values (Read multi-line CSV values that cross a block in ArrowCursor #1045; newlines_in_values=True cost about 0.03–0.04 s per 250 MiB when measured on master), the Polars iterator close (Stop a closed PolarsDataFrameIterator for every reader kind #1029), and PandasCursor DDL results with engine="pyarrow" (Use the C engine for Pandas DDL text results #1075).

TEST

All runs were at 5bf0490. The branch head 683658f has the same tree; only three commit messages changed afterwards.

• just lint (ruff check, ruff format --check, mypy): passed.
• markdownlint-cli2 on the changed docs: 0 errors.
• Offline, on Python 3.10.16 with the 3.x lock (pandas 2.2.3), using --noconftest: 342 passed. The run covered:
  • tests/pyathena/test_converter.py
  • aio/test_common.py and aio/test_result_set.py
  • spark/test_common.py
  • arrow/test_util.py
  • pandas/test_result_set.py and polars/test_result_set.py
  • sqlalchemy/test_compiler.py and sqlalchemy/test_base.py::TestAthenaDialect
• Real AWS, in the maintainer's test account, Python 3.10.16, -n 8: 1178 passed.
  • The run covered tests/pyathena/test_cursor.py and aio/test_cursor.py, tests/pyathena/pandas/, arrow/, polars/, filesystem/, sqlalchemy/test_base.py, test_converter.py, the aio result-set and common tests, and spark/test_common.py.
  • On pandas 2.2.3, PandasDataFrameIterator.as_pandas() (Return the whole result from as_pandas() when auto chunking applies #950) emits pandas' FutureWarning about concatenating all-NA chunks in test_as_pandas_matches_whole_read[category_null_chunk]. Master does not show it because it requires pandas 3; the test passes.
• Not run locally: the Spark integration suite, the full tests/pyathena/ suite, and just test sqla / sqla-async. The PR's AWS jobs run them on the newest Python once the PR is Ready, and the release gate runs every Python version for the tag.

🤖 Generated with Claude Code

[laughingman7743]

Self-review round 1: implementation behavior: CLEAN (no actionable findings)

Scope: git diff f09cf4269052b7dfa4157f10a5f37a5cbfa6148a..5bf0490e450e56ca9a1be3840e94d86234beae4f, the 20 commits, 36 files.

Covered:

• Faithfulness. Each commit's added and removed lines were compared with its master PR's first-parent diff, using a line-multiset diff.
  • Clean picks match exactly: Find an existing table in to_sql whatever the name's case #800, Assert cache hits in the test work group instead of primary #817, Shut down the AsyncSparkCursor executor when session termination fails #830, Fix stale S3FileSystem listing and object caches #928, Merge a short trailing block into the previous multipart part #947, Do not convert ArrowCursor fallback values twice #1023, Keep NULL rows of single-column CSV results in ArrowCursor #1031, Read multi-line CSV values that cross a block in ArrowCursor #1045.
  • Every other difference is one of the resolutions recorded in the commit messages:
    • docstrings from Answer throttled metadata requests from Glue in the cursor #803/Document every public API in pyathena/ and check docstrings with ruff #919 left out (Stop Spark session readiness polling on failure states #831, Keep the existing object when appending within a larger block size #929, Return the whole result from as_pandas() when auto chunking applies #950, Skip the result cache for qmark queries with parameters #959);
    • 3.x's unannotated request (Terminate a newly started Spark session when cursor setup fails #832);
    • 3.x's typing import (Reject synchronous iteration of the asyncio cursors and result sets #899);
    • the mock import for Keep multipart copy parts within the S3 part size limits #955's test;
    • timezone.utc instead of datetime.UTC (Skip the result cache for qmark queries with parameters #959);
    • the new 3.x test file (Stop a closed PolarsDataFrameIterator for every reader kind #1029);
    • the partial ports Map Arrow decimal32/64/256 to Athena decimal #900, Reflect the field and value types of top-level STRUCT and MAP columns #995, Convert time zone values to aware times and empty TIME/JSON text to NULL #1033 and Use the C engine for Pandas DDL text results #1075.
• Behavior and failure paths. Each item below was traced into its 3.x callers.
  • Reject synchronous iteration of the asyncio cursors and result sets #899: every 3.x asyncio SQL cursor (AioCursor, AioDictCursor, AioArrowCursor, AioPandasCursor, AioPolarsCursor, AioS3FSCursor) overrides fetchone() with a coroutine. Before this change, CursorIterator.__next__ got a coroutine that is never None, so the loop never ended.
  • Skip the result cache for qmark queries with parameters #959: 3.x's _build_start_query_execution_request sets ExecutionParameters only for qmark parameters (common.py:263), so request.get("ExecutionParameters") guards the same case as master.
  • Terminate a newly started Spark session when cursor setup fails #832: _start_session() calls the name-mangled __terminate_session() synchronously, so AioSparkCursor and AsyncSparkCursor, which override _terminate_session() as a coroutine or future, still release the session.
• Data and resource boundaries.
  • Convert time zone values to aware times and empty TIME/JSON text to NULL #1033: _parse_utc_offset(tz) or gettz(tz) keeps zone names.
  • Map Arrow decimal32/64/256 to Athena decimal #900: Type_DECIMAL128 and Type_DECIMAL256 exist on pyarrow>=10.
  • Return the whole result from as_pandas() when auto chunking applies #950: the CategoricalDtype handling also works on pandas 2.2.3 (3.x lock).
• Tests. The live and offline runs listed in the PR body pass on Python 3.10.16. tests/pyathena/spark/test_async_cursor.py's new offline tests also pass (4).

Recorded limitations (3.x consequences of the agreed scope, not defects of this PR) are in the threads below.

[laughingman7743]

Round 1 note (#899):

• 3.x's WithAsyncFetch still carries the default synchronous fetch methods, which master removed in Share the pure cursor-base logic between the sync and asyncio cursors #883.
• A user subclass that keeps them and relied on synchronous iteration now gets this TypeError.
• No shipped 3.x cursor does that; all of them override the fetch methods with coroutines.
• Recorded rather than changed: Reject synchronous iteration of the asyncio cursors and result sets #899's contract is that asyncio cursors are not synchronously iterable.

[laughingman7743]

Round 1 note (#995):

• visit_null is kept, so NullType is not rejected in DDL on 3.x.
• A reflected MAP or STRUCT with an unrecognized nested type renders NULL in CREATE TABLE, and Athena rejects the statement. Master raises CompileError at compile time instead.
• Before this change, the whole top-level column reflected as String.
• 3.x's AthenaArray(NullType) already behaved this way.
• This is the agreed partial port: removing visit_null without Split SQLAlchemy type rendering into Hive DDL and Trino DML compilers #961 would break CAST(x AS NULL).

[laughingman7743]

Self-review round 2: claims, callers and operations: FINDINGS, repaired (wording in commit messages and the PR body only; no code change)

Scope: git diff f09cf4269052b7dfa4157f10a5f37a5cbfa6148a..683658f24dff765e0cc6df1bffcfebdd9f7797a8, the PR body, and the 20 commit messages.

Claims checked against evidence:

• pyarrow (Map Arrow decimal32/64/256 to Athena decimal #900): measured with uv run --with pyarrow==X.
  • pyarrow.lib.Type_DECIMAL32 is absent on 10.0.1 and 18.1.0 and present on 19.0.0.
  • Type_DECIMAL128 = 23 and Type_DECIMAL256 = 24 exist on all three.
  • On pyarrow 10.0.1 with this branch, get_athena_type() returns ('decimal', 38, 4), ('decimal', 40, 5), ('array', 0, 0) and ('varchar', …) for decimal128, decimal256, list and string.
• Provenance in the resolution notes: checked with git log --first-parent -S on master.
  • Confirmed:
    • Answer throttled metadata requests from Glue in the cursor #803: _exists_session docstring, _terminate_session annotation
    • Document every public API in pyathena/ and check docstrings with ruff #919: S3File.__init__ and as_pandas docstrings
    • Accept max_workers in S3FileSystem.open() and pipe_file() #948: unittest.mock import
    • Require polars>=1.39.0 for chunked PolarsCursor reads #828: polars test_result_set.py
    • Correct user guide claims that contradict the implementation #940: aio paragraph
    • Keep result-set filesystems out of the fsspec instance cache and read pandas results through one filesystem #1001: _fs.open in _read_csv
    • Support SQLAlchemy datetime literals and microsecond timestamps #818: _parse_datetime
  • Corrected (three commit messages, rewritten with --force-with-lease; tree unchanged against 5bf0490):
    • Skip the result cache for qmark queries with parameters #959: _execute()'s docstring and _build_execute_request() come from Share the pure cursor-base logic between the sync and asyncio cursors #883 and _start_execution() from Re-raise and stop interrupted queries in the SQL cursors, sharing the Spark handling #853. The message wrongly also named Replace name-mangled helpers with single-underscore methods #881/Document every public API in pyathena/ and check docstrings with ruff #919.
    • Reflect the field and value types of top-level STRUCT and MAP columns #995: test_columns_from_information_schema comes from Fix metadata reflection throttling and reuse listed metadata #777, not Read the dialect's own queries through an API cursor and resolve federated absence from information_schema #798.
    • Use the C engine for Pandas DDL text results #1075: master's PyArrow-engine paragraph was added by Fix PandasCursor performance tuning examples #902 (extended by Convert time zone values to aware times and empty TIME/JSON text to NULL #1033/Read CSV results for the pandas PyArrow engine with pyarrow.csv #1057), not Read CSV results for the pandas PyArrow engine with pyarrow.csv #1057.
  • The PR table row for Use the C engine for Pandas DDL text results #1075 was corrected the same way.
• Callers:
  • Versions: no picked diff uses an API newer than 3.x's floors. Checked: Python 3.10 (no datetime.UTC, Self, TaskGroup, Task.cancelling, @override), pandas CategoricalDtype(ordered=), polars, and the pyarrow.csv.ParseOptions(newlines_in_values=) / ignore_empty_lines options.
  • New imports: no new fsspec imports.
• CI claim: 3.x test.yaml path filters select the pyathena, sqla and spark suites for this diff, since it touches pyathena/*.py, pyathena/sqlalchemy/** and pyathena/spark/**. They run once the PR is Ready.
• Evidence:
  • The local AWS and offline results in the PR body belong to tree 5bf0490, which equals the head.
  • The Spark live suite and just test sqla / sqla-async were not run locally; that is stated in the PR body.

[laughingman7743]

Round 2 deferral (docs reader):

• 3.x's sentence above, that the convenience methods "operate on already-loaded data", was already inaccurate for chunked results before this PR. Return the whole result from as_pandas() when auto chunking applies #950's added sentence makes that visible.
• Master corrected the paragraph in Correct user guide claims that contradict the implementation #940, a docs-only PR that is not in this backport's scope.
• Left as is. The added sentence is accurate.

[laughingman7743]

Independent review (relayed): Codex CLI 0.160.0, model gpt-6.1-sol, codex exec -s read-only, effort high. Session 01a11134-c83c-7aa2-96a0-26d06ed51cb0. Static review only: no builds, tests, edits or network.

• Snapshot: detached worktree at head 683658f24dff765e0cc6df1bffcfebdd9f7797a8, base f09cf4269052b7dfa4157f10a5f37a5cbfa6148a. The prompt contained the diff range, the mapping from commit to master merge commit, the 3.x dependency floors and the required scope. It contained no PR text, commit messages or prior findings.
• The snapshot and the PR worktree were unchanged afterwards.
• Coverage reported:
  • all 20 commits against their master patches, including the three partial ports;
  • sync, threaded and asyncio cursors; pandas, Arrow, Polars and S3FS result sets;
  • Spark lifecycle;
  • S3 caches, append, multipart upload/copy and failure paths;
  • SQLAlchemy reflection and compiler, dependency bounds, tests and docs.
• Verdict: FINDINGS, with two P2 findings: one introduced (this thread) and one pre-existing (next thread).

Finding (introduced, P2):

• With managed result storage (no output location), the GetQueryResults fallback no longer applies the result set's converter mapping in the fetch methods.
• So a custom mapping such as DefaultArrowTypeConverter().set("varchar", str.upper) stops applying: fetchall() returns ("hello",) instead of ("HELLO",). The same holds on master since Do not convert ArrowCursor fallback values twice #1023.

Author verification: confirmed, deferred.

• Before Do not convert ArrowCursor fallback values twice #1023, _fetch() applied self.converters to values that _fetch_all_rows() had already converted with DefaultTypeConverter.
• With the default Arrow converter, that raised TypeError for TIME, VARBINARY and JSON objects and decoded JSON twice (ArrowCursor converts GetQueryResults fallback values twice in the fetch methods #1021, measured since v3.28.0). A custom mapping received already-converted values, not the strings the Converter contract specifies.
• as_arrow() never applied custom mappings on this path, before or after.
• Master removed the double path in Read managed query results as a CSV result file in the pandas, Arrow, and Polars cursors #1061, which reads managed results as a CSV result file through the cursor's converter. That is a 4.0.0 behavior change outside this backport.
• So 3.x keeps Do not convert ArrowCursor fallback values twice #1023 as ported: the default conversions work, and custom mappings on the managed-storage fetch path are a known limitation, to be release-noted together with Do not convert ArrowCursor fallback values twice #1023.

[laughingman7743]

Finding (pre-existing, P2), relayed from the independent review:

• Location: pyathena/filesystem/s3_async.py:276, AioS3FileSystem._copy_object_with_multipart_upload.
• After CreateMultipartUpload, an UploadPartCopy error, a cancellation, or a completion failure propagates without AbortMultipartUpload, so the uploaded parts are left behind.

Author verification: pre-existing at the review base, not introduced.

• Keep multipart copy parts within the S3 part size limits #955 only changed the part ranges here.
• Master fixed it in Cancelling an async multipart copy leaves its multipart upload behind #1046 / Abort a cancelled async multipart copy after its running parts #1056, which builds on Copy metadata, tags and annotations in multipart copies #1036's copy rework (botocore>=1.43.31). That work is excluded from 3.x along with the other filesystem chain, by maintainer decision. Deferred.

[laughingman7743]

Second independent review (relayed, requested by the maintainer): Claude Code CLI, model claude-fable-5-1, max profile (first-party Max auth verified), effort high, --permission-mode plan.

• Tools: read and git only (Read/Grep/Glob, git show/diff/log/grep/blame). Session 40ff7b56-3d85-4a99-88a5-7cebd4deb23d.
• Static review on a detached snapshot at 683658f24dff765e0cc6df1bffcfebdd9f7797a8 (base f09cf4269052b7dfa4157f10a5f37a5cbfa6148a). The snapshot was unchanged afterwards.
• The prompt matched the Codex one, plus a compatibility audit:
  • breaking changes;
  • every user-visible behavior change, classified (a) fix of a wrong or failing result, or (b) change to previously correct, working behavior.

Coverage reported:

• all 20 commits against master, hunk by hunk;
• cursor bases and the cache;
• aio iteration, including pyathena/aio/sqlalchemy, which awaits fetchall() and never iterates;
• Spark lifecycle;
• the Arrow, pandas and Polars result sets;
• the converters, with gettz("+05:30") is None confirmed on dateutil 2.9;
• SQLAlchemy reflection traces;
• the filesystem cache, append, multipart and copy paths, against the fsspec 2024.12 flush() contract;
• the tests (each traced to fail without its fix);
• the docs.

Result:

• Correctness, faithfulness, tests and docs: CLEAN. No defects introduced.
• Breaking changes: none. No public name removed or renamed; no signature, default, return shape or dependency floor changed. New exceptions occur only on paths that used to hang, loop forever or return wrong data.

Category (b) behavior changes it listed:

• Skip the result cache for qmark queries with parameters #959: a qmark query with the same parameters is no longer served from the client cache. This is documented in docs/usage.md and options.py.
• Merge a short trailing block into the previous multipart part #947 / Keep multipart copy parts within the S3 part size limits #955: part boundaries change when the tail is short, so multipart ETags of identical content can differ (for example a -3 suffix becomes -2).
• Return the whole result from as_pandas() when auto chunking applies #950: PandasDataFrameIterator.as_pandas() keeps the chunks' index. With an explicit chunksize and index_col, the index is the index_col values instead of RangeIndex(0..n-1).
• Return the whole result from as_pandas() when auto chunking applies #950: with chunksize=None, a second as_pandas() call (or one after fetchone()) returns an empty DataFrame instead of raising a bare StopIteration.
• Do not convert ArrowCursor fallback values twice #1023: managed-storage ArrowCursor results no longer pass through a custom converter (already recorded in the thread on arrow/result_set.py).
• Read multi-line CSV values that cross a block in ArrowCursor #1045: newlines_in_values=True can slow multi-threaded Arrow CSV parsing. This is the same trade-off as on master, measured there at about 0.03–0.04 s per 250 MiB.

Pre-existing 3.x defects it noted (not introduced; fixed on master by excluded PRs):

• _ls_from_cache vs tuple listing keys (Answer info() and exists() from cached parent listings and fix bucket lookups #1006);
• CreateMultipartUpload receiving non-input fields (Per-file request parameters are not applied consistently to multipart upload requests #946 / Route S3 request parameters to the operations that accept them #1000);
• commit() dropping a failed upload (Keep a multipart upload whose abort fails so that discard() retries it #1047);
• no 10,000-part limit (Keep multipart uploads within the 10,000-part limit #968);
• top-level varchar(x) raising in get_columns;
• an unreachable base __anext__ without await.

Minor docs note: docs/aio.md:192-194, already recorded in round 2.

Author verification and actions:

• The Return the whole result from as_pandas() when auto chunking applies #950 second-call behavior was checked against the code: 3.x used next(self._df_iter), and PandasDataFrameIterator.as_pandas() returns pd.DataFrame() when nothing remains.
• All category (b) items were added to the release notes in the PR body.
• No code change: each item is either the documented contract of its fix or the same behavior as master.