Skip to content

Replace name-mangled helper methods with shared single-underscore helpers #879

Description

@laughingman7743

Use case

Name-mangled (__name) helper methods hide shared behavior from subclasses. Because a subclass cannot override or reuse them, each class that needs the behavior keeps its own copy, and the copies drift.
PyAthena used few of them, but the count has grown recently.
Counting non-dunder def __name( in pyathena/:

Ref Mangled helpers
v3.20.0 (2025-11-08) 5
v3.36.0 (2026-09-02) 10
master 7b77a53 (2026-09-28) 16

All six added since v3.36.0 come from the Spark lifecycle work in #791:

Open PR #853 would add eight more and remove one: __start_query_execution, __stop_started_query, __cancel_and_wait and __stop_started_calculation, each in a sync and an async version.

The other mangled helpers on master:

  • __poll in BaseCursor (pyathena/common.py), AioBaseCursor (pyathena/aio/common.py), SparkBaseCursor (pyathena/spark/common.py) and AioSparkCursor (pyathena/aio/spark/cursor.py). The four have almost the same body: poll the status, call on_poll, stop at a terminal state, sleep. Each subclass also has to override _poll because it cannot reach the parent's __poll.
  • AthenaResultSet.__get_query_results and __fetch (pyathena/result_set.py), and AthenaAioResultSet.__async_get_query_results and __async_fetch (pyathena/aio/result_set.py).
  • __s3_file_system in AthenaArrowResultSet and AthenaPandasResultSet. The two bodies are different: the Arrow one returns pyarrow's S3FileSystem, the pandas one returns PyAthena's. AthenaS3FSResultSet names the same role _create_s3_file_system.

Neither the tests nor the package access a mangled name from outside its class (no _ClassName__name references), so renaming them is internal.

Proposed change

Validation plan (if implementing)

  • just lint.
  • The affected PyAthena tests on AWS: tests/pyathena/test_cursor.py, tests/pyathena/aio/test_cursor.py, tests/pyathena/spark/, tests/pyathena/aio/spark/ (existing interrupt, cancel and session-cleanup tests), and the result-set tests of the Arrow, pandas, Polars and S3FS cursors.
  • Re-count the mangled helpers, and diff the remaining polling loops, to show the duplication was removed.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions