Skip to content

fix(google-api-core): make RetryError picklable and copyable - #18947

Open
Mahalaxmibejugam wants to merge 2 commits into
googleapis:mainfrom
Mahalaxmibejugam:fix-retryerror-pickle
Open

Mahalaxmibejugam wants to merge 2 commits into
googleapis:mainfrom
Mahalaxmibejugam:fix-retryerror-pickle

Conversation

@Mahalaxmibejugam

@Mahalaxmibejugam Mahalaxmibejugam commented Nov 1, 2026 •

Copy link
Copy Markdown
  • Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
  • Ensure the tests and linter pass
  • Code coverage does not decrease (if any source code was changed)
  • Appropriate docs were updated (if necessary)

Fixes #18946 🦕

Problem

RetryError.__init__(self, message, cause) requires cause, but only message is passed to Exception.__init__, so self.args == (message,). The default BaseException.__reduce__ rebuilds the exception as RetryError(message), which fails:

TypeError: RetryError.__init__() missing 1 required positional argument: 'cause'

As a result, a RetryError raised in a worker process can't be returned to the parent. multiprocessing.Pool result handling crashes and AsyncResult.get() blocks, and ProcessPoolExecutor raises BrokenProcessPool. copy.copy() / copy.deepcopy() of a RetryError fail the same way.

Fix

Define RetryError.__reduce__ so the exception is rebuilt with both message and cause, and return self.__dict__ as the state (as BaseException does by default) so other instance attributes, such as notes added with add_note(), are preserved.

Testing

Added unit tests in tests/unit/test_exceptions.py:

  • pickle round-trip for every pickle protocol (message, cause and str() preserved);
  • round-trip with cause=None;
  • instance attributes preserved across a round-trip;
  • copy.copy and copy.deepcopy.

The new tests fail without the fix and pass with it. tests/unit/retry and tests/unit/test_exceptions.py pass (198 tests), and ruff format --check / ruff check are clean.

RetryError.__init__ requires a 'cause' argument that isn't stored in
self.args, so the default BaseException.__reduce__ rebuilds it as
RetryError(message) and fails with:

    TypeError: RetryError.__init__() missing 1 required positional
    argument: 'cause'

This breaks pickling (e.g. returning the error from multiprocessing or
concurrent.futures workers) as well as copy.copy/copy.deepcopy.

Define __reduce__ so the exception is rebuilt with both message and
cause.
@Mahalaxmibejugam
Mahalaxmibejugam requested a review from a team as a code owner November 1, 2026 19:35
@Mahalaxmibejugam
Mahalaxmibejugam marked this pull request as draft November 1, 2026 19:36

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request implements the reduce method for exceptions to support proper pickling and copying across processes, and adds corresponding unit tests. The reviewer noted that returning only a 2-tuple from reduce will cause any additional instance attributes to be lost, and suggested returning self.dict as the third element to preserve them.

…ckling

Return self.__dict__ as the state from RetryError.__reduce__, matching
BaseException's default behavior, so attributes set on the instance
(including notes added with add_note()) survive pickling and copying.
@Mahalaxmibejugam
Mahalaxmibejugam marked this pull request as ready for review November 1, 2026 19:53

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant