Repository navigation
perf: replace tenacity with a plain retry loop - #72
Merged
Merged
Conversation
A first attempt that succeeds no longer enters tenacity's attempt loop, so postgres_retry costs about 0.2 us per call instead of 22 us. After a retriable first failure, the retryer replays that error as attempt 1, so waits, attempt counts and logging stay the same.
The loop keeps the attempt count, the predicate-first decision, the jittered exponential wait and re-raising, and drops the tenacity dependency.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #71.
postgres_retrynow runs a plain retry loop instead of tenacity, and tenacity is no longer a dependency. A call that succeeds first time, which is almost every call, is now one directawaitinside atry.What stays the same
retries=NorDB_RETRY_RETRIES_NUMBERcaps the calls of the body. It is still read on every call (ADR 0001), but only once an attempt has failed._backoffis the formula behind tenacity'swait_exponential_jitter()at its defaults:2 ** (attempt - 1)plus uniform jitter in[0, 1). The cap is dropped, because the default cap issys.maxsize / 2seconds.reraise=True.What changes
tenacityis removed fromdependencies. Code that imported it only because db-retry pulled it in has to declare it itself. That belongs in the release notes.before_logDEBUG line ("Starting call to ..., this is the Nth time calling it") is gone. The two db-retry debug lines remain.Exception. Under tenacity, aBaseExceptionsuch asCancelledErrorwent through the predicate and logged "giving up on retry" before propagating. Now it propagates without the log line. It was never retried in either version.An earlier revision of this PR kept tenacity and replayed the first error into it. That worked, but it took 24 lines to call the function once outside tenacity. The loop is 10 lines and has no dependency.
Benchmarks
Wrapping a coroutine that returns immediately, 50,000 calls, Python 3.14:
postgres_retryoverheadA one-row
session.geton an autocommit engine against Postgres 17 in Docker, with SQLAlchemy 2.0.54 and asyncpg 0.31.0. The table shows µs per request over two rounds with 1 client and one round with 50 clients. "Before" and "after" differ only in the db-retry code:postgres_retry+TransactionBefore this change, replacing pre-ping with the retry made requests slower; now it makes them faster.
Tests and docs
test_a_retriable_first_failure_waits_once_then_succeedschecks that one retriable failure costs exactly one non-zero wait and two attempts.README.mdno longer lists tenacity as a requirement.CONTEXT.mdandAGENTS.mdno longer define "attempt" through tenacity'sstop_after_attempt.