Skip to content

fix(pubsub): release listener subscriptions and pool connections - #144

Draft
cw-jlu wants to merge 1 commit into
taskiq-python:mainfrom
cw-jlu:fix/pubsub-listener-cleanup
Draft

cw-jlu wants to merge 1 commit into
taskiq-python:mainfrom
cw-jlu:fix/pubsub-listener-cleanup

Conversation

@cw-jlu

@cw-jlu cw-jlu commented Nov 11, 2026 •

Copy link
Copy Markdown

Closing a PubSubBroker.listen() generator previously left the Redis subscription and its dedicated pool connection checked out. After one listener ended, a broker with a single-connection pool could no longer publish or open a replacement listener (ConnectionError: No connection available). Cancellation and a failed receive leaked the slot too. The listener now owns the PubSub object through its async context manager, so it disconnects and releases the slot on every exit before the surrounding Redis client exits.

Fixes #143. Assignment has been requested; the issue is not yet assigned. This is limited to PubSub listener resource ownership and does not change delivery guarantees, retries, Redis Streams, or the public API.

Actual local verification on Windows 11, Python 3.14.7, taskiq 0.13.0, redis-py 8.0.0, and an isolated Redis 5.0.14.1 over TCP/RESP2:

  • The three new real-Redis regressions fail on unchanged main 8f70bc85dd6a469cae7d3b467af7dea720c92434 because the next broker.kick() cannot acquire a connection; all three pass with the fix. They cover generator close, cancellation while waiting, and a server-side CLIENT KILL of the test-owned connection, then verify publishing and receiving through a replacement listener.
  • uv run --frozen pytest tests/test_pubsub_cleanup.py tests/test_broker.py -k 'pubsub or (pub_sub and not sentinel)' -q: 5 passed, 13 deselected, including the existing broadcast and connection-limit tests.
  • Additional real-server verification on a local Docker Redis 8.2.11 with default RESP3: the same selection returned 5 passed, 13 deselected.
  • uv run --frozen prek run --files taskiq_redis/redis_broker.py tests/test_pubsub_cleanup.py: all applicable hooks passed, including repository-wide Ruff and mypy (16 source files), and Black on the changed files. git diff --check passed.

The real Redis checks exercise broker methods, connections and wire behavior, not a full Taskiq worker process. Sentinel, Cluster, Streams, the full runtime suite, and the Python/OS matrix were not run locally; those remain CI work. RESP2 was explicitly selected because the locally available Redis 5 server does not support RESP3.

Prepared with OpenAI Codex assistance for investigation, implementation, tests and this description. This PR is intentionally a draft pending independent review and account-owner review; no human review is claimed.

Prepared with OpenAI Codex assistance; account-owner review is pending.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

PubSubBroker listener close leaks subscriptions and exhausts its connection pool

1 participant