Skip to content

Add TLS session resumption via SSLSessionCache - #789

Open
sylwiaszunejko wants to merge 6 commits into
scylladb:masterfrom
sylwiaszunejko:tls-ticket
Open

Add TLS session resumption via SSLSessionCache#789
sylwiaszunejko wants to merge 6 commits into
scylladb:masterfrom
sylwiaszunejko:tls-ticket

Conversation

@sylwiaszunejko

@sylwiaszunejko sylwiaszunejko commented Apr 3, 2026

Copy link
Copy Markdown

What and why

A shard-aware driver opens one TLS connection per shard to every node, and each one currently
pays for a full handshake — certificate exchange plus a signature, which is the expensive part,
especially with certificate authentication. TLS lets a client skip that by replaying a session
established earlier with the same peer (RFC 5077 tickets for TLS 1.2, RFC 8446 PSKs for
TLS 1.3), but OpenSSL never does this on its own: the client has to hold on to the session and
offer it explicitly on the next connection. Neither the stdlib ssl module nor pyOpenSSL
exposes SSL_CTX_sess_set_new_cb, so there is no way around doing it by hand.

This adds that: one SSLSessionCache per Cluster, offered to every connection before its
handshake and refreshed after. On by default whenever ssl_context is set.

cluster = Cluster(ssl_context=ssl_context)                                  # resumption on
cluster = Cluster(ssl_context=ssl_context, ssl_session_cache=None)          # off
cluster = Cluster(ssl_context=ssl_context,
                  ssl_session_cache=SSLSessionCache(max_size=64))           # sized, or shared

Design notes

  • A cached session is not consumed by being used. get() leaves the entry in place, and
    each successful handshake stores a fresh session over it. Measured: one session is accepted by
    four concurrent connections on TLS 1.2 and 1.3, stdlib and pyOpenSSL, and against real Scylla.
    Treating tickets as single-use (removing on get()) would mean only the first connection of a
    per-shard burst resumes — precisely the case this ticket is about. RFC 8446's "SHOULD NOT
    reuse" concerns 0-RTT replay and tracking; the driver sends no early data.
  • The session is stored from the ReadyMessage / AuthSuccessMessage handlers, not right after
    the handshake.
    A TLS 1.3 server sends its NewSessionTicket as a post-handshake message;
    confirmed against Scylla that has_ticket is False immediately after connect() and True
    after the first CQL exchange. Storing is idempotent, so nothing needs to track whether it
    already happened, and every failure in this path is logged and dropped — both call sites are
    wrapped in @defunct_on_error, where a raised exception would kill a healthy connection over
    an optimisation.
  • The SSLContext is part of the cache key. A session cannot be replayed onto a different
    context — the stdlib rejects it with ValueError: Session refers to a different SSLContext.
    That is also why the deprecated ssl_options-only path does not participate: each of those
    connections builds its own context.
  • No TTL. OpenSSL enforces session lifetime itself; a session the server no longer accepts
    costs one full handshake, which is the fallback anyway.
  • Policy is separated from accessors (_get_resumable_tls_session / _set_tls_session) so a
    reactor not using the stdlib ssl module overrides only those.

Not covered

  • asyncio — the handshake happens inside loop.create_connection(..., ssl=...), which offers
    no point at which a session could be restored. AsyncioConnection declares
    supports_tls_session_resumption = False and no cache is created for it.

Fixes: https://scylladb.atlassian.net/browse/DRIVER-165

Pre-review checklist

  • I have split my patch into logically separate commits.
  • All commit messages clearly explain what they change and why.
  • I added relevant tests for new features and bug fixes.
  • All commits compile, pass static checks and pass test.
  • PR description sums up the changes and reasons why they should be introduced.
  • I have provided docstrings for the public items that I want to introduce.
  • I have adjusted the documentation in ./docs/source/.
  • I added appropriate Fixes: annotations to PR description.

@Lorak-mmk

Copy link
Copy Markdown

This reduces reconnection latency and CPU overhead, especially in
deployments with short-lived connections or frequent reconnects.

Such claims would ideally be supported by benchmarks. Could you try to create some?
I very vaguely remember this feature being postponed because the performance gains were underwhelming (but perhaps memory is failing me).

@sylwiaszunejko

Copy link
Copy Markdown
Author

This reduces reconnection latency and CPU overhead, especially in
deployments with short-lived connections or frequent reconnects.

Such claims would ideally be supported by benchmarks. Could you try to create some? I very vaguely remember this feature being postponed because the performance gains were underwhelming (but perhaps memory is failing me).

That's the goal, but you're right, I don't have any tests to prove that, removed this claim from the PR description. If I manage to create proper benchmarks I will update on that

@mykaul

mykaul commented Apr 3, 2026

Copy link
Copy Markdown

We could, if it helps, only support this for TLS 1.3.

@sylwiaszunejko

Copy link
Copy Markdown
Author

@dkropachev @Lorak-mmk I pushed changes with improvement from older Dmitry's PR, will update PR description soon

@dkropachev dkropachev left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I rechecked the TLS session-resumption path against the current branch. The ssl_options configuration still builds a fresh SSLContext per Connection, and a cached stdlib session from the previous connection is incompatible with that new context. I reproduced the failure locally on Python 3.10.12; the session restore path raises ValueError: Session refers to a different SSLContext. Since the new code only catches AttributeError and ssl.SSLError, reconnects fail instead of falling back to a full handshake, and the regression is enabled by default because Cluster auto-creates SSLSessionCache for ssl_options.

Comment thread cassandra/connection.py Outdated

@dkropachev dkropachev left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two blocking issues from local validation:

  1. Twisted caches a TLS session even after hostname verification has already failed, which lets an untrusted peer populate the resumption cache.
  2. SSLSessionCache accepts max_size <= 0 and then crashes on the first insert (KeyError from popitem() on an empty OrderedDict).

Comment thread cassandra/io/twistedreactor.py Outdated
Comment thread cassandra/connection.py Outdated

@dkropachev dkropachev left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two correctness issues need attention before this lands: the PyOpenSSL TLS 1.3 cache point is too early to capture the resumable session, and the cache can evict a live entry while expired ones remain resident.

Comment thread cassandra/io/twistedreactor.py Outdated
Comment thread cassandra/connection.py Outdated
Comment thread cassandra/io/twistedreactor.py Outdated
Comment thread tests/integration/standard/test_tls_resumption.py Outdated
Comment thread cassandra/cluster.py Outdated
Comment thread tests/integration/standard/test_tls_resumption.py Outdated
@coderabbitai

coderabbitai Bot commented Jul 15, 2026

Copy link
Copy Markdown

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Adds SSLSessionCache, a thread-safe bounded LRU cache for TLS sessions. Cluster creates, disables, or accepts a cache and passes it to connections. Connections derive endpoint-specific keys, restore sessions before handshakes, and store sessions after startup or authentication. Reactor implementations declare unsupported resumption. Tests cover cache behavior, TLS 1.2 and TLS 1.3, concurrency, cache isolation, and shard-aware connections.

Possibly related PRs

Suggested reviewers: mykaul, dkropachev, lorak-mmk

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 19.83% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly identifies the primary change: adding TLS session resumption through SSLSessionCache.
Description check ✅ Passed The description explains the motivation, design, configuration, limitations, tests, and linked issue. It follows the repository template and is sufficiently complete despite unchecked documentation-re…
Linked Issues check ✅ Passed The description includes a valid Fixes annotation for DRIVER-165.
Out of Scope Changes check ✅ Passed The changes support the stated TLS session resumption objective and add related implementation, reactor declarations, documentation, and tests. No unrelated changes are evident.
Full details: Description check

Explanation

The description explains the motivation, design, configuration, limitations, tests, and linked issue. It follows the repository template and is sufficiently complete despite unchecked documentation-related items.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai
coderabbitai Bot requested a review from dkropachev July 15, 2026 08:11
Comment thread cassandra/connection.py Outdated
Comment thread cassandra/connection.py Outdated
Comment thread cassandra/connection.py

@dkropachev dkropachev left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One inline correctness issue. Branch also needs a rebase onto current master; PR is currently conflicting.

Comment thread cassandra/connection.py
Comment thread cassandra/cluster.py Outdated
Comment thread tests/unit/test_tls_resumption.py
Comment thread tests/integration/standard/test_tls_resumption.py Outdated
@nikagra

nikagra commented Sep 7, 2026

Copy link
Copy Markdown

The design is right where it matters — one cache per Cluster, restore before the handshake, store from Ready/AuthSuccess, context and hostname in the key. None of that is what I want to revisit.

What I would like to settle before the next push is the shape of the bookkeeping around it, because each round so far has been answered with a parameter or a method rather than a change of mechanism:

round what was added
ticket lifetime ignored set(lifetime=)
deadline slides on reuse set(extend=) + _tls_session_was_renewed()
context retention weakref → discard_context()acquire_context/release_context + _context_owners + two Cluster attributes + a shutdown hook
bad entry re-offered discard(key, session) + _tls_session_offered cleared in three places
shard-aware keying _tls_session_cache_key_override set from pool.py

Two changes would collapse most of that.

1. The cache has an owner, and the driver knows which it is. An auto-created cache is a Cluster attribute, so its lifetime is already the cluster's and it needs no refcount. A user-supplied one is the user's object, and the driver deleting rows in it is the part I would question. Recording which of the two it is at construction removes acquire_context, release_context, _context_owners, both Cluster attributes, the shutdown hook and the k[0] is ssl_context scan — along with the cases each of those had to handle: cluster that never connected, shutdown that raised, cache assigned after construction, keys that are not tuples.

2. The accessor boundary should hold. _tls_session_lifetime and _tls_session_was_renewed read self._socket directly, against the note at :1352. @dkropachev's open twisted and eventlet threads are the same boundary from the reactor side.

Order I would suggest: 1, then 2 (which also removes the version-guessing in _get_resumable_tls_session), then move the extend decision inside set(), where the lock is.

Separately: five threads are resolved while the code at each is unchanged — cluster.py, connection.py, pool.py, cluster.py, test_tls_resumption.py. If any of those was a decision rather than an oversight, say so and I will drop it.

Deliberately not raising this round, to keep it to the shape above: docs for an on-by-default change, SSLSessionCache.set/discard edge behaviour, the asyncio hostname duplication, the SniEndPoint key override, unit-suite cost.

@nikagra nikagra left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Detail under the comment above — the two changes there would remove most of what these three sit on.

Comment thread cassandra/connection.py Outdated
Comment thread cassandra/connection.py
Comment thread cassandra/connection.py Outdated
@sylwiaszunejko
sylwiaszunejko force-pushed the tls-ticket branch 2 times, most recently from b540378 to a9124b7 Compare September 8, 2026 10:53

@nikagra nikagra left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both findings below are from the 2026-09-08 rework of the cache core, not the earlier rounds -- the rest of the backlog reads as closed to me. One follow-up on the TLS 1.3 discard thread separately.

Comment thread cassandra/connection.py Outdated
Comment thread cassandra/cluster.py Outdated
Comment thread cassandra/connection.py Outdated
Comment thread cassandra/cluster.py Outdated
Comment thread cassandra/connection.py Outdated
TLS clients can skip the expensive part of a handshake by replaying a
session established earlier with the same peer (RFC 5077 tickets for
TLS 1.2, RFC 8446 PSKs for TLS 1.3), but OpenSSL never does this on its
own: the client has to hold on to the session and offer it explicitly on
the next connection.

Add the storage half of that: a bounded, thread-safe LRU of TLS sessions
keyed by TLS peer identity, plus an EndPoint.tls_session_cache_key
property that produces the key.  The cache is a module of its own,
cassandra.ssl_session_cache, because keeping sessions is all it does: what
may be offered to whom, and for how long, belongs with the connections that
work it out.  A cached session is not consumed by
being used -- one session can be replayed by any number of concurrent
connections -- so get() leaves the entry in place and each successful
handshake stores back over it whatever the peer handed over.

Entries carry the lifetime the caller gives them and are dropped once it
runs out, so a session is never offered past the point the peer said it
would honour it; what that lifetime should be is for the caller to work
out, since it depends on how the session resumes.  Whether a store moves an
existing deadline is decided here rather than asked of the caller: a session
the peer handed back unchanged keeps the deadline it had, since its lifetime
runs from when the peer issued it and not from when it was last replayed, and
re-stamping a full lifetime on every reuse would let one ticket be offered
for as long as connections keep being opened.  Below TLS 1.3 an abbreviated
handshake hands back the offered session itself, so that is the ordinary
case; TLS 1.3 normally issues a fresh one, which is entitled to a lifetime of
its own.  The two are told apart by session id, compared under the same lock
that stores the result, so no concurrent store can land in between.

An entry recognised this way keeps the object holding it as well as its
deadline.  SSLSocket.session builds a new wrapper on every access, so what a
resumed connection stores back is another handle on the credential already
there; swapping one for the other would change nothing but the identity a
caller compares against when it asks for its own session to be dropped.
Keeping it means an entry changes identity only when it changes credential.

A caller can also say which session it offered on the handshake it is
storing the result of, and a store of that same session is skipped where the
entry no longer holds it: another connection has stored a session the peer
reissued to it, or the deadline passed and a lookup dropped the entry.  In
neither case has this caller anything to add, while storing it would put a
deadline running from now on a session the peer issued at some earlier
point -- and would replace a fresher session with an older one.

Neither comparison can say anything about a ticket whose session id is
empty, which RFC 5077 section 3.4 lets a server send and for which
SSLSession offers no ticket to compare instead.  Two of those are reported
as the same session, the conservative reading: the deadline then stays where
it is, where calling them different would re-stamp a full lifetime on what
may well be the ticket already held.  What that costs is resumption rather
than correctness -- a reissued ticket inherits its predecessor's deadline,
and one arriving where the entry has since gone is not stored, which the
next connection puts right by offering nothing and storing afresh.

SNI endpoints add the server name to their key, since they all share a
proxy address and port but are distinct TLS peers.  Client-routes
endpoints key on the node's host_id rather than the proxy address they
happen to resolve to at the moment.

Nothing uses the cache yet.

Refs DRIVER-165
Offer the cached session for the endpoint before the handshake, and
store the negotiated session once the connection is up, so that the next
connection to the same node -- in particular the burst of per-shard
connections a pool opens at once -- can skip the certificate exchange
and signature of a full handshake.

The session is stored from the ReadyMessage / AuthSuccessMessage
handlers rather than right after the handshake.  A TLS 1.3 server sends
its NewSessionTicket as a post-handshake message, so a session read
straight after connect() carries no ticket and would not resume; by the
time the CQL handshake has completed the ticket has been read off the
socket.  Storing is idempotent, so nothing needs to track whether it
already happened, and every failure in this path is logged and dropped:
resumption is an optimisation, and both call sites are wrapped in
@defunct_on_error, where a raised exception would kill a healthy
connection.

How long a session may be offered is worked out here, because it depends on
how the session resumes: a ticket's lifetime is the one the server
announced, while SSLSession.timeout is only the local context's default and
says nothing about what the peer will still accept, so it is used solely for
a session that resumes by id.  RFC 8446 section 4.6.1 also caps the
client at seven days however long the server asked for.  A zero lifetime is
read against the negotiated version, since the two RFCs disagree on it: TLS
1.3 says discard the ticket immediately, while RFC 5077 section 3.3 reserves
zero for "lifetime unspecified" and leaves retention to local policy, so a
TLS 1.2 ticket is kept and timed by the local timeout.  The version also
rules out caching a TLS 1.3 session that carries only an id: resumption
there is the ticket's pre-shared key, while the id a TLS 1.3 handshake
carries is the legacy_session_id_echo a server sends back for the middlebox
compatibility mode of RFC 8446 appendix D.4, which resumes nothing.  OpenSSL
reports no id at all until a NewSessionTicket has been read, so that session
is not one it produces; stating the rule against the negotiated version
rather than against what one library exposes is what makes it hold
regardless.
OpenSSL does not apply either limit on the client's behalf -- it will offer
an expired ticket and let the server refuse it.  The announced lifetime is
taken whole rather than reduced by the session's age: this connection
established the session moments ago, so that age is one CQL handshake, and
SSLSession.time is a wall-clock stamp, so subtracting it would let a clock
step landing in between decide the answer -- far enough forward and nothing
is cached at all.  The deadline the cache keeps is monotonic, so nothing
after the store can skew it either.

A pool reaches a shard-aware node on a second port, which would otherwise
key those connections separately from the one the control connection
established, leaving the whole per-shard burst to handshake in full.  The
endpoint alias that _get_shard_aware_endpoint already builds for that port
therefore carries the node's cache key, so both listeners share one session
and nothing has to be threaded through the connection factory.  The port
stays part of the key by default, so two unrelated TLS servers on one
address still cannot share a session; only an endpoint that names another
node is exempt.

The SSLContext is part of the key because a session cannot be replayed onto
a different one and a cache may be shared by several clusters.  It is held
strongly there: a cached session already keeps its context alive on its own
-- CPython's SSLSession holds a reference to the context it was established
with -- so holding it weakly here would buy nothing.

The key also carries the name wrap_socket() is given, which is the name
the peer certificate is verified against.  A resumed handshake sends no
Certificate, so that name is never checked again; offering a session to a
connection expecting a different name would silently skip hostname
verification for it.  Both the key and wrap_socket() take the name from
one accessor so the two cannot drift apart.

A session offered on a connection whose handshake then failed is dropped from
the cache.  Both RFCs have a server fall back to a full handshake rather than
fail when it will not resume, so this should not happen; but nothing stores a
fresh session for a connection that never came up, so an entry that did
provoke a failure would otherwise be offered again by every later connection
until its lifetime ran out.  Only a TLS error counts: a refused or reset
connection says nothing about the session.  And only the session this
connection offered goes: connections to one node are opened together, so
another may have stored a session the peer issued in its place, and that one
failed nothing.  A sibling storing the same session back is not that, and
leaves the entry retractable, because the cache keeps the object it already
holds for a session it recognises.

What was offered is kept past a handshake that succeeded as well, because
the store hands it to the cache.  Telling a session the peer reissued from
the one that came back unchanged needs both sides of the exchange, and where
connections to a node are opened together only the connection that offered
one knows the second.

Three accessors are the whole of what a reactor whose TLS does not go
through the stdlib ssl module has to reimplement to take part: the policy
around them asks one for the session to store, one for the negotiated
version and one to restore a session onto a socket, and reads nothing off a
socket itself.  Connections whose SSLContext is
derived from ssl_options do not participate, because a session cannot be
replayed onto a different context and each of those connections builds
its own.  The asyncio reactor opts out entirely: its handshake happens
inside loop.create_connection(), with no point at which a session could
be restored.

Refs DRIVER-165
Create an SSLSessionCache per Cluster whenever TLS is configured through
ssl_context, and hand it to every connection the cluster opens, so that
resumption is on by default with no configuration.  Pass
ssl_session_cache=None to turn it off, or an instance of your own to size
it or share it between clusters.

No cache is created where resumption cannot work: the deprecated
ssl_options-only path, whose per-connection SSLContexts a session cannot
be replayed onto, and reactors that report they cannot restore a session
before the handshake, which today means asyncio -- which is also what the
default connection class resolves to on Python 3.12 and newer with no libev
extension installed, asyncore having left the standard library there.
connection_class is not required to derive from Connection, so one that does
not report the capability at all is treated as lacking it rather than
raising.

Nothing is settled in advance.  ssl_session_cache is a property, answering
against whatever ssl_context and connection_class are in force when it is
read and making a cache on first use where one is wanted.  Both of those are
public attributes, so a decision taken at construction would leave resumption
off on a reactor that does support it, or hand the keyword to a connection
class that does not take it -- and one retaken later needs the first to be
remembered, which is how an explicit None came to be overwritten.  Behind the
property is what the caller set and nothing else: assigning None turns
resumption off and leaves it off, and assigning a cache asks for one as much
as passing it to the constructor does, whenever it is done.

A cache the caller asked for that cannot be used reads back as None, and
connect() says why.  Asking for resumption and silently getting none is
worse than not having it: a cache answered back here would be handed to
every connection -- which a connection class that does not take the keyword
cannot even accept -- and would sit reachable and empty for anyone reading
it back, which is also what a server that issues no tickets looks like.
Only what the caller set is worth a word, since a cache made for a cluster
is only ever made where it can be used; turning TLS off afterwards is not
something to complain about.  connect() is the one place that says it, so it
is said once without anything having to record that it was.

Whose the cache is settles what becomes of it, and nothing has to track that
either: the field behind the property holds what the caller set, and one
made for the cluster is kept apart from it.  One created here is reachable
only through the attribute, so it and the sessions in it go when the cluster
does and shutdown has nothing to do.  One the caller supplied stays the
caller's: shutdown leaves its entries alone, which is what lets clusters
share sessions -- at the same time, or one after another, so that a cluster
replacing an earlier one resumes rather than handshaking in full -- and
keeps the driver from deleting rows in an object it does not own.  Its
entries hold the SSLContext their session was established with, bounded by
the cache's max_size, and clear() is there for a caller who wants them gone
sooner.

Refs DRIVER-165
Stand up a TLS server on loopback and connect to it with the driver's own
socket setup, so the restore-before-handshake and store-after-startup
paths run for real and the result is read back the way OpenSSL reports
it, through SSLSocket.session_reused.  Covers TLS 1.2 and TLS 1.3, the
latter skipped where the local OpenSSL does not offer it -- skipping the
subclass rather than the base, since a skipped base would take its
subclasses with it.

Two of these pin down behaviour that is easy to regress: that four
connections opened at once all resume from the single cached session --
the per-shard burst DRIVER-165 is about -- and that on TLS 1.3 nothing is
cached until the server's NewSessionTicket has actually been read off the
socket.

Refs DRIVER-165
Restart the cluster with client encryption on, warm a session cache with
one cluster, then hand it to a second one and require every connection it
opens to have resumed -- which is the question only a real server can
answer: whether it accepts one session offered concurrently by the whole
batch of per-shard connections.

The cluster is given a shard-aware TLS port, since that is the port those
per-shard connections use and therefore where resumption has to pay off;
Scylla leaves it unset by default.  The certificate names every node
rather than only the contact point, or the driver could not build pools to
the rest of the cluster and the test would quietly examine a single host.
Each Session is held for the duration of a test: Cluster.sessions is a
WeakSet, so a dropped Session takes its pools -- everything worth
inspecting -- with it and leaves only the control connection behind.  The
number of connections collected is asserted before their resumption
flags, so the test cannot pass by examining almost nothing.

Whether there is anything here to test at all depends on the reactor, so the
skip asks the connection class Cluster will instantiate rather than reading
EVENT_LOOP_MANAGER: with no selector set and no libev to import, that class
is the asyncio reactor, which cannot restore a session before the handshake.

Follows the reconfigure-and-remove pattern the other modules here use for
cluster-level options, and generates the server certificate with
cryptography so the test does not depend on an openssl binary.

Refs DRIVER-165
Scylla only issues session tickets when enable_session_tickets is set in
client_encryption_options, and that is off by default -- without it the
cache stays empty and every connection performs a full handshake, with no
indication of why.

Refs DRIVER-165

@nikagra nikagra left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Major] 🟠 The description still contradicts the code on the point @dkropachev asked to have updated in #789 (comment):

  • "No TTL. OpenSSL enforces session lifetime itself" — it doesn't, and the driver now does. _tls_session_lifetime() takes the lifetime the server announced, capped at _MAX_TLS_SESSION_LIFETIME, and SSLSessionCache drops an entry past its deadline both on lookup and on eviction. That bullet needs to describe the rule that shipped, including the TLS 1.2 / 1.3 split on a zero lifetime.
  • "Policy is separated from accessors (_get_resumable_tls_session / _set_tls_session)" — three now; _tls_negotiated_version is the one the retention rules read.
  • "Measured: … stdlib and pyOpenSSL" — no pyOpenSSL reactor is left in the tree. Worth saying the measurement predates their removal, or dropping that half.
  • The deferrals are agreed but unlinked: #984 (test capability detection), #985 (multi-ticket caching — #789 (comment) asked for this link explicitly), #986 (tickets after CQL startup). A "Deferred" section naming the three closes two bot threads with it.

The checklist's docstring and docs boxes are also unticked on a PR that adds docs/api/cassandra/ssl_session_cache.rst.


Notes on the process

Separately, and about process rather than code. I have closed six of my seven open threads with this pass; the one left is #789 (comment), which is now on its third restatement. Worth noting why that happens, because the code here is converging and the threads are not: git diff 637f384c 8c74beec -- cassandra/connection.py is five hunks — one import, one docstring cross-reference, the 190-line SSLSessionCache move into its own module, one docstring reword, and one line from the rebase. No functional change at all, yet 15 threads are open and only three of them anchor to current code.

Three things would end this:

  • @coderabbitai pause. 33 of the 100 review threads on this PR are bot-opened (copilot 25, coderabbit 8), several were stale when posted, and they bury the human blockers.
  • Settle the retention policy in DRIVER-165, not here. Lifetime source, cache ownership and failed-handshake behaviour have each been designed twice in review — two complete mechanisms were built and then deleted (SSLContext retention went strong key → weakref key → discard_context → none of it; _decide_tls_session_cache plus three bookkeeping flags → a property). Get an explicit ack from @dkropachev on the policy, then review conformance only.
  • Cut the prose to contract level. cassandra/ssl_session_cache.py is 70 lines of code to 141 of docstring and comment; _is_same_session is 23 lines of docstring over 4 of code. Rationale that is not a caller's contract belongs in DRIVER-165 or a commit message. As it stands every behaviour change forces a paragraph rewrite, which manufactures new review surface — two of my findings this round, and a good share of every round before it, are prose contradicting code rather than defects.

If the next round is not green, the way out is three stacked PRs — (a) ssl_session_cache.py + EndPoint.tls_session_cache_key + tests, no behaviour change; (b) connection wiring; (c) the Cluster surface — one reviewer concern each.

Comment thread cassandra/cluster.py
return None
if self._ssl_session_cache is not _NOT_SET:
return self._ssl_session_cache
if self._ssl_session_cache_created is None:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Minor] 🟡 Check-then-act. Two threads reaching this with _ssl_session_cache_created still None both construct a cache and the second assignment wins; the connections the first thread is opening then cache into an object nothing else can reach, and never resume from it.

The docstring above is what makes this reachable rather than theoretical — "configuring TLS at any point still gets a cache" means a cluster whose ssl_context is set after connect() first reads this from whichever pool threads happen to be reconnecting.

A dedicated Lock() around the create rather than self._lock: it holds nothing else while the constructor runs, so it cannot take part in an ordering with the pool locks.

kwargs = cluster._make_connection_kwargs(DefaultEndPoint('127.0.0.1'), {})
assert 'ssl_session_cache' not in kwargs

def test_does_not_warn_where_resumption_works_or_was_declined(self):

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Minor] 🟡 This went vacuous when the warning moved from __init__ to connect(). make_cluster does not connect, so it now asserts that construction does not warn — which it cannot. Its four positive siblings (:1240, :1253, :1266, :1280) all call _warn_if_tls_session_cache_unusable() directly, so nothing covers the call at cluster.py:1893 either: delete that line and the unit suite stays green.

Calling the method on each of the three clusters here restores it, and one test driving connect() past _is_setup would pin the wiring.

Related, same line of code: the warn runs only inside if not self._is_setup, so a cache assigned after connect() gets the None read-back but never the log — the docstring at cluster.py:1718-1720 promises both. Either call it from the setter too, or drop that half of the sentence.


# One session as far as this can tell, so the entry keeps both the
# deadline it had and the object holding it.
assert cache.get('key') is first

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Minor] 🟡 assert cache.get('key') is first has to run within 50 ms of the set two lines up, or the entry has expired and this is None. A GC pause or a loaded runner is enough.

test_the_offered_session_does_not_displace_a_siblings (:152-165) has the non-timing idiom for exactly this — give the first set a real lifetime and assert cache._sessions['key'][1] is unchanged. The sleep(0.06) half is fine; only the pre-sleep assertion races.

What OpenSSL reports for each of the cluster's live connections: a list of
``session_reused`` flags, one per connection.
"""
return [bool(connection._socket.session_reused)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Nit] 🟢 Still unguarded, though the thread that raised it (#789 (comment)) is closed. A connection defuncted between the wait_until and the read at :198 has _socket is None while still reachable through get_connections(), so this raises AttributeError out of the lambda instead of reporting a result. Skipping connections with no live socket keeps the count assertion meaningful.

replayed by any number of concurrent connections, and each successful
handshake stores back whatever the peer handed over -- a fresh session
where one was issued, otherwise the same one again, which keeps the
deadline it already had rather than starting a new one. An entry whose lifetime has run out is never handed out again, and

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Nit] 🟢 127 chars; the rest of the file wraps at ~79. Reflow artifact from the move.

Comment thread cassandra/connection.py
# to a connection expecting a different name would silently skip
# hostname verification for it. Deriving the name from the same place
# _wrap_socket_from_context does is what keeps the two from drifting.
return (self.ssl_context, self.endpoint.tls_session_cache_key,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] Bind cached sessions to TLS security configuration. The key uses SSLContext identity but ignores mutable verification, cipher, and client-certificate state. Reproduced outcomes include CERT_NONE to CERT_REQUIRED still resuming an unverified session, mTLS certificate rotation retaining the old client identity, and cipher tightening failing the next reconnect. Track configuration generation or invalidate sessions when context security state changes.

Comment thread cassandra/connection.py
# overrides those accessors -- asyncio hands the handshake to
# loop.create_connection(), which offers no point to restore a session at
# all.
supports_tls_session_resumption = True

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] Make resumption capability opt-in. Base Connection defaults support to True, so existing custom subclasses inherit it and receive the new ssl_session_cache keyword. A factory accepting every pre-existing keyword then fails with TypeError. Default to False and enable explicitly on supported reactors.

Comment thread docs/api/index.rst
cassandra/decoder
cassandra/concurrent
cassandra/connection
cassandra/ssl_session_cache

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] Rename this Sphinx document. A clean warning-as-error build fails with Document name contains underscores: api/cassandra/ssl_session_cache. Rename the file to ssl-session-cache.rst and update this entry.

Comment thread cassandra/connection.py
# nothing about the session, and dropping it would cost a later
# connection a full handshake for no reason. Whether anything
# was offered to retract is _discard_tls_session's own business.
if isinstance(err, ssl.SSLError):

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Discard sessions after deferred handshake failures. With do_handshake_on_connect=False, TLS failures occur in reactor I/O and bypass this cleanup. The failed session remains cached and is retried until expiry. Compare-and-discard it from fatal incomplete-handshake paths, including EOF and timeout.

Comment thread cassandra/connection.py
# deferred its ticket past this point is what would call for a
# later hook than this one.
return
lifetime = self._tls_session_lifetime(session)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Start ticket expiry when the ticket arrives. The full lifetime currently starts at CQL READY or AuthSuccess, extending it by startup and authentication time. Capture a monotonic timestamp before the TLS handshake and derive remaining lifetime from that bound.

Comment thread cassandra/cluster.py
session. In those cases no cache is created and connections handshake in
full.

It equally requires the server to hand out something it will honour later.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Version-qualify the Scylla ticket default. enable_session_tickets is not always off by default; newer Scylla releases default it on. Qualify this statement by release.

raise


def teardown_module():

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Preserve TLS files when KEEP_TEST_CLUSTER retains the cluster. In that mode remove_cluster() is a no-op, but teardown still deletes certificate files referenced by the retained TLS configuration. Skip this test mode, restore configuration, or retain the files.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants