[v9r0] Use SSL/cryptography instead of M2Crypto/pyasn1#8691
Draft
chrisburr wants to merge 7 commits into
Draft
Conversation
Reimplement the X509 layer (X509Certificate, X509Chain, X509Request, X509CRL) with pyca/cryptography, moving the modules to the top of DIRAC.Core.Security and dropping the m2crypto subpackage and its extend_path import hack. Decode the VOMS and diracGroup extensions with the declarative ASN.1 API of cryptography (>= 47) instead of pyasn1, which is 40-80x faster. Reimplement the DISET SSLTransport with ssl.SSLContext: RFC 3820 proxies are accepted via ssl.VERIFY_ALLOW_PROXY_CERTS and the peer credentials are extracted from SSLSocket.get_unverified_chain(), which requires python >= 3.13 on the server side (clients work on 3.11+). Run the Tornado HTTPS server with a standard SSLContext and read the peer chain directly as DER in BaseRequestHandler, removing the need for the patched tornado fork and tornado-m2crypto. Rewrite DIRACCAProxyProvider with cryptography, replacing the M2Crypto NID lookups with OID-based tables. The TLS floor is now 1.2 and the legacy default cipher list is no longer applied; ciphers can be pinned with DIRAC_SSL_CIPHERS (the old DIRAC_M2CRYPTO_SSL_CIPHERS is still honoured).
…edentials X509Chain.getCredentials already resolves DN (identity for proxies, subject otherwise), isProxy and isLimitedProxy, so both the DISET getPeerInfo and the Tornado _authzSSL only need to attach the chain to the credential dict instead of re-deriving these fields.
Building a client SSL context reads and parses the credential files for every RPC connection. Cache the contexts in a small LRU keyed on the credential file stat info (invalidated by proxy renewal), the CA location and the cipher configuration, with a TTL so changes in the CA directory are eventually picked up. Contexts are safe to share between connections and threads, as already done on the server side.
Reinstate, unchanged, the M2Crypto/pyasn1 based implementation that was removed by the migration to pyca/cryptography and the standard library ssl module: the DIRAC.Core.Security.m2crypto subpackage, the DISET M2SSLTransport/M2Utils modules and the default SSL cipher/method constants they rely on. The code is not used yet: a follow-up commit makes it selectable at runtime with DIRAC_USE_M2CRYPTO.
Move the pyca/cryptography based X509 modules into the DIRAC.Core.Security.pyca subpackage and select the implementation once, at import time, by extending the DIRAC.Core.Security search path with either the pyca or the m2crypto subpackage, based on the DIRAC_USE_M2CRYPTO environment variable (default No, i.e. pyca). This is the same pkgutil.extend_path mechanism that was used during the pyGSI to M2Crypto transition. The DISET SSLTransport module becomes a selector: the implementation agnostic delegate() and checkSanity() helpers stay there, while the standard library ssl implementation moves to StdSSLTransport, and the SSLTransport class and the getSSLContext/getPeerInfo helpers are picked between it and the M2SSLTransport/M2Utils ones. The HTTPS services honour the switch as well: TornadoServer only configures the tornado_m2crypto iostream and builds the dict style ssl_options when the fallback is enabled (so that the default path never imports M2Crypto or tornado_m2crypto, which remain optional dependencies, listed in the new m2crypto extra), and BaseRequestHandler._authzSSL and DIRACCAProxyProvider dispatch between the two implementations the same way.
Re-add the M2Crypto variants to the implementation matrices (X509CHAINTYPES, X509REQUESTTYPES, X509CERTTYPES and TRANSPORTTESTS). Both implementations are imported through their fully qualified subpackage paths so they can be tested in the same process, whatever DIRAC_USE_M2CRYPTO says; the M2 variants are skipped when M2Crypto or pyasn1 are not installed, as they are optional dependencies. The standard library ssl transport pair is skipped on python < 3.13, where SSLSocket.get_unverified_chain is not available.
Describe the switch between the default pyca/cryptography based implementation and the legacy M2Crypto one, re-add the entries for the M2Crypto specific environment variables it brings back, and mention the tornado fork and tornado-m2crypto requirements of the fallback in the Tornado services guide.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is a proof of concept for now.
BEGINRELEASENOTES
Thank you for writing the text to appear in the release notes. It will show up
exactly as it appears between the two bold lines
Please follow the template:
*Subsystem
NEW/CHANGE/FIX: explanation
For examples look into release.notes
ENDRELEASENOTES