diff --git a/README.md b/README.md index 5f023604..47decc64 100644 --- a/README.md +++ b/README.md @@ -535,5 +535,6 @@ API query: ## Additional chapters - [**Adding custom plugins**](documentation/adding_custom_plugins.md) - [**Configuration file**](documentation/configuration_file.md) +- [**Encrypting the connection to Zabbix**](TLS_ENCRYPTION.md) - [**Metrics**](documentation/metrics.md) - [**Tools**](documentation/tools.md) diff --git a/TLS_ENCRYPTION.md b/TLS_ENCRYPTION.md new file mode 100644 index 00000000..6b6272d9 --- /dev/null +++ b/TLS_ENCRYPTION.md @@ -0,0 +1,144 @@ +# Encrypting the connection to Zabbix (mamonsu 3.5.17.1) + +This document describes what changed compared to upstream **3.5.17**. + +## Why + +In 3.5.17 `ZbxSender` sends metrics to the Zabbix server over a plain TCP socket: + +```python +sock = socket.socket() +sock.connect((self.host, self.port)) +sock.sendall(packet) +``` + +When a host in Zabbix is set to **Connections from host: PSK**, the server rejects such connections and mamonsu cannot deliver a single metric, even though the stock `zabbix_sender` works with the same PSK settings. + +Starting with 3.5.17.1 mamonsu encrypts the connection itself, with no external processes and no additional Python packages. + +## What changed + +| File | Change | +| --- | --- | +| `mamonsu/lib/senders/tls.py` | new module: the TLS-PSK and TLS-cert transports | +| `mamonsu/lib/senders/zbx.py` | reads the new settings and picks the transport in `_connect()` | +| `mamonsu/lib/config.py` | defaults for the `tls_*` parameters | +| `mamonsu/lib/parser.py`, `mamonsu/lib/runner.py` | the `--zabbix-tls-*` command line options | +| `packaging/conf/example_linux.conf` | commented example of the settings | +| `documentation/configuration_file.md` | description of the parameters | +| `tests/unit/` | tests that need neither docker nor a Zabbix server | + +The wire format and the queue logic are untouched: `_send_data()` still sends `ZBXD\x01` + length + JSON. The only difference is where the socket comes from. + +## Compatibility + +**Without the new parameters the behaviour is exactly that of 3.5.17.** An existing `agent.conf` needs no changes: `tls_connect` defaults to `unencrypted`, and in that case the code path is the previous one. + +If the settings are wrong (unknown mode, missing identity or key file, unreadable file), the sender is **disabled with an error in the log** instead of falling back to an unencrypted connection: mamonsu never quietly sends metrics in the clear. + +## Configuring PSK + +In `/etc/mamonsu/agent.conf`: + +```ini +[zabbix] +address = zabbix-5 +port = 10051 +client = db2 + +tls_connect = psk +tls_psk_identity = PSK DB2 +tls_psk_file = /etc/zabbix/zabbix_agentd.psk +``` + +`tls_psk_identity` and `tls_psk_file` have to match what is configured for this host in Zabbix (and the `TLSPSKIdentity` / `TLSPSKFile` of the Zabbix agent, if one runs on the same host). + +How it works: + +* on Python **3.13 and newer** — the standard `ssl` module (`SSLContext.set_psk_client_callback`); +* on Python **3.7 to 3.12** — a TLS client of our own through `ctypes` to the system libssl. Only the public OpenSSL API is used and no CPython internals are touched, so the same code works on Astra Linux 1.7.6 (OpenSSL 1.1.1, `libssl.so.1.1`) and Astra Linux 1.8 (OpenSSL 3.x, `libssl.so.3`). + +A PSK connection is negotiated as **TLS 1.2**: TLS 1.3 carries the PSK through a different mechanism (`psk_use_session`), which is not implemented yet. Every Zabbix version that supports encryption accepts TLS 1.2. + +## Configuring certificates + +```ini +[zabbix] +tls_connect = cert +tls_ca_file = /etc/zabbix/ca.crt +tls_cert_file = /etc/zabbix/mamonsu.crt +tls_key_file = /etc/zabbix/mamonsu.key +# tls_crl_file = /etc/zabbix/ca.crl +# tls_server_cert_issuer = CN=Zabbix CA,O=Company +# tls_server_cert_subject = CN=zabbix server,O=Company +``` + +The server is validated the way the Zabbix agent does it: the chain is verified against the CA file, while the host name is **not** matched against the certificate — the server is pinned by its issuer and subject instead. Escaped commas in a DN are honoured (`CN=Company\, Inc`), the order of the attributes does not matter, and both short (`CN`, `O`, `OU`, `C`, ...) and long (`commonName`) names are accepted. + +The `cert` mode works on any Python 3.x and uses the standard library only. + +## Command line + +Any of the settings except the cipher ones (`tls_cipher_psk`, `tls_cipher_cert`) can be overridden without touching the config file, which is convenient for checking a setup: + +```bash +mamonsu -c /etc/mamonsu/agent.conf \ + --zabbix-tls-connect psk \ + --zabbix-tls-psk-identity 'PSK DB2' \ + --zabbix-tls-psk-file /etc/zabbix/zabbix_agentd.psk +``` + +The full list: `--zabbix-tls-connect`, `--zabbix-tls-psk-identity`, `--zabbix-tls-psk-file`, `--zabbix-tls-ca-file`, `--zabbix-tls-crl-file`, `--zabbix-tls-cert-file`, `--zabbix-tls-key-file`, `--zabbix-tls-server-cert-issuer`, `--zabbix-tls-server-cert-subject`. Only the options actually passed are overridden, the rest come from the config file. They work both for the daemon and for `mamonsu upload`. + +## Security + +* the PSK is read from the file once at startup and is kept in the memory of the process only; +* the content of the PSK file never reaches the log or the text of an error (there is a test for that); +* the PSK is not duplicated in `agent.conf`, which only holds the path to the file and the identity; +* the file has to be readable by the user mamonsu runs as (`mamonsu` in the systemd unit). + +## Testing + +The tests live in `tests/unit` and need neither docker nor Zabbix: + +```bash +python -m pytest tests/unit # with pytest +python3 tests/unit/test_tls_openssl.py # without it, e.g. on the monitored host +python3 tests/unit/test_zbx_sender_socket.py +``` + +What they cover: + +* a real TLS handshake against `openssl s_server`, both PSK and certificates, with data going both ways; +* a wrong PSK, an unknown CA and a certificate subject that does not match are all rejected; +* parsing of the PSK file, and the absence of the secret in error messages; +* parsing and comparison of distinguished names; +* the unencrypted path: the frame is byte for byte the previous one and `failed: N` is still detected; +* the choice of the transport and the command line overrides. + +Verified on Linux with Python 3.12 and OpenSSL 3.0.13 (the ctypes path) and on Windows with Python 3.12 (the cert mode on the standard library; the PSK tests are skipped there). The `psk` mode has also been run in a pilot on Astra Linux 1.7.6 with Python 3.7 and OpenSSL 1.1.1, the oldest combination the ctypes path is meant to cover, with metrics reaching a Zabbix 7.4 server. + +The reference check with the stock utility: + +```bash +zabbix_sender -c /etc/zabbix/zabbix_agent2.conf -z zabbix-5 -p 10051 -s db2 -k 'pgsql.ping[]' -o 1 +# processed: 1; failed: 0; total: 1 +``` + +After the settings are in place, mamonsu has to reach the same result without allowing `No encryption` on the Zabbix side. + +## Known limitations + +* TLS 1.3 with a PSK is not supported: the connection is pinned to TLS 1.2; +* the PSK file is read once at startup, so the agent has to be restarted after the key is changed; +* on Windows the `psk` mode requires Python 3.13 or newer, as there is no system libssl there; +* the `cert` mode has only been tested against `openssl s_server`, not against a production Zabbix server. + +## Building + +The version comes from `mamonsu/__init__.py`, and the same value is set in `packaging/debian/changelog` and `packaging/rpm/SPECS/mamonsu.spec` — `3.5.17.1`. + +```bash +make -f Makefile.pkg deb +make -f Makefile.pkg rpm +``` diff --git a/documentation/configuration_file.md b/documentation/configuration_file.md index 0cabd352..c3d9890d 100644 --- a/documentation/configuration_file.md +++ b/documentation/configuration_file.md @@ -99,6 +99,53 @@ The [zabbix] section provides connection settings for the Zabbix server and can         Default: 15 +**tls_connect** +        How _mamonsu_ connects to the Zabbix server: `unencrypted` for a plain TCP connection, `psk` for a TLS connection with a pre-shared key, or `cert` for a TLS connection with certificates. The value has to match the _Connections from host_ setting of this host in Zabbix. + +        A PSK connection is negotiated as TLS 1.2, which Zabbix accepts in every version that supports encryption; a certificate connection uses TLS 1.2 or newer. PSK needs no additional Python package: on Python 3.13 and newer the handshake uses the standard `ssl` module, on older versions it goes through the system libssl (OpenSSL 1.1.1 or 3.x) directly. + +        Default: unencrypted + +**tls_psk_identity** +        The PSK identity string, exactly as configured for this host in Zabbix. Required when tls_connect is psk. + +**tls_psk_file** +        Path to the file with the pre-shared key, a single line of hexadecimal digits — the same file the Zabbix agent uses in its TLSPSKFile parameter. The file is read once at startup and must be readable by the user _mamonsu_ runs as; its content is never written to the mamonsu log. Required when tls_connect is psk. + +**tls_cipher_psk** +        OpenSSL cipher string used to select the PSK cipher suites, for the rare case when the Zabbix server is restricted to a specific one. + +        Default: PSK + +**tls_ca_file** +        Path to the top-level CA certificate that signed the certificate of the Zabbix server. Required when tls_connect is cert. + +**tls_cert_file** +        Path to the certificate _mamonsu_ presents to the Zabbix server. Required when tls_connect is cert. + +**tls_key_file** +        Path to the private key of that certificate. Required when tls_connect is cert. + +**tls_crl_file** +        Path to a certificate revocation list. When set, the certificate of the Zabbix server is checked against it. + +**tls_server_cert_issuer** +        Expected issuer of the server certificate, for example `CN=Zabbix CA,O=Company`. The attributes listed here must all be present in the certificate; their order does not matter, and a comma inside a value is escaped with a backslash. As in the Zabbix agent, the host name is not matched against the certificate — the issuer and subject are what identify the server. + +**tls_server_cert_subject** +        Expected subject of the server certificate, in the same format as tls_server_cert_issuer. + +**tls_cipher_cert** +        OpenSSL cipher string used to select the certificate cipher suites, for the rare case when the Zabbix server is restricted to a specific one. + +

 

+ +All of the tls_* parameters except the cipher ones (tls_cipher_psk, tls_cipher_cert) can be overridden from the command line, which is convenient for checking a setting without editing the config file: + +```bash +mamonsu -c /etc/mamonsu/agent.conf --zabbix-tls-connect psk --zabbix-tls-psk-identity 'PSK 001' --zabbix-tls-psk-file /etc/zabbix/zabbix_agentd.psk +``` +

 

**[agent]** diff --git a/mamonsu/__init__.py b/mamonsu/__init__.py index 1ecb369d..fd86da8a 100644 --- a/mamonsu/__init__.py +++ b/mamonsu/__init__.py @@ -1,7 +1,7 @@ __author__ = 'Dmitry Vasilyev' __author_email__ = 'info@postgrespro.ru' __description__ = 'Monitoring agent for PostgreSQL' -__version__ = '3.5.17' +__version__ = '3.5.17.1' __licence__ = 'BSD' __url__ = 'https://github.com/postgrespro/mamonsu' diff --git a/mamonsu/lib/config.py b/mamonsu/lib/config.py index 4d358b84..79efc852 100644 --- a/mamonsu/lib/config.py +++ b/mamonsu/lib/config.py @@ -56,6 +56,18 @@ def __init__(self, cfg_file=None, plugin_directories=None): config.set('zabbix', 'port', str(10051)) config.set('zabbix', 'timeout', str(15)) config.set('zabbix', 're_send', str(False)) + # unencrypted keeps the plain TCP connection used before 3.5.18 + config.set('zabbix', 'tls_connect', 'unencrypted') + config.set('zabbix', 'tls_psk_identity', str(None)) + config.set('zabbix', 'tls_psk_file', str(None)) + config.set('zabbix', 'tls_cipher_psk', str(None)) + config.set('zabbix', 'tls_cipher_cert', str(None)) + config.set('zabbix', 'tls_ca_file', str(None)) + config.set('zabbix', 'tls_crl_file', str(None)) + config.set('zabbix', 'tls_cert_file', str(None)) + config.set('zabbix', 'tls_key_file', str(None)) + config.set('zabbix', 'tls_server_cert_issuer', str(None)) + config.set('zabbix', 'tls_server_cert_subject', str(None)) config.add_section('metric_log') config.set('metric_log', 'enabled', str(False)) diff --git a/mamonsu/lib/parser.py b/mamonsu/lib/parser.py index 97726032..4e51f17b 100644 --- a/mamonsu/lib/parser.py +++ b/mamonsu/lib/parser.py @@ -15,6 +15,19 @@ usage_msg += """ -d daemonize """ +usage_msg += """ Encryption of the connection to the Zabbix server, overrides the + [zabbix] section of the config file: + --zabbix-tls-connect + --zabbix-tls-psk-identity + --zabbix-tls-psk-file + --zabbix-tls-ca-file + --zabbix-tls-crl-file + --zabbix-tls-cert-file + --zabbix-tls-key-file + --zabbix-tls-server-cert-issuer + --zabbix-tls-server-cert-subject +""" + usage_msg += """ --version prints version information, then exits --help shows this help message, then exits @@ -296,6 +309,19 @@ def parse_args(): parser.add_option('--zabbix-file', dest='zabbix_file', default='/var/log/mamonsu/localhost.log') # log level to send metrics parser.add_option('--zabbix-log-level', dest='zabbix_log_level', default='INFO') + # encryption of the connection to the Zabbix server, overrides [zabbix] of the + # config file; without these options the settings from the config file are used + parser.add_option('--zabbix-tls-connect', dest='zabbix_tls_connect', default=None) + parser.add_option('--zabbix-tls-psk-identity', dest='zabbix_tls_psk_identity', default=None) + parser.add_option('--zabbix-tls-psk-file', dest='zabbix_tls_psk_file', default=None) + parser.add_option('--zabbix-tls-ca-file', dest='zabbix_tls_ca_file', default=None) + parser.add_option('--zabbix-tls-crl-file', dest='zabbix_tls_crl_file', default=None) + parser.add_option('--zabbix-tls-cert-file', dest='zabbix_tls_cert_file', default=None) + parser.add_option('--zabbix-tls-key-file', dest='zabbix_tls_key_file', default=None) + parser.add_option( + '--zabbix-tls-server-cert-issuer', dest='zabbix_tls_server_cert_issuer', default=None) + parser.add_option( + '--zabbix-tls-server-cert-subject', dest='zabbix_tls_server_cert_subject', default=None) # check unknown options args, commands = parser.parse_args() diff --git a/mamonsu/lib/runner.py b/mamonsu/lib/runner.py index e79c1747..32c93d86 100644 --- a/mamonsu/lib/runner.py +++ b/mamonsu/lib/runner.py @@ -19,6 +19,23 @@ from mamonsu.plugins.system.linux.scripts import Scripts +def apply_zabbix_tls_args(cfg, args): + """Let the command line override the TLS settings of the config file.""" + options = ( + ('tls_connect', args.zabbix_tls_connect), + ('tls_psk_identity', args.zabbix_tls_psk_identity), + ('tls_psk_file', args.zabbix_tls_psk_file), + ('tls_ca_file', args.zabbix_tls_ca_file), + ('tls_crl_file', args.zabbix_tls_crl_file), + ('tls_cert_file', args.zabbix_tls_cert_file), + ('tls_key_file', args.zabbix_tls_key_file), + ('tls_server_cert_issuer', args.zabbix_tls_server_cert_issuer), + ('tls_server_cert_subject', args.zabbix_tls_server_cert_subject)) + for key, value in options: + if value is not None: + cfg.config.set('zabbix', key, value) + + def start(): def quit_handler(_signo=None, _stack_frame=None): logging.info("Bye bye!") @@ -67,6 +84,7 @@ def quit_handler(_signo=None, _stack_frame=None): cfg.config.set('zabbix', 'port', args.zabbix_port) cfg.config.set('zabbix', 'client', args.zabbix_client) cfg.config.set('log', 'level', args.zabbix_log_level) + apply_zabbix_tls_args(cfg, args) supervisor = Supervisor(cfg) supervisor.send_file_zabbix(cfg, args.zabbix_file) @@ -186,6 +204,7 @@ def quit_handler(_signo=None, _stack_frame=None): if len(commands) > 0: print_total_help() cfg = Config(args.config_file, args.plugins_dirs) + apply_zabbix_tls_args(cfg, args) # simple daemon if args.daemon: diff --git a/mamonsu/lib/senders/tls.py b/mamonsu/lib/senders/tls.py new file mode 100644 index 00000000..60ebadd7 --- /dev/null +++ b/mamonsu/lib/senders/tls.py @@ -0,0 +1,440 @@ +# -*- coding: utf-8 -*- + +# TLS-PSK transport for the Zabbix sender. +# +# ssl.SSLContext.set_psk_client_callback() appeared in Python 3.13, while the +# distributions mamonsu is packaged for still ship 3.7 (Astra Linux 1.7) or +# 3.11 (Astra Linux 1.8). For those the handshake is done through libssl +# directly: only the public OpenSSL API is used, no CPython internals, so the +# same code works with libssl.so.1.1 (OpenSSL 1.1.1) and libssl.so.3 +# (OpenSSL 3.x). + +import ctypes +import ctypes.util +import os +import socket +import ssl +import struct +import sys + +# TLS 1.3 negotiates PSK through a different callback (psk_use_session), so the +# PSK connection is pinned to TLS 1.2, which every Zabbix server built with +# OpenSSL supports. +TLS1_2_VERSION = 0x0303 + +# ssl/ssl.h +SSL_CTRL_SET_MIN_PROTO_VERSION = 123 +SSL_CTRL_SET_MAX_PROTO_VERSION = 124 + +SSL_ERROR_NONE = 0 +SSL_ERROR_SSL = 1 +SSL_ERROR_WANT_READ = 2 +SSL_ERROR_WANT_WRITE = 3 +SSL_ERROR_SYSCALL = 5 +SSL_ERROR_ZERO_RETURN = 6 + +# minimal length Zabbix accepts for a PSK: 128 bits, i.e. 32 hex digits +MIN_PSK_HEX_LEN = 32 + +DEFAULT_PSK_CIPHERS = 'PSK' + +# short names of the DN attributes Zabbix uses in TLSServerCertIssuer / +# TLSServerCertSubject, mapped to the long names OpenSSL reports +DN_ATTRIBUTES = { + 'CN': 'commonName', + 'O': 'organizationName', + 'OU': 'organizationalUnitName', + 'C': 'countryName', + 'ST': 'stateOrProvinceName', + 'L': 'localityName', + 'DC': 'domainComponent', + 'STREET': 'streetAddress', + 'UID': 'userId', +} + +# OpenSSL 1.0.x is not listed: it has neither TLS_client_method() nor +# SSL_CTRL_SET_MIN/MAX_PROTO_VERSION, so it can't be used here anyway +LIBSSL_NAMES = ['libssl.so.3', 'libssl.so.1.1', 'libssl.so'] +LIBCRYPTO_NAMES = ['libcrypto.so.3', 'libcrypto.so.1.1', 'libcrypto.so'] + + +class TLSError(Exception): + pass + + +def stdlib_psk_supported(): + return hasattr(ssl.SSLContext, 'set_psk_client_callback') + + +def read_psk_file(path): + """Read a Zabbix PSK file (a single line of hex digits) and return raw bytes. + + The value itself is never logged or included in exception messages. + """ + try: + with open(path, 'r') as fd: + content = fd.read() + except (IOError, OSError) as e: + raise TLSError('can\'t read PSK file {0}: {1}'.format(path, e.strerror or e)) + psk_hex = ''.join(content.split()) + if len(psk_hex) < MIN_PSK_HEX_LEN: + raise TLSError( + 'PSK file {0} contains less than {1} hex digits'.format(path, MIN_PSK_HEX_LEN)) + if len(psk_hex) % 2 != 0: + raise TLSError('PSK file {0} contains an odd number of hex digits'.format(path)) + try: + psk = bytes(bytearray.fromhex(psk_hex)) + except (ValueError, TypeError): + raise TLSError('PSK file {0} is not a valid hex string'.format(path)) + return psk + + +def connect_psk(host, port, identity, psk, timeout, ciphers=None): + """Open a TLS-PSK connection and return a socket-like object. + + The result implements sendall()/recv()/close(), which is all the Zabbix + sender needs from a socket. + """ + ciphers = ciphers or DEFAULT_PSK_CIPHERS + if not identity: + raise TLSError('tls_psk_identity is not set') + if not psk: + raise TLSError('PSK is empty') + sock = socket.create_connection((host, port), timeout=timeout) + try: + if stdlib_psk_supported(): + return _wrap_stdlib(sock, identity, psk, ciphers) + return OpenSSLSocket(sock, identity, psk, timeout, ciphers) + except Exception: + sock.close() + raise + + +def _wrap_stdlib(sock, identity, psk, ciphers): + context = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT) + # a PSK handshake carries no certificates: both sides are authenticated by + # the pre-shared key itself, and the connection fails if the keys differ + context.check_hostname = False + context.verify_mode = ssl.CERT_NONE + context.minimum_version = ssl.TLSVersion.TLSv1_2 + context.maximum_version = ssl.TLSVersion.TLSv1_2 + try: + context.set_ciphers(ciphers) + except ssl.SSLError as e: + raise TLSError('no PSK cipher suite available for "{0}": {1}'.format(ciphers, e)) + context.set_psk_client_callback(lambda hint: (identity, psk)) + try: + return context.wrap_socket(sock) + except ssl.SSLError as e: + raise TLSError('TLS handshake failed: {0}'.format(e)) + + +def connect_cert(host, port, ca_file, cert_file, key_file, timeout, + crl_file=None, ciphers=None, + server_cert_issuer=None, server_cert_subject=None): + """Open a TLS connection authenticated by certificates. + + Like the Zabbix agent, the server is trusted through the CA file and, + optionally, pinned by the issuer and subject of its certificate; the host + name is deliberately not matched against the certificate, because Zabbix + certificates are issued for the server, not for the address the agent uses. + """ + for name, path in (('tls_ca_file', ca_file), + ('tls_cert_file', cert_file), + ('tls_key_file', key_file)): + if not path: + raise TLSError('{0} is not set'.format(name)) + context = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT) + context.check_hostname = False + context.verify_mode = ssl.CERT_REQUIRED + context.minimum_version = ssl.TLSVersion.TLSv1_2 + try: + context.load_verify_locations(cafile=ca_file) + if crl_file: + context.load_verify_locations(cafile=crl_file) + context.verify_flags |= ssl.VERIFY_CRL_CHECK_LEAF + context.load_cert_chain(certfile=cert_file, keyfile=key_file) + except (ssl.SSLError, IOError, OSError) as e: + raise TLSError("can't load TLS certificates: {0}".format(e)) + if ciphers: + try: + context.set_ciphers(ciphers) + except ssl.SSLError as e: + raise TLSError('no cipher suite available for "{0}": {1}'.format(ciphers, e)) + sock = socket.create_connection((host, port), timeout=timeout) + try: + connection = context.wrap_socket(sock) + except ssl.SSLError as e: + sock.close() + raise TLSError('TLS handshake failed: {0}'.format(e)) + except Exception: + sock.close() + raise + try: + peer = connection.getpeercert() + check_dn(peer.get('issuer'), server_cert_issuer, 'issuer') + check_dn(peer.get('subject'), server_cert_subject, 'subject') + except Exception: + connection.close() + raise + return connection + + +def parse_dn(dn): + """Split "CN=zabbix,O=company" into pairs, honouring backslash escapes.""" + pairs, item, escaped = [], '', False + for char in dn: + if escaped: + item += char + escaped = False + elif char == '\\': + escaped = True + elif char == ',': + pairs.append(item) + item = '' + else: + item += char + pairs.append(item) + result = [] + for pair in pairs: + if not pair.strip(): + continue + if '=' not in pair: + raise TLSError("can't parse {0} as a distinguished name".format(dn)) + key, value = pair.split('=', 1) + key = key.strip() + result.append((DN_ATTRIBUTES.get(key.upper(), key), value.strip())) + return result + + +def check_dn(peer_dn, expected, what): + """Verify that every attribute of the configured DN is in the peer's one.""" + if not expected: + return + peer = set() + for rdn in peer_dn or (): + for key, value in rdn: + peer.add((key, value)) + missing = [pair for pair in parse_dn(expected) if pair not in peer] + if missing: + raise TLSError( + 'certificate {0} does not match tls_server_cert_{0}: {1}'.format( + what, ', '.join('{0}={1}'.format(*pair) for pair in missing))) + + +def _load_library(names, what): + errors = [] + candidates = list(names) + found = ctypes.util.find_library(what) + if found is not None and found not in candidates: + candidates.append(found) + for name in candidates: + try: + return ctypes.CDLL(name, use_errno=True) + except OSError as e: + errors.append('{0}: {1}'.format(name, e)) + raise TLSError('can\'t load {0} ({1})'.format(what, '; '.join(errors))) + + +_libssl = None +_libcrypto = None + + +def _libs(): + """Load libssl/libcrypto once and declare the prototypes that are used.""" + global _libssl, _libcrypto + if _libssl is not None: + return _libssl, _libcrypto + if sys.platform in ('win32', 'darwin'): + # there is no loadable system libssl on these platforms (the macOS + # /usr/lib/libssl.dylib stub aborts the process when loaded) + raise TLSError('TLS-PSK requires Python 3.13 or newer on this platform') + libssl = _load_library(LIBSSL_NAMES, 'ssl') + libcrypto = _load_library(LIBCRYPTO_NAMES, 'crypto') + + # looking a symbol up raises AttributeError when the loaded library does not + # export it, which is what happens on OpenSSL 1.0.x: report that plainly + # instead of letting a bare AttributeError escape + try: + libssl.TLS_client_method.restype = ctypes.c_void_p + libssl.SSL_CTX_new.argtypes = [ctypes.c_void_p] + libssl.SSL_CTX_new.restype = ctypes.c_void_p + libssl.SSL_CTX_free.argtypes = [ctypes.c_void_p] + libssl.SSL_CTX_free.restype = None + libssl.SSL_CTX_ctrl.argtypes = [ctypes.c_void_p, ctypes.c_int, ctypes.c_long, ctypes.c_void_p] + libssl.SSL_CTX_ctrl.restype = ctypes.c_long + libssl.SSL_CTX_set_cipher_list.argtypes = [ctypes.c_void_p, ctypes.c_char_p] + libssl.SSL_CTX_set_cipher_list.restype = ctypes.c_int + libssl.SSL_CTX_set_psk_client_callback.argtypes = [ctypes.c_void_p, PSK_CLIENT_CALLBACK] + libssl.SSL_CTX_set_psk_client_callback.restype = None + libssl.SSL_new.argtypes = [ctypes.c_void_p] + libssl.SSL_new.restype = ctypes.c_void_p + libssl.SSL_free.argtypes = [ctypes.c_void_p] + libssl.SSL_free.restype = None + libssl.SSL_set_fd.argtypes = [ctypes.c_void_p, ctypes.c_int] + libssl.SSL_set_fd.restype = ctypes.c_int + libssl.SSL_connect.argtypes = [ctypes.c_void_p] + libssl.SSL_connect.restype = ctypes.c_int + libssl.SSL_read.argtypes = [ctypes.c_void_p, ctypes.c_void_p, ctypes.c_int] + libssl.SSL_read.restype = ctypes.c_int + libssl.SSL_write.argtypes = [ctypes.c_void_p, ctypes.c_void_p, ctypes.c_int] + libssl.SSL_write.restype = ctypes.c_int + libssl.SSL_shutdown.argtypes = [ctypes.c_void_p] + libssl.SSL_shutdown.restype = ctypes.c_int + libssl.SSL_get_error.argtypes = [ctypes.c_void_p, ctypes.c_int] + libssl.SSL_get_error.restype = ctypes.c_int + libssl.SSL_get_version.argtypes = [ctypes.c_void_p] + libssl.SSL_get_version.restype = ctypes.c_char_p + + libcrypto.ERR_get_error.restype = ctypes.c_ulong + libcrypto.ERR_error_string_n.argtypes = [ctypes.c_ulong, ctypes.c_char_p, ctypes.c_size_t] + libcrypto.ERR_error_string_n.restype = None + libcrypto.ERR_clear_error.restype = None + except AttributeError as e: + raise TLSError( + 'system OpenSSL is too old, version 1.1.0 or newer is required ({0})'.format(e)) + + _libssl, _libcrypto = libssl, libcrypto + return _libssl, _libcrypto + + +# unsigned int (*)(SSL *ssl, const char *hint, char *identity, +# unsigned int max_identity_len, +# unsigned char *psk, unsigned int max_psk_len) +PSK_CLIENT_CALLBACK = ctypes.CFUNCTYPE( + ctypes.c_uint, + ctypes.c_void_p, + ctypes.c_char_p, + ctypes.POINTER(ctypes.c_char), + ctypes.c_uint, + ctypes.POINTER(ctypes.c_ubyte), + ctypes.c_uint) + + +def _errors(): + """Drain the OpenSSL error queue into a printable string.""" + _, libcrypto = _libs() + messages = [] + while True: + code = libcrypto.ERR_get_error() + if code == 0: + break + buf = ctypes.create_string_buffer(256) + libcrypto.ERR_error_string_n(code, buf, len(buf)) + messages.append(buf.value.decode('utf-8', 'replace')) + return ', '.join(messages) or 'unknown error' + + +def _timeval(seconds): + seconds = int(seconds) + # struct timeval { time_t tv_sec; suseconds_t tv_usec; } + return struct.pack('@ll', seconds, 0) + + +class OpenSSLSocket(object): + """TLS-PSK connection over libssl, with the socket API the sender uses.""" + + def __init__(self, sock, identity, psk, timeout, ciphers): + libssl, _ = _libs() + self._libssl = libssl + self._sock = sock + self._ssl = None + self._ctx = None + # OpenSSL does the I/O itself, so the socket must stay in blocking mode + # and the timeout has to be enforced by the kernel + sock.settimeout(None) + if timeout: + sock.setsockopt(socket.SOL_SOCKET, socket.SO_RCVTIMEO, _timeval(timeout)) + sock.setsockopt(socket.SOL_SOCKET, socket.SO_SNDTIMEO, _timeval(timeout)) + + identity = identity.encode('utf-8') if not isinstance(identity, bytes) else identity + + def psk_client_callback(_ssl, _hint, identity_buf, max_identity_len, psk_buf, max_psk_len): + if len(identity) + 1 > max_identity_len or len(psk) > max_psk_len: + return 0 + ctypes.memmove(identity_buf, identity + b'\0', len(identity) + 1) + ctypes.memmove(psk_buf, psk, len(psk)) + return len(psk) + + # the callback must outlive the context, otherwise libssl calls freed memory + self._callback = PSK_CLIENT_CALLBACK(psk_client_callback) + + ctx = libssl.SSL_CTX_new(libssl.TLS_client_method()) + if not ctx: + raise TLSError('SSL_CTX_new failed: {0}'.format(_errors())) + self._ctx = ctx + libssl.SSL_CTX_ctrl(ctx, SSL_CTRL_SET_MIN_PROTO_VERSION, TLS1_2_VERSION, None) + libssl.SSL_CTX_ctrl(ctx, SSL_CTRL_SET_MAX_PROTO_VERSION, TLS1_2_VERSION, None) + if libssl.SSL_CTX_set_cipher_list(ctx, ciphers.encode('utf-8')) != 1: + self.close() + raise TLSError('no PSK cipher suite available for "{0}": {1}'.format(ciphers, _errors())) + libssl.SSL_CTX_set_psk_client_callback(ctx, self._callback) + + ssl_obj = libssl.SSL_new(ctx) + if not ssl_obj: + self.close() + raise TLSError('SSL_new failed: {0}'.format(_errors())) + self._ssl = ssl_obj + if libssl.SSL_set_fd(ssl_obj, sock.fileno()) != 1: + self.close() + raise TLSError('SSL_set_fd failed: {0}'.format(_errors())) + ret = libssl.SSL_connect(ssl_obj) + if ret != 1: + error = self._describe(ret) + self.close() + raise TLSError('TLS handshake failed: {0}'.format(error)) + + def version(self): + if self._ssl is None: + return None + version = self._libssl.SSL_get_version(self._ssl) + return version.decode('utf-8') if version else None + + def _describe(self, ret): + code = self._libssl.SSL_get_error(self._ssl, ret) + if code in (SSL_ERROR_WANT_READ, SSL_ERROR_WANT_WRITE): + # the socket BIO reports the SO_RCVTIMEO/SO_SNDTIMEO expiry + # as a retryable read/write, not as a syscall error + return 'timed out' + if code == SSL_ERROR_SYSCALL: + errno = ctypes.get_errno() + if errno in (11, 110): # EAGAIN, ETIMEDOUT + return 'timed out' + return 'system error: {0}'.format(os.strerror(errno) if errno else 'connection closed') + if code == SSL_ERROR_ZERO_RETURN: + return 'connection closed by peer' + return _errors() + + def sendall(self, data): + buf = ctypes.create_string_buffer(bytes(data), len(data)) + sent, total = 0, len(data) + while sent < total: + written = self._libssl.SSL_write( + self._ssl, ctypes.byref(buf, sent), total - sent) + if written <= 0: + raise TLSError('TLS write failed: {0}'.format(self._describe(written))) + sent += written + + def recv(self, count): + buf = ctypes.create_string_buffer(count) + read = self._libssl.SSL_read(self._ssl, buf, count) + if read < 0: + raise TLSError('TLS read failed: {0}'.format(self._describe(read))) + if read == 0: + return b'' + return buf.raw[:read] + + def close(self): + if self._ssl is not None: + try: + self._libssl.SSL_shutdown(self._ssl) + except Exception: + pass + self._libssl.SSL_free(self._ssl) + self._ssl = None + if self._ctx is not None: + self._libssl.SSL_CTX_free(self._ctx) + self._ctx = None + if self._sock is not None: + self._sock.close() + self._sock = None diff --git a/mamonsu/lib/senders/zbx.py b/mamonsu/lib/senders/zbx.py index 039a41e0..1d7b270c 100644 --- a/mamonsu/lib/senders/zbx.py +++ b/mamonsu/lib/senders/zbx.py @@ -10,9 +10,15 @@ from mamonsu.lib.plugin import Plugin from mamonsu.lib.queue import Queue +from mamonsu.lib.senders import tls from itertools import islice +TLS_UNENCRYPTED = 'unencrypted' +TLS_PSK = 'psk' +TLS_CERT = 'cert' + + class ZbxSender(Plugin): Interval = 10 _sender = True @@ -32,6 +38,7 @@ def __init__(self, config): self.queue = Queue() self.log = logging.getLogger( 'ZBX-{0}:{1}'.format(self.host, self.port)) + self._setup_tls(config) def send(self, key, value, host=None, clock=None): if host is None: @@ -110,14 +117,94 @@ def send_file_to_zabbix(self, path): if not lines: break + def _setup_tls(self, config): + """Read TLS settings. Without them the sender behaves exactly as before.""" + # raw=True: identities, distinguished names and paths are free text, + # a '%' in them must not trigger configparser interpolation + self.tls_connect = (config.fetch('zabbix', 'tls_connect', raw=True) or TLS_UNENCRYPTED).lower() + self.tls_psk_identity = config.fetch('zabbix', 'tls_psk_identity', raw=True) + self.tls_psk_file = config.fetch('zabbix', 'tls_psk_file', raw=True) + self.tls_cipher_psk = config.fetch('zabbix', 'tls_cipher_psk', raw=True) + self.tls_cipher_cert = config.fetch('zabbix', 'tls_cipher_cert', raw=True) + self.tls_ca_file = config.fetch('zabbix', 'tls_ca_file', raw=True) + self.tls_crl_file = config.fetch('zabbix', 'tls_crl_file', raw=True) + self.tls_cert_file = config.fetch('zabbix', 'tls_cert_file', raw=True) + self.tls_key_file = config.fetch('zabbix', 'tls_key_file', raw=True) + self.tls_server_cert_issuer = config.fetch('zabbix', 'tls_server_cert_issuer', raw=True) + self.tls_server_cert_subject = config.fetch('zabbix', 'tls_server_cert_subject', raw=True) + self._psk = None + if self.tls_connect == TLS_UNENCRYPTED: + return + # a misconfigured encrypted sender must never quietly fall back to + # plaintext, so the plugin is disabled instead + if self.tls_connect not in (TLS_PSK, TLS_CERT): + self._disable( + 'unknown tls_connect value "{0}", expected one of: {1}'.format( + self.tls_connect, ', '.join([TLS_UNENCRYPTED, TLS_PSK, TLS_CERT]))) + return + if self.tls_connect == TLS_CERT: + missing = [name for name, value in ( + ('tls_ca_file', self.tls_ca_file), + ('tls_cert_file', self.tls_cert_file), + ('tls_key_file', self.tls_key_file)) if not value] + if missing: + self._disable( + 'tls_connect = cert requires {0}'.format(', '.join(missing))) + return + self.log.info('sending metrics over TLS with a certificate') + return + if not self.tls_psk_identity or not self.tls_psk_file: + self._disable( + 'tls_connect = psk requires both tls_psk_identity and tls_psk_file') + return + try: + self._psk = tls.read_psk_file(self.tls_psk_file) + except tls.TLSError as e: + self._disable('{0}'.format(e)) + return + self.log.info( + 'sending metrics over TLS-PSK, identity: {0}'.format(self.tls_psk_identity)) + + def _disable(self, reason): + self._enabled = False + self.log.error(reason) + + def _connect(self): + if self.tls_connect == TLS_PSK: + return tls.connect_psk( + self.host, self.port, self.tls_psk_identity, self._psk, + int(self.timeout), self.tls_cipher_psk) + if self.tls_connect == TLS_CERT: + return tls.connect_cert( + self.host, self.port, self.tls_ca_file, self.tls_cert_file, + self.tls_key_file, int(self.timeout), + crl_file=self.tls_crl_file, ciphers=self.tls_cipher_cert, + server_cert_issuer=self.tls_server_cert_issuer, + server_cert_subject=self.tls_server_cert_subject) + if self.tls_connect != TLS_UNENCRYPTED: + # _setup_tls() disables the plugin, but send_file_to_zabbix() does + # not honour that flag - a misconfigured sender must fail here too + # rather than fall through to an unencrypted connection + raise tls.TLSError( + 'unknown tls_connect value "{0}", refusing to send data' + ' unencrypted'.format(self.tls_connect)) + sock = socket.socket() + try: + sock.settimeout(int(self.timeout)) + sock.connect((self.host, self.port)) + except Exception: + # a failed connect() would otherwise leak the file descriptor + # until the socket is garbage collected + sock.close() + raise + return sock + def _send_data(self, data): sent_all = True data_len = struct.pack(' - 3.5.17.1-1 +- send metrics to the Zabbix server over TLS with a pre-shared key or a certificate + * Thu Jul 23 2026 Maxim Styushin - 3.5.17-1 - prevent Instance plugin Items list mutation on each run() cycle (#233); diff --git a/tests/README.md b/tests/README.md index d82efad1..bcbe5984 100644 --- a/tests/README.md +++ b/tests/README.md @@ -11,6 +11,16 @@ Mamonsu testing with different Postgres version, different operation systems(not pip3 install -e requirement.txt ``` +## Unit tests + +The tests under `tests/unit` need neither docker nor a Zabbix server and are run from the repository root: + +```bash +python -m pytest tests/unit +``` + +`tests/unit/test_tls_openssl.py` checks the TLS-PSK handshake against `openssl s_server`, so it is skipped when the openssl binary is missing. Both files can also be run directly with `python3`, which is handy on a host that has no pytest installed. + ## Usage/Examples You can simly run tests with only pytest mark "bash" and it will be ran with Postgres version from env variable POSTGRES_VERSION which is specified in .env file diff --git a/tests/unit/test_tls_openssl.py b/tests/unit/test_tls_openssl.py new file mode 100644 index 00000000..3b375c5e --- /dev/null +++ b/tests/unit/test_tls_openssl.py @@ -0,0 +1,250 @@ +# -*- coding: utf-8 -*- + +"""TLS handshakes against a real TLS server ("openssl s_server"). + +These are the only tests that exercise the transports end to end, so they are +worth running on every platform mamonsu is packaged for. They need the openssl +binary and a loopback socket, no Zabbix server and no docker. + + python -m pytest tests/unit/test_tls_openssl.py # with pytest + python3 tests/unit/test_tls_openssl.py # without it +""" + +import os +import socket +import subprocess +import sys +import time + +sys.path.insert(0, os.path.join(os.path.dirname(os.path.abspath(__file__)), '..', '..')) + +from mamonsu.lib.senders import tls # noqa: E402 + +try: + import pytest +except ImportError: # the file is also runnable as a plain script + pytest = None + +PSK_HEX = 'd1e4b7a9c2f30516a8b7c6d5e4f30219a8b7c6d5e4f302198a7b6c5d4e3f2011' +PSK_BYTES = bytes(bytearray.fromhex(PSK_HEX)) +WRONG_PSK = bytes(bytearray.fromhex('00' * 32)) +IDENTITY = 'PSK 001' + +CA_DN = 'CN=Mamonsu Test CA,O=Mamonsu' +SERVER_DN = 'CN=zabbix server,O=Mamonsu' +CLIENT_DN = 'CN=mamonsu,O=Mamonsu' + + +def no_openssl_binary(): + try: + subprocess.check_output(['openssl', 'version'], stderr=subprocess.STDOUT) + except (OSError, subprocess.CalledProcessError): + return 'the openssl binary is not available' + return None + + +def no_libssl(): + if sys.platform == 'win32': + return 'the libssl transport is not used on Windows' + return no_openssl_binary() + + +def skip_unless(reason_func): + """Skip under pytest, and let main() report the same reason without it.""" + def decorator(function): + function.skip_reason = reason_func + if pytest is None: + return function + reason = reason_func() + return pytest.mark.skipif(reason is not None, reason=reason or '')(function) + return decorator + + +def free_port(): + sock = socket.socket() + try: + sock.bind(('127.0.0.1', 0)) + return sock.getsockname()[1] + finally: + sock.close() + + +def start_server(port, options): + """Start an s_server that echoes back reversed text.""" + process = subprocess.Popen( + ['openssl', 's_server', '-accept', str(port), '-tls1_2', '-rev', '-quiet'] + options, + stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL) + deadline = time.time() + 10 + while time.time() < deadline: + probe = socket.socket() + try: + probe.settimeout(0.5) + probe.connect(('127.0.0.1', port)) + return process + except socket.error: + time.sleep(0.1) + finally: + probe.close() + process.kill() + raise AssertionError('openssl s_server did not start') + + +def stop_server(process): + if process.poll() is None: + process.kill() + process.wait() + + +def openssl(*args): + subprocess.check_output(['openssl'] + list(args), stderr=subprocess.STDOUT) + + +def make_certificates(directory): + """Issue a CA, a server and a client certificate, as Zabbix would need.""" + path = lambda name: os.path.join(str(directory), name) # noqa: E731 + + def subject(dn): + # openssl wants /CN=x/O=y, the config file format is CN=x,O=y + return '/' + '/'.join(reversed(dn.split(','))) + + openssl('req', '-x509', '-newkey', 'rsa:2048', '-nodes', '-days', '1', + '-keyout', path('ca.key'), '-out', path('ca.crt'), '-subj', subject(CA_DN)) + for name, dn in (('server', SERVER_DN), ('client', CLIENT_DN)): + openssl('req', '-newkey', 'rsa:2048', '-nodes', + '-keyout', path(name + '.key'), '-out', path(name + '.csr'), + '-subj', subject(dn)) + openssl('x509', '-req', '-in', path(name + '.csr'), '-days', '1', + '-CA', path('ca.crt'), '-CAkey', path('ca.key'), '-CAcreateserial', + '-out', path(name + '.crt')) + return path + + +# PSK + + +@skip_unless(no_libssl) +def test_psk_handshake_and_data(): + """The connection is established and carries data in both directions.""" + port = free_port() + process = start_server(port, ['-nocert', '-psk_identity', IDENTITY, '-psk', PSK_HEX]) + try: + sock = tls.connect_psk('127.0.0.1', port, IDENTITY, PSK_BYTES, 10) + try: + assert sock.version() == 'TLSv1.2' + sock.sendall(b'ping\n') + assert sock.recv(64).strip() == b'gnip' + finally: + sock.close() + finally: + stop_server(process) + + +@skip_unless(no_libssl) +def test_wrong_psk_is_reported(): + """A key mismatch fails the handshake instead of sending anything.""" + port = free_port() + process = start_server(port, ['-nocert', '-psk_identity', IDENTITY, '-psk', PSK_HEX]) + try: + try: + tls.connect_psk('127.0.0.1', port, IDENTITY, WRONG_PSK, 10) + except tls.TLSError as error: + assert 'handshake failed' in str(error) + else: + raise AssertionError('a wrong PSK must not produce a connection') + finally: + stop_server(process) + + +# certificates + + +def cert_server(port, path): + return start_server(port, [ + '-cert', path('server.crt'), '-key', path('server.key'), + '-CAfile', path('ca.crt'), '-Verify', '1']) + + +@skip_unless(no_openssl_binary) +def test_cert_handshake_and_data(tmp_path): + path = make_certificates(tmp_path) + port = free_port() + process = cert_server(port, path) + try: + sock = tls.connect_cert( + '127.0.0.1', port, path('ca.crt'), path('client.crt'), path('client.key'), 10, + server_cert_issuer=CA_DN, server_cert_subject=SERVER_DN) + try: + sock.sendall(b'ping\n') + assert sock.recv(64).strip() == b'gnip' + finally: + sock.close() + finally: + stop_server(process) + + +@skip_unless(no_openssl_binary) +def test_cert_subject_mismatch_is_reported(tmp_path): + path = make_certificates(tmp_path) + port = free_port() + process = cert_server(port, path) + try: + try: + tls.connect_cert( + '127.0.0.1', port, path('ca.crt'), path('client.crt'), path('client.key'), 10, + server_cert_subject='CN=someone else,O=Mamonsu') + except tls.TLSError as error: + assert 'does not match tls_server_cert_subject' in str(error) + else: + raise AssertionError('a foreign server certificate must be rejected') + finally: + stop_server(process) + + +@skip_unless(no_openssl_binary) +def test_unknown_ca_is_rejected(tmp_path): + path = make_certificates(tmp_path) + other = tmp_path / 'other' + other.mkdir() + other_path = make_certificates(other) + port = free_port() + process = cert_server(port, path) + try: + try: + tls.connect_cert( + '127.0.0.1', port, other_path('ca.crt'), + path('client.crt'), path('client.key'), 10) + except tls.TLSError as error: + assert 'handshake failed' in str(error) + else: + raise AssertionError('a server signed by an unknown CA must be rejected') + finally: + stop_server(process) + + +def main(): + import tempfile + import shutil + from pathlib import Path + + failures = 0 + for test in (test_psk_handshake_and_data, test_wrong_psk_is_reported, + test_cert_handshake_and_data, test_cert_subject_mismatch_is_reported, + test_unknown_ca_is_rejected): + reason = test.skip_reason() + if reason: + print('skipped: {0} ({1})'.format(test.__name__, reason)) + continue + if test.__code__.co_argcount: + directory = Path(tempfile.mkdtemp()) + try: + test(directory) + finally: + shutil.rmtree(directory, ignore_errors=True) + else: + test() + print('ok: {0}'.format(test.__name__)) + return failures + + +if __name__ == '__main__': + sys.exit(main()) diff --git a/tests/unit/test_tls_settings.py b/tests/unit/test_tls_settings.py new file mode 100644 index 00000000..ed9a4416 --- /dev/null +++ b/tests/unit/test_tls_settings.py @@ -0,0 +1,350 @@ +# -*- coding: utf-8 -*- + +"""Unit tests for the TLS settings of the Zabbix sender. + +They need neither docker nor a Zabbix server. Run from the repository root: + + python -m pytest tests/unit +""" + +import logging +import socket + +import pytest + +from mamonsu.lib.senders import tls +from mamonsu.lib.senders.zbx import ZbxSender, TLS_CERT, TLS_PSK, TLS_UNENCRYPTED + +PSK_HEX = 'd1e4b7a9c2f30516a8b7c6d5e4f30219a8b7c6d5e4f302198a7b6c5d4e3f2011' +PSK_BYTES = bytes(bytearray.fromhex(PSK_HEX)) + + +class FakeConfig(object): + """Just enough of Config for ZbxSender._setup_tls().""" + + def __init__(self, **options): + self.options = options + + def fetch(self, section, key, klass=None, raw=False): + assert section == 'zabbix' + value = self.options.get(key) + if not raw and isinstance(value, str) and '%' in value: + # mimic configparser: a bare '%' in an interpolated read raises + raise ValueError('interpolation syntax error in {0}'.format(key)) + return value + + +def make_sender(**options): + sender = ZbxSender.__new__(ZbxSender) + sender.log = logging.getLogger('test-zbx-sender') + sender._enabled = True + sender.host, sender.port, sender.timeout = 'zabbix', 10051, 15 + sender._setup_tls(FakeConfig(**options)) + return sender + + +# PSK file parsing + + +def test_read_psk_file(tmp_path): + path = tmp_path / 'zabbix_agentd.psk' + path.write_text(PSK_HEX + '\n') + assert tls.read_psk_file(str(path)) == PSK_BYTES + + +def test_read_psk_file_ignores_whitespace(tmp_path): + path = tmp_path / 'zabbix_agentd.psk' + path.write_text(' ' + PSK_HEX[:32] + '\n' + PSK_HEX[32:] + ' \n\n') + assert tls.read_psk_file(str(path)) == PSK_BYTES + + +def test_read_psk_file_missing(tmp_path): + with pytest.raises(tls.TLSError) as error: + tls.read_psk_file(str(tmp_path / 'nope.psk')) + assert 'can\'t read PSK file' in str(error.value) + + +def test_read_psk_file_too_short(tmp_path): + path = tmp_path / 'short.psk' + path.write_text('abcdef\n') + with pytest.raises(tls.TLSError) as error: + tls.read_psk_file(str(path)) + assert 'hex digits' in str(error.value) + + +def test_read_psk_file_odd_length(tmp_path): + path = tmp_path / 'odd.psk' + path.write_text(PSK_HEX + 'a') + with pytest.raises(tls.TLSError) as error: + tls.read_psk_file(str(path)) + assert 'odd number' in str(error.value) + + +def test_read_psk_file_not_hex(tmp_path): + path = tmp_path / 'garbage.psk' + secret = 'zzzz' + PSK_HEX[4:] + path.write_text(secret) + with pytest.raises(tls.TLSError) as error: + tls.read_psk_file(str(path)) + # the file content must never end up in a message that goes to the log + assert secret not in str(error.value) + assert 'not a valid hex string' in str(error.value) + + +# configuration of the sender + + +def test_no_tls_options_keeps_previous_behaviour(): + sender = make_sender() + assert sender.tls_connect == TLS_UNENCRYPTED + assert sender._enabled is True + assert sender._psk is None + + +def test_psk_options_are_loaded(tmp_path): + path = tmp_path / 'zabbix_agentd.psk' + path.write_text(PSK_HEX) + sender = make_sender( + tls_connect='psk', tls_psk_identity='PSK 001', tls_psk_file=str(path)) + assert sender.tls_connect == TLS_PSK + assert sender._enabled is True + assert sender._psk == PSK_BYTES + + +def test_psk_value_is_case_insensitive(tmp_path): + path = tmp_path / 'zabbix_agentd.psk' + path.write_text(PSK_HEX) + sender = make_sender( + tls_connect='PSK', tls_psk_identity='PSK 001', tls_psk_file=str(path)) + assert sender.tls_connect == TLS_PSK + + +def test_unknown_tls_connect_disables_sender(): + sender = make_sender(tls_connect='ssl') + assert sender._enabled is False + + +def test_psk_without_identity_disables_sender(tmp_path): + path = tmp_path / 'zabbix_agentd.psk' + path.write_text(PSK_HEX) + sender = make_sender(tls_connect='psk', tls_psk_file=str(path)) + assert sender._enabled is False + + +def test_psk_without_file_disables_sender(): + sender = make_sender(tls_connect='psk', tls_psk_identity='PSK 001') + assert sender._enabled is False + + +def test_unreadable_psk_file_disables_sender(tmp_path): + sender = make_sender( + tls_connect='psk', tls_psk_identity='PSK 001', + tls_psk_file=str(tmp_path / 'nope.psk')) + assert sender._enabled is False + + +# transport selection + + +def test_connect_without_tls_uses_plain_socket(monkeypatch): + calls = {} + + class FakeSocket(object): + def settimeout(self, timeout): + calls['timeout'] = timeout + + def connect(self, address): + calls['address'] = address + + monkeypatch.setattr(socket, 'socket', lambda *a, **kw: FakeSocket()) + monkeypatch.setattr( + tls, 'connect_psk', + lambda *a, **kw: pytest.fail('TLS must not be used without tls_connect = psk')) + sender = make_sender() + assert isinstance(sender._connect(), FakeSocket) + assert calls == {'timeout': 15, 'address': ('zabbix', 10051)} + + +def test_connect_with_psk_uses_tls(monkeypatch, tmp_path): + path = tmp_path / 'zabbix_agentd.psk' + path.write_text(PSK_HEX) + calls = {} + marker = object() + + def fake_connect_psk(host, port, identity, psk, timeout, ciphers=None): + calls.update( + host=host, port=port, identity=identity, psk=psk, + timeout=timeout, ciphers=ciphers) + return marker + + monkeypatch.setattr(tls, 'connect_psk', fake_connect_psk) + monkeypatch.setattr( + socket, 'socket', + lambda *a, **kw: pytest.fail('plain socket must not be used with tls_connect = psk')) + sender = make_sender( + tls_connect='psk', tls_psk_identity='PSK 001', tls_psk_file=str(path)) + assert sender._connect() is marker + assert calls == { + 'host': 'zabbix', 'port': 10051, 'identity': 'PSK 001', + 'psk': PSK_BYTES, 'timeout': 15, 'ciphers': None} + + +def test_connect_psk_rejects_empty_identity(): + with pytest.raises(tls.TLSError) as error: + tls.connect_psk('zabbix', 10051, '', PSK_BYTES, 15) + assert 'tls_psk_identity' in str(error.value) + + +# certificates + + +def test_cert_options_are_loaded(): + sender = make_sender( + tls_connect='cert', tls_ca_file='/etc/zabbix/ca.crt', + tls_cert_file='/etc/zabbix/mamonsu.crt', tls_key_file='/etc/zabbix/mamonsu.key', + tls_server_cert_issuer='CN=CA,O=Company') + assert sender.tls_connect == TLS_CERT + assert sender._enabled is True + assert sender.tls_server_cert_issuer == 'CN=CA,O=Company' + + +def test_cert_without_key_disables_sender(): + sender = make_sender( + tls_connect='cert', tls_ca_file='/etc/zabbix/ca.crt', + tls_cert_file='/etc/zabbix/mamonsu.crt') + assert sender._enabled is False + + +def test_connect_with_cert_uses_tls(monkeypatch): + calls = {} + marker = object() + + def fake_connect_cert(host, port, ca_file, cert_file, key_file, timeout, **kwargs): + calls.update( + host=host, port=port, ca_file=ca_file, cert_file=cert_file, + key_file=key_file, timeout=timeout, **kwargs) + return marker + + monkeypatch.setattr(tls, 'connect_cert', fake_connect_cert) + monkeypatch.setattr( + socket, 'socket', + lambda *a, **kw: pytest.fail('plain socket must not be used with tls_connect = cert')) + sender = make_sender( + tls_connect='cert', tls_ca_file='ca.crt', tls_cert_file='m.crt', + tls_key_file='m.key', tls_crl_file='ca.crl', + tls_server_cert_subject='CN=zabbix,O=Company') + assert sender._connect() is marker + assert calls == { + 'host': 'zabbix', 'port': 10051, 'ca_file': 'ca.crt', 'cert_file': 'm.crt', + 'key_file': 'm.key', 'timeout': 15, 'crl_file': 'ca.crl', 'ciphers': None, + 'server_cert_issuer': None, 'server_cert_subject': 'CN=zabbix,O=Company'} + + +def test_connect_cert_requires_files(): + with pytest.raises(tls.TLSError) as error: + tls.connect_cert('zabbix', 10051, None, 'm.crt', 'm.key', 15) + assert 'tls_ca_file' in str(error.value) + + +# distinguished names + + +def test_parse_dn(): + assert tls.parse_dn('CN=zabbix server,O=Company') == [ + ('commonName', 'zabbix server'), ('organizationName', 'Company')] + + +def test_parse_dn_keeps_escaped_commas(): + assert tls.parse_dn(r'CN=Company\, Inc,OU=IT') == [ + ('commonName', 'Company, Inc'), ('organizationalUnitName', 'IT')] + + +def test_parse_dn_accepts_long_names(): + assert tls.parse_dn('commonName=zabbix') == [('commonName', 'zabbix')] + + +def test_check_dn_ignores_attribute_order(): + peer = ((('organizationName', 'Company'),), (('commonName', 'zabbix'),)) + tls.check_dn(peer, 'CN=zabbix,O=Company', 'subject') + + +def test_check_dn_reports_a_mismatch(): + peer = ((('commonName', 'zabbix'),),) + with pytest.raises(tls.TLSError) as error: + tls.check_dn(peer, 'CN=other', 'issuer') + assert 'does not match tls_server_cert_issuer' in str(error.value) + + +def test_check_dn_without_expectation_accepts_anything(): + tls.check_dn(None, None, 'subject') + + +# command line + + +def test_command_line_overrides_config(): + from mamonsu.lib.runner import apply_zabbix_tls_args + + class FakeArgs(object): + zabbix_tls_connect = 'psk' + zabbix_tls_psk_identity = 'PSK 001' + zabbix_tls_psk_file = '/etc/zabbix/zabbix_agentd.psk' + zabbix_tls_ca_file = None + zabbix_tls_crl_file = None + zabbix_tls_cert_file = None + zabbix_tls_key_file = None + zabbix_tls_server_cert_issuer = None + zabbix_tls_server_cert_subject = None + + class FakeCfg(object): + def __init__(self): + self.values = {} + self.config = self + + def set(self, section, key, value): + self.values[(section, key)] = value + + cfg = FakeCfg() + apply_zabbix_tls_args(cfg, FakeArgs()) + # options that were not given on the command line keep the config file value + assert cfg.values == { + ('zabbix', 'tls_connect'): 'psk', + ('zabbix', 'tls_psk_identity'): 'PSK 001', + ('zabbix', 'tls_psk_file'): '/etc/zabbix/zabbix_agentd.psk'} + + +# review fixes + + +def test_percent_in_tls_values_is_read_literally(tmp_path): + """tls_* settings are free text and must be fetched without interpolation.""" + path = tmp_path / 'zabbix_agentd.psk' + path.write_text(PSK_HEX) + sender = make_sender( + tls_connect='psk', tls_psk_identity='PSK%01', tls_psk_file=str(path)) + assert sender._enabled is True + assert sender.tls_psk_identity == 'PSK%01' + + +def test_connect_refuses_unknown_tls_mode(monkeypatch): + """Even a caller that ignores _enabled (mamonsu upload) gets no plaintext.""" + monkeypatch.setattr( + socket, 'socket', + lambda *a, **kw: pytest.fail( + 'an unknown tls_connect must not fall back to a plain socket')) + sender = make_sender(tls_connect='ssl') + assert sender._enabled is False + with pytest.raises(tls.TLSError) as error: + sender._connect() + assert 'refusing to send data unencrypted' in str(error.value) + + +def test_libssl_transport_is_refused_on_macos(monkeypatch): + """The macOS libssl stub aborts the process, so it must never be loaded.""" + import sys as _sys + monkeypatch.setattr(tls, '_libssl', None) + monkeypatch.setattr(tls, '_libcrypto', None) + monkeypatch.setattr(_sys, 'platform', 'darwin') + with pytest.raises(tls.TLSError) as error: + tls._libs() + assert 'Python 3.13' in str(error.value) diff --git a/tests/unit/test_zbx_sender_socket.py b/tests/unit/test_zbx_sender_socket.py new file mode 100644 index 00000000..4c00ef6d --- /dev/null +++ b/tests/unit/test_zbx_sender_socket.py @@ -0,0 +1,120 @@ +# -*- coding: utf-8 -*- + +"""The unencrypted sender must behave exactly as it did before TLS was added. + +A fake Zabbix trapper is started on the loopback interface, so these tests need +neither a Zabbix server nor docker: + + python -m pytest tests/unit/test_zbx_sender_socket.py + python3 tests/unit/test_zbx_sender_socket.py +""" + +import json +import logging +import os +import socket +import struct +import sys +import threading + +sys.path.insert(0, os.path.join(os.path.dirname(os.path.abspath(__file__)), '..', '..')) + +from mamonsu.lib.senders.zbx import ZbxSender # noqa: E402 + +HEADER = b'ZBXD' + b'\x01' +SUCCESS = '{"response":"success","info":"processed: 1; failed: 0; total: 1; seconds spent: 0.1"}' +PARTIAL = '{"response":"success","info":"processed: 1; failed: 1; total: 2; seconds spent: 0.1"}' + + +class FakeConfig(object): + def fetch(self, section, key, klass=None, raw=False): + return None + + +class FakeTrapper(object): + """Accepts one connection, replies in the Zabbix sender protocol.""" + + def __init__(self, response): + self.response = response + self.request = None + self.socket = socket.socket() + self.socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) + self.socket.bind(('127.0.0.1', 0)) + self.socket.listen(1) + self.port = self.socket.getsockname()[1] + self.thread = threading.Thread(target=self._serve) + self.thread.daemon = True + self.thread.start() + + def _receive(self, conn, count): + buf = b'' + while len(buf) < count: + chunk = conn.recv(count - len(buf)) + if not chunk: + break + buf += chunk + return buf + + def _serve(self): + conn, _ = self.socket.accept() + try: + header = self._receive(conn, 13) + body = self._receive(conn, struct.unpack('