diff --git a/CHANGELOG.md b/CHANGELOG.md index 86b7083f8..8992472d4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,6 +10,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Breaking Changes +* **role:postgresql_server**: Databases are created with the `C.UTF-8` locale instead of `en_US.UTF-8`, which failed on hosts with only the minimal glibc language pack. `C.UTF-8` sorts text by code point instead of by language rules. Existing databases are unaffected. To create new databases as before, set `lc_collate: 'en_US.UTF-8'` and `lc_ctype: 'en_US.UTF-8'` on their entries in `postgresql_server__databases__*_var`. +* **role:postgresql_server, playbook:setup_mastodon**: `postgresql_server__version` is mandatory, and PostgreSQL is only installed from the PostgreSQL Yum Repository instead of optionally from the distribution packages. Set the major version in your inventory, for example `postgresql_server__version: '18'`. On a host that already runs PostgreSQL from the distribution packages, the run aborts before it installs anything; move the data to the PostgreSQL Yum Repository release first, as described in the role README. +* **role:postgresql_server**: `postgresql_server__pg_hba_entries` is replaced by `postgresql_server__pg_hba_local_entries__host_var` / `__group_var` for `local` records and `postgresql_server__pg_hba_host_entries__host_var` / `__group_var` for all `host*` records. Both are merged with the role defaults instead of replacing them, and an entry with `state: 'absent'` removes a default. Move your `local` entries to the first variable without their `type` key, and all other entries to the second, where `address` is mandatory. Entries from the inventory are written before the role defaults. The run aborts as long as the old variable is set. * **role:icinga2_agent**: If the agent cannot get a PKI ticket from the Icinga2 master, the run aborts and names the cause and the fix. Until now the role set the agent up anyway, which also replaced the signed certificate of an agent that was already connected with an unsigned one. If you sign agent certificates on the master by hand, or do not set `icinga2_agent__icinga2_api_user_login`, set `icinga2_agent__skip_pki_ticket: true`. * **role:dnf_makecache**: `dnf_makecache__service_enabled` and `dnf_makecache__service_state` are gone; remove them from your inventory. The role only manages `dnf-makecache.timer` now, since `dnf-makecache.service` cannot be enabled at boot and only runs when the timer triggers it. `dnf_makecache__service_enabled` never had an effect, but a run against an unchanged host reported a change for it. A host that set `dnf_makecache__service_state: 'started'` no longer runs `dnf makecache` on every run of the role. Use `dnf_makecache__timer_enabled` and `dnf_makecache__timer_state` for the periodic cache refresh. * **role:kibana**: The session cookie always carries the `Secure` flag, also behind a reverse proxy that terminates TLS, where Kibana left the flag off. A Kibana that browsers reach over plain HTTP no longer logs anyone in until `kibana__xpack_security_secure_cookies: false` is set. Remove `xpack.security.secureCookies` from `kibana__raw` if you set it there. @@ -35,6 +38,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Changed +* **role:postgresql_server**: A changed `pg_hba.conf` or `postgresql_server__conf_password_encryption` reloads PostgreSQL instead of restarting it, so open connections are kept; the first run after the update still restarts it once. +* **role:postgresql_server**: The configuration is checked before PostgreSQL is restarted, so a broken setting in `postgresql.conf` or `pg_hba.conf` aborts the run with the error message instead of leaving PostgreSQL down after the restart. * **role:repo_postgresql**: The PostgreSQL version repositories take precedence over the distribution's packages of the same name, so on RHEL 10 an install or update no longer switches a PostgreSQL server from the PGDG build to the AppStream build, which uses a different file layout. * **role:apache_httpd**: Responses of type `text/markdown` are compressed like HTML, so the Markdown versions of pages that CMSs such as Grav hand to AI agents no longer go out uncompressed. * **role:grav**: The README lists setting `session.secure` as a manual step behind a reverse proxy that terminates TLS, where Grav sends its session cookies without the `Secure` flag. @@ -45,6 +50,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Fixed +* **role:postgresql_server**: `postgresql-dump` also saves the roles and tablespaces (`globals.sql.gz`), without which a restored database fails on every object owner, no longer skips the `postgres` database and databases whose name starts with `template`, and uses the client tools of the installed PostgreSQL version. +* **role:postgresql_server**: Clients connecting to `localhost` are accepted over IPv6 as well, where they were rejected with `no pg_hba.conf entry for host "::1"`. * **role:postfix**: `postfix__compatibility_level` takes effect on Debian and Ubuntu as well, and defaults to the level the distribution ships, so Debian 13 and Ubuntu 26.04 run at `3.9` instead of `3.6` ([#364](https://github.com/Linuxfabrik/lfops/issues/364)). * **role:mariadb_server**: On RHEL 10 with a `selinux-policy-targeted` older than `42.1.18-4.el10_2.3`, MariaDB runs confined in `mysqld_t` again, so web applications such as WordPress or Nextcloud reach its socket. Until now it ran in `initrc_t` there, and PHP-FPM failed to connect until the SELinux policy was updated and MariaDB restarted. * **role:apache_httpd**: The role hands the content of the document root to the web server user, but leaves the directory itself to the httpd package, whose tmpfiles rule resets it to `root` on every boot and after some package installations. A second run on a fresh host no longer reports the ownership as changed. diff --git a/COMPATIBILITY.md b/COMPATIBILITY.md index c528d1721..0644d9db0 100644 --- a/COMPATIBILITY.md +++ b/COMPATIBILITY.md @@ -122,7 +122,7 @@ Which Ansible role is proven to run on which OS? | podman_containers | | | (x) | x | (x) | | | | | | policycoreutils | | | x | x | x | | | | Fedora 35 | | postfix | x | x | x | x | x | (x) | (x) | (x) | Fedora 35 | -| postgresql_server | | | x | (x) | (x) | | | | | +| postgresql_server | | | x | x | x | | | | | | proxysql | | | x | (x) | (x) | | | | | | python | x | x | x | x | x | (x) | (x) | (x) | Windows | | python_venv | x | x | x | x | x | (x) | (x) | (x) | Fedora 35 | diff --git a/extensions/molecule/postgresql_server/converge.yml b/extensions/molecule/postgresql_server/converge.yml new file mode 100644 index 000000000..ddc14d158 --- /dev/null +++ b/extensions/molecule/postgresql_server/converge.yml @@ -0,0 +1,2 @@ +- name: 'Converge postgresql_server playbook' + ansible.builtin.import_playbook: 'linuxfabrik.lfops.postgresql_server' diff --git a/extensions/molecule/postgresql_server/inventory/group_vars/systems_under_test.yml b/extensions/molecule/postgresql_server/inventory/group_vars/systems_under_test.yml new file mode 100644 index 000000000..c211ca853 --- /dev/null +++ b/extensions/molecule/postgresql_server/inventory/group_vars/systems_under_test.yml @@ -0,0 +1,31 @@ +# Variables the playbook under test needs, applied to every system under test. +# +# Where a value is compared in verify.yml, it deliberately differs from the role default, so the +# check can tell "the running server uses what the inventory asked for" apart from "the running +# server happens to use the packaged default". +postgresql_server__version: '18' + +postgresql_server__conf_listen_addresses: + - '*' +postgresql_server__conf_max_connections: 150 + +postgresql_server__databases__group_var: + - name: 'app' + owner: 'app' + # the dump used to skip every database whose name starts with "template" + - name: 'template_app' + owner: 'app' + +# Replaces the catch-all default for Unix-socket connections in place, and adds a more specific +# entry that only works if the role renders it before the defaults. +postgresql_server__pg_hba_local_entries__group_var: + - database: 'all' + user: 'all' + auth_method: 'reject' + - database: 'app' + user: 'app' + auth_method: 'scram-sha-256' + +postgresql_server__users__group_var: + - username: 'app' + password: 'linuxfabrik' diff --git a/extensions/molecule/postgresql_server/inventory/hosts.yml b/extensions/molecule/postgresql_server/inventory/hosts.yml new file mode 100644 index 000000000..1e0f4bc01 --- /dev/null +++ b/extensions/molecule/postgresql_server/inventory/hosts.yml @@ -0,0 +1,15 @@ +# yamllint disable rule:empty-values + +# postgresql_server targets 'lfops_postgresql_server' (see playbooks/postgresql_server.yml: hosts). +# +# Only Red Hat-family hosts run: the role installs PostgreSQL from the PostgreSQL Yum Repository +# and has no Debian support (see COMPATIBILITY.md). +lfops_postgresql_server: + children: + systems_under_test: + +systems_under_test: + hosts: + rocky8-vm: + rocky9-vm: + rocky10-vm: diff --git a/extensions/molecule/postgresql_server/molecule.yml b/extensions/molecule/postgresql_server/molecule.yml new file mode 100644 index 000000000..1e47cbff8 --- /dev/null +++ b/extensions/molecule/postgresql_server/molecule.yml @@ -0,0 +1 @@ +# Molecule scenario marker diff --git a/extensions/molecule/postgresql_server/verify.yml b/extensions/molecule/postgresql_server/verify.yml new file mode 100644 index 000000000..8f7cc6744 --- /dev/null +++ b/extensions/molecule/postgresql_server/verify.yml @@ -0,0 +1,198 @@ +# verify.yml runs after converge and again after the idempotence step. It asserts the observable +# end state, not that individual tasks ran: every setting is read back from the running server, +# and pg_hba.conf is proven by connections that have to succeed or be rejected. +- name: 'Verify postgresql_server is running and uses the configured values' + hosts: 'systems_under_test' + gather_facts: true + become: true + + vars: + + __molecule__postgresql_bin_dir: '/usr/pgsql-{{ postgresql_server__version }}/bin' + __molecule__postgresql_service_name: 'postgresql-{{ postgresql_server__version }}.service' + __molecule__psql: '{{ __molecule__postgresql_bin_dir }}/psql --no-align --tuples-only --no-psqlrc' + __molecule__dump_directory: '/backup/postgresql-dump' + + tasks: + + - name: 'Get the service facts' + ansible.builtin.service_facts: # yamllint disable-line rule:empty-values + + - name: 'Assert the PostgreSQL service is enabled and running' + ansible.builtin.assert: + that: + - 'ansible_facts["services"][__molecule__postgresql_service_name]["state"] == "running"' + - 'ansible_facts["services"][__molecule__postgresql_service_name]["status"] == "enabled"' + fail_msg: '{{ __molecule__postgresql_service_name }} is {{ ansible_facts["services"][__molecule__postgresql_service_name] | d("absent") }}' + + # Ask the running server what it applied, which proves the chain from group_vars through + # conf.d/z00-linuxfabrik.conf and the include_dir line to the server. + - name: 'psql --command="SHOW ..."' + ansible.builtin.command: '{{ __molecule__psql }} --command="SHOW {{ item["setting"] }}"' + become: true + become_user: 'postgres' + register: '__molecule__postgresql_show_result' + changed_when: false + loop: + - setting: 'listen_addresses' + expected: '*' + - setting: 'max_connections' + expected: '{{ postgresql_server__conf_max_connections | string }}' + loop_control: + label: '{{ item["setting"] }}' + + - name: 'Assert the running server applied the configured values' + ansible.builtin.assert: + that: 'item["stdout"] == item["item"]["expected"]' + fail_msg: '{{ item["item"]["setting"] }} is {{ item["stdout"] }}, expected {{ item["item"]["expected"] }}' + loop: '{{ __molecule__postgresql_show_result["results"] }}' + loop_control: + label: '{{ item["item"]["setting"] }}' + + # The databases in the inventory set no locale, so they get the role default. + - name: 'psql --command="SELECT datcollate, datctype FROM pg_database WHERE datname = ''app''"' + ansible.builtin.command: '{{ __molecule__psql }} --field-separator=" " --command="SELECT datcollate, datctype FROM pg_database WHERE datname = ''app''"' + become: true + become_user: 'postgres' + register: '__molecule__postgresql_locale_result' + changed_when: false + + - name: 'Assert the database was created with the C.UTF-8 locale' + ansible.builtin.assert: + that: '__molecule__postgresql_locale_result["stdout"] == "C.UTF-8 C.UTF-8"' + fail_msg: 'datcollate and datctype of app are {{ __molecule__postgresql_locale_result["stdout"] }}' + + - name: 'psql --command="SELECT ... FROM pg_hba_file_rules WHERE error IS NOT NULL"' + ansible.builtin.command: '{{ __molecule__psql }} --command="SELECT line_number, error FROM pg_hba_file_rules WHERE error IS NOT NULL"' + become: true + become_user: 'postgres' + register: '__molecule__postgresql_hba_errors_result' + changed_when: false + + - name: 'Assert pg_hba.conf has no errors' + ansible.builtin.assert: + that: '__molecule__postgresql_hba_errors_result["stdout"] | length == 0' + fail_msg: 'pg_hba.conf contains errors: {{ __molecule__postgresql_hba_errors_result["stdout"] }}' + + # The IPv4 and IPv6 loopback defaults. `localhost` resolves to ::1 first, and libpq does not + # fall back to 127.0.0.1 when the server rejects the connection on ::1. + - name: 'psql over TCP as app' + ansible.builtin.command: '{{ __molecule__psql }} "host={{ item }} user=app dbname=app" --command="SELECT inet_client_addr()"' + environment: + PGPASSWORD: 'linuxfabrik' + register: '__molecule__postgresql_tcp_result' + changed_when: false + loop: '{{ ["127.0.0.1", "localhost"] + (__molecule__has_ipv6_loopback | ternary(["::1"], [])) }}' + vars: + # Gate on the loopback interface, not on ansible_facts['all_ipv6_addresses']: that fact + # lists the real addresses only and never contains ::1. + __molecule__has_ipv6_loopback: '{{ "::1" in (ansible_facts["lo"]["ipv6"] | d([]) | map(attribute="address") | list) }}' + + - name: 'Assert app connects over TCP from the expected address' + ansible.builtin.assert: + that: 'item["stdout"] in (item["item"] == "localhost") | ternary(["127.0.0.1", "::1"], [item["item"]])' + fail_msg: 'connecting to {{ item["item"] }} came from {{ item["stdout"] }}' + loop: '{{ __molecule__postgresql_tcp_result["results"] }}' + loop_control: + label: '{{ item["item"] }}' + + # The inventory replaces the `local all all` default with `reject` and adds `local app app`. + # The connection to app only works if the added entry is rendered before the defaults, the + # one to postgres is only rejected if the default was replaced. + - name: 'psql over the Unix socket as app to the database app' + ansible.builtin.command: '{{ __molecule__psql }} "user=app dbname=app" --command="SELECT current_user"' + environment: + PGPASSWORD: 'linuxfabrik' + register: '__molecule__postgresql_socket_app_result' + changed_when: false + + - name: 'psql over the Unix socket as app to the database postgres' + ansible.builtin.command: '{{ __molecule__psql }} "user=app dbname=postgres" --command="SELECT current_user"' + environment: + PGPASSWORD: 'linuxfabrik' + register: '__molecule__postgresql_socket_postgres_result' + changed_when: false + failed_when: false + + - name: 'Assert the local pg_hba entries are applied in the expected order' + ansible.builtin.assert: + that: + - '__molecule__postgresql_socket_app_result["stdout"] == "app"' + - '__molecule__postgresql_socket_postgres_result["rc"] != 0' + - '"rejects connection" in __molecule__postgresql_socket_postgres_result["stderr"]' + fail_msg: 'socket to postgres as app: rc={{ __molecule__postgresql_socket_postgres_result["rc"] }} {{ __molecule__postgresql_socket_postgres_result["stderr"] }}' + + # postgresql-dump: dump a database with content and restore it, with the tools the script is + # expected to use. + - name: 'Create a table with a row in the database app' + ansible.builtin.command: >- + {{ __molecule__psql }} "user=app dbname=app" + --command="CREATE TABLE IF NOT EXISTS molecule_marker (id int)" + --command="INSERT INTO molecule_marker SELECT 1 WHERE NOT EXISTS (SELECT 1 FROM molecule_marker)" + environment: + PGPASSWORD: 'linuxfabrik' + changed_when: false + + - name: 'systemctl start postgresql-dump.service' + ansible.builtin.systemd: + name: 'postgresql-dump.service' + state: 'started' + changed_when: false + + - name: 'stat {{ __molecule__dump_directory }}/*.sql.gz' + ansible.builtin.stat: + path: '{{ __molecule__dump_directory }}/{{ item }}' + register: '__molecule__postgresql_dump_stat_result' + loop: + - 'dump-app.sql.gz' + - 'dump-postgres.sql.gz' + - 'dump-template1.sql.gz' + - 'dump-template_app.sql.gz' + - 'globals.sql.gz' + + - name: 'Assert every database and the globals were dumped' + ansible.builtin.assert: + that: 'item["stat"]["exists"]' + fail_msg: '{{ item["item"] }} is missing' + loop: '{{ __molecule__postgresql_dump_stat_result["results"] }}' + loop_control: + label: '{{ item["item"] }}' + + - name: 'zcat globals.sql.gz; zcat dump-app.sql.gz' + ansible.builtin.command: 'zcat {{ __molecule__dump_directory }}/{{ item }}' + register: '__molecule__postgresql_dump_content_result' + changed_when: false + loop: + - 'globals.sql.gz' + - 'dump-app.sql.gz' + + - name: 'Assert the dumps contain the role and were made with the matching pg_dump' + ansible.builtin.assert: + that: + - '__molecule__postgresql_dump_content_result["results"][0]["stdout"] is search("CREATE ROLE app;")' + - '__molecule__postgresql_dump_content_result["results"][1]["stdout"] is search("Dumped from database version " ~ __molecule__server_major ~ "\.")' + - '__molecule__postgresql_dump_content_result["results"][1]["stdout"] is search("Dumped by pg_dump version " ~ __molecule__server_major ~ "\.")' + fail_msg: 'unexpected dump content, expected PostgreSQL {{ __molecule__server_major }}' + vars: + __molecule__server_major: '{{ + __molecule__postgresql_dump_content_result["results"][1]["stdout"] + | regex_search("Dumped from database version (\d+)", "\1") | first + }}' + + # The dumps are readable by root only, so zcat runs as root and only psql as postgres. + - name: 'Restore dump-app.sql.gz into the database app_restore' + ansible.builtin.shell: + cmd: >- + set -o pipefail; + runuser --user=postgres -- {{ __molecule__psql }} --command="DROP DATABASE IF EXISTS app_restore" && + runuser --user=postgres -- {{ __molecule__psql }} --command="CREATE DATABASE app_restore OWNER app" && + zcat {{ __molecule__dump_directory }}/dump-app.sql.gz | runuser --user=postgres -- {{ __molecule__psql }} --quiet --dbname=app_restore && + runuser --user=postgres -- {{ __molecule__psql }} --dbname=app_restore --command="SELECT tableowner || ':' || (SELECT count(*) FROM molecule_marker) FROM pg_tables WHERE tablename = 'molecule_marker'" + executable: '/bin/bash' + register: '__molecule__postgresql_restore_result' + changed_when: false + + - name: 'Assert the restored table has its owner and its row' + ansible.builtin.assert: + that: '__molecule__postgresql_restore_result["stdout_lines"][-1] == "app:1"' + fail_msg: 'restore returned {{ __molecule__postgresql_restore_result["stdout"] }} {{ __molecule__postgresql_restore_result["stderr"] }}' diff --git a/roles/postgresql_server/README.md b/roles/postgresql_server/README.md index 9cec62940..345d203be 100644 --- a/roles/postgresql_server/README.md +++ b/roles/postgresql_server/README.md @@ -6,20 +6,27 @@ This role installs and configures a [PostgreSQL](https://www.postgresql.org/) se *Available since LFOps `2.0.0`.* +## How the Role Behaves + +* PostgreSQL is installed from the PostgreSQL Yum Repository, not from the distribution packages. The cluster lives in `/var/lib/pgsql//data` and runs as `postgresql-.service`. The role does not upgrade the data: changing `postgresql_server__version` on an existing host installs and initializes a second, empty cluster next to the old one, which fails to start while the old one listens on the same port. +* Settings that only take effect after a restart (`postgresql_server__conf_listen_addresses`, `__conf_max_connections`, `__conf_port`) are written to `conf.d/z00-linuxfabrik.conf`, and changing them restarts PostgreSQL. `postgresql_server__conf_password_encryption` is written to `conf.d/z00-linuxfabrik-reload.conf`, and a change there or in `pg_hba.conf` only reloads PostgreSQL, which keeps open connections. +* Before the reload or restart, the role checks the configuration files with `postgres -C` and asks the running server for errors in `pg_hba.conf` through the `pg_hba_file_rules` view. A broken setting aborts the run with the error message, and the running server keeps its current configuration. The file with the error is already deployed at that point: fix the inventory and run the role again before PostgreSQL is restarted for any other reason. + + ## Dependent Roles Any [LFOps playbook](https://github.com/Linuxfabrik/lfops/blob/main/playbooks/README.md) that installs this role runs these for you. Optional ones can be disabled via the playbook's skip variables. * The `python3-psycopg2` library must be installed (role: [linuxfabrik.lfops.python](https://github.com/Linuxfabrik/lfops/tree/main/roles/python)). -* Optional: the official [PostgreSQL Yum Repository](https://yum.postgresql.org/) enabled (role: [linuxfabrik.lfops.repo_postgresql](https://github.com/Linuxfabrik/lfops/tree/main/roles/repo_postgresql)). +* The [PostgreSQL Yum Repository](https://yum.postgresql.org/) must be enabled (role: [linuxfabrik.lfops.repo_postgresql](https://github.com/Linuxfabrik/lfops/tree/main/roles/repo_postgresql)). ## Tags `postgresql_server` -* Installs and configures PostgreSQL. -* Triggers: posgresql.service restart. +* Installs and configures PostgreSQL, and manages its users, databases, privileges and the dump timer. +* Triggers: postgresql-.service reload or restart, after the configuration check. `postgresql_server:state` @@ -47,6 +54,20 @@ Any [LFOps playbook](https://github.com/Linuxfabrik/lfops/blob/main/playbooks/RE * Triggers: none. +## Mandatory Role Variables + +`postgresql_server__version` + +* The major version of PostgreSQL to install from the PostgreSQL Yum Repository, for example `'18'`. The latest minor release of that version is installed. +* Type: String. + +Example: +```yaml +# mandatory +postgresql_server__version: '18' +``` + + ## Optional Role Variables `postgresql_server__conf_listen_addresses` @@ -63,7 +84,7 @@ Any [LFOps playbook](https://github.com/Linuxfabrik/lfops/blob/main/playbooks/RE `postgresql_server__conf_password_encryption` -* Determines the algorithm to use to encrypt passwords when creating new users / roles. Possible options: `'scram-sha-256'`, `'md5'`. +* Determines the algorithm to use to encrypt passwords when creating new users / roles or changing their passwords. Possible options: `'scram-sha-256'`, `'md5'`. PostgreSQL deprecates MD5 passwords and logs a warning for them. * Type: String. * Default: `'scram-sha-256'` @@ -88,15 +109,17 @@ Any [LFOps playbook](https://github.com/Linuxfabrik/lfops/blob/main/playbooks/RE * `lc_collate`: - * Optional. DB Collation order. + * Optional. Collation order of the database. Only used when the database is created, PostgreSQL cannot change it afterwards. * Type: String. - * Default: `'en_US.UTF-8'` + * Default: `'C.UTF-8'` + * Deviates from the upstream default, the locale of the template database, which `initdb` takes from the environment of the host: `C.UTF-8` exists on every host, also where only the minimal glibc language pack is installed, and does not change its sort order with a glibc update. It sorts by code point, not by language rules (`B` before `a`); set `'en_US.UTF-8'` or another installed locale for linguistic sorting. * `lc_ctype`: - * Optional. DB Character classification. + * Optional. Character classification of the database. Only used when the database is created, PostgreSQL cannot change it afterwards. * Type: String. - * Default: `'en_US.UTF-8'` + * Default: `'C.UTF-8'` + * Deviates from the upstream default for the same reason as `lc_collate`. * `encoding`: @@ -124,7 +147,7 @@ Any [LFOps playbook](https://github.com/Linuxfabrik/lfops/blob/main/playbooks/RE `postgresql_server__dump_directory` -* The directory where `postgresql-dump` stores its database dumps. +* The directory where `postgresql-dump` stores its dumps: one file `dump-.sql.gz` per database, and `globals.sql.gz` with the roles and tablespaces, which pg_dump does not include in a database dump. Restore `globals.sql.gz` first. The directory is emptied at the start of every dump. * Type: String. * Default: `'/backup/postgresql-dump'` @@ -146,37 +169,99 @@ Any [LFOps playbook](https://github.com/Linuxfabrik/lfops/blob/main/playbooks/RE * Type: String. * Default: unset -`postgresql_server__pg_hba_entries` +`postgresql_server__pg_hba_host_entries__host_var` / `postgresql_server__pg_hba_host_entries__group_var` -* List of [host based authentication](https://www.postgresql.org/docs/current/static/auth-pg-hba-conf.html) entries. +* [Client authentication](https://www.postgresql.org/docs/current/auth-pg-hba-conf.html) records for TCP/IP connections (`host`, `hostssl`, `hostnossl`, `hostgssenc`, `hostnogssenc`). Entries are identified by `type`, `database`, `user` and `address` together. +* PostgreSQL uses the first record that matches a connection. The role renders the entries from the inventory in their inventory order, followed by the role defaults, which are catch-all records. An entry that uses the same identifying keys as a default replaces that default in place. +* For the usage in `host_vars` / `group_vars` (can only be used in one group at a time). * Type: List of dictionaries. -* Default: Allow `scram-sha-256` for all `local` and `host` +* Default: + + ```yaml + - type: 'host' + database: 'all' + user: 'all' + address: '127.0.0.1/32' + auth_method: 'scram-sha-256' + - type: 'host' + database: 'all' + user: 'all' + address: '::1/128' + auth_method: 'scram-sha-256' + ``` + * Subkeys: * `type`: - * Mandatory. Record type. + * Mandatory. Record type. One of `host`, `hostgssenc`, `hostnogssenc`, `hostnossl`, `hostssl`. * Type: String. * `database`: - * Mandatory. Specifies which database name(s) this record matches. + * Mandatory. Database name(s) this record matches, for example `all` or `db1,db2`. * Type: String. * `user`: - * Mandatory. Specifies which database user name(s) this record matches. + * Mandatory. Database user name(s) this record matches, for example `all` or `user1`. * Type: String. * `address`: - * Optional. Specifies the client machine address(es) that this record matches. + * Mandatory. Client address(es) this record matches, for example `192.0.2.0/24`, `all`, `samenet` or a host name. + * Type: String. + + * `auth_method`: + + * Optional. Authentication method for a connection that matches this record. + * Type: String. + * Default: `'scram-sha-256'` + + * `auth_options`: + + * Optional. Options for the `auth_method`. * Type: String. * Default: `''` + * `state`: + + * Optional. `present` or `absent`. + * Type: String. + * Default: `'present'` + +`postgresql_server__pg_hba_local_entries__host_var` / `postgresql_server__pg_hba_local_entries__group_var` + +* [Client authentication](https://www.postgresql.org/docs/current/auth-pg-hba-conf.html) records of type `local`, for connections over the Unix-domain socket. Entries are identified by `database` and `user` together. +* PostgreSQL uses the first record that matches a connection. The role renders the entries from the inventory in their inventory order, followed by the role defaults, which are catch-all records. An entry that uses the same identifying keys as a default replaces that default in place. +* For the usage in `host_vars` / `group_vars` (can only be used in one group at a time). +* Type: List of dictionaries. +* Default: + + ```yaml + - database: 'all' + user: 'postgres' + auth_method: 'peer' + - database: 'all' + user: 'all' + auth_method: 'scram-sha-256' + ``` + +* Subkeys: + + * `database`: + + * Mandatory. Database name(s) this record matches, for example `all` or `db1,db2`. + * Type: String. + + * `user`: + + * Mandatory. Database user name(s) this record matches, for example `all` or `user1`. + * Type: String. + * `auth_method`: - * Optional. Specifies the authentication method to use when a connection matches this record. + * Optional. Authentication method for a connection that matches this record. * Type: String. * Default: `'scram-sha-256'` @@ -186,6 +271,12 @@ Any [LFOps playbook](https://github.com/Linuxfabrik/lfops/blob/main/playbooks/RE * Type: String. * Default: `''` + * `state`: + + * Optional. `present` or `absent`. + * Type: String. + * Default: `'present'` + `postgresql_server__privs__host_var` / `postgresql_server__privs__group_var` * List of dictionaries containing PostgreSQL privileges to apply. @@ -265,12 +356,6 @@ Any [LFOps playbook](https://github.com/Linuxfabrik/lfops/blob/main/playbooks/RE * Type: String. * Default: `'present'` -`postgresql_server__version` - -* Specifies the PostgreSQL verison to install (use only the major version number like `'14'`. The latest minor version is used). Set this when using the official PostgreSQL Repo. -* Type: String. -* Default: `''` - Example: ```yaml @@ -290,13 +375,17 @@ postgresql_server__dump_directory: '/backup/postgresql-dump' postgresql_server__dump_on_calendar: '*-*-* 21:30:00' postgresql_server__enabled: true postgresql_server__login_password: 'linuxfabrik' -postgresql_server__pg_hba_entries: - - type: 'local' - database: 'all' - user: 'all' - - type: 'host' - database: 'all' +postgresql_server__pg_hba_host_entries__host_var: + - type: 'hostssl' + database: 'database1' + user: 'user1' + address: '192.0.2.0/24' +postgresql_server__pg_hba_local_entries__host_var: + - database: 'all' user: 'all' + auth_method: 'reject' + - database: 'database1' + user: 'user1' postgresql_server__privs__host_var: - privs: - 'CONNECT' @@ -311,10 +400,16 @@ postgresql_server__users__host_var: - username: 'user1' password: 'linuxfabrik' state: 'present' -postgresql_server__version: '14' ``` +## Troubleshooting + +**The run aborts with `This host has a PostgreSQL cluster of the distribution package in /var/lib/pgsql/data`** + +* The host runs PostgreSQL from the distribution packages, which the role does not manage. Move the data to the PostgreSQL Yum Repository release before running the role, for example: dump the cluster with `pg_dumpall`, stop and disable `postgresql.service`, rename `/var/lib/pgsql/data`, run the role, and restore the dump into the new cluster. + + ## License [The Unlicense](https://unlicense.org/) diff --git a/roles/postgresql_server/defaults/main.yml b/roles/postgresql_server/defaults/main.yml index f44c4e3c2..4659d9441 100644 --- a/roles/postgresql_server/defaults/main.yml +++ b/roles/postgresql_server/defaults/main.yml @@ -18,18 +18,52 @@ postgresql_server__databases__role_var: [] postgresql_server__dump_directory: '/backup/postgresql-dump' postgresql_server__dump_on_calendar: '*-*-* 21:{{ 59 | random(start=0, seed=inventory_hostname) }}:00' postgresql_server__enabled: true -postgresql_server__pg_hba_entries: - - type: 'local' - database: 'all' - user: 'postgres' - auth_method: 'peer' - - type: 'local' +# pg_hba.conf is split by connection kind because a `local` record has no address. A `local` +# record only matches Unix-socket connections and a `host*` record only TCP connections +# (check_hba() in src/backend/libpq/hba.c), so rendering all local records before all host +# records does not change which record a connection matches. +# `type` is part of the host key because `host` matches the same TCP connection as `hostssl` or +# `hostnossl`. +postgresql_server__pg_hba_host_entries__combined_var: '{{ ( + postgresql_server__pg_hba_host_entries__role_var + + postgresql_server__pg_hba_host_entries__dependent_var + + postgresql_server__pg_hba_host_entries__group_var + + postgresql_server__pg_hba_host_entries__host_var + ) | linuxfabrik.lfops.combine_lod(unique_key=["type", "database", "user", "address"]) + }}' +postgresql_server__pg_hba_host_entries__dependent_var: [] +postgresql_server__pg_hba_host_entries__group_var: [] +postgresql_server__pg_hba_host_entries__host_var: [] +postgresql_server__pg_hba_host_entries__role_var: + - type: 'host' database: 'all' user: 'all' + address: '127.0.0.1/32' + auth_method: 'scram-sha-256' + # `localhost` also resolves to ::1, and libpq does not fall back to 127.0.0.1 when the server + # rejects the connection on ::1 (connection_failed() in src/interfaces/libpq/fe-connect.c). - type: 'host' database: 'all' user: 'all' - address: '127.0.0.1/32' + address: '::1/128' + auth_method: 'scram-sha-256' +postgresql_server__pg_hba_local_entries__combined_var: '{{ ( + postgresql_server__pg_hba_local_entries__role_var + + postgresql_server__pg_hba_local_entries__dependent_var + + postgresql_server__pg_hba_local_entries__group_var + + postgresql_server__pg_hba_local_entries__host_var + ) | linuxfabrik.lfops.combine_lod(unique_key=["database", "user"]) + }}' +postgresql_server__pg_hba_local_entries__dependent_var: [] +postgresql_server__pg_hba_local_entries__group_var: [] +postgresql_server__pg_hba_local_entries__host_var: [] +postgresql_server__pg_hba_local_entries__role_var: + - database: 'all' + user: 'postgres' + auth_method: 'peer' + - database: 'all' + user: 'all' + auth_method: 'scram-sha-256' postgresql_server__privs__combined_var: '{{ ( postgresql_server__privs__role_var + postgresql_server__privs__dependent_var + @@ -55,7 +89,6 @@ postgresql_server__users__dependent_var: [] postgresql_server__users__group_var: [] postgresql_server__users__host_var: [] postgresql_server__users__role_var: [] -postgresql_server__version: '' # ----------------------------------------------------------------------------- postgresql_server__python__modules__dependent_var: diff --git a/roles/postgresql_server/handlers/main.yml b/roles/postgresql_server/handlers/main.yml index ed754f667..91bf0da9c 100644 --- a/roles/postgresql_server/handlers/main.yml +++ b/roles/postgresql_server/handlers/main.yml @@ -1,13 +1,95 @@ +# The config tasks notify 'postgresql_server: validate config; restart postgresql' for settings +# that need a restart, and 'postgresql_server: validate config; reload postgresql' for pg_hba.conf +# and settings that a reload applies. The checks listen to both topics, and handlers run in the +# order they are defined in this file, so the reload and the restart at the end only run once +# every check passed. A failing check aborts the run while the running server keeps its current +# configuration. With both topics notified, the checks run once, followed by the reload and the +# restart. + +# `postgres -C` reads postgresql.conf, conf.d and postgresql.auto.conf the way the postmaster does +# at startup, where every error is fatal, prints the value and exits before it touches the data +# directory lock, so it is safe next to a running server (PostmasterMain() in +# src/backend/postmaster/postmaster.c). It does not read pg_hba.conf. +- name: 'postgresql_server: postgres -C data_directory' + ansible.builtin.command: '{{ __postgresql_server__bin_dir }}/postgres -D {{ postgresql_server__data_dir | quote }} -C data_directory' + become: true + become_user: 'postgres' + changed_when: false # read-only check + check_mode: false # run task even if `--check` is specified + listen: + - 'postgresql_server: validate config; reload postgresql' + - 'postgresql_server: validate config; restart postgresql' + +# pg_hba.conf has no offline check. The pg_hba_file_rules view parses the file on disk on every +# query and reports each broken line (fill_hba_view() in src/backend/utils/adt/hbafuncs.c), so it +# needs the running server. Its port is taken from line 4 of postmaster.pid +# (LOCK_FILE_LINE_PORT in src/include/utils/pidfile.h) instead of postgresql_server__conf_port, +# because a changed port only takes effect with the restart below. +# Skipped where the restart is skipped as well: the service was just started with this +# configuration, or it is supposed to be stopped. +- name: 'postgresql_server: cat postmaster.pid' + ansible.builtin.slurp: + src: '{{ postgresql_server__data_dir }}/postmaster.pid' + register: '__postgresql_server__postmaster_pid_result' + failed_when: false # the server is not running, nothing to check against + listen: + - 'postgresql_server: validate config; reload postgresql' + - 'postgresql_server: validate config; restart postgresql' + when: + - '__postgresql_server__service_state_result is not defined or __postgresql_server__service_state_result is not changed' + - 'postgresql_server__state != "stopped"' + +- name: 'postgresql_server: SELECT line_number, error FROM pg_hba_file_rules WHERE error IS NOT NULL' + community.postgresql.postgresql_query: + port: '{{ (__postgresql_server__postmaster_pid_result["content"] | b64decode).splitlines()[3] }}' + login_password: '{{ postgresql_server__login_password | default(omit) }}' + query: 'SELECT line_number, error FROM pg_hba_file_rules WHERE error IS NOT NULL' + become: true + become_user: 'postgres' + register: '__postgresql_server__pg_hba_errors_result' + check_mode: false # read-only query, run it even if `--check` is specified + listen: + - 'postgresql_server: validate config; reload postgresql' + - 'postgresql_server: validate config; restart postgresql' + when: + - '__postgresql_server__postmaster_pid_result["content"] is defined' + +- name: 'postgresql_server: Assert that pg_hba.conf has no errors' + ansible.builtin.assert: + that: + - '__postgresql_server__pg_hba_errors_result["query_result"] | length == 0' + quiet: true + fail_msg: >- + {{ postgresql_server__data_dir }}/pg_hba.conf contains errors, PostgreSQL was neither reloaded nor restarted: + {% for row in __postgresql_server__pg_hba_errors_result["query_result"] %}line {{ row["line_number"] }}: {{ row["error"] }}; {% endfor %} + listen: + - 'postgresql_server: validate config; reload postgresql' + - 'postgresql_server: validate config; restart postgresql' + when: + - '__postgresql_server__pg_hba_errors_result is not skipped' + +# A reload applies the configuration without an outage, so it is not skipped by +# lfops__skip_restart_handlers. The remaining guards are the same as for the restart below. +- name: 'postgresql_server: reload postgresql' + ansible.builtin.service: + name: '{{ postgresql_server__service_name }}' + state: 'reloaded' + listen: 'postgresql_server: validate config; reload postgresql' + when: + - '__postgresql_server__service_state_result is not defined or __postgresql_server__service_state_result is not changed' + - 'postgresql_server__state != "stopped"' + # skip restart if the operator deferred restarts LFOps-wide (lfops__skip_restart_handlers), if # the service was just started (redundant), or if the user wants it stopped. # __postgresql_server__service_state_result is registered on a dedicated state-only task (not the # enabled task), so that enabled-only changes don't prevent the restart. # `is not defined` covers a run that skips the state block (`--skip-tags postgresql_server:state`), # where the result is never registered. -- name: 'postgresql_server: restart posgresql' +- name: 'postgresql_server: restart postgresql' ansible.builtin.service: - name: 'postgresql-{{ postgresql_server__version }}' + name: '{{ postgresql_server__service_name }}' state: 'restarted' + listen: 'postgresql_server: validate config; restart postgresql' when: - 'not (lfops__skip_restart_handlers | d(false) | bool)' - '__postgresql_server__service_state_result is not defined or __postgresql_server__service_state_result is not changed' diff --git a/roles/postgresql_server/tasks/main.yml b/roles/postgresql_server/tasks/main.yml index 7e49a9304..d4d1203b8 100644 --- a/roles/postgresql_server/tasks/main.yml +++ b/roles/postgresql_server/tasks/main.yml @@ -1,5 +1,56 @@ - block: + - name: 'Assert that postgresql_server__version is a major version' + ansible.builtin.assert: + that: + - 'postgresql_server__version is defined' + - 'postgresql_server__version | string is match("^\\d+$")' + quiet: true + fail_msg: 'postgresql_server__version is mandatory and takes the major version of PostgreSQL, for example "18".' + + # Ansible ignores an unknown inventory variable without a word, so an inventory that still sets + # the replaced variable would silently deploy the role defaults instead. + - name: 'Assert that postgresql_server__pg_hba_entries is not set' + ansible.builtin.assert: + that: + - 'postgresql_server__pg_hba_entries is not defined' + quiet: true + fail_msg: 'postgresql_server__pg_hba_entries is replaced by postgresql_server__pg_hba_local_entries__*_var and postgresql_server__pg_hba_host_entries__*_var. Move the entries there, see the role README.' + + # A host record without an address, or with a type PostgreSQL does not know, keeps + # PostgreSQL from starting at all: postmaster aborts with "could not load pg_hba.conf". + - name: 'Assert that every pg_hba host entry has a valid type and an address' + ansible.builtin.assert: + that: + - 'item["type"] in ["host", "hostgssenc", "hostnogssenc", "hostnossl", "hostssl"]' + - 'item["address"] | string | length > 0' + quiet: true + fail_msg: 'postgresql_server__pg_hba_host_entries: "type" must be one of host, hostgssenc, hostnogssenc, hostnossl or hostssl, and "address" must not be empty.' + loop: '{{ postgresql_server__pg_hba_host_entries__combined_var }}' + loop_control: + label: '{{ item["type"] }} {{ item["database"] }} {{ item["user"] }} {{ item["address"] }}' + + tags: + - 'always' + + +- block: + + # The distribution package keeps its cluster in /var/lib/pgsql/data and listens on the same port. + # Installing the PostgreSQL Yum Repository release next to it would initialize a second, empty + # cluster that cannot start while the first one runs. + - name: 'stat /var/lib/pgsql/data/PG_VERSION' + ansible.builtin.stat: + path: '/var/lib/pgsql/data/PG_VERSION' + register: '__postgresql_server__distribution_cluster_result' + + - name: 'Assert that there is no cluster of the distribution package' + ansible.builtin.assert: + that: + - 'not __postgresql_server__distribution_cluster_result["stat"]["exists"]' + quiet: true + fail_msg: 'This host has a PostgreSQL cluster of the distribution package in /var/lib/pgsql/data. The role only installs PostgreSQL from the PostgreSQL Yum Repository. Migrate the data first, see the Troubleshooting section of the role README.' + - name: 'Install postgresql{{ postgresql_server__version }}-server' ansible.builtin.package: name: @@ -10,43 +61,19 @@ - 'postgresql_server' +# The names and paths only depend on postgresql_server__version, not on the state of the host, so +# they are safe to set under every tag. - block: - name: 'set internal vars for role' ansible.builtin.set_fact: - postgresql_server__installed_version: '{{ postgresql_server__version }}' - postgresql_server__service_name: 'postgresql-{{ postgresql_server__version }}' + __postgresql_server__bin_dir: '/usr/pgsql-{{ postgresql_server__version }}/bin' postgresql_server__data_dir: '/var/lib/pgsql/{{ postgresql_server__version }}/data' postgresql_server__initdb_cmd: '/usr/pgsql-{{ postgresql_server__version }}/bin/postgresql-{{ postgresql_server__version }}-setup initdb' + postgresql_server__service_name: 'postgresql-{{ postgresql_server__version }}' - when: - - 'postgresql_server__version | string | length > 0' - tags: - - 'postgresql_server' - - 'postgresql_server:state' - - -- block: - - - name: 'Get the list of installed packages' - ansible.builtin.package_facts: # yamllint disable-line rule:empty-values - check_mode: false # run task even if `--check` is specified - - - name: 'Get postgresql-server version' - ansible.builtin.set_fact: - postgresql_server__installed_version: '{{ (ansible_facts["packages"]["postgresql-server"][0]["version"] | d(ansible_facts["packages"]["postgresql-server"][0]["version"])) | regex_replace("\.\d+$", "") }}' # strip the patch version - - - name: 'set internal vars for role' - ansible.builtin.set_fact: - postgresql_server__service_name: 'postgresql' - postgresql_server__data_dir: '/var/lib/pgsql/data' - postgresql_server__initdb_cmd: '/usr/bin/postgresql-setup --initdb' - - when: - - 'not postgresql_server__version | string | length > 0' tags: - - 'postgresql_server' - - 'postgresql_server:state' + - 'always' - block: @@ -61,7 +88,7 @@ line: "include_dir = 'conf.d'" regexp: '^include_dir(\s+|\s*=\s*)?''conf.d''\s*(#.*)?$' path: '{{ postgresql_server__data_dir }}/postgresql.conf' - notify: 'postgresql_server: restart posgresql' + notify: 'postgresql_server: validate config; restart postgresql' - name: 'mkdir -p {{ postgresql_server__data_dir }}/conf.d' ansible.builtin.file: @@ -71,6 +98,16 @@ mode: 0o700 state: 'directory' + - name: 'Deploy {{ postgresql_server__data_dir }}/conf.d/z00-linuxfabrik-reload.conf' + ansible.builtin.template: + backup: true + src: 'var/lib/pgsql/data/conf.d/z00-linuxfabrik-reload.conf.j2' + dest: '{{ postgresql_server__data_dir }}/conf.d/z00-linuxfabrik-reload.conf' + owner: 'postgres' + group: 'postgres' + mode: 0o600 + notify: 'postgresql_server: validate config; reload postgresql' + - name: 'Deploy {{ postgresql_server__data_dir }}/conf.d/z00-linuxfabrik.conf' ansible.builtin.template: backup: true @@ -79,7 +116,7 @@ owner: 'postgres' group: 'postgres' mode: 0o600 - notify: 'postgresql_server: restart posgresql' + notify: 'postgresql_server: validate config; restart postgresql' - name: 'Deploy {{ postgresql_server__data_dir }}/pg_hba.conf' ansible.builtin.template: @@ -89,7 +126,7 @@ owner: 'postgres' group: 'postgres' mode: 0o600 - notify: 'postgresql_server: restart posgresql' + notify: 'postgresql_server: validate config; reload postgresql' - name: 'Remove rpmnew / rpmsave (and Debian equivalents)' ansible.builtin.include_role: @@ -143,12 +180,28 @@ - block: + - name: 'SELECT datname FROM pg_database' + community.postgresql.postgresql_query: + port: '{{ postgresql_server__conf_port }}' + login_password: '{{ postgresql_server__login_password | default(omit) }}' + query: 'SELECT datname FROM pg_database' + become: true + become_user: 'postgres' + register: '__postgresql_server__existing_databases_result' + changed_when: false # read-only query + check_mode: false # run task even if `--check` is specified + + # The locale default only applies when a database is created. postgresql_db aborts with + # "Changing LC_COLLATE is not supported" when it is given a locale that differs from the one of + # an existing database, which would break every database created with an earlier default. + # C.UTF-8 sorts by code point on RHEL 8, 9 and 10 alike (`LC_ALL=C.utf8 sort` verified on Rocky + # 8, 9 and 10), so the order does not change with an OS upgrade. - name: 'Create or delete PostgreSQL databases' community.postgresql.postgresql_db: port: '{{ postgresql_server__conf_port }}' name: '{{ item["name"] }}' - lc_collate: '{{ item["lc_collate"] | default("en_US.UTF-8") }}' - lc_ctype: '{{ item["lc_ctype"] | default("en_US.UTF-8") }}' + lc_collate: '{{ item["lc_collate"] | default(omit if item["name"] in __postgresql_server__existing_databases else "C.UTF-8") }}' + lc_ctype: '{{ item["lc_ctype"] | default(omit if item["name"] in __postgresql_server__existing_databases else "C.UTF-8") }}' login_password: '{{ postgresql_server__login_password | default(omit) }}' encoding: '{{ item["encoding"] | default("UTF-8") }}' template: '{{ item["template"] | default("template0") }}' @@ -157,6 +210,8 @@ loop: '{{ postgresql_server__databases__combined_var }}' become: true become_user: 'postgres' + vars: + __postgresql_server__existing_databases: '{{ __postgresql_server__existing_databases_result["query_result"] | map(attribute="datname") | list }}' tags: - 'postgresql_server' diff --git a/roles/postgresql_server/templates/usr/local/sbin/postgresql-dump.j2 b/roles/postgresql_server/templates/usr/local/sbin/postgresql-dump.j2 index d835ecdc9..fa229367b 100755 --- a/roles/postgresql_server/templates/usr/local/sbin/postgresql-dump.j2 +++ b/roles/postgresql_server/templates/usr/local/sbin/postgresql-dump.j2 @@ -6,7 +6,7 @@ # License: The Unlicense, see LICENSE file. # {{ ansible_managed }} -# 2022111701 +# 2026091301 # make sure the backup is not readable by others old_umask=$(umask) @@ -14,10 +14,30 @@ umask 027 source /etc/postgresql-dump.conf +# the client tools of the running server: pg_dump refuses to dump a server that is newer than +# itself, and /usr/bin may hold the tools of another installed PostgreSQL version +BIN_DIR='{{ __postgresql_server__bin_dir }}' + rm -rf "$BACKUP_DIR" mkdir -p "$BACKUP_DIR" -DBS=$(sudo --user "$USERNAME" --login psql --tuples-only --command='SELECT datname FROM pg_database;') +final_rc=0 + +# roles and tablespaces belong to the cluster, not to a database, so pg_dump never includes them. +# restore this file first, otherwise every `ALTER ... OWNER TO` of a database dump fails. +sudo --user "$USERNAME" --login "$BIN_DIR/pg_dumpall" --globals-only > "$BACKUP_DIR/globals.sql" +rc=$? +if [ $rc -ne 0 ]; then + echo "Failed to dump the roles and tablespaces (Return Code $rc)." + final_rc=$rc +else + echo 'Roles and tablespaces dumped.' +fi +gzip -9fn "$BACKUP_DIR/globals.sql" + +# the same selection as pg_dumpall: every database that accepts connections and is not left +# invalid by an interrupted DROP DATABASE (datconnlimit -2). template0 does not accept connections. +DBS=$(sudo --user "$USERNAME" --login "$BIN_DIR/psql" --no-align --tuples-only --command='SELECT datname FROM pg_database WHERE datallowconn AND datconnlimit != -2 ORDER BY datname;') if [ $? -ne 0 ]; then echo "Cannot connect to PostgreSQL with user '$USERNAME'." exit 1 @@ -28,27 +48,18 @@ if [ "$DBS" == '' ]; then exit 0 fi -final_rc=0 -for DB in $DBS; do - case $DB in - template* ) - # do nothing - ;; - postgres ) - # do nothing - ;; - *) - sudo --user "$USERNAME" --login pg_dump --clean "$DB" > "$BACKUP_DIR/dump-$DB.sql" - rc=$? - if [ $rc -ne 0 ]; then - echo "Failed to dump $DB (Return Code $rc)." - final_rc=$rc - else - echo "Database $DB dumped." - fi - gzip -9fn "$BACKUP_DIR/dump-$DB.sql" - esac -done +# one database name per line, so names containing spaces stay intact +while IFS= read -r DB; do + sudo --user "$USERNAME" --login "$BIN_DIR/pg_dump" --clean "$DB" > "$BACKUP_DIR/dump-$DB.sql" + rc=$? + if [ $rc -ne 0 ]; then + echo "Failed to dump $DB (Return Code $rc)." + final_rc=$rc + else + echo "Database $DB dumped." + fi + gzip -9fn "$BACKUP_DIR/dump-$DB.sql" +done <<< "$DBS" umask "$old_umask" diff --git a/roles/postgresql_server/templates/var/lib/pgsql/data/conf.d/z00-linuxfabrik-reload.conf.j2 b/roles/postgresql_server/templates/var/lib/pgsql/data/conf.d/z00-linuxfabrik-reload.conf.j2 new file mode 100644 index 000000000..fad1f85db --- /dev/null +++ b/roles/postgresql_server/templates/var/lib/pgsql/data/conf.d/z00-linuxfabrik-reload.conf.j2 @@ -0,0 +1,13 @@ +# {{ ansible_managed }} +# 2026091302 + +# Settings that PostgreSQL applies on a reload (every context other than "postmaster" and +# "internal" in pg_settings). Settings that need a restart belong in z00-linuxfabrik.conf. + +#------------------------------------------------------------------------------ +# CONNECTIONS AND AUTHENTICATION +#------------------------------------------------------------------------------ + +# - Authentication - + +password_encryption = {{ postgresql_server__conf_password_encryption | d('scram-sha-256') }} diff --git a/roles/postgresql_server/templates/var/lib/pgsql/data/conf.d/z00-linuxfabrik.conf.j2 b/roles/postgresql_server/templates/var/lib/pgsql/data/conf.d/z00-linuxfabrik.conf.j2 index 54006a88b..b191d6868 100644 --- a/roles/postgresql_server/templates/var/lib/pgsql/data/conf.d/z00-linuxfabrik.conf.j2 +++ b/roles/postgresql_server/templates/var/lib/pgsql/data/conf.d/z00-linuxfabrik.conf.j2 @@ -1,17 +1,16 @@ # {{ ansible_managed }} -# 2024061001 +# 2026091302 + +# Settings that only take effect when PostgreSQL is restarted (context "postmaster" in +# pg_settings). Settings that a reload applies belong in z00-linuxfabrik-reload.conf, so that +# changing them does not restart the server. #------------------------------------------------------------------------------ # CONNECTIONS AND AUTHENTICATION #------------------------------------------------------------------------------ +# - Connection Settings - + listen_addresses = '{{ postgresql_server__conf_listen_addresses | d(['localhost']) | join(',') }}' port = {{ postgresql_server__conf_port | d(5432) }} max_connections = {{ postgresql_server__conf_max_connections | d(100) }} - - -#------------------------------------------------------------------------------ -# CONNECTIONS AND AUTHENTICATION -#------------------------------------------------------------------------------ - -password_encryption = {{ postgresql_server__conf_password_encryption | d('scram-sha-256') }} diff --git a/roles/postgresql_server/templates/var/lib/pgsql/data/pg_hba.conf.j2 b/roles/postgresql_server/templates/var/lib/pgsql/data/pg_hba.conf.j2 index fd6580c63..4b11829ca 100644 --- a/roles/postgresql_server/templates/var/lib/pgsql/data/pg_hba.conf.j2 +++ b/roles/postgresql_server/templates/var/lib/pgsql/data/pg_hba.conf.j2 @@ -1,7 +1,40 @@ # {{ ansible_managed }} -# 2022111701 +# 2026091301 +{# +PostgreSQL uses the first record that matches a connection. The role defaults are catch-all +records, so the entries that do not replace a default are rendered first, otherwise a more +specific entry from the inventory could never match. combine_lod keeps an entry at the position +it was first seen, which is why the order cannot be left to the combined variable. +#} +{% set __local = namespace(defaults=[], own=[]) %} +{% for item in postgresql_server__pg_hba_local_entries__combined_var if item['state'] | d('present') != 'absent' %} +{% if postgresql_server__pg_hba_local_entries__role_var + | selectattr('database', 'equalto', item['database']) + | selectattr('user', 'equalto', item['user']) + | list | length > 0 %} +{% set __local.defaults = __local.defaults + [item] %} +{% else %} +{% set __local.own = __local.own + [item] %} +{% endif %} +{% endfor %} +{% set __host = namespace(defaults=[], own=[]) %} +{% for item in postgresql_server__pg_hba_host_entries__combined_var if item['state'] | d('present') != 'absent' %} +{% if postgresql_server__pg_hba_host_entries__role_var + | selectattr('type', 'equalto', item['type']) + | selectattr('database', 'equalto', item['database']) + | selectattr('user', 'equalto', item['user']) + | selectattr('address', 'equalto', item['address']) + | list | length > 0 %} +{% set __host.defaults = __host.defaults + [item] %} +{% else %} +{% set __host.own = __host.own + [item] %} +{% endif %} +{% endfor %} # TYPE DATABASE USER ADDRESS METHOD -{% for item in postgresql_server__pg_hba_entries %} -{{ item['type'] }} {{ item['database'] }} {{ item['user'] }} {{ item['address'] | d('') }} {{ item['auth_method'] | d('scram-sha-256') }} {{ item['auth_options'] | d('') }} +{% for item in __local.own + __local.defaults %} +local {{ item['database'] }} {{ item['user'] }} {{ item['auth_method'] | d('scram-sha-256') }} {{ item['auth_options'] | d('') }} +{% endfor %} +{% for item in __host.own + __host.defaults %} +{{ item['type'] }} {{ item['database'] }} {{ item['user'] }} {{ item['address'] }} {{ item['auth_method'] | d('scram-sha-256') }} {{ item['auth_options'] | d('') }} {% endfor %}