Skip to content

fix(mail): unbreak join@ registration when the Mailu cert lapses - #129

Merged
ralyodio merged 1 commit into
mainfrom
fix/mailu-cert-refresh-registration
Sep 22, 2026
Merged

ralyodio merged 1 commit into
mainfrom
fix/mailu-cert-refresh-registration

Conversation

@ralyodio

@ralyodio ralyodio commented Sep 22, 2026

Copy link
Copy Markdown
Contributor

Closes #130

Registration is down, and has been since 2026-09-13

ssh join@bbs.profullstack.com creates the account, then dies at the email step:

  Email: someone@example.com
  couldn't email the code; please retry or contact an admin.
Connection to bbs.profullstack.com closed.

Reproduced against production with a throwaway key. No one can complete a signup.

Cause

Caddy owns ACME for mail.profullstack.com and renewed on 2026-08-14 (valid to Nov 12).
Mailu kept serving the certificate it loaded at container start — Jun 15 → Sep 13. On the live host right now:

endpoint serves expires
:443 (Caddy) renewed cert Nov 12 2026
:25 / :465 / :993 (Mailu) stale cert Sep 13 2026 ❌ expired

When it lapsed, internal/mail's STARTTLS handshake started failing verification, and every
transactional send went with it: join@ confirmation codes, signup notifications, notify-creds.
The last signup notification email is dated Sep 2 — the flow has been silently broken ever since.

Why one stale cert could do this

1. setup.sh never ran the refresher. It installed the script and enabled the timer
but systemctl enable --now <timer> starts the timer, not the service. A redeploy left the stale
cert untouched, and did nothing at all if the timer was never scheduled. The news and IRC sections
already run their refresher at provision time; the Mailu section now does too. This is the change
that repairs the live host on the next deploy.

2. refresh-certs.sh only compared files. A copy whose reload silently failed leaves a fresh
cert on disk and an expiring one on the wire, and nothing notices. It now reads back what the relay
actually serves, forces a reload when that disagrees with /certs, refuses to copy a source cert
that is itself expired, and no longer swallows the docker compose restart failure. It restarts
front alone — the only container that mounts ./certs.

3. internal/mail verified the certificate even on loopback, where there is nothing to
intercept. It now skips verification for a loopback relay — the same reasoning docs/mail.md
already applies to the plaintext Dovecot hand-off — and gains AGENTBBS_SMTP_SERVERNAME
(mirroring AGENTBBS_MAIL_SMTP_SERVERNAME), so the documented 127.0.0.1:25 config verifies
against the mail host rather than an IP literal. A non-loopback relay is still verified.
Errors are wrapped with the address and failing stage, so the next failure is one journal line
to diagnose instead of nine days of silence.

Tests

internal/mail had no test file. Added: envelope/CRLF handling, the unreachable-relay message,
and both halves of the TLS decision — a loopback relay with an expired cert delivers, a
non-loopback relay with the same cert is refused. Full suite green.

Repairing production

Merging and letting agentbbs-update.timer redeploy runs the refresher and fixes the live cert.
To do it immediately on the box:

sudo /usr/local/bin/agentbbs-mailu-certs     # copies Caddy's cert, reloads front, loud on failure
# verify what is actually on the wire:
printf 'QUIT\r\n' | openssl s_client -quiet -starttls smtp \
  -connect 127.0.0.1:25 -servername mail.profullstack.com 2>&1 | grep -i notAfter

Note: my reproduction left an unverified account regtest0922 on prod — safe to delete.

🤖 Generated with Claude Code

Registration has been dead since 2026-09-13. `ssh join@bbs.profullstack.com`
creates the account, then fails at the confirmation step with "couldn't email
the code" and disconnects, so nobody can finish signing up.

Cause: Caddy owns ACME for mail.profullstack.com and renewed on 2026-08-14
(valid to Nov 12), but Mailu went on serving the certificate it loaded at
container start (Jun 15 -> Sep 13). When that lapsed, the STARTTLS handshake
from internal/mail started failing verification and every transactional send
died with it -- confirmation codes, signup notifications, credential mail.
Reproduced against production; 25/465/993 all still present the expired cert
while :443 serves the renewed one.

Three things let a single stale certificate take registration down:

- setup.sh installed the refresher and enabled its *timer*, but never ran it.
  `systemctl enable --now <timer>` starts the timer, not the service, so a
  redeploy left a stale cert in place (and did nothing at all if the timer was
  never scheduled). The news and IRC sections already run theirs at provision
  time; the Mailu section now does too, which is what repairs the live host.

- refresh-certs.sh only compared files, so a copy whose reload silently failed
  left a fresh cert on disk and an expiring one on the wire -- invisible. It now
  reads back what the relay actually serves, forces a reload when that disagrees
  with /certs, refuses to copy a source cert that is itself expired, and no
  longer swallows the `docker compose restart` failure. It restarts `front`
  alone, the only container that mounts ./certs.

- internal/mail verified the relay's certificate even on loopback, where there
  is nothing to intercept. It now skips verification for a loopback relay (the
  reasoning docs/mail.md already applies to the plaintext Dovecot hand-off) and
  gains AGENTBBS_SMTP_SERVERNAME, mirroring AGENTBBS_MAIL_SMTP_SERVERNAME, so
  the documented 127.0.0.1:25 config can verify against the mail host instead of
  an IP literal. A non-loopback relay is still verified. Errors are wrapped with
  the address and the failing stage so the next failure is one journal line to
  diagnose rather than nine days of silence.

Tests cover the envelope, the unreachable-relay message, and both halves of the
TLS decision: a loopback relay with an expired cert delivers, a non-loopback one
with the same cert is refused.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Comment thread internal/mail/mail.go
name := c.tlsServerName()
conf := &tls.Config{ServerName: name}
if isLoopback(c.Host) {
conf = &tls.Config{ServerName: name, InsecureSkipVerify: true} // #nosec G402 -- loopback relay, see doc comment
@github-actions

Copy link
Copy Markdown

ThreatCrush Security Scan

10 finding(s)

HIGH/CRITICAL: 7 | LOW: 3

Severity Rule Location
HIGH secret-generic-credential deploy/ergo/ircd.yaml:162
HIGH secret-generic-credential deploy/ergo/ircd.yaml:221
HIGH secret-generic-credential deploy/ergo/ircd.yaml:638
HIGH secret-generic-credential deploy/ergo/ircd.yaml:787
HIGH secret-database-url deploy/ergo/ircd.yaml:893
HIGH secret-generic-credential deploy/ergo/ircd.yaml:1025
HIGH secret-generic-credential deploy/ergo/ircd.yaml:1033
LOW secret-generic-credential deploy/ergo/ircd.yaml:772
LOW tls-verification-disabled internal/mail/mail.go:109
LOW secret-generic-credential internal/mailu/mailu_test.go:44

Snippets are redacted; ThreatCrush never prints matched credential material.

@ralyodio
ralyodio merged commit 249a4e6 into main Sep 22, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Registration is broken: join@ can't send confirmation codes (expired Mailu TLS cert)

2 participants