[TeleporterGuy]

What

A server install had no way to update itself. Someone had to SSH in and run the four "Upgrading" commands by hand. This adds an opt-in nightly auto-update for servers, as a counterpart to the desktop self-update (#19).

• deploy/auto-update.sh: one run is the documented upgrade (backup, git pull, docker compose build, up -d), plus the checks a person would make by eye:
  • Nothing new on the tracked branch: it exits without touching anything.
  • Build fails: the running containers are left alone, and the checkout goes back to the old commit.
  • New server doesn't log listening on within 180s, or stops right after: it retags the images it ran before (saved as :before-update before building), so going back is a restart, not a rebuild.
  • After a failed update: that commit is recorded in .auto-update-failed and skipped until the branch moves past it. A broken commit on main costs one restart, not one every night.
  • Checkout has commits that aren't on the remote branch: it refuses to touch it.
  • Two runs at once: flock lets only one through, so a manual run can't race the timer.
  • Cleanup: it keeps the newest 7 of its own backups (other files in that folder are left alone) and prunes dangling images.
• --install / --uninstall write and remove a systemd service plus timer: nightly at 04:00 with a 30-minute random delay, and Persistent=true so a VM that was off catches up. The service runs as the checkout's owner, not root. Someone who isn't a Linux expert only needs sudo /opt/stem/deploy/auto-update.sh --install.
• Settings: three optional values from the environment or .env: STEM_BACKUP_DIR, STEM_KEEP_BACKUPS, STEM_UPDATE_TIMEOUT.
• The script updates itself on each pull. The whole body is inside main(), which is called on the last line, so bash has read the entire file before git pull can replace it.
• Docs: a new "Upgrading automatically" subsection in docs/running-on-a-server.md, plus a 0.7.2 (Unreleased) release-notes entry. .gitignore now covers the default backups/, the lock file and the failed-commit marker.

Testing

All runs were against a real Tailscale server install (Ubuntu 24.04, systemd 255, Docker Compose v2) at /opt/stem:

• Nothing new: prints up to date and exits 0.
• One-commit update with a real image change: backup written, container recreated, build <sha> in the server log matches the new commit, exits 0 in about 15s with cached layers.
• Update whose image is byte-identical (docs/release-notes-only commit): passes without a restart. The first version of the health check failed on this case and is why the check now reads logs from the container's own StartedAt.
• Forced failure (STEM_UPDATE_TIMEOUT=1): rolled back to the previous images and commit, the server came back on the old build, exit 1. The next run printed skipping <sha>: it did not come up last time and exited 0. Removing the marker let the update go through.
• Two runs in the same minute: this collided on the archive name (export refuses to overwrite and the run stopped safely). Archive names now include seconds.
• --install: the unit files it writes, generated into a temp dir, pass systemd-analyze verify. Not tested: an actual install on a host, because the test server needs a sudo password. That's worth one sudo … --install plus systemctl start stem-auto-update.service after merge.
• No shellcheck run (not installed on either machine). bash -n is clean.

🤖 Generated with Claude Code