docs: rewrite the docs and code comments in ASD-STE100

CLAUDE.md

@@ -1,76 +1,106 @@
 # dispatch
 
-Starts the katoptra mirrors' workflows on UTC slots, from a systemd timer on a NixOS host.
-`README.md` is for users; this file is the design.
+This repository is the scheduler. It starts the workflows of the katoptra mirrors at UTC
+slots. A systemd timer on a NixOS host starts it. `README.md` is for users. This file is
+the design.
 
 ## The files
 
-- `schedules/` -- `Slot` (a bitmask of UTC hours, each at `HH:42`), the named aliases,
-  `Latest`, `Job`, and one file per katoptra repo registering its jobs. Routine changes
-  touch only this directory.
-- `tick.go` -- the state file, `Plan`, and `Tick`: lock, load, plan, write ahead, dispatch.
-- `github.go` -- App JWT, the org's installation, installation token, `workflow_dispatch`,
-  retries. No SDK.
-- `main.go` -- flags, credentials from `$CREDENTIALS_DIRECTORY`, the healthcheck ping, the
-  exit code.
-- `module.nix` -- the timer and the hardened oneshot. `flake.nix` -- package, module, checks.
-- `Taskfile.yml`, `Dockerfile` -- every Go command runs in the toolbox image.
+- `schedules/`: `Slot` (a bitmask of UTC hours, each at `HH:42`), the slot names (`Hourly`,
+  `Evening`, `Overnight`, `Morning`, `Afternoon`), `Latest`, `Job`, and one file for each
+  katoptra repository. Each of these files adds the jobs of its repository. A usual change
+  is only in this directory.
+- `tick.go`: the state file, `Plan`, and `Tick`. `Tick` does these steps in this sequence:
+  1. It locks the state directory.
+  2. It loads the state.
+  3. It uses `Plan` to find the jobs to dispatch.
+  4. It gets the installation token (`Dispatcher.Prepare`).
+  5. It records the slots in `state.json`, before the first POST.
+  6. It dispatches the workflows.
+- `github.go`: the App JWT, the installation of the App on the organization, the
+  installation token, `workflow_dispatch`, and the retries. It uses no SDK.
+- `main.go`: the flags, the credentials from `$CREDENTIALS_DIRECTORY`, the healthcheck ping,
+  and the exit code.
+- `module.nix`: the timer, and the oneshot service with its systemd protections.
+- `flake.nix`: the package, the module, and the checks.
+- `Taskfile.yml` and `Dockerfile`: each Go command operates in the toolbox image.
 
 ## Constraints
 
-- It dispatches. It does not run work or watch outcomes; each mirror pings its own
-  healthcheck.
-- katoptra repositories only; the App is installed on the `katoptra` org alone.
-- Standard library only (`vendorHash = null`). Go 1.26, matching nixos-26.05's
-  `buildGoModule`; the toolbox image pins the same.
-- UTC throughout. No DST handling anywhere.
-- No secrets in the repo, the store, or the state file. The module takes file paths.
+- The scheduler only dispatches. It does not do the work of a run, and it does not monitor
+  the result of a run. Each mirror sends pings to its healthcheck.
+- It dispatches workflows only in katoptra repositories. The App is installed only on the
+  `katoptra` organization.
+- It uses only the Go standard library (`vendorHash = null`).
+- It uses Go 1.26, the same version as the `buildGoModule` of nixos-26.05. The toolbox image
+  also uses Go 1.26: its `FROM` line contains that version and a digest.
+- All times are in UTC. No part of the code adjusts a time for DST.
+- The repository, the Nix store and the state file contain no secrets. The module gets file
+  paths, not secrets.
+- Writing: obey ASD-STE100 and the rules in the
+  [Writing section of the org CONTRIBUTING](https://github.com/katoptra/.github/blob/main/CONTRIBUTING.md#writing).
+  Read that section before you write.
 
 ## Must knows
 
-- **At most once, by writing ahead.** `Tick` records every due slot in `state.json` before
-  the first POST. A crash or a failed dispatch loses that slot; nothing retries it, the next
-  slot is the retry. Moving the write after the dispatch turns this into at-least-once, and
-  `TestSlotIsRecordedBeforeTheDispatch` fails.
-- **The token comes before the write.** `Dispatcher.Prepare` mints the installation token
-  before anything is recorded, since minting starts no run. A GitHub or network outage there
-  leaves the slots unrecorded and the next tick tries again, instead of losing a daily
-  mirror's day. `TestPrepareFailureRecordsNothing` holds it.
-- **Catch-up is `Plan`, not systemd.** The timer has no `Persistent=`. Every tick compares
-  each job's latest slot with its recorded one, so any number of missed slots fire once.
-  A timer per slot with `Persistent=true` would fire each missed slot instead.
-- **A corrupt `state.json` stops everything.** It is never treated as empty: empty state
-  fires every job for slots already sent. Repair or delete it by hand; deleting fires each
-  job once.
-- **A new job fires at the next tick**, for its latest slot, because it has no entry.
-- **The timer is `*:02/5 UTC`.** It starts at :02 so `:42` is a tick; `UTC` because the
-  jgrid.net hosts run in `America/Los_Angeles`. Changing `schedules.Minute` means changing
-  the timer too.
-- **Every dispatch is `ref: main`, empty inputs.** A repo whose default branch is not
-  `main` answers 404, which is fatal.
-- **Retries (10 s, 20 s, 40 s) depend on whether a repeat is harmless.** A 4xx other than
-  408/429 is never retried. The two mint requests retry any 5xx, 408, 429 or network error.
-  A dispatch POST retries only 408, 429 and a connection that never opened: a 5xx or a
-  timeout can follow a dispatch GitHub already accepted, and a retry would be a second run.
-- **GitHub gets two minutes a tick** (`githubBudget`), retries included; a retry that would
-  pass the deadline is not made. The rest of `TimeoutStartSec=4min` is the ping's, so a
-  failing tick still reaches `/fail`.
-- **The binary is renamed in `postInstall`.** `buildGoModule` names it after the module
-  path's last element, `dispatch`; the unit runs `katoptra-dispatch`. The `module` check
-  fails if `ExecStart` is not an executable.
-- **Journal priorities** (`<3>`, `<4>`) are written only when `JOURNAL_STREAM` is set, so
-  a hand run prints plain lines.
+- **At-most-once, because `Tick` records the slots first.** Before the first POST, `Tick`
+  records in `state.json` the slot of each job that it will dispatch. After a crash or an
+  error in a workflow start, that slot gets no run. The scheduler does not try that slot
+  again: the next slot is the retry. Do not put the code that records the slots after the
+  POST. A slot can then get two runs (at-least-once), and
+  `TestSlotIsRecordedBeforeTheDispatch` finds that error.
+- **`Tick` gets the token before it records a slot.** `Dispatcher.Prepare` gets the
+  installation token. `Tick` can do this step first because a token request starts no run.
+  If GitHub or the network is not available at that step, `Tick` records no slot, and the
+  next tick tries again. Thus, a daily mirror gets its run for that day.
+  `TestPrepareFailureRecordsNothing` makes sure that this sequence does not change.
+- **`Plan`, not systemd, finds the slots that had no run.** The timer has no
+  `Persistent=`. Each tick compares the latest slot of each job with the slot that the
+  state records for it. Thus, a job that had no run for one or more slots gets one run. If
+  each slot has its timer with `Persistent=true`, systemd starts a run for each slot that
+  had no run.
+- **A corrupted `state.json` stops all jobs.** `LoadState` gives an error for it, not empty
+  state, because empty state starts each job again for slots that had a run. Repair or
+  delete the file manually. If you delete it, each job starts one time.
+- **A new job starts at the next tick**, for its latest slot, because the state has no entry
+  for it.
+- **The timer is `*:02/5 UTC`.** The timer starts at :02. Thus, `:42` is a tick. The
+  calendar has `UTC` because the time zone of the jgrid.net hosts is `America/Los_Angeles`.
+  If you change `schedules.Minute`, also change the timer.
+- **The scheduler dispatches on `ref: main`, with empty inputs.** If the default branch of
+  a repository is not `main`, GitHub sends a 404. The scheduler does not try again, and
+  that slot gets no run.
+- **A retry (after 10 s, 20 s, 40 s) occurs only when a second request cannot cause a
+  second run.**
+  - A 4xx other than 408 and 429 gets no retry.
+  - The two token requests try again after a 5xx, 408, 429, or network error.
+  - A workflow start tries again only after a 408, a 429, or a connection that did not
+    open. A 5xx or a timeout can follow a workflow start that GitHub accepted, and a retry
+    then starts a second run.
+- **GitHub gets two minutes in each tick** (`githubBudget`), with the retries included. If
+  a retry cannot start in the two minutes, the scheduler does not make it. The other two
+  minutes of `TimeoutStartSec=4min` are for the ping. Thus, a tick with errors can send its
+  ping to `/fail`.
+- **`postInstall` changes the name of the binary.** `buildGoModule` gives the binary the
+  name of the last part of the module path: `dispatch`. The unit starts
+  `katoptra-dispatch`. If `ExecStart` is not an executable, the `module` check stops with
+  an error.
+- **`Log` writes the journal priorities** (`<3>`, `<4>`) only when `JOURNAL_STREAM` is set.
+  Thus, a manual start of the binary prints its lines without these prefixes.
 
 ## Verifying a change
 
-`task check` is what CI runs: gofmt, go vet, go test, in the toolbox image.
+CI uses `task check`: gofmt, go vet and go test, in the toolbox image.
 
-`nix flake check` builds the package (its tests run in `checkPhase`) and renders the two
-units from a minimal system. Without nix on the laptop:
+`nix flake check` builds the package and makes the two units from a small NixOS system
+that has the module. The tests of the package operate in `checkPhase`. If the laptop does
+not have nix, use this command:
 
     container run --rm --memory 4G -v "$PWD":/work -w /work nixos/nix sh -c \
       'git config --global --add safe.directory /work &&
        NIX_CONFIG="experimental-features = nix-command flakes" nix flake check'
 
-Files must be `git add`ed for the flake to see them. Only a real tick proves the GitHub App
-contract: on the host, `sudo systemctl start katoptra-dispatch` and read the journal.
+The flake uses only the files in the git index. Use `git add` on each new file first.
+
+Only a tick on the host can show that the GitHub App contract is correct. On the host, use
+`sudo systemctl start katoptra-dispatch`. Then read the journal.

Dockerfile

@@ -1,6 +1,7 @@
-# The toolbox: every check runs in here, on a laptop and in CI alike. Go 1.26 because the
-# host builds with nixos-26.05's buildGoModule, which is Go 1.26; a newer toolchain here
-# could accept code the host's build refuses.
+# The toolbox: each check operates in this image, on a laptop and in CI. It uses Go 1.26
+# because the host builds with the buildGoModule of nixos-26.05, which is Go 1.26. Do not
+# use a newer Go toolchain here. A newer toolchain can accept code that the build on the
+# host rejects.
 FROM golang:1.26-alpine@sha256:8ac98ca534ac3f51e1f420a1dd2c15e74c75cfa0f23f3ad27eb5d7236c349a0c
-# No .git inside the mount is guaranteed, and nothing reads the stamp.
+# The mount does not always contain .git, and no code reads the VCS stamp.
 ENV GOFLAGS=-buildvcs=false