diff --git a/CLAUDE.md b/CLAUDE.md
index dddfcd7..ff825d9 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -1,76 +1,106 @@
# dispatch
-Starts the katoptra mirrors' workflows on UTC slots, from a systemd timer on a NixOS host.
-`README.md` is for users; this file is the design.
+This repository is the scheduler. It starts the workflows of the katoptra mirrors at UTC
+slots. A systemd timer on a NixOS host starts it. `README.md` is for users. This file is
+the design.
## The files
-- `schedules/` -- `Slot` (a bitmask of UTC hours, each at `HH:42`), the named aliases,
- `Latest`, `Job`, and one file per katoptra repo registering its jobs. Routine changes
- touch only this directory.
-- `tick.go` -- the state file, `Plan`, and `Tick`: lock, load, plan, write ahead, dispatch.
-- `github.go` -- App JWT, the org's installation, installation token, `workflow_dispatch`,
- retries. No SDK.
-- `main.go` -- flags, credentials from `$CREDENTIALS_DIRECTORY`, the healthcheck ping, the
- exit code.
-- `module.nix` -- the timer and the hardened oneshot. `flake.nix` -- package, module, checks.
-- `Taskfile.yml`, `Dockerfile` -- every Go command runs in the toolbox image.
+- `schedules/`: `Slot` (a bitmask of UTC hours, each at `HH:42`), the slot names (`Hourly`,
+ `Evening`, `Overnight`, `Morning`, `Afternoon`), `Latest`, `Job`, and one file for each
+ katoptra repository. Each of these files adds the jobs of its repository. A usual change
+ is only in this directory.
+- `tick.go`: the state file, `Plan`, and `Tick`. `Tick` does these steps in this sequence:
+ 1. It locks the state directory.
+ 2. It loads the state.
+ 3. It uses `Plan` to find the jobs to dispatch.
+ 4. It gets the installation token (`Dispatcher.Prepare`).
+ 5. It records the slots in `state.json`, before the first POST.
+ 6. It dispatches the workflows.
+- `github.go`: the App JWT, the installation of the App on the organization, the
+ installation token, `workflow_dispatch`, and the retries. It uses no SDK.
+- `main.go`: the flags, the credentials from `$CREDENTIALS_DIRECTORY`, the healthcheck ping,
+ and the exit code.
+- `module.nix`: the timer, and the oneshot service with its systemd protections.
+- `flake.nix`: the package, the module, and the checks.
+- `Taskfile.yml` and `Dockerfile`: each Go command operates in the toolbox image.
## Constraints
-- It dispatches. It does not run work or watch outcomes; each mirror pings its own
- healthcheck.
-- katoptra repositories only; the App is installed on the `katoptra` org alone.
-- Standard library only (`vendorHash = null`). Go 1.26, matching nixos-26.05's
- `buildGoModule`; the toolbox image pins the same.
-- UTC throughout. No DST handling anywhere.
-- No secrets in the repo, the store, or the state file. The module takes file paths.
+- The scheduler only dispatches. It does not do the work of a run, and it does not monitor
+ the result of a run. Each mirror sends pings to its healthcheck.
+- It dispatches workflows only in katoptra repositories. The App is installed only on the
+ `katoptra` organization.
+- It uses only the Go standard library (`vendorHash = null`).
+- It uses Go 1.26, the same version as the `buildGoModule` of nixos-26.05. The toolbox image
+ also uses Go 1.26: its `FROM` line contains that version and a digest.
+- All times are in UTC. No part of the code adjusts a time for DST.
+- The repository, the Nix store and the state file contain no secrets. The module gets file
+ paths, not secrets.
+- Writing: obey ASD-STE100 and the rules in the
+ [Writing section of the org CONTRIBUTING](https://github.com/katoptra/.github/blob/main/CONTRIBUTING.md#writing).
+ Read that section before you write.
## Must knows
-- **At most once, by writing ahead.** `Tick` records every due slot in `state.json` before
- the first POST. A crash or a failed dispatch loses that slot; nothing retries it, the next
- slot is the retry. Moving the write after the dispatch turns this into at-least-once, and
- `TestSlotIsRecordedBeforeTheDispatch` fails.
-- **The token comes before the write.** `Dispatcher.Prepare` mints the installation token
- before anything is recorded, since minting starts no run. A GitHub or network outage there
- leaves the slots unrecorded and the next tick tries again, instead of losing a daily
- mirror's day. `TestPrepareFailureRecordsNothing` holds it.
-- **Catch-up is `Plan`, not systemd.** The timer has no `Persistent=`. Every tick compares
- each job's latest slot with its recorded one, so any number of missed slots fire once.
- A timer per slot with `Persistent=true` would fire each missed slot instead.
-- **A corrupt `state.json` stops everything.** It is never treated as empty: empty state
- fires every job for slots already sent. Repair or delete it by hand; deleting fires each
- job once.
-- **A new job fires at the next tick**, for its latest slot, because it has no entry.
-- **The timer is `*:02/5 UTC`.** It starts at :02 so `:42` is a tick; `UTC` because the
- jgrid.net hosts run in `America/Los_Angeles`. Changing `schedules.Minute` means changing
- the timer too.
-- **Every dispatch is `ref: main`, empty inputs.** A repo whose default branch is not
- `main` answers 404, which is fatal.
-- **Retries (10 s, 20 s, 40 s) depend on whether a repeat is harmless.** A 4xx other than
- 408/429 is never retried. The two mint requests retry any 5xx, 408, 429 or network error.
- A dispatch POST retries only 408, 429 and a connection that never opened: a 5xx or a
- timeout can follow a dispatch GitHub already accepted, and a retry would be a second run.
-- **GitHub gets two minutes a tick** (`githubBudget`), retries included; a retry that would
- pass the deadline is not made. The rest of `TimeoutStartSec=4min` is the ping's, so a
- failing tick still reaches `/fail`.
-- **The binary is renamed in `postInstall`.** `buildGoModule` names it after the module
- path's last element, `dispatch`; the unit runs `katoptra-dispatch`. The `module` check
- fails if `ExecStart` is not an executable.
-- **Journal priorities** (`<3>`, `<4>`) are written only when `JOURNAL_STREAM` is set, so
- a hand run prints plain lines.
+- **At-most-once, because `Tick` records the slots first.** Before the first POST, `Tick`
+ records in `state.json` the slot of each job that it will dispatch. After a crash or an
+ error in a workflow start, that slot gets no run. The scheduler does not try that slot
+ again: the next slot is the retry. Do not put the code that records the slots after the
+ POST. A slot can then get two runs (at-least-once), and
+ `TestSlotIsRecordedBeforeTheDispatch` finds that error.
+- **`Tick` gets the token before it records a slot.** `Dispatcher.Prepare` gets the
+ installation token. `Tick` can do this step first because a token request starts no run.
+ If GitHub or the network is not available at that step, `Tick` records no slot, and the
+ next tick tries again. Thus, a daily mirror gets its run for that day.
+ `TestPrepareFailureRecordsNothing` makes sure that this sequence does not change.
+- **`Plan`, not systemd, finds the slots that had no run.** The timer has no
+ `Persistent=`. Each tick compares the latest slot of each job with the slot that the
+ state records for it. Thus, a job that had no run for one or more slots gets one run. If
+ each slot has its timer with `Persistent=true`, systemd starts a run for each slot that
+ had no run.
+- **A corrupted `state.json` stops all jobs.** `LoadState` gives an error for it, not empty
+ state, because empty state starts each job again for slots that had a run. Repair or
+ delete the file manually. If you delete it, each job starts one time.
+- **A new job starts at the next tick**, for its latest slot, because the state has no entry
+ for it.
+- **The timer is `*:02/5 UTC`.** The timer starts at :02. Thus, `:42` is a tick. The
+ calendar has `UTC` because the time zone of the jgrid.net hosts is `America/Los_Angeles`.
+ If you change `schedules.Minute`, also change the timer.
+- **The scheduler dispatches on `ref: main`, with empty inputs.** If the default branch of
+ a repository is not `main`, GitHub sends a 404. The scheduler does not try again, and
+ that slot gets no run.
+- **A retry (after 10 s, 20 s, 40 s) occurs only when a second request cannot cause a
+ second run.**
+ - A 4xx other than 408 and 429 gets no retry.
+ - The two token requests try again after a 5xx, 408, 429, or network error.
+ - A workflow start tries again only after a 408, a 429, or a connection that did not
+ open. A 5xx or a timeout can follow a workflow start that GitHub accepted, and a retry
+ then starts a second run.
+- **GitHub gets two minutes in each tick** (`githubBudget`), with the retries included. If
+ a retry cannot start in the two minutes, the scheduler does not make it. The other two
+ minutes of `TimeoutStartSec=4min` are for the ping. Thus, a tick with errors can send its
+ ping to `/fail`.
+- **`postInstall` changes the name of the binary.** `buildGoModule` gives the binary the
+ name of the last part of the module path: `dispatch`. The unit starts
+ `katoptra-dispatch`. If `ExecStart` is not an executable, the `module` check stops with
+ an error.
+- **`Log` writes the journal priorities** (`<3>`, `<4>`) only when `JOURNAL_STREAM` is set.
+ Thus, a manual start of the binary prints its lines without these prefixes.
## Verifying a change
-`task check` is what CI runs: gofmt, go vet, go test, in the toolbox image.
+CI uses `task check`: gofmt, go vet and go test, in the toolbox image.
-`nix flake check` builds the package (its tests run in `checkPhase`) and renders the two
-units from a minimal system. Without nix on the laptop:
+`nix flake check` builds the package and makes the two units from a small NixOS system
+that has the module. The tests of the package operate in `checkPhase`. If the laptop does
+not have nix, use this command:
container run --rm --memory 4G -v "$PWD":/work -w /work nixos/nix sh -c \
'git config --global --add safe.directory /work &&
NIX_CONFIG="experimental-features = nix-command flakes" nix flake check'
-Files must be `git add`ed for the flake to see them. Only a real tick proves the GitHub App
-contract: on the host, `sudo systemctl start katoptra-dispatch` and read the journal.
+The flake uses only the files in the git index. Use `git add` on each new file first.
+
+Only a tick on the host can show that the GitHub App contract is correct. On the host, use
+`sudo systemctl start katoptra-dispatch`. Then read the journal.
diff --git a/Dockerfile b/Dockerfile
index 59437bc..01e0bd4 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -1,6 +1,7 @@
-# The toolbox: every check runs in here, on a laptop and in CI alike. Go 1.26 because the
-# host builds with nixos-26.05's buildGoModule, which is Go 1.26; a newer toolchain here
-# could accept code the host's build refuses.
+# The toolbox: each check operates in this image, on a laptop and in CI. It uses Go 1.26
+# because the host builds with the buildGoModule of nixos-26.05, which is Go 1.26. Do not
+# use a newer Go toolchain here. A newer toolchain can accept code that the build on the
+# host rejects.
FROM golang:1.26-alpine@sha256:8ac98ca534ac3f51e1f420a1dd2c15e74c75cfa0f23f3ad27eb5d7236c349a0c
-# No .git inside the mount is guaranteed, and nothing reads the stamp.
+# The mount does not always contain .git, and no code reads the VCS stamp.
ENV GOFLAGS=-buildvcs=false
diff --git a/README.md b/README.md
index e275886..35cc180 100644
--- a/README.md
+++ b/README.md
@@ -9,7 +9,7 @@
dispatch
-Runs the katoptra mirrors on a schedule.
+The scheduler that starts the katoptra mirrors.
@@ -17,48 +17,53 @@
-Each katoptra mirror is a GitHub Actions workflow with no schedule of its own. This repo
-starts them.
+Each katoptra mirror is a GitHub Actions workflow that has no schedule. The scheduler in
+this repository starts these workflows.
-- A systemd timer on a NixOS host runs it every five minutes.
-- Each run starts any workflow that is due, then pings a healthcheck.
-- If the host goes down, each job that missed a run fires once when it comes back.
-- A job never fires twice for the same time slot.
-- It is one Go binary with no dependencies outside the standard library.
+- A systemd timer on a NixOS host starts a tick at intervals of five minutes.
+- Each tick finds the jobs to start and starts their workflows. Then the tick sends a ping
+ to a healthcheck.
+- If the host stops, each job that had no run for its slots gets one run when the host
+ starts again.
+- A job does not get two runs for the same slot.
+- The scheduler is one Go binary, and it uses only the Go standard library.
## Adding a job
-Each repo gets one file in [`schedules/`](schedules), named after the repo:
+Each repository has one file in [`schedules/`](schedules). The file has the name of the
+repository:
```go
// schedules/ctan.go
var _ = register(Job{Repo: "katoptra/ctan", File: "sync.yml", Slots: Hourly})
```
-Every slot runs at 42 minutes past the hour, UTC:
+Each slot is at minute 42 of the hour, UTC:
| Slot | UTC | Pacific (winter) |
|---|---|---|
-| `Hourly` | every hour at :42 | every hour at :42 |
+| `Hourly` | each hour at :42 | each hour at :42 |
| `Evening` | 05:42 | 21:42 |
| `Overnight` | 11:42 | 03:42 |
| `Morning` | 17:42 | 09:42 |
| `Afternoon` | 23:42 | 15:42 |
-- `S0` to `S23` name each hour. The four daily names are shortcuts for `S5`, `S11`, `S17`
- and `S23`.
-- Combine slots with `|`, like `Morning | Evening`.
-- A misspelled slot won't compile.
+- `S0` to `S23` are the names of the hours. The four daily names are the same slots as
+ `S5`, `S11`, `S17` and `S23`.
+- To put slots together, use `|`, for example `Morning | Evening`.
+- An incorrect slot name causes a compile error.
-The workflow needs three things that this repo can't check for you:
+This repository cannot examine the workflow. Make sure that the workflow has these three
+items:
-- `workflow_dispatch:` under `on:`.
-- A `concurrency` group with `cancel-in-progress: false`, so a second start waits for the
- first run to finish.
-- Its own healthcheck ping. The scheduler starts runs but never sees whether they pass.
+- It has `workflow_dispatch:` in `on:`.
+- It has a `concurrency` group with `cancel-in-progress: false`. Thus, a second start waits
+ for the end of the first run.
+- It sends a ping to its healthcheck. The scheduler starts runs, but it does not know the
+ result of a run.
-A change goes live when the host's flake lock moves to the new commit. A new job fires on
-the next tick.
+The host gets a change when the host's flake moves its lock to a new commit of this flake.
+A new job starts at the next tick.
## How it works
@@ -70,41 +75,46 @@ flowchart LR
token -. "GitHub down, nothing saved" .-> ping
```
-The timer fires at :02, :07 and every five minutes after that. Each tick:
+The timer starts a tick at :02, at :07, and at intervals of five minutes after that. Each
+tick does these steps:
-1. Loads `state.json`, which holds the last slot each job fired for.
-2. Works out which jobs are due. A job that missed several slots is due once, for the
- latest one.
-3. Gets a token for the GitHub App.
-4. Saves the new state.
-5. Starts each due workflow on `main`.
-6. Pings the healthcheck: the plain URL if everything worked, or `/fail` with the errors.
+1. It loads `state.json`, which contains the last dispatched slot of each job.
+2. It finds the jobs to start. If a job had no run for some slots, it starts one time, for
+ the latest slot.
+3. It gets a token for the GitHub App.
+4. It saves the new state.
+5. It starts the workflow of each of these jobs on `main`.
+6. It sends a ping to the healthcheck: to the URL if the tick has no errors, or to `/fail`
+ with the errors.
-The state is saved before any workflow starts. If a tick crashes after that, the run for
-that slot is lost and the job runs again at its next slot. No run ever starts twice. If
-GitHub is down when the token is requested, nothing is saved and the next tick tries again.
+The tick saves the state before it starts a workflow. If a crash stops the tick after that
+step, the job gets no run for that slot. The job starts again at its next slot. Thus, a run
+does not start two times. If GitHub is not available when the tick sends the token request,
+the tick does not save the state. Then the next tick tries again.
-Retries wait 10, 20 and then 40 seconds:
+Before each retry, the scheduler waits 10, 20, and then 40 seconds. The retry rules are:
-- Token requests retry on any 5xx, 408, 429 or network error.
-- Workflow starts retry only on 408, 429 or a connection that never opened. GitHub can
- return a 5xx after it has already started the run, so a retry could start it twice.
-- GitHub gets two minutes per tick, which leaves time to ping the healthcheck.
+- A token request tries again after a 5xx, 408, 429, or network error.
+- A workflow start tries again only after a 408, a 429, or a connection that did not open.
+ GitHub can send a 5xx after it started the run. Then a retry can start a second run.
+- GitHub gets two minutes in each tick. Thus, the tick has time to send its ping to the
+ healthcheck.
-The design and the reasons behind it are in [`CLAUDE.md`](CLAUDE.md).
+[`CLAUDE.md`](CLAUDE.md) gives the design and the cause of each rule.
-## Running it
+## Operating it
-`task` on its own prints the menu. From a laptop:
+`task` without a task name prints the menu. Use these commands on a laptop:
```sh
-task check # gofmt, go vet and go test in the toolbox image, same as CI
-task targets # every job and when it runs
-task runs # recent runs of each job on GitHub (needs gh logged in)
-task runs LIMIT=10 # more of them
+task check # gofmt, go vet and go test in the toolbox image, the same as CI
+task targets # each job and its slot times, in UTC
+task runs # the last 3 runs of each job on GitHub (gh must have a login)
+task runs LIMIT=10 # more runs for each job
```
-Runs started by this repo show up as `workflow_dispatch`. On the host:
+On GitHub, a run that the scheduler starts has the event `workflow_dispatch`. Use these
+commands on the host:
```sh
systemctl list-timers katoptra-dispatch.timer
@@ -114,30 +124,39 @@ sudo cat /var/lib/private/katoptra-dispatch/state.json
sudo STATE_DIRECTORY=/var/lib/private/katoptra-dispatch katoptra-dispatch --dry-run
```
-`--dry-run` shows what is due and why, without saving, starting or pinging anything.
+`--dry-run` shows the jobs to start, with the slot of each and its last dispatched slot. It
+does not save the state, start a workflow, or send a ping.
### When something goes wrong
-- **The healthcheck goes quiet.** The host or its timer is down. When it comes back, each
- job that missed a slot fires once.
-- **The healthcheck gets a `/fail` ping.** The error is in the ping body and in
- `journalctl -u katoptra-dispatch -p err`. That slot is skipped, and the job runs again
- at its next slot.
-- **A workflow start returns 404.** The repo's default branch isn't `main`, or the workflow
- file isn't there.
-- **Every tick fails on `state.json`.** The file is corrupt, and nothing runs until it is
- fixed. Repair it by hand or delete it. Deleting it makes every job fire once.
-
-## Run your own
-
-1. **Fork this repo** and replace the files in `schedules/` with your own jobs. The tests
- only accept `katoptra/` repos, so change that prefix to your organization's.
-2. **Create a GitHub App** owned by your organization, with one permission: Actions, read
- and write. It needs no webhook. Install it on every repo in the organization so new
- repos are covered automatically. Note the App ID and generate a private key. The key
- works as GitHub issues it.
-3. **Create a healthchecks.io check** with a 5 minute period and a 10 minute grace.
-4. **Add the flake to your NixOS host** and give the module three files:
+- **The healthcheck gets no pings.** The host or its timer does not operate. When the host
+ and the timer operate again, each job that had no run for a slot gets one run.
+- **The healthcheck gets a `/fail` ping.** The body of the ping and
+ `journalctl -u katoptra-dispatch -p err` show the error. If the error is from a workflow
+ start, that job gets no run for that slot, and it starts again at its next slot. If the
+ error occurs before the tick saves the state, the tick saves no slot, and the next tick
+ tries again.
+- **A workflow start gets a 404.** The default branch of the repository is not `main`, or
+ the repository does not have the workflow file.
+- **Each tick stops with an error about `state.json`.** The file is corrupted. No job starts
+ until you repair the file. Repair or delete the file manually. If you delete it, each job
+ starts one time.
+
+## Want your own?
+
+1. **Fork this repository.** Replace the files in `schedules/` with files for your jobs.
+ The tests accept only `katoptra/` repositories. Change that prefix in the tests to the
+ name of your organization. In `main.go`, change `Org: "katoptra"` to the name of your
+ organization.
+2. **Make a GitHub App in your organization:**
+ 1. Give the App one permission: Actions, read and write. A webhook is not necessary.
+ 2. Install the App on all repositories in the organization. Then each new repository
+ also gets the App automatically.
+ 3. Record the App ID.
+ 4. Make a private key. You can use the key in the format that GitHub gives.
+3. **Make a healthchecks.io check** with a period of 5 minutes and a grace time of 10
+ minutes.
+4. **Add the flake to your NixOS host.** Give the module the paths of three files:
```nix
{
@@ -151,15 +170,20 @@ sudo STATE_DIRECTORY=/var/lib/private/katoptra-dispatch katoptra-dispatch --dry-
}
```
- Keep the paths as strings. A Nix path would copy the secrets into the Nix store, where
- anyone on the host can read them. The files are loaded with `LoadCredential=`, so
- root-owned files with mode 0400 work.
-5. **Check it.** On a laptop with go-task and Docker or Apple `container`, run `task check`
- and `task targets`. `nix flake check` builds the package and the systemd units;
- [`CLAUDE.md`](CLAUDE.md) shows how to run it without nix installed. The only real test
- of the App is a live tick: run `sudo systemctl start katoptra-dispatch` on the host and
- read the journal.
+ **Caution:** Write each path as a string, not as a Nix path. A Nix path puts a copy of the
+ secret in the Nix store. All users on the host can read the Nix store.
+
+ systemd loads the files with `LoadCredential=`. Thus, the files can have the owner `root`
+ and the mode 0400.
+5. **Make sure that the scheduler operates:**
+ 1. On a laptop with go-task and Docker or Apple `container`, use `task check`. Then use
+ `task targets`.
+ 2. Use `nix flake check`. It builds the package and the systemd units.
+ [`CLAUDE.md`](CLAUDE.md) shows how to use it without nix.
+ 3. Only a tick on the host is a full test of the App. On the host, use
+ `sudo systemctl start katoptra-dispatch`. Then read the journal.
## License
-MIT. Built by [Josh Vaughen](https://ijosh.com). Pull requests are welcome.
+The license is MIT. [Josh Vaughen](https://ijosh.com) made this repository. You can send a
+pull request.
diff --git a/Taskfile.yml b/Taskfile.yml
index 3f94119..170af07 100644
--- a/Taskfile.yml
+++ b/Taskfile.yml
@@ -1,9 +1,10 @@
-# katoptra-dispatch: starts the katoptra mirrors' workflows on UTC slots, from a systemd
-# timer on a jgrid.net host. Everything here is a local check or a read; nothing deploys.
-# The host picks up a change when jshvn/jgrid.net bumps this flake in its lock.
+# katoptra-dispatch starts the workflows of the katoptra mirrors at UTC slots, from a
+# systemd timer on a jgrid.net host. Each task here is a local check or a read. No task
+# deploys a change. The host gets a change when the host's flake moves its lock to a new
+# commit of this flake.
#
-# Every Go command runs inside the toolbox image (Dockerfile), so the host supplies a
-# container engine and go-task and nothing else. The design is CLAUDE.md.
+# Each Go command operates in the toolbox image (Dockerfile). Thus, a laptop or a CI runner
+# must supply only a container engine and go-task. CLAUDE.md gives the design.
version: '3'
vars:
@@ -16,12 +17,13 @@ vars:
else echo docker
fi
IMAGE: katoptra-dispatch-toolbox:local
- # Apple container draws a progress spinner on a terminal; Docker's run has no such flag.
+ # Apple container shows a progress spinner on a terminal. `docker run` has no such flag.
QUIET: '{{if eq .ENGINE "container"}}--progress none{{end}}'
tasks:
- # The menu. ponytail: hand-maintained listing -- go-task can neither group nor annotate,
- # so a new task means a line below. `task --list` stays the generated view.
+ # The menu. ponytail: a menu that we keep manually. go-task cannot put tasks in groups or
+ # add notes to them. Thus, add a line below for each new task. `task --list` gives the
+ # view that go-task makes.
default:
desc: Print the menu
silent: true
@@ -64,8 +66,8 @@ tasks:
desc: Build the toolbox image
run: once
silent: true
- # Presence is the freshness check, so a Dockerfile edit does not rebuild on its own:
- # task clean, then any task.
+ # If the image is there, task does not build it again. Thus, a change to the Dockerfile
+ # does not cause a new build. Do `task clean`. Then the next task builds a new image.
status:
- '{{.ENGINE}} image inspect {{.IMAGE}} >/dev/null 2>&1'
cmds:
@@ -99,8 +101,9 @@ tasks:
vars:
LIMIT: '{{.LIMIT | default 3}}'
cmds:
- # GH_PAGER=cat: on a terminal gh opens its pager for each target and waits on it.
- # An unfinished run has an empty conclusion, so its status stands in.
+ # GH_PAGER=cat: on a terminal, gh opens its pager for each target and waits for it.
+ # If the `status` of a run is not `completed`, its `conclusion` is empty. Thus, the
+ # table shows the `status` as an alternative.
- |
task run -- go run . --list | awk 'NR > 1 { print $1 }' | while read -r id; do
GH_PAGER=cat gh run list --repo "${id%/*}" --workflow "${id##*/}" --limit {{.LIMIT}} \
diff --git a/flake.nix b/flake.nix
index b714148..c9ebf20 100644
--- a/flake.nix
+++ b/flake.nix
@@ -1,7 +1,8 @@
{
description = "katoptra-dispatch: starts the katoptra mirrors' workflows on UTC slots";
- # The same release jshvn/jgrid.net builds its hosts from, so the host's lock can follow it.
+ # This is the same release that the host's flake builds its hosts from. Thus, the host's
+ # lock can use the same release.
inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixos-26.05";
outputs =
@@ -26,10 +27,11 @@
(lib.fileset.fileFilter (f: f.hasExt "go") ./.)
];
};
- vendorHash = null; # standard library only
- # go test runs in buildGoModule's checkPhase; vet first, so either fails the host's build
+ vendorHash = null; # only the standard library
+ # go test operates in the checkPhase of buildGoModule. go vet operates first. Thus,
+ # an error from go vet or go test stops the build on the host.
preCheck = "go vet ./...";
- # the module path's last element would name it `dispatch`
+ # without this step, the name is `dispatch`, the last part of the module path
postInstall = "mv $out/bin/dispatch $out/bin/katoptra-dispatch";
meta.mainProgram = "katoptra-dispatch";
};
@@ -37,9 +39,10 @@
nixosModules.default = import ./module.nix self;
- # The package (its tests included), and the module's two units rendered from a minimal
- # system: evaluation alone, no VM. ponytail: a nixosTest would boot the timer for real;
- # add one if a unit mistake ever gets past this.
+ # These checks are the package (with its tests) and the two units of the module, made
+ # from a small system. Nix makes the units, but it does not boot a VM. ponytail: a
+ # nixosTest boots the timer in a VM. If a unit error occurs on a host and this check
+ # did not find it, add a nixosTest.
checks = forAll (
pkgs:
let
@@ -68,7 +71,8 @@
in
{
package = self.packages.${system}.default;
- # the rendered units, failing if ExecStart names anything but an executable
+ # the two units. This check stops with an error if ExecStart is not an
+ # executable.
module = pkgs.runCommand "katoptra-dispatch-units" { } ''
service=${units."katoptra-dispatch.service".unit}/katoptra-dispatch.service
exe=$(sed -n 's/^ExecStart=//p' "$service")
diff --git a/github.go b/github.go
index 481dca7..eeb3e3c 100644
--- a/github.go
+++ b/github.go
@@ -1,7 +1,13 @@
package main
-// The GitHub App's side: sign a JWT, find the App's installation on the org, mint an
-// installation token, dispatch a workflow. No SDK, three requests.
+// This file does four steps as the GitHub App:
+//
+// 1. It signs a JWT.
+// 2. It finds the installation of the App on the organization.
+// 3. It gets an installation token.
+// 4. It dispatches a workflow.
+//
+// It uses no SDK, only three HTTP requests.
import (
"bytes"
@@ -22,11 +28,11 @@ import (
"time"
)
-// backoff is the wait before each retry of a transient failure: three retries, 70 s in all.
-// A retry whose wait would pass the context's deadline is not made.
+// backoff is the interval before each retry of a temporary error: three retries, 70 s in
+// total. If a retry cannot start before the deadline of the context, call does not make it.
var backoff = []time.Duration{10 * time.Second, 20 * time.Second, 40 * time.Second}
-// StatusError is GitHub answering with something other than 2xx.
+// StatusError is a GitHub response with a status other than 2xx.
type StatusError struct {
Status int
Msg string
@@ -34,12 +40,14 @@ type StatusError struct {
func (e *StatusError) Error() string { return e.Msg }
-// retryable is a failure another attempt may clear. Any other 4xx is a refusal no retry can
-// change: a missing workflow file, an App without the permission, no installation.
+// retryable tells if call can try again after err. With a 4xx other than 408 and 429,
+// GitHub rejected the request, and a retry cannot change that. For example: a missing
+// workflow file, an App without the permission, or no installation.
//
-// A request that starts a run (safe false) is retried only when GitHub cannot have acted on
-// it: the connection never opened, or GitHub turned it away unread (408, 429). A 5xx or a
-// timeout can follow a dispatch GitHub already accepted, and a retry would be a second run.
+// call tries a request that starts a run (safe false) again only if GitHub did not start
+// the run. GitHub did not start the run if the connection did not open, or if GitHub
+// rejected the request before it read it (408, 429). A 5xx or a timeout can follow a
+// request that GitHub accepted. Then a retry starts a second run.
func retryable(err error, safe bool) bool {
var s *StatusError
if errors.As(err, &s) {
@@ -49,8 +57,8 @@ func retryable(err error, safe bool) bool {
return safe || errors.As(err, &op) && op.Op == "dial"
}
-// ParseKey reads the App's private key. GitHub issues PKCS#1; PKCS#8 is accepted too, so a
-// converted key still works.
+// ParseKey reads the private key of the App. GitHub gives a PKCS#1 key. ParseKey also
+// accepts PKCS#8. Thus, you can also use a converted key.
func ParseKey(data []byte) (*rsa.PrivateKey, error) {
block, _ := pem.Decode(data)
if block == nil {
@@ -70,8 +78,9 @@ func ParseKey(data []byte) (*rsa.PrivateKey, error) {
return rk, nil
}
-// AppJWT signs a short-lived App JWT. Backdated 60 s because GitHub rejects a future iat,
-// and 9 minutes long against GitHub's 10-minute ceiling.
+// AppJWT signs an App JWT with a short life. Its iat is 60 s before now, because GitHub
+// rejects an iat after the time on its servers. Its life is 9 minutes, less than the
+// 10-minute maximum of GitHub.
func AppJWT(appID string, key *rsa.PrivateKey, now time.Time) (string, error) {
enc := base64.RawURLEncoding
iat := now.Unix() - 60
@@ -88,12 +97,12 @@ func AppJWT(appID string, key *rsa.PrivateKey, now time.Time) (string, error) {
return signed + "." + enc.EncodeToString(sig), nil
}
-// GitHub dispatches workflows as the App. Prepare mints one installation token and keeps it
-// in memory only.
+// GitHub dispatches workflows as the App. Prepare gets one installation token and keeps it
+// only in memory.
type GitHub struct {
API string // https://api.github.com
AppID string
- Org string // the one installation this App has
+ Org string // the one installation of this App
Key *rsa.PrivateKey
HTTP *http.Client
Sleep func(time.Duration)
@@ -102,7 +111,7 @@ type GitHub struct {
token string
}
-// Prepare mints the installation token, once per process.
+// Prepare gets the installation token one time in each process.
func (g *GitHub) Prepare(ctx context.Context) error {
if g.token != "" {
return nil
@@ -148,8 +157,9 @@ func (g *GitHub) mint(ctx context.Context) (string, error) {
return tok.Token, nil
}
-// call sends one request, retrying transient failures after each backoff step. safe says a
-// repeat has no effect beyond the first; see retryable.
+// call sends one request. After a temporary error, it waits for the next backoff step and
+// tries again. safe is true when a second copy of the request has no more effect than the
+// first copy. Refer to retryable.
func (g *GitHub) call(ctx context.Context, method, path, auth string, body []byte, safe bool) ([]byte, error) {
for attempt := 0; ; attempt++ {
b, err := g.once(ctx, method, path, auth, body)
diff --git a/github_test.go b/github_test.go
index 5953c8a..48b9d7b 100644
--- a/github_test.go
+++ b/github_test.go
@@ -28,7 +28,8 @@ var testKey = func() *rsa.PrivateKey {
return k
}()
-// github serves the three endpoints; dispatch answers with the statuses in order, then 204.
+// github is a test server for the three endpoints. The /dispatches endpoint sends the
+// statuses in dispatch, in sequence, and then 204.
func github(t *testing.T, dispatch ...int) (*GitHub, *[]string, *[]time.Duration) {
t.Helper()
var hits []string
diff --git a/main.go b/main.go
index 8918de6..63ac2d5 100644
--- a/main.go
+++ b/main.go
@@ -1,6 +1,7 @@
-// katoptra-dispatch is one tick of the katoptra scheduler: fire every job whose latest slot
-// has not been fired yet, record it first, then ping the scheduler's healthcheck. A systemd
-// timer runs it every five minutes; see module.nix.
+// katoptra-dispatch is one tick of the katoptra scheduler. It finds each job with a latest
+// slot that the state does not contain. It records that slot first, and then it starts the
+// job. At the end, it sends a ping to the healthcheck of the scheduler. A systemd timer
+// starts it at intervals of five minutes (refer to module.nix).
package main
import (
@@ -20,11 +21,13 @@ import (
"github.com/katoptra/dispatch/schedules"
)
-// pingLimit caps the error text sent with a /fail ping. healthchecks.io keeps 100 kB.
+// pingLimit is the maximum length of the error text in a /fail ping. healthchecks.io keeps
+// 100 kB.
const pingLimit = 10 << 10
-// githubBudget bounds everything GitHub gets in a tick, retries included, so the ping
-// (at most 3 x 30 s + 4 s) still lands inside the unit's TimeoutStartSec=4min.
+// githubBudget is the maximum time for all GitHub requests in a tick, with the retries
+// included. Thus, the ping (a maximum of 3 x 30 s + 4 s) can complete in the
+// TimeoutStartSec=4min of the unit.
const githubBudget = 2 * time.Minute
func main() {
@@ -39,7 +42,7 @@ func main() {
os.Exit(run(*dryRun, log))
}
-// printJobs is the table `task targets` shows; `task runs` reads its first column.
+// printJobs prints the table that `task targets` shows. `task runs` reads its first column.
func printJobs(w io.Writer) {
tw := tabwriter.NewWriter(w, 0, 0, 2, ' ', 0)
fmt.Fprintln(tw, "JOB\tSLOT\tUTC")
@@ -110,7 +113,7 @@ func newGitHub(creds string, client *http.Client) (*GitHub, error) {
}, nil
}
-// credential reads one file systemd's LoadCredential= put in $CREDENTIALS_DIRECTORY.
+// credential reads one file that systemd put in $CREDENTIALS_DIRECTORY with LoadCredential=.
func credential(dir, name string) (string, error) {
if dir == "" {
return "", fmt.Errorf("credential %s: CREDENTIALS_DIRECTORY is not set", name)
@@ -126,9 +129,13 @@ func credential(dir, name string) (string, error) {
return v, nil
}
-// ping reports the tick to healthchecks.io: the URL itself when clean, /fail with the errors
-// as the body otherwise. Three attempts; a miss is the caller's warning, not a failed tick.
-// The URL is the check's credential, so no error returned here carries it.
+// ping sends the result of the tick to healthchecks.io. If the tick has no errors, ping
+// sends the request to the URL. If the tick has errors, ping sends the request to /fail,
+// with the errors as the body.
+//
+// ping sends the request a maximum of three times. If the ping does not get to
+// healthchecks.io, the caller records a warning, not an error. The URL is the credential of
+// the check. Thus, no error from ping contains the URL.
func ping(client *http.Client, url string, errs []string, sleep func(time.Duration)) error {
body := strings.Join(errs, "\n")
if len(errs) > 0 {
diff --git a/module.nix b/module.nix
index 933bbf8..4b315ac 100644
--- a/module.nix
+++ b/module.nix
@@ -1,6 +1,6 @@
-# The scheduler as a NixOS service: a timer every five minutes starting one oneshot tick.
-# It takes three file paths and knows nothing of where they come from; the host renders them
-# (jshvn/jgrid.net: jgrid.secretTemplates into /run/secrets).
+# The scheduler as a NixOS service: a timer that starts one oneshot tick at intervals of
+# five minutes. It gets three file paths, and it does not know the source of the files. The
+# host makes them (the host's flake: jgrid.secretTemplates into /run/secrets).
self:
{
config,
@@ -11,7 +11,8 @@ self:
let
cfg = config.services.katoptra-dispatch;
package = self.packages.${pkgs.stdenv.hostPlatform.system}.default;
- # A string, never a Nix path: a path literal would copy the secret into the world-readable store.
+ # A string, not a Nix path. A path literal puts a copy of the secret in the Nix store,
+ # which all users can read.
secretPath =
description:
lib.mkOption {
@@ -31,9 +32,10 @@ in
systemd.timers.katoptra-dispatch = {
description = "katoptra-dispatch tick, every five minutes";
wantedBy = [ "timers.target" ];
- # :02, :07 .. :42 .. :57, so the slot minute is always a tick. UTC whatever the host's
- # zone. No Persistent=: every tick reconciles from the state file, so a missed tick is
- # caught up by the next one.
+ # :02, :07 .. :42 .. :57. Thus, the slot minute is always a tick. The calendar is in
+ # UTC, and the time zone of the host has no effect. No Persistent=: each tick compares
+ # the slots with the state file. Thus, the next tick does the work of a tick that did
+ # not occur.
timerConfig.OnCalendar = "*:02/5 UTC";
};
diff --git a/schedules/ctan.go b/schedules/ctan.go
index c562c6e..a989a6e 100644
--- a/schedules/ctan.go
+++ b/schedules/ctan.go
@@ -1,4 +1,4 @@
package schedules
-// katoptra/ctan mirrors CTAN into R2.
+// katoptra/ctan is a mirror of CTAN in R2.
var _ = register(Job{Repo: "katoptra/ctan", File: "sync.yml", Slots: Hourly})
diff --git a/schedules/dropbox.go b/schedules/dropbox.go
index 8ebc349..576e32e 100644
--- a/schedules/dropbox.go
+++ b/schedules/dropbox.go
@@ -1,5 +1,6 @@
package schedules
-// katoptra/dropbox mirrors a Dropbox account into Proton Drive. The run chains itself until
-// every batch is done, so one dispatch a day starts the whole pass.
+// katoptra/dropbox is a mirror of a Dropbox account in Proton Drive. Each run starts the
+// next run in a chain until all batches are done. Thus, the scheduler dispatches it daily,
+// and that starts the runs for all batches.
var _ = register(Job{Repo: "katoptra/dropbox", File: "sync.yml", Slots: Overnight})
diff --git a/schedules/github.go b/schedules/github.go
index 63c5e95..8e19380 100644
--- a/schedules/github.go
+++ b/schedules/github.go
@@ -1,4 +1,4 @@
package schedules
-// katoptra/github mirrors every repository under jshvn and katoptra into Proton Drive.
+// katoptra/github is a mirror of all repositories of jshvn and katoptra in Proton Drive.
var _ = register(Job{Repo: "katoptra/github", File: "sync.yml", Slots: Morning})
diff --git a/schedules/gnu.go b/schedules/gnu.go
index b00d783..48d7b43 100644
--- a/schedules/gnu.go
+++ b/schedules/gnu.go
@@ -1,4 +1,5 @@
package schedules
-// katoptra/gnu mirrors GNU's release tree from a secondary, twice a day.
+// katoptra/gnu is a mirror of GNU's release tree from a secondary mirror. The scheduler
+// starts it twice a day.
var _ = register(Job{Repo: "katoptra/gnu", File: "sync.yml", Slots: S3 | S15})
diff --git a/schedules/nongnu.go b/schedules/nongnu.go
index ac0c9b8..052b165 100644
--- a/schedules/nongnu.go
+++ b/schedules/nongnu.go
@@ -1,4 +1,5 @@
package schedules
-// katoptra/nongnu mirrors Savannah's nongnu release tree, twice a day.
+// katoptra/nongnu is a mirror of Savannah's nongnu release tree. The scheduler starts it
+// twice a day.
var _ = register(Job{Repo: "katoptra/nongnu", File: "sync.yml", Slots: S3 | S15})
diff --git a/schedules/schedules.go b/schedules/schedules.go
index 5c76794..1cbe562 100644
--- a/schedules/schedules.go
+++ b/schedules/schedules.go
@@ -1,14 +1,18 @@
-// Package schedules is what the scheduler starts, and when. One file per katoptra repo,
-// named for the half of "owner/name" after the slash, each adding its jobs to the list
-// below. Changing what runs touches only this directory.
+// Package schedules tells the scheduler which jobs to start, and when. Each katoptra
+// repository has one file here. The file name is the part of "owner/name" after the slash.
+// Each file adds the jobs of its repository to the list below. A change to the jobs or to
+// their slots is only in this directory.
//
-// Before adding a workflow, confirm all three. Nothing here can check them, and a workflow
-// that fails any of them is dispatched into silence:
+// Before you add a workflow, make sure that it has these three items:
//
-// 1. It declares `workflow_dispatch:` in `on:`.
-// 2. It declares a `concurrency` group with `cancel-in-progress: false`, so a dispatch
-// arriving during a run queues instead of doubling up.
-// 3. The workload pings its own healthcheck. This repo never learns whether a run passed.
+// 1. It has `workflow_dispatch:` in `on:`.
+// 2. It has a `concurrency` group with `cancel-in-progress: false`. Thus, a workflow start
+// during a run waits in the queue, and two runs do not operate at the same time.
+// 3. The workflow sends a ping to its healthcheck. This repository does not get the
+// result of a run.
+//
+// No code here can examine these items. If a workflow does not have all three, a problem
+// with its runs can stay unknown.
package schedules
import (
@@ -17,12 +21,12 @@ import (
"time"
)
-// Slot is a set of hours of the UTC day. Each hour's slot fires at HH:42. Slots combine
-// with |, so a job in two slots says `Morning | Evening`, and a misspelt name fails to
-// compile.
+// Slot is a set of hours of the UTC day. The slot of each hour is at HH:42. To put slots
+// together, use |. For example, a job in two slots has `Morning | Evening`. An incorrect
+// name causes a compile error.
type Slot uint32
-// The 24 hourly slots, S0 at 00:42 UTC through S23 at 23:42 UTC.
+// The 24 hourly slots, S0 at 00:42 UTC to S23 at 23:42 UTC.
const (
S0 Slot = 1 << iota
S1
@@ -50,8 +54,8 @@ const (
S23
)
-// Names for the daily slots, six hours apart. Pacific in the comments is winter time; in
-// summer each is an hour later.
+// Names for the daily slots, six hours apart. The Pacific times in the comments are winter
+// times. In summer, each Pacific time is one hour after the time in the comment.
const (
Evening = S5 // 05:42 UTC, 21:42 PST
Overnight = S11 // 11:42 UTC, 03:42 PST
@@ -60,12 +64,14 @@ const (
Hourly = Slot(1<<24 - 1)
)
-// Minute is the minute past the hour every slot fires at: off the hour, which GitHub sheds
-// first, and on a tick of the */5 timer that starts at :02.
+// Minute is the minute of the hour for each slot. It is not minute 0: when the load on
+// GitHub is high, GitHub decreases the load at minute 0 first. It is also on a tick of the
+// */5 timer that starts at :02.
const Minute = 42
-// Latest is the most recent slot time in s at or before now, in UTC. Today's and
-// yesterday's slots are always enough to find it. False when s holds no hour.
+// Latest gives the latest slot time in s: the last slot time at or before now, in UTC. The
+// slots of the same day and of the day before are always sufficient to find it. The bool is
+// false when s contains no hour.
func (s Slot) Latest(now time.Time) (time.Time, bool) {
now = now.UTC()
for day := 0; day < 2; day++ {
@@ -83,7 +89,7 @@ func (s Slot) Latest(now time.Time) (time.Time, bool) {
return time.Time{}, false
}
-// Times is each slot's firing time, "HH:42" UTC, earliest first.
+// Times gives the time of each slot in s as "HH:42" UTC, in the sequence of the day.
func (s Slot) Times() []string {
var out []string
for hour := 0; hour < 24; hour++ {
@@ -98,7 +104,8 @@ var names = map[Slot]string{
Hourly: "Hourly", Evening: "Evening", Overnight: "Overnight", Morning: "Morning", Afternoon: "Afternoon",
}
-// String names s as a schedule file would: "Hourly", "Morning", "Evening | Morning", "S3".
+// String gives the name of s as a schedule file writes it: "Hourly", "Morning",
+// "Evening | Morning", "S3".
func (s Slot) String() string {
if n, ok := names[s]; ok {
return n
@@ -118,14 +125,15 @@ func (s Slot) String() string {
return strings.Join(parts, " | ")
}
-// Job is one workflow in one repo, dispatched on `main` in every slot it holds.
+// Job is one workflow in one repository. The scheduler dispatches it on `main` in each of
+// its slots.
type Job struct {
Repo string // "katoptra/"
- File string // workflow file name, e.g. "sync.yml"
+ File string // the name of the workflow file, for example "sync.yml"
Slots Slot
}
-// ID is the job's key in the state file: "katoptra/ctan/sync.yml".
+// ID is the key of the job in the state file: "katoptra/ctan/sync.yml".
func (j Job) ID() string { return j.Repo + "/" + j.File }
var jobs []Job
@@ -135,5 +143,5 @@ func register(j ...Job) bool {
return true
}
-// Jobs is every registered job.
+// Jobs gives all jobs that register added, in a new slice.
func Jobs() []Job { return append([]Job(nil), jobs...) }
diff --git a/schedules/tlnet.go b/schedules/tlnet.go
index 07167ec..03dab43 100644
--- a/schedules/tlnet.go
+++ b/schedules/tlnet.go
@@ -1,4 +1,4 @@
package schedules
-// katoptra/tlnet mirrors the TeX Live network installation tree.
+// katoptra/tlnet is a mirror of the TeX Live network installation tree.
var _ = register(Job{Repo: "katoptra/tlnet", File: "sync.yml", Slots: Evening})
diff --git a/tick.go b/tick.go
index 1fd520e..dfbd960 100644
--- a/tick.go
+++ b/tick.go
@@ -14,15 +14,15 @@ import (
"github.com/katoptra/dispatch/schedules"
)
-// stateFile holds, per job id, the last slot that job was fired for.
+// stateFile contains the last dispatched slot of each job, by job id.
const stateFile = "state.json"
-// State maps a job id to the last slot fired for it, in UTC.
+// State is a map from a job id to the last dispatched slot of that job, in UTC.
type State map[string]time.Time
-// LoadState reads dir's state file. A missing file is empty state, the first run; anything
-// unreadable or malformed is an error, never empty state, because empty state would fire
-// every job again for slots already sent.
+// LoadState reads the state file in dir. A missing file is empty state: the first tick. A
+// file that it cannot read, or that has an incorrect format, is an error, not empty state.
+// This is because empty state starts each job again for dispatched slots.
func LoadState(dir string) (State, error) {
b, err := os.ReadFile(filepath.Join(dir, stateFile))
if errors.Is(err, os.ErrNotExist) {
@@ -35,14 +35,16 @@ func LoadState(dir string) (State, error) {
if err := json.Unmarshal(b, &s); err != nil {
return nil, fmt.Errorf("state file %s is corrupt: %w", filepath.Join(dir, stateFile), err)
}
- if s == nil { // `null` parses; it is not a missing file, and must not act like one
+ // json.Unmarshal accepts `null`. But a `null` file is not a missing file: LoadState
+ // gives an error for it, not empty state.
+ if s == nil {
return nil, fmt.Errorf("state file %s holds no object", filepath.Join(dir, stateFile))
}
return s, nil
}
-// SaveState replaces dir's state file atomically: a crash leaves the old file or the new
-// one, never half of either.
+// SaveState replaces the state file in dir in one step. After a crash, dir contains the
+// full previous file or the full new file.
func SaveState(dir string, s State) error {
b, err := json.MarshalIndent(s, "", " ")
if err != nil {
@@ -52,7 +54,7 @@ func SaveState(dir string, s State) error {
if err != nil {
return err
}
- defer os.Remove(tmp.Name()) // a no-op once renamed
+ defer os.Remove(tmp.Name()) // a no-op after os.Rename
if _, err := tmp.Write(append(b, '\n')); err != nil {
tmp.Close()
return err
@@ -75,14 +77,15 @@ func SaveState(dir string, s State) error {
return d.Sync()
}
-// Fire is one job due for one slot.
+// Fire is one job to dispatch for one slot.
type Fire struct {
Job schedules.Job
Slot time.Time
}
-// Plan returns the jobs due at now: those with no recorded slot, or whose latest slot is
-// after the one recorded. However many slots a job missed, it is due once, for the latest.
+// Plan gives the jobs to dispatch at now. A job is in the result if it has no recorded
+// slot, or if its latest slot is after its recorded slot. If there are one or more slots
+// after the recorded slot, the job is in the result one time, for the latest slot.
func Plan(jobs []schedules.Job, s State, now time.Time) (fires []Fire, warnings []string) {
for _, j := range jobs {
due, ok := j.Slots.Latest(now)
@@ -102,8 +105,9 @@ func Plan(jobs []schedules.Job, s State, now time.Time) (fires []Fire, warnings
return fires, warnings
}
-// next is the state to write before dispatching fires: every current job's entry, with the
-// fired slots recorded. Entries for jobs no longer registered are dropped.
+// next is the state to write before Tick dispatches fires. It keeps the entry of each job
+// in jobs, and it records the slot of each fire. It does not keep the entry of a job that
+// is not in jobs.
func next(jobs []schedules.Job, s State, fires []Fire) State {
n := State{}
for _, j := range jobs {
@@ -117,20 +121,22 @@ func next(jobs []schedules.Job, s State, fires []Fire) State {
return n
}
-// Dispatcher starts workflow runs. Prepare does everything that starts nothing (credentials),
-// so its failure can leave the slots unrecorded; Dispatch starts one run.
+// Dispatcher starts workflow runs. Prepare does the steps that start no run (the
+// credentials). Thus, if Prepare gives an error, Tick can stop before it records the slots.
+// Dispatch starts one run.
type Dispatcher interface {
Prepare(ctx context.Context) error
Dispatch(ctx context.Context, repo, file string) error
}
-// Tick is one run of the timer: lock, plan, prepare, record, dispatch. Every failure goes to log;
-// the caller reads log.Errors() for the exit code and the ping.
+// Tick is one tick of the timer: lock, plan, prepare, record, dispatch. Each error goes to
+// log. The caller reads log.Errors() for the exit code and the ping.
//
-// The slots are recorded before anything is dispatched, so a crash or a failed dispatch
-// loses that slot rather than repeating it: at most once per slot. Prepare comes before the
-// record, so a GitHub or network outage before any run could start costs five minutes, not
-// the slot.
+// Tick records the slots before it dispatches. Thus, after a crash or an error from
+// Dispatch, that slot gets no run. A slot does not get two runs: a maximum of one run for
+// each slot. Tick does Prepare before it records the slots. Thus, if GitHub or the network
+// is not available before a run can start, the slot gets its run at the next tick, after
+// five minutes.
func Tick(ctx context.Context, dir string, jobs []schedules.Job, now time.Time, d Dispatcher, dryRun bool, log *Log) {
unlock, err := lock(dir)
if err != nil {
@@ -177,8 +183,9 @@ func Tick(ctx context.Context, dir string, jobs []schedules.Job, now time.Time,
}
}
-// lock takes an exclusive flock on dir itself. systemd already refuses to start a oneshot
-// that is still running; this covers the binary run by hand beside it.
+// lock gets an exclusive flock on the directory dir, not on a file in it. systemd does not
+// start a oneshot unit while that unit operates. The flock also prevents two ticks at the
+// same time when you start the binary manually.
func lock(dir string) (func(), error) {
f, err := os.Open(dir)
if err != nil {
@@ -200,9 +207,9 @@ func lastFired(s State, j schedules.Job) string {
return "never"
}
-// Log writes to the journal through stdout and stderr, and keeps the errors for the ping.
-// Under systemd (JOURNAL_STREAM set) lines carry the sd-daemon priority prefix, so
-// `journalctl -p err` finds the errors.
+// Log writes to the journal through stdout and stderr. It keeps the errors for the ping.
+// When systemd starts the binary (JOURNAL_STREAM is set), the warning and error lines have
+// the sd-daemon priority prefix. Thus, `journalctl -p err` finds the errors.
type Log struct {
Out, Err io.Writer
Journal bool
@@ -225,5 +232,5 @@ func (l *Log) write(w io.Writer, prio, format string, a ...any) {
fmt.Fprintf(w, prio+format+"\n", a...)
}
-// Errors is every error logged so far.
+// Errors gives each error that Error recorded before this call.
func (l *Log) Errors() []string { return l.errs }
diff --git a/tick_test.go b/tick_test.go
index e1d7f23..0fc5cb4 100644
--- a/tick_test.go
+++ b/tick_test.go
@@ -26,8 +26,8 @@ func utc(s string) time.Time {
return t
}
-// fake records each dispatch, reads the state file as it stood at that moment, and fails
-// the jobs named in fail. Prepare returns prepErr.
+// fake records each job that Dispatch gets, and reads the state file as it is at that time.
+// It gives an error for each job in fail. Prepare gives prepErr.
type fake struct {
dir string
fail map[string]bool