feat(report): status/v1 and check/v1 schemas; a compatibility page

CONTRIBUTING.md

@@ -169,11 +169,11 @@ The first push creates the `ghcr.io/bedrock-python/touchmark` package private; m
 public in the organisation's package settings once, or every `docker pull`, every GitLab
 and Gitea job and the Action are denied.
 
-- `feat:` bumps the minor version, `fix:` the patch version, `feat!:` / a `BREAKING CHANGE:`
-  footer bumps the major version (before 1.0 a breaking change bumps the minor version —
-  `bump-minor-pre-major` is on).
-- To force a specific version, add a `Release-As: x.y.z` footer to a commit — this is how
-  1.0.0 is cut.
+- `feat:` bumps the minor version, `fix:` the patch version. touchmark is at 0.x, where a
+  breaking change (`feat!:`, a `BREAKING CHANGE:` footer) bumps the minor version too
+  (`bump-minor-pre-major` is on). What counts as breaking is on the
+  [Compatibility](https://bedrock-python.github.io/touchmark/reference/compatibility/) page.
+- To force a specific version, add a `Release-As: x.y.z` footer to a commit.
 - Tags are `vX.Y.Z`, without the component prefix the other bedrock-python repositories
   use: Go modules and the Action resolve bare `v` tags only.
 - release-please writes each release above the first version heading of `CHANGELOG.md`.

docs/project/release.md

@@ -44,7 +44,7 @@ Every job after `check` checks out the tag's commit by its id and stops if the c
 
 **The vulnerability scan.** The scan that blocks is CI's: the job `release-snapshot` builds the image a release would publish and scans it on every pull request, the release pull request included, and every night, so a new vulnerability in the base image shows up before a release does. The scan in `publish.yml` only reports: by then the tag and the release exist, the same base image is in every earlier release, and holding the image back would leave the Action at the tag broken for good.
 
-**Versions.** `feat:` bumps the minor version, `fix:` the patch version; before 1.0 a breaking change (`feat!:`, a `BREAKING CHANGE:` footer) bumps the minor version too (`bump-minor-pre-major`). A `Release-As: x.y.z` footer on a commit forces a version. The manifest starts at `0.0.0`, so the first release, from the commit `feat: initial release`, is `0.1.0`.
+**Versions.** `feat:` bumps the minor version, `fix:` the patch version; touchmark is at 0.x, where a breaking change (`feat!:`, a `BREAKING CHANGE:` footer) bumps the minor version too (`bump-minor-pre-major`). What counts as breaking is in [Compatibility](../reference/compatibility.md). A `Release-As: x.y.z` footer on a commit forces a version. The manifest starts at `0.0.0`, so the first release, from the commit `feat: initial release`, is `0.1.0`.
 
 **Tags.** A release tag is `vX.Y.Z`, without the component prefix of the org's libraries (`pg-partsmith-v1.7.2`): the Go module proxy and `uses: bedrock-python/touchmark@vX.Y.Z` need bare semver tags (`include-component-in-tag: false`). The release is named `vX.Y.Z` too.
 
@@ -110,7 +110,7 @@ touchmark adds nothing stricter to them on purpose: no required approval (a main
 5. **The first release pull request.** `CHANGELOG.md` has no release yet, so Release Please writes `# Changelog`, the 0.1.0 section, and then the file's own header, demoted to `## Changelog`. Before merging, edit `CHANGELOG.md` in the pull request: the header back on top, the 0.1.0 section below it. Later releases go in above the previous one.
 6. **The package.** The first Publish run creates `ghcr.io/bedrock-python/touchmark`, linked to the repository by the index's `org.opencontainers.image.source`. With public packages allowed in the organisation, it takes the public repository's visibility: v0.1.0's package was public from its first push, and `verify` passed on the first run. Should a package come out private, `verify` fails at the anonymous steps: in the package's settings, check the link to the repository and that the repository's Actions have write access, then **Change visibility → Public**, and re-run the failed jobs. Hubs and the Action pull anonymously.
 
-PyPI is not part of a release yet: see [Adding PyPI later](#adding-pypi-later).
+touchmark is not published to PyPI: a release is the binaries, the image and the Action.
 
 ## Release, step by step
 
@@ -202,12 +202,3 @@ docker buildx imagetools inspect ghcr.io/bedrock-python/touchmark@sha256:<digest
 ```
 
 Add `--bundle-from-oci` to the `gh attestation verify` of the image for the check the Action makes, from the attestation stored in GHCR.
-
-## Adding PyPI later
-
-The org's libraries publish to PyPI; touchmark does not yet. To let Python projects run `uvx touchmark`, add to `publish.yml`, after `assets`:
-
-1. a job that downloads the release's archives, checks them against the signed `checksums.txt` (as the `image` job does), and wraps each binary into a wheel with [go-to-wheel](https://pypi.org/project/go-to-wheel/), pinned by hash: 8 platform tags (manylinux and musllinux for amd64 and arm64, macOS, Windows), each wheel's binary byte for byte the archive's;
-2. a job `pypi` in an environment `pypi` with `id-token: write`, which uploads the wheels with `pypa/gh-action-pypi-publish` (trusted publishing; `skip-existing` so that a re-run completes a failed upload), and, if the wheels should be on the GitHub release too, uploads them there with their own build provenance attestation.
-
-On pypi.org, add a pending trusted publisher: project `touchmark`, owner `bedrock-python`, repository `touchmark`, workflow `publish.yml`, environment `pypi`; and add the badge and the install line to the README.

docs/reference/commands.md

@@ -291,7 +291,7 @@ their commands. Point an editor's YAML language server at a schema for completio
 <!-- generated: help schema -->
 
 ```text
-Usage: touchmark schema hub|targets|opt-in|operations|report|doctor|setup
+Usage: touchmark schema hub|targets|opt-in|operations|report|doctor|setup|status|check
 
 Print the JSON Schema of a configuration file or of a report.
 ```

docs/reference/compatibility.md

@@ -0,0 +1,64 @@
+# Compatibility
+
+touchmark is at 0.x and stays there for now. This page says what a hub, a target and a
+script around touchmark can rely on from one release to the next.
+
+## Versions
+
+A patch release (`fix:`) breaks nothing listed below. A minor release (`feat:`) adds
+things. Before 1.0, a breaking change also bumps the minor version (`feat!:` or a
+`BREAKING CHANGE:` footer), and the changelog lists it under **⚠ BREAKING CHANGES**.
+A hub pins touchmark to a release (the template pins the Action by commit and the image by
+digest), so a new minor version arrives as a pull request: read that section of the
+changelog before you merge it.
+
+## Stable
+
+- **Configuration files at `version: 1`:** `hub.yml`, `targets.yml`, the opt-in file and
+  `.touchmark/operations.yml`. A file that a release accepts keeps working, with the same
+  meaning, in every later release that reads `version: 1`. Changes only add: new optional
+  keys and new values. An older release rejects a key it does not know, so a new key needs
+  the release that added it; the reference says which release that is. The opt-in file
+  lives in repositories the hub does not control, so it gains keys rarely. Removing a key
+  or changing what it means takes a `version: 2` of that file, and touchmark then keeps
+  reading `version: 1`, with a warning.
+- **Versioned JSON reports:** `report/v1` (`plan`, `distribute`, and the lines of
+  `--stream`), `doctor/v1`, `setup/v1`, `status/v1` (`status`, `apply`) and `check/v1`.
+  Changes only add: new fields, and new values where a field lists its values (outcomes,
+  reasons, states, check and step names). Ignore fields you do not know and treat values
+  you do not know as opaque. Anything else is a new version, such as `report/v2`, released
+  as a breaking change. Validate a report against the schema of the release that wrote it
+  (`touchmark schema <name>`): the schemas published on `master` describe the next release
+  and reject fields they do not know.
+- **What touchmark leaves in repositories:** the marker in pull request bodies
+  (`touchmark:v1`), the commit trailers (`Touchmark-Hub`, `Touchmark-Stream`,
+  `Touchmark-Content`, `Touchmark-Hub-Commit`), the control lines of a pull request body,
+  the marker of the plan comment, and the sync branch's name. Every release reads what an
+  earlier one wrote. The marker's payload is frozen: a change to it is a `touchmark:v2`
+  marker, which releases read alongside `v1`.
+- **The command line and CI:** command and flag names, flag defaults, exit codes,
+  environment variable names, the names of the report files in CI, the GitHub Action's
+  inputs, and the image's tags (`X.Y.Z`, `X.Y`, `latest`) and what the hub template's jobs
+  use in it (`sh`, `touchmark` on the `PATH`, user 65532).
+
+## Not stable
+
+Text and Markdown output, step summaries and annotations, the wording of pull request
+bodies and comments, warnings and error messages, the output of `manifest` and `migrate`,
+and the Go packages, `schemas` included. Parse the JSON reports, not the text.
+
+## Deprecation
+
+A deprecated key, flag or input is marked deprecated in its schema and in the reference,
+with what replaces it. Using it warns: in `check`, and in the warnings of `status`, `plan`
+and `distribute`, which reach the hub's pull request comment and the step summary. The
+changelog announces the deprecation. A flag or an input keeps working for at least two
+minor releases and six months; a configuration key keeps working for as long as touchmark
+reads that version of its file. Removing either is a breaking change.
+
+Deprecated now:
+
+| What | Since | Instead |
+|---|---|---|
+| `commit.author` in `hub.yml` | before 0.1.0 | Nothing: the writer account authors every commit. The key is ignored, with a warning. |
+| a bare `repos:` list in `targets.yml` | before 0.1.0 | `version: 1` with `targets:` entries. The list is read as `repo:` entries, with a warning. |

docs/reference/hub.md

@@ -84,7 +84,7 @@ In a hub pull request, `providers` and `ca_file` are read from the default branc
 | `opt_in_file` | string | Path of the opt-in file in targets. Default: .engineering-assets.yml. |
 | `commit` | object | Commit settings. The commit author is always the provider's writer account. |
 | `commit.message` | string | Commit message. Default: "chore: sync engineering assets". Must not start with Draft: or WIP:, which turns a GitLab merge request into a draft. |
-| `commit.author` | object | Deprecated. Removed in v1 and ignored with a warning: commits are always authored by the writer account. |
+| `commit.author` | object | Deprecated. Ignored with a warning: commits are always authored by the writer account. |
 | `commit.author.name` | string | — |
 | `commit.author.email` | string | — |
 | `pr` | object | Pull request settings. After creation, the title, draft state and labels belong to people. |

docs/reference/output.md

@@ -9,7 +9,11 @@ Every command that inspects targets prints a report on stdout: text by default,
 | `report/v1` | `plan`, `distribute` | `touchmark schema report` | `touchmark-report.json`, `touchmark-report.md`; `distribute` also `touchmark-report.jsonl` |
 | `doctor/v1` | `doctor` | `touchmark schema doctor` | `touchmark-doctor.json`, `touchmark-doctor.md` |
 | `setup/v1` | `setup` | `touchmark schema setup` | — |
-| status | `status`, `apply` | — | — |
+| `status/v1` | `status`, `apply` | `touchmark schema status` | — |
+| `check/v1` | `check` | `touchmark schema check` | — |
+
+What may change in these formats between releases, and what may not, is in
+[Compatibility](compatibility.md).
 
 `--report FILE` (`distribute`, `doctor`) also writes the JSON report to a file;
 `--stream FILE` (`distribute`) writes one JSON line per finished target as the run goes,
@@ -279,6 +283,7 @@ pack selection and where each pack came from, one entry per managed path with it
 
 ```json
 {
+  "schema": "status/v1",
   "command": "status",
   "dry_run": false,
   "hub": { "dir": "/src/engineering-assets", "id": "acme-eng", "commit": "ffae0953c140298cf045e600f15710104e9d5d0d" },
@@ -295,5 +300,82 @@ pack selection and where each pack came from, one entry per managed path with it
 }
 ```
 
-This format has no schema yet and may change before 1.0; the `report/v1`, `doctor/v1` and
-`setup/v1` formats change only with their version.
+`apply` adds `results`, one per entry whose action is not `keep`, with its `outcome`
+(`done`, `skipped`, `failed`); `apply --dry-run` has none.
+
+<!-- generated: schema status -->
+
+| Key | Type | Description |
+|---|---|---|
+| `schema` | `status/v1`, required | Report format: status/v1. |
+| `command` | one of `status`, `apply`, required | — |
+| `dry_run` | boolean, required | apply --dry-run: the plan only, nothing written. Always false for status. |
+| `hub` | object, required | The hub the report was made from. |
+| `hub.dir` | string, required | The hub's directory. |
+| `hub.id` | string, required | hub.yml's id; empty for a hub without hub.yml. |
+| `hub.commit` | string, required | The hub commit read; empty when touchmark stopped before reading it. |
+| `target` | object, required | The target working tree. |
+| `target.root` | string, required | The target's working tree. |
+| `target.ref` | string, required | The target in targets.yml, path or provider:path; empty when it could not be determined. |
+| `target.opt_in_file` | string, required | The opt-in file's path in the target. |
+| `target.opted_in` | boolean, required | The target gets packs: opt_in is file, assumed or flag. |
+| `target.opt_in` | one of `file`, `assumed`, `flag`, `opted-out`, `none`, required | Why the target is opted in or not. file: the opt-in file is there; assumed: there is none, and a repo: entry with opt_in: assumed subscribes it; flag: there is none, and --assume-opt-in counts it as opted in; opted-out: the file says enabled: false; none: neither. |
+| `selection` | object, required | The resolved pack list; null when the target has not opted in. |
+| `selection.packs` | list of strings, required | The packs the target gets, requires included. |
+| `selection.complete` | boolean, required | Every pack named was found in the hub. |
+| `selection.unresolved` | list of strings, required | Packs named that the hub does not have. |
+| `selection.sources` | map of name to list of strings, required | Per pack, where it was selected: the opt-in file, targets.yml, defaults or requires &lt;pack&gt;. |
+| `entries` | list of objects, required | The whole plan in plan order, one entry per managed path, current entries included. |
+| `entries[].path` | string, not empty, required | The path in the target. |
+| `entries[].state` | one of `missing`, `current`, `outdated`, `local`, `ignored`, `retired`, `retired-local`, `unsafe`, `orphaned`, required | The file's state in the target. |
+| `entries[].action` | one of `keep`, `create`, `update`, `delete`, `adopt`, `chmod`, required | What apply does with it. |
+| `entries[].pack` | string, required | The pack that ships the path, or shipped it. |
+| `entries[].from` | string, required | The blob id the target has; empty when it has none. |
+| `entries[].to` | string, required | The blob id apply writes; empty for keep and delete. |
+| `entries[].mode` | one of ``, `100644`, `100755`, required | The file mode apply writes; empty when it writes nothing. |
+| `entries[].detail` | string, required | Why, for states and actions that need a reason. |
+| `entries[].after_deletes` | boolean, required | Written after the deletes, because a path it needs is deleted first. |
+| `results` | list of objects | apply without --dry-run only: what it did with each entry whose action is not keep, in execution order. |
+| `results[].path` | string, not empty, required | — |
+| `results[].action` | one of `keep`, `create`, `update`, `delete`, `adopt`, `chmod`, required | — |
+| `results[].state` | one of `missing`, `current`, `outdated`, `local`, `ignored`, `retired`, `retired-local`, `unsafe`, `orphaned`, required | — |
+| `results[].outcome` | one of `done`, `skipped`, `failed`, required | — |
+| `results[].detail` | string, required | The reason for skipped and the error for failed. |
+| `summary` | object, required | Entries per state, the changes (entries whose action is not keep) and, for apply, the outcomes. |
+| `summary.missing` | integer ≥ 0, required | — |
+| `summary.current` | integer ≥ 0, required | — |
+| `summary.outdated` | integer ≥ 0, required | — |
+| `summary.local` | integer ≥ 0, required | — |
+| `summary.ignored` | integer ≥ 0, required | — |
+| `summary.retired` | integer ≥ 0, required | — |
+| `summary.retired_local` | integer ≥ 0, required | — |
+| `summary.unsafe` | integer ≥ 0, required | — |
+| `summary.orphaned` | integer ≥ 0, required | — |
+| `summary.changes` | integer ≥ 0, required | — |
+| `summary.done` | integer ≥ 0, required | — |
+| `summary.skipped` | integer ≥ 0, required | — |
+| `summary.failed` | integer ≥ 0, required | — |
+| `warnings` | list of strings, required | — |
+
+<!-- end generated -->
+
+## check
+
+`--format json` prints the hub, its packs, and every error and warning; `errors` is empty
+when the hub passes.
+
+<!-- generated: schema check -->
+
+| Key | Type | Description |
+|---|---|---|
+| `schema` | `check/v1`, required | Report format: check/v1. |
+| `command` | `check`, required | — |
+| `hub` | object, required | The hub checked. |
+| `hub.dir` | string, required | The hub's directory. |
+| `hub.id` | string, required | hub.yml's id; empty for a hub without hub.yml. |
+| `hub.commit` | string, required | The hub commit read; empty when touchmark stopped before reading it. |
+| `packs` | list of strings, required | The hub's packs. |
+| `errors` | list of strings, required | What makes check exit 2; empty when the hub passes. |
+| `warnings` | list of strings, required | — |
+
+<!-- end generated -->

internal/cli/check.go

@@ -29,6 +29,7 @@ func runCheck(ctx context.Context, e *env, o *options) error {
 		return err
 	}
 	c := &checkRun{hub: h, rep: &report.Check{
+		Schema:   report.CheckSchema,
 		Command:  "check",
 		Packs:    []string{},
 		Errors:   []string{},

internal/cli/cli.go

@@ -14,7 +14,7 @@
 //	touchmark status   [--hub DIR] [--dir DIR] [--repo REF] [--packs a,b] [--worktree] [--format text|json]
 //	touchmark apply    [same as status] [--adopt GLOB]... [--dry-run]
 //	touchmark manifest [--hub DIR] [--format json]
-//	touchmark schema   hub|targets|opt-in|operations|report|doctor|setup
+//	touchmark schema   hub|targets|opt-in|operations|report|doctor|setup|status|check
 //	touchmark version
 //
 // --hub defaults to $TOUCHMARK_HUB; --dir defaults to the current directory.
@@ -102,7 +102,7 @@ func commands() []*command {
 		{name: "manifest", synopsis: "[--hub DIR] [--format json]",
 			summary: "print the ownership manifest built from the hub's history",
 			flags:   manifestFlags, run: runManifest},
-		{name: "schema", synopsis: "hub|targets|opt-in|operations|report|doctor|setup",
+		{name: "schema", synopsis: "hub|targets|opt-in|operations|report|doctor|setup|status|check",
 			summary: "print the JSON Schema of a configuration file or of a report",
 			flags:   schemaFlags, run: runSchema},
 		{name: "version", synopsis: "",

internal/cli/flags.go

@@ -209,7 +209,7 @@ func manifestFlags() (*flagSet, *options) {
 
 func schemaFlags() (*flagSet, *options) {
 	f, o := newFlagSet("schema")
-	f.nargs, f.argName = 1, "schema name: hub, targets, opt-in, operations, report, doctor or setup"
+	f.nargs, f.argName = 1, "schema name: hub, targets, opt-in, operations, report, doctor, setup, status or check"
 	return f, o
 }
 

internal/cli/sync.go

@@ -44,6 +44,7 @@ func runSync(ctx context.Context, e *env, o *options, write bool) error {
 		return err
 	}
 	s.rep = &report.Sync{
+		Schema:   report.SyncSchema,
 		Command:  "status",
 		Hub:      s.hub.report(),
 		Target:   s.tgt.report(),

internal/cli/testdata/golden/assumed/apply.json

@@ -1,4 +1,5 @@
 {
+  "schema": "status/v1",
   "command": "apply",
   "dry_run": false,
   "hub": {

internal/cli/testdata/golden/check-defects/check.json

@@ -1,4 +1,5 @@
 {
+  "schema": "check/v1",
   "command": "check",
   "hub": {
     "dir": "$HUB",

internal/cli/testdata/golden/crlf-committed/apply-adopt.json

@@ -1,4 +1,5 @@
 {
+  "schema": "status/v1",
   "command": "apply",
   "dry_run": false,
   "hub": {

internal/cli/testdata/golden/crlf-committed/apply.json

@@ -1,4 +1,5 @@
 {
+  "schema": "status/v1",
   "command": "apply",
   "dry_run": false,
   "hub": {

internal/cli/testdata/golden/crlf/apply.json

@@ -1,4 +1,5 @@
 {
+  "schema": "status/v1",
   "command": "apply",
   "dry_run": false,
   "hub": {

internal/cli/testdata/golden/file-directory/apply.json

@@ -1,4 +1,5 @@
 {
+  "schema": "status/v1",
   "command": "apply",
   "dry_run": false,
   "hub": {