diff --git a/HISTORY.md b/HISTORY.md index 9468844..b10c877 100644 --- a/HISTORY.md +++ b/HISTORY.md @@ -41,6 +41,13 @@ pass. No native tests, CI jobs or release artifacts were rebuilt or rerun; product sources, workflows, pins and project locks are unchanged. Current scope: [CI acceptance](docs/acceptance/ci_acceptance.md#current-hosted-coverage-2026-11-11). +### Add the protected release workflow and the CI pin promotion policy + +`release.yml` packages a commit that is on `main` with a passing `CI gate`, verifies the archive on a separate runner (including a real GitHub SDK acquisition and the C and C++ workflows) and publishes only through the `release` environment, which needs a reviewer. +A design flaw was fixed before the first run (the notes needed a digest that exists only after the build; they are now a token template). A hosted dry run produced a verified draft whose archive, sums file and notes agreed; the draft was deleted, and an unchecked commit +was rejected at the first job. The real publish step has not been run. The pin promotion policy and `tools/check_ci_pins.py` are in the development guide; every current pin matches the published bytes. The dry run carried an approval record the assistant did not create. +[CI acceptance](docs/acceptance/ci_acceptance.md#release-workflow-2026-11-11). + ### Add the manual integration workflow `integration.yml` runs a pin gate, the 54-case restricted-redirect replay and the port suites on Linux x64 and arm64, with every input pinned and size/SHA-256 checked by a unit-tested driver. The redirect replay passes diff --git a/TODO.md b/TODO.md index aec3ece..61eb017 100644 --- a/TODO.md +++ b/TODO.md @@ -65,10 +65,13 @@ paths and the Linux/arm64 dispatch, push and pull-request runs. - [ ] `integration.yml` (manual; the 54-case redirect replay passes on Linux x64 and arm64 in hosted run `42957820373`): the five port suites still need CRT to publish and pin a per-host independent host-tools bundle and a hosted-runner adaptation of `tools/replay_port_linux.py`; until then `pins` and `ports` fail on purpose. Add `schedule:` with the pin; macOS/Windows jobs follow. -- [ ] `release.yml`: protected, manually approved packaging from a tested CI artifact, independent extraction and - GitHub SDK acquisition before publication. CI success is not publication authority. -- [ ] Define how CI pins are promoted (new Preview release, digest review) and add a Windows/arm64 leg - once its public SDK and CLI package exist (the Linux/arm64 leg was added 2026-11-11). +- [ ] `release.yml` (manual; `package` → `verify` → approval-gated `publish`): the dry run stops at a verified draft and the entry + gates reject an unchecked commit ([record](docs/acceptance/ci_acceptance.md#release-workflow-2026-11-11)). Not yet exercised: the + real publish step (`dry_run: false`), whose first use should be the next genuine Preview; it needs the reviewer's approval on the + `release` environment, and the approval's author is to be recorded with a comment. Linux x64 and arm64 only; macOS and Windows + jobs follow the same host-key pattern. +- [ ] Add a Windows/arm64 leg under the [pin promotion policy](docs/guides/development.md#ci-pin-promotion) once its public SDK, its + CLI Preview and a hosted runner exist. The policy and `tools/check_ci_pins.py` are in place (2026-11-11). ## Planned diff --git a/docs/acceptance/ci_acceptance.md b/docs/acceptance/ci_acceptance.md index a09000d..8b1d15a 100644 --- a/docs/acceptance/ci_acceptance.md +++ b/docs/acceptance/ci_acceptance.md @@ -487,3 +487,38 @@ source is pinned (`zlib-1.3.1.tar.gz`, 1,512,791 bytes, SHA-256 equal to the rec Not covered: macOS and Windows integration jobs, the oracle's pinned Python, the port suites themselves, and any repository secret (none is used). +## Release workflow (2026-11-11) + +`release.yml` is started by hand (`workflow_dispatch`: host, a full commit SHA, date, serial, a notes template path, `dry_run` defaulting to true). Three jobs, driven by `tools/ci_release.py` (14 unit tests): + +| Job | Permission | Does | +| --- | --- | --- | +| `package` | read | refuses a commit that is not on `main` or has no successful `CI gate`; builds exactly that commit with the pinned published CLI and public SDK through `ci_bootstrap.py`; assembles the notices; packs twice and requires identical bytes | +| `verify` | read | on a separate runner: archive digest against its sums file, `package_cli_archive.py verify`, `--version`, self `doctor` with the manifest pin, a wrong pin refused, a real GitHub `sdk acquire`, then C and C++ `new`/`build`/`test`/`run`/warm build | +| `publish` | `contents: write` | waits for the `release` environment; re-checks the digest, renders the notes template, stages a draft, downloads it and compares byte for byte, then publishes a non-latest prerelease (or stops at the draft) | + +The `release` environment requires reviewer `webos21` and deploys from `main` only. `prevent_self_review` is off because a one-person repository could not publish otherwise; the approval is therefore a deliberate pause, not independent review. + +**A design flaw found before the first run.** `publish` required the notes to contain the archive SHA-256, which only exists after the build, so a note committed in advance could never pass. The notes are now a reviewed template with explicit tokens +(`{{ARCHIVE_SHA256}}`, `{{ARCHIVE_BYTES}}`, `{{COMMIT}}`, `{{TAG}}`); a template without the digest token or with an unknown token is refused, and a digest typed by hand does not stand in for the token. + +**Local run first.** `ci_release.py package` on the native aarch64 machine produced the archive, with a first attempt (`release-ci-check-r1-failed`) stopping on a bare `StopIteration` because the artifact is named `crt-cli-`; the executable is now located by +its verified digest and the SDK by the identity the summary records. `verify` passed all steps against that archive. + +**Hosted dry run.** Run `42963992535` (`main`, `8063bbf`, `linux-arm64`, serial 9, `dry_run: true`): `package`, `verify` and `publish` succeeded. The draft `v0.1.0-preview.20261111.linux-arm64.9` targeted `8063bbf`, held the 5,036,724-byte archive and the sums file, +and its notes carried the real digest (`67d2074d...d703`) with no leftover token; the archive, the sums file and the notes agreed after an independent download. The draft was then deleted; no tag was created and no release other than the four published ones remains. +**An approval record exists for that run** (`approved`, `webos21`) that the assistant did not create; who pressed it and how is not known from the record. Future runs should add an approval comment. + +**Rejections.** Run `42965008964` with `6a00885`, which has no `CI gate` check, stopped in `package` ("CI gate on ...: none"); `verify` and `publish` were skipped and nothing was created. Locally, the publish driver also refused a branch name instead of a SHA, notes without the digest, +a tampered archive and an already-published tag (checked against the real serial-1 release, read only). + +Not covered: an actual publication (`dry_run: false`), the other two hosts, the environment's behaviour for a second reviewer, and workflow-level tests for `actions/download-artifact` across a failed upload. + +## CI pin promotion (2026-11-11) + +[The policy](../guides/development.md#ci-pin-promotion) fixes when a pin may move, what moves with it, the evidence a pull request must quote, what must pass and how a new host is added. `tools/check_ci_pins.py` re-downloads each host's published CLI archive and SDK manifest, archive and +contract and compares size, SHA-256 and the manifest's listing; it reported `OK` for `linux-x64`, `linux-arm64`, `macos-arm64` and `win-x64` against `.github/ci/pins.json`. Six unit tests make it report a changed bootstrap archive, a size-only change, a changed SDK archive or contract, +an archive missing from the manifest and a contract digest the manifest disagrees with, so that `OK` is not the only output ever seen. + +Not covered: automatic detection that a newer Preview exists, a scheduled run of the checker, and any second reviewer. + diff --git a/docs/guides/development.md b/docs/guides/development.md index 1929237..27862ec 100644 --- a/docs/guides/development.md +++ b/docs/guides/development.md @@ -291,8 +291,8 @@ On macOS the Xcode tools come from `xcrun -f`; on Windows pass `--llvm-bin` and, not wanted, `--os-sdk`. `--tests` restricts the suites for a quick local pass; CI always runs the pinned set. The Python driver is a build-host orchestrator like the other `tools/*.py` replays, not a CLI prerequisite. -Pins change only in a reviewed commit that cites the release record and re-checks the asset digests against the -published `SHA256SUMS`; never use a moving tag or `latest`. Linux/x86_64 has two required +Pins change only through a reviewed pull request that follows the [pin promotion policy](#ci-pin-promotion); never use a moving tag +or `latest`. Linux/x86_64 has two required legs, `ubuntu-24.04` and `ubuntu-26.04`, consuming the same repaired Dev SDK pin. A separate required `linux-arm64` leg (`ubuntu-26.04-arm`) consumes the Linux/aarch64 Dev SDK and CLI Preview pins ([CI acceptance](../acceptance/ci_acceptance.md#linuxarm64-leg-2026-11-11)). All Linux legs use LLVM 21: noble provisions the explicitly keyed apt.llvm.org repository; 26.04 uses Ubuntu's archive. Existing macOS/arm64 and Windows/x64 checks remain @@ -304,6 +304,35 @@ The original `.3` failure remains historical evidence; the producer's TLSDESC fix is shipped in `.4`, rather than hidden by a runner upgrade. [TODO](../../TODO.md#github-ci) owns integration/release work and any still-pending hosted qualification. +### CI Pin Promotion + +`.github/ci/pins.json` (bootstrap CLI and SDK per host) and `.github/ci/integration-pins.json` (integration inputs) are the only +inputs `ci.yml` and `integration.yml` trust. A pin moves in a pull request, never automatically and never from a branch, tag or `latest`. + +1. **When.** Only after the new input is published and its release record exists in [preview acceptance](../acceptance/preview_acceptance.md) + or the producer's acceptance. A pin is not raised to a release that is only a draft, a rehearsal or a local candidate. +2. **What changes.** The release tag, asset name, size and SHA-256 of the bootstrap CLI, and the SDK manifest, archive, contract and + content identity together; never one without the others. Do not touch product source in the same pull request. +3. **Evidence in the pull request.** Run `python tools/check_ci_pins.py` and quote its output: it re-downloads each host's published + bytes and fails on a size, digest, manifest or contract mismatch. `OK` for every host is required; a host you did not change must + still be `OK`, because a published asset can be withdrawn or replaced. Cite the release record that justifies the move. +4. **What must pass.** `CI gate` on the pull request (every required leg builds the checked-out sources with the new pins) and, for + an SDK change, the integration workflow's redirect replay for the affected hosts. A pin change that only some legs exercise is split + by host. +5. **After merge.** The `push` run on `main` must be green. If the bootstrap CLI changed, record in acceptance that stage 1 and stage 2 + still match or why they differ; if the SDK changed, record the new suite counts. A failed promotion is reverted, not patched with a + retry, and the previous known-good pin stays in history. +6. **Who approves.** The `main` ruleset requires `CI gate`; the repository administrator can merge. That is a deliberate-pause control for + a one-person repository, not independent review. A second reviewer, when one exists, should approve pin changes. +7. **Adding a host.** A new leg (for example Windows/arm64) needs, in order: a public SDK release for that host with its record, a + published CLI Preview for that host built by that SDK alone, a hosted runner label that exists for the repository, and a local + `tools/ci_bootstrap.py --host ` pass. Only then add the matrix row, the `pins.json` host and the driver's host key in one pull + request, and run it through a manual dispatch before treating it as required. A host with no hosted runner is recorded as unsupported, + not simulated. +8. **Release pins are separate.** Publishing a Preview does not move `ci.yml` pins. After publication, raise the pin in its own pull + request under this policy, because the published bytes are what the pin must match. + + The reviewed `src/test/data/zlib.recipe.json` fixture is checked out with LF through `.gitattributes` on every host. Recipe verification hashes exact bytes; line-ending conversion produces a different, unreviewed input. diff --git a/tools/check_ci_pins.py b/tools/check_ci_pins.py new file mode 100644 index 0000000..c12e865 --- /dev/null +++ b/tools/check_ci_pins.py @@ -0,0 +1,92 @@ +#!/usr/bin/env python3 +"""Review aid for .github/ci/pins.json: re-derive every pin from the published release bytes. + +A pin is only as good as the check that produced it. This downloads each host's pinned bootstrap CLI archive and the SDK +manifest, archive and contract from their public releases and compares size and SHA-256 (and the SDK content identity named by +the manifest) with the pins. It changes nothing and publishes nothing; a mismatch exits 1. Run it in the same pull request that +changes a pin, and quote its output in the review. +""" +import argparse +import hashlib +import json +import ssl +import sys +import urllib.request +from pathlib import Path + +ROOT = Path(__file__).resolve().parent.parent + + +class PinError(Exception): + pass + + +def fetch(url): + with urllib.request.urlopen(url, timeout=300, context=ssl.create_default_context()) as response: + return response.read() + + +def compare(label, data, size, sha256): + actual = hashlib.sha256(data).hexdigest() + problems = [] + if len(data) != size: + problems.append('%s: size %d differs from the pin %d' % (label, len(data), size)) + if actual != sha256: + problems.append('%s: SHA-256 %s differs from the pin %s' % (label, actual, sha256)) + return problems + + +def manifest_asset(manifest, name): + for asset in manifest.get('assets', []): + if asset.get('name') == name: + return asset + return None + + +def review_host(host, pins, get=fetch): + """Return the list of problems for one host; an empty list means every pin matches the published bytes.""" + problems = [] + boot, sdk = pins['bootstrap'], pins['sdk'] + base = 'https://github.com/webos21/crt-cli/releases/download/%s/' % boot['release'] + problems += compare('%s bootstrap archive' % host, get(base + boot['asset']), boot['size'], boot['sha256']) + manifest_bytes = get(sdk['release_base'] + sdk['manifest']['name']) + problems += compare('%s SDK manifest' % host, manifest_bytes, sdk['manifest']['size'], sdk['manifest']['sha256']) + manifest = json.loads(manifest_bytes) + archive = manifest_asset(manifest, sdk['archive']) + if archive is None: + return problems + ['%s: the SDK manifest does not list the pinned archive %s' % (host, sdk['archive'])] + problems += compare('%s SDK archive' % host, get(sdk['release_base'] + sdk['archive']), archive['size'], archive['sha256']) + contract = get(sdk['release_base'] + sdk['contract']) + if hashlib.sha256(contract).hexdigest() != sdk['contract_sha256']: + problems.append('%s: SDK contract SHA-256 differs from the pin' % host) + listed = manifest_asset(manifest, sdk['contract']) + if listed is not None and listed.get('sha256') != sdk['contract_sha256']: + problems.append('%s: the pinned contract digest differs from the one the SDK manifest lists' % host) + for key in ('content_sha256',): + content = manifest.get('content_sha256') or manifest.get('sdk', {}).get(key) + if content is not None and content != sdk[key]: + problems.append('%s: the pinned %s differs from the SDK manifest' % (host, key)) + return problems + + +def main(): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument('--pins', default=str(ROOT / '.github/ci/pins.json')) + parser.add_argument('--host', action='append', help='limit to these hosts (default: all)') + args = parser.parse_args() + pins = json.loads(Path(args.pins).read_text(encoding='utf-8'))['hosts'] + failed = False + for host in args.host or sorted(pins): + try: + problems = review_host(host, pins[host]) + except (OSError, ValueError, KeyError) as error: + problems = ['%s: could not be checked: %s' % (host, error)] + for problem in problems: + print('MISMATCH ' + problem, file=sys.stderr) + print('%s %s' % ('FAIL' if problems else 'OK ', host), flush=True) + failed = failed or bool(problems) + return 1 if failed else 0 + + +if __name__ == '__main__': + sys.exit(main()) diff --git a/tools/test_check_ci_pins.py b/tools/test_check_ci_pins.py new file mode 100644 index 0000000..0857c50 --- /dev/null +++ b/tools/test_check_ci_pins.py @@ -0,0 +1,60 @@ +"""The pin reviewer must report a mismatch, not only print OK.""" +import hashlib +import json +import unittest + +from check_ci_pins import review_host + +def sha(data): + return hashlib.sha256(data).hexdigest() + +BOOT, ARCHIVE, CONTRACT = b'bootstrap-archive', b'sdk-archive', b'sdk-contract' + +def world(**changes): + manifest = {'assets': [{'name': 'sdk.tar.xz', 'size': len(ARCHIVE), 'sha256': sha(ARCHIVE)}, + {'name': 'sdk-toolchain.json', 'size': len(CONTRACT), 'sha256': sha(CONTRACT)}]} + manifest.update(changes.pop('manifest', {})) + manifest_bytes = json.dumps(manifest).encode() + pins = {'bootstrap': {'release': 'v1', 'asset': 'cli.tar.xz', 'size': len(BOOT), 'sha256': sha(BOOT)}, + 'sdk': {'release_base': 'https://example.invalid/sdk/', 'manifest': {'name': 'm.json', 'size': len(manifest_bytes), 'sha256': sha(manifest_bytes)}, + 'archive': 'sdk.tar.xz', 'contract': 'sdk-toolchain.json', 'contract_sha256': sha(CONTRACT), 'content_sha256': 'c' * 64}} + files = {'https://github.com/webos21/crt-cli/releases/download/v1/cli.tar.xz': BOOT, + 'https://example.invalid/sdk/m.json': manifest_bytes, 'https://example.invalid/sdk/sdk.tar.xz': ARCHIVE, + 'https://example.invalid/sdk/sdk-toolchain.json': CONTRACT} + files.update(changes.get('files', {})) + return pins, (lambda url: files[url]) + +class ReviewTest(unittest.TestCase): + def test_matching_pins_report_nothing(self): + pins, get = world() + self.assertEqual(review_host('h', pins, get), []) + + def test_a_changed_bootstrap_archive_is_reported(self): + pins, get = world(files={'https://github.com/webos21/crt-cli/releases/download/v1/cli.tar.xz': b'tampered-bootstrap'}) + problems = review_host('h', pins, get) + self.assertTrue(any('bootstrap archive' in p and 'SHA-256' in p for p in problems), problems) + + def test_a_changed_size_alone_is_reported(self): + pins, get = world() + pins['bootstrap']['size'] += 1 + self.assertTrue(any('size' in p for p in review_host('h', pins, get))) + + def test_a_changed_sdk_archive_or_contract_is_reported(self): + pins, get = world(files={'https://example.invalid/sdk/sdk.tar.xz': b'other-sdk-archive!'}) + self.assertTrue(any('SDK archive' in p for p in review_host('h', pins, get))) + pins, get = world(files={'https://example.invalid/sdk/sdk-toolchain.json': b'other-contract'}) + self.assertTrue(any('contract' in p for p in review_host('h', pins, get))) + + def test_an_archive_the_manifest_does_not_list_is_reported(self): + pins, get = world() + pins['sdk']['archive'] = 'unlisted.tar.xz' + self.assertTrue(any('does not list the pinned archive' in p for p in review_host('h', pins, get))) + + def test_a_contract_digest_that_differs_from_the_manifest_is_reported(self): + wrong = {'assets': [{'name': 'sdk.tar.xz', 'size': len(ARCHIVE), 'sha256': sha(ARCHIVE)}, + {'name': 'sdk-toolchain.json', 'size': len(CONTRACT), 'sha256': '0' * 64}]} + pins, get = world(manifest=wrong) + self.assertTrue(any('differs from the one the SDK manifest lists' in p for p in review_host('h', pins, get))) + +if __name__ == '__main__': + unittest.main()