Skip to content

ci(flow): commit artifact paths only; pin their hashes at release - #1342

Merged
abrichr merged 4 commits into
mainfrom
ci/inventory-classify-on-prs
Oct 19, 2026
Merged

abrichr merged 4 commits into
mainfrom
ci/inventory-classify-on-prs

Conversation

@abrichr

@abrichr abrichr commented Oct 19, 2026

Copy link
Copy Markdown
Member

Why

packages/flow/public-artifacts.json pinned the SHA-256 of every public data or configuration file. Any pull request that edited claims.yaml, docs/verification.json or a fixture rewrote the same hash rows. Two such pull requests always conflicted, and when git merged them cleanly the hashes on the merge commit were wrong anyway.

Since 2026-10-17, 70 commits on PR branches touched this file. 28 of them were merges from main and 6 were inventory refreshes. Each one was a push that reran the full PR pipeline (about 40 jobs).

What changes

  • Committed file: paths only (schema 2). --write-public-artifact-inventory writes a sorted path list. Editing a listed file no longer touches the inventory. Adding, removing or renaming an artifact still requires a one-line edit to the list.
  • Release: hashes are stamped on the exact commit being built. release-package.yml (package flow) runs --stamp-release-inventory. That command first validates the tree against the reviewed list, then rewrites the list as schema 1 (path + SHA-256), as before. After python -m build, it runs --validate-dist-dir. The stamp hashes only the reviewed paths, so a file that appears only at build time is still flagged as unregistered in the archive check.
  • Archives that embed the unstamped list are refused. Wheel and sdist validation now requires the hashed form (require_hashes=True).

What stays exactly as strict

  • On PRs and on main, python scripts/check_release_consistency.py still fails on any unlisted or missing artifact path, untracked listed paths, symlinks, private path segments, private-corpus content signatures, repository-only evaluation paths and the bounded lending evidence shape. Only the hash comparison moved to the release.
  • A tree that carries a stamped (schema 1) inventory is still hash-checked, as before.
  • The bytes a pull request changes are reviewed in its own diff. Before this PR, the hash rows recorded that a human had regenerated them, not that anyone had reviewed the bytes.

This also adds a check. release-package.yml did not run flow's archive boundary checks before; python -m build output went straight to PyPI. It now runs --validate-dist-dir, which checks the license, the open-core boundary and artifact provenance.

Verified

  • The new list has exactly the same 1,636 paths as main's inventory.
  • Release steps on a clean clone (--stamp-release-inventory, python -m build packages/flow, --validate-dist-dir ../../dist): pass. The stamped rows are identical to the hashed rows on main today.
  • A wheel or sdist built without stamping is rejected ("embeds the unstamped path list").
  • The script still runs with the standard library only, in a bare venv, as the release job requires.
  • tests/test_release_contract.py and tests/test_source_policy_binding.py: 81 passed. packages/privacy/tests/test_release_config.py: passed. Ruff and mypy are clean.

Savings

Each avoided conflict saves one push and one full PR pipeline rerun (about 40 jobs). From the counts above, that was around 30 reruns over two days.

One-time cost: open PRs that edit public-artifacts.json will conflict once with this change. To resolve, take main's file and run python scripts/check_release_consistency.py --write-public-artifact-inventory.

No changes to packages.yml, desktop-release.yml or main.yml; the Desktop 0.21.0 release does not depend on anything here.

🤖 Generated with Claude Code

public-artifacts.json pinned the SHA-256 of every public data/config file,
so each pull request that edited claims.yaml, verification.json or a fixture
rewrote the same hash rows, and parallel pull requests conflicted on them.

The committed file is now the reviewed path list (schema 2). Pull requests and
main still fail on any unlisted or missing artifact path, and every open-core
path, signature and license check is unchanged. The release workflow runs
--stamp-release-inventory on the exact commit it builds, which rewrites the
list as schema 1 (path + SHA-256), and then validates the wheel and sdist with
--validate-dist-dir. Archives that embed the unstamped list are refused.
release-package.yml did not run the flow archive checks at all before this.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…-on-prs

# Conflicts:
#	packages/flow/public-artifacts.json
abrichr and others added 2 commits October 19, 2026 15:00
…ed list in archive tests

The archive tests built their stamped inventory by walking packages/flow, so a
symlink another test left under runs/ failed them (flow core unit shard 3/4).
They now hash the committed path list directly.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…-on-prs

# Conflicts:
#	packages/flow/public-artifacts.json
@abrichr
abrichr merged commit 9cfc7ee into main Oct 19, 2026
40 checks passed
@abrichr
abrichr deleted the ci/inventory-classify-on-prs branch October 19, 2026 22:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant