Skip to content

ci(flow): one file per claim; deterministic verification report - #1344

Merged
abrichr merged 8 commits into
mainfrom
ci/claims-one-file-each
Oct 19, 2026
Merged

abrichr merged 8 commits into
mainfrom
ci/claims-one-file-each

Conversation

@abrichr

@abrichr abrichr commented Oct 19, 2026

Copy link
Copy Markdown
Member

Merge after #1342. This branch is stacked on it. Until #1342 merges, this diff also shows #1342's commit. This PR targets main so that dependency-review and CodeQL, which run only on PRs into main, still report.

Why

Any two pull requests that added a claim conflicted on three files:

  • claims.yaml: every claim lived in one list, and new claims were appended at the end of the same section.
  • docs/VERIFICATION.md and docs/verification.json: line one held the HEAD commit date, which changed on every regeneration. The summary listed all claim ids, with counts, on one line per tier.

Since 2026-10-17, PR branches have made 8 commits to claims.yaml and 12 to docs/verification.json, not counting the PR merges themselves.

What changes

  • claims/<id>.yaml, one claim per file. The loader refuses a file whose id doesn't match its file name, which also keeps ids unique. The header contract from claims.yaml moves to claims/README.md. The parsed content of all 25 claims is unchanged.
  • The report is deterministic and does not depend on registry order.
    • It carries a fixed stamp. The legacy validate-claims.yml sync check already used this same stamp.
    • Claims are grouped by tier (strongest first) and sorted by id.
    • The summary lists one id per line, with no counts.
    • Every claim's entry in verification.json is identical to main's; only the order and the stamp changed.
  • New tests in tests/test_validate_claims.py:
    • The committed report must equal the rendered one. Before this PR, nothing in the monorepo CI checked that the report was in sync.
    • Rendering forward and reversed registries gives identical output.
    • The registry must have one file per claim, each named by its id.
    • The default stamp is fixed.

The tier/evidence gate is unchanged (test_real_registry_passes, --check --structure-only).

Verified

  • python scripts/validate_claims.py --check --structure-only: passes, 25 claims, 16 supported.
  • tests/test_validate_claims.py: 29 passed.
  • Merge simulation on a scratch clone: two branches each added a claim, regenerated the report and refreshed the inventory list, then were merged.
    • All three generated files auto-merged.
    • The sync test passed on the merge result.
    • check_release_consistency.py passed.
    • On main today, the same two branches conflict on claims.yaml, both report files and the inventory hash rows.

Savings

A PR that adds a claim no longer forces a rebase, push and full PR pipeline rerun (about 40 jobs) on every other PR that adds or edits a claim.

A conflict can still happen when two new claims are in the same tier and sort next to each other (no existing claim between them). That is now the exception rather than the rule, and the fix is to run --report again.

One-time cost: open PRs that edit claims.yaml will conflict once. Move the claim into claims/<id>.yaml and run python scripts/validate_claims.py --report.

No changes to packages.yml, desktop-release.yml or main.yml.

🤖 Generated with Claude Code

abrichr and others added 4 commits October 19, 2026 11:18
public-artifacts.json pinned the SHA-256 of every public data/config file,
so each pull request that edited claims.yaml, verification.json or a fixture
rewrote the same hash rows, and parallel pull requests conflicted on them.

The committed file is now the reviewed path list (schema 2). Pull requests and
main still fail on any unlisted or missing artifact path, and every open-core
path, signature and license check is unchanged. The release workflow runs
--stamp-release-inventory on the exact commit it builds, which rewrites the
list as schema 1 (path + SHA-256), and then validates the wheel and sdist with
--validate-dist-dir. Archives that embed the unstamped list are refused.
release-package.yml did not run the flow archive checks at all before this.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
claims.yaml held every claim in one list, and docs/VERIFICATION.md and
docs/verification.json stamped the HEAD commit date on line one and listed
all claim ids on one summary line with counts. Any two pull requests that
added a claim conflicted on all three files.

- claims/<id>.yaml: one claim per file; the file must be named after its id.
  The old header contract moves to claims/README.md. Parsed content of all 25
  claims is unchanged.
- The report carries a fixed stamp, groups claims by tier and sorts them by
  id, and lists one id per line with no counts. Every claim's entry is
  identical to main's.
- tests/test_validate_claims.py now fails when the committed report is stale
  and checks that the render does not depend on registry order.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…-on-prs

# Conflicts:
#	packages/flow/public-artifacts.json
abrichr and others added 4 commits October 19, 2026 15:00
…ed list in archive tests

The archive tests built their stamped inventory by walking packages/flow, so a
symlink another test left under runs/ failed them (flow core unit shard 3/4).
They now hash the committed path list directly.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…to ci/claims-one-file-each

# Conflicts:
#	packages/flow/claims.yaml
#	packages/flow/docs/VERIFICATION.md
#	packages/flow/docs/verification.json
…-on-prs

# Conflicts:
#	packages/flow/public-artifacts.json
@abrichr
abrichr merged commit c80d3f0 into main Oct 19, 2026
40 checks passed
@abrichr
abrichr deleted the ci/claims-one-file-each branch October 19, 2026 22:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant