Skip to content

fix: add a daily check that main is branch-protected - #48

Open
abrichr wants to merge 2 commits into
mainfrom
fix/ledger-2026-10-10
Open

abrichr wants to merge 2 commits into
mainfrom
fix/ledger-2026-10-10

Conversation

@abrichr

@abrichr abrichr commented Oct 10, 2026

Copy link
Copy Markdown
Member

The main branch of this repository has no branch protection and no ruleset. Anyone with write access can push to it directly, force-push over its history, or delete it, with no review and no checks. That branch holds the admission ledgers and the workflows that issue and verify admissions. This change adds a daily check that fails and opens an issue until main is protected, plus the ruleset that satisfies the check. The protection itself is a repository setting that an owner applies after merge.

Fixes

Problem Change Regression test Ref
main accepted direct pushes, force pushes, and deletion with no review or required checks, and nothing reported it. scripts/check_branch_protection.py reads the ruleset rules for main. When the token has Administration read access, it also reads classic branch protection. It fails unless main requires a pull request, requires the validate-profile check from GitHub Actions (app 15368), and blocks force pushes and deletion. When it can read who may bypass those rules (a ruleset's bypass list, enforce_admins, or the pull request bypass allowances), it also fails if anyone can bypass them. In profile-consistency.yml, the new check-branch-protection job runs the check daily and on manual dispatch, and report-unprotected-branch opens or updates an issue when it fails. The check doesn't run on pull requests. tests/test_branch_protection.py: TodayTests.test_command_exits_nonzero_on_the_observed_github_responses, TodayTests.test_unprotected_main_without_rulesets_fails_every_requirement, ClassicProtectionTests.test_each_missing_classic_setting_fails, ClassicProtectionTests.test_pull_request_bypass_allowance_fails_only_that_requirement, RulesetTests.test_ruleset_with_a_bypass_actor_fails, RulesetTests.test_status_check_must_come_from_github_actions, and 11 more B247
The repository didn't define the protection main should have. main-branch-ruleset.json is an active ruleset for refs/heads/main with no bypass actors. It blocks deletion and force pushes, requires a pull request with no minimum approvals, and requires validate-profile from GitHub Actions. The required settings live in the script, so editing this file can't weaken the check. CommittedRulesetTests.test_committed_ruleset_satisfies_the_check B247

Not fixed here

  • main stays unprotected until an owner applies the ruleset. The scheduled check fails every day until then.
  • A pull request needs no approvals, not one. The only maintainer can't approve their own pull request, and the other OpenAdapt repositories also require 0 approvals. Requiring one approval needs a second reviewer, plus a change to MINIMUM_APPROVALS and the ruleset file.
  • verify-production-release-admission isn't a required check. Other workflows call it, and it never reports on pull requests in this repository, so requiring it would block every merge.
  • With the default Actions token, GitHub hides a ruleset's bypass list. The check then prints a note that bypass wasn't checked, instead of failing. A BRANCH_PROTECTION_READ_TOKEN secret with read access to Administration lets the check verify the bypass list.
  • Pull requests opened by the lifecycle App skip validate-profile because of its actor condition, and GitHub treats a skipped required check as passed. Those pull requests rely on review, not on the check.

How it was tested

  • The new tests fail on main, where the check doesn't exist, and pass on this branch.
  • Full suite: 387 tests pass (python3 -m unittest discover -s tests -p 'test_*.py'), up from 370 on main.
  • scripts/validate_evidence_registry.py, scripts/check_profile.py, and scripts/check_benchmark_claims.py --allow-recorded-drift pass.
  • ruff check reports nothing in the new files. actionlint reports one SC2016 info finding at line 62 of profile-consistency.yml, which also exists on main.
  • Against GitHub, the check exits 1 for OpenAdaptAI/.github main with an owner token and with no token, and reports all four requirements as not set. Against the classic protection on OpenAdapt, openadapt-flow, and openadapt-desktop, it reads the real settings and reports that administrators can bypass them, because enforce_admins is off there.

Before merging

  • After merge, an organization owner applies the ruleset from the repository root, then runs the Profile consistency workflow by hand and confirms that check-branch-protection passes:

    gh api -X POST repos/OpenAdaptAI/.github/rulesets --input main-branch-ruleset.json

    Until then, each daily run fails and comments on the issue "Branch protection on main needs attention".

  • The ruleset lets nobody bypass it, owners included. Every change to main, including an owner's, then needs a pull request with validate-profile passing. The lifecycle workflows already push review branches and open pull requests, and production-lifecycle-ref.yml writes only to production-lifecycle-feed, so they keep working.

  • Classic branch protection also satisfies the check, but only with enforce_admins on and a BRANCH_PROTECTION_READ_TOKEN secret that has read access to Administration. Without that secret, the check can't see classic protection and keeps failing.

  • Decide whether to require one approval. That needs a second reviewer account.

🤖 Generated with Claude Code

abrichr and others added 2 commits October 10, 2026 00:15
Main had no branch protection and no ruleset. Anyone with write access
could push to it directly, force-push over its history, or delete it.
That covers the admission ledgers and the workflows that issue and verify
admissions, and nothing reported it.

What changed:
- scripts/check_branch_protection.py reads the ruleset rules for main and,
  when the token allows, its classic branch protection. It fails unless
  main requires a pull request and the validate-profile check from GitHub
  Actions, and blocks force pushes and deletion. When the token can read
  who may bypass those rules, it also fails if anyone can.
- Profile consistency runs the check on its daily schedule and on manual
  dispatch, and files an issue when it fails. It doesn't run on pull
  requests, so it can't block unrelated changes.
- main-branch-ruleset.json is a ruleset body that satisfies the check. An
  admin applies it with:
  gh api -X POST repos/OpenAdaptAI/.github/rulesets --input main-branch-ruleset.json
- Tests cover the responses GitHub returns today (the check fails),
  classic protection, rulesets, rules split across both, and the
  committed ruleset body.

This commit doesn't change any GitHub setting. The check stays red until
an admin turns on protection for main.

Ledger: B247

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Classic branch protection can list users, teams, or apps in
bypass_pull_request_allowances. They push to main without a pull request
even when enforce_admins is on, so the branch protection check treated
that setup as fully bound. The check now reports the pull request
requirement as bypassable when any allowance is listed, or when the
allowance has a shape it doesn't recognize.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant