-
Notifications
You must be signed in to change notification settings - Fork 0
feat(ci): add strict admission and anchored revocation ledger #2869
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
7a7c07a
feat(ci): add fail-closed admission and anchored evidence ledger
Chris0Jeky e1c8116
Merge branch 'codex/2336-observer-review' into codex/2336-admission-r…
Chris0Jeky 73c98e8
Merge branch 'main' into ci/continuation-06-admission
Chris0Jeky cb7ae41
Merge branch 'main' into ci/continuation-06-admission
Chris0Jeky 3c76464
Merge branch 'main' into ci/continuation-06-admission
Chris0Jeky 10c27c3
Merge branch 'main' into ci/continuation-06-admission
Chris0Jeky File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,43 @@ | ||
| # Admissibility, revocation and bounded exposure | ||
|
|
||
| Date: 2026-09-10. Related: #2327, #2336 and #2339. Parent: [engineering contract](README.md). | ||
|
|
||
| ## Auxiliary decision, not a second required gate | ||
|
|
||
| `core/admission.mjs` recomputes the continuation plan from protected inputs rather than trusting an uploaded selected list or digest. The entire claimed plan must match. Every task is accounted for once: fresh verified execution, still-valid signed reuse, or explicitly qualified unaffected selection. Missing, duplicate, unknown, failed, skipped, cancelled, wrong-candidate, wrong-policy, empty-test and retry-erased outcomes reject admission. | ||
|
|
||
| Output is `authority:none`, not a ci-run.v1 replacement, and is never posted as a required GitHub check. Taskdeck's canonical gate remains authoritative; shadow mode is unchanged. | ||
|
|
||
| `verifyFresh` is a protected-controller callback, not a JSON field. Its default refuses proof. Expected command/environment/input identities are recomputed; producer contracts bind reviewed workflow revision/path/ID and run/job IDs must be present. The callback must authenticate actual execution independently. The fixture's callback returning true simulates that verifier; it is not production provenance. | ||
|
|
||
| Omission also requires explicit selection qualification and reviewed contracts. The package cannot grant those approvals itself. Taskdeck contracts remain unreviewed and production reuse disabled. Full qualification bypasses reuse; reused proof retains its original completion/expiry. | ||
|
|
||
| ## Durable reference ledger | ||
|
|
||
| `core/ledger.mjs` supplies bounded single-host JSONL append storage, canonical hash chaining, exclusive writer lock, expected-anchor comparison, file fsync, revocation/circuit reduction and age-plus-merge-exposure decisions. Events record revocation, trip, successful complete baseline, explicit recovery or landed exposure. Invalid recovery is rejected before append; historical revocations are not silently cleared. | ||
|
|
||
| **Hash chaining is not authentication.** An independently protected latest anchor must be supplied and each returned digest published through a protected compare-and-swap store. Trusting a PR artifact's own final hash protects nothing. The directory, parent filesystem, tooling and anchor provider must be trusted and inaccessible to candidate writes. No production store/key/secret/admin setting is provisioned here. | ||
|
|
||
| Append validates the whole current chain against the supplied anchor, validates the new semantic transition, appends/fsyncs and returns the new digest. Concurrent stale-anchor writers fail. Partial writes, corruption, rollback/stale anchors, contention and the 8 MiB capacity ceiling fail closed. Archival/checkpointing requires explicit review; history is never silently dropped. | ||
|
|
||
| This is not a distributed database. Shared network filesystems, multi-host locking, power-loss guarantees beyond file fsync and transactional external-anchor publication are not claimed. A crash between append and anchor publication leaves an unanchored tail: disable optimisation and reconcile it against protected execution records before advancing the anchor. Do not auto-truncate tails or break stale locks merely because they are old. | ||
|
|
||
| ## Circuit and exposure rules | ||
|
|
||
| The protected controller should append revocation/trip events after an omitted oracle failure. A trip remains until an explicitly referenced NEWER complete successful baseline covers the affected task. An older baseline or selective green retry cannot recover it. Duplicate baseline identities fail. | ||
|
|
||
| `qualificationRequired` checks authenticated state, current policy, full current task-universe coverage, maximum age, maximum landed merges since full baseline and open circuits. Missing/unverified/partial/expired/future state or exhausted limits requires full qualification. Release/R4/other mandatory policy passes mandatory:true; this utility cannot downgrade it. | ||
|
|
||
| Permission to use ordinary policy is not successful CI. Landed exposure must come from authenticated events. No event broker is installed; the metadata observer cannot issue full-baseline events. A protected controller must verify actual checkout, complete suite execution and bypassed reuse first. | ||
|
|
||
| ## Activation and remaining integration | ||
|
|
||
| Validate immutable inputs and event/merge bindings, establish independent execution provenance, provide protected anchor/key/revocation storage, qualify selection with frozen-plan/full-oracle recall, rehearse corruption/cancellation/audit misses, then request maintainer review for one family. Merely changing mode to enforce is not activation approval. | ||
|
|
||
| Administrative and evidence gates remain open. This PR does not provide a production GitHub fresh-execution verifier or change canonical gate handling. Those integrations remain explicitly outstanding, not represented by a permissive fake verifier. | ||
|
|
||
| ## Validation and rollback | ||
|
|
||
| Combined continuation/placement suite: **352 passed, zero failed/skipped/cancelled**, local Node 22.16.0/Linux. New regressions cover recomputation, stale/empty/skipped/cancelled/retried outcomes, missing verifier and exception redaction, unqualified omission, contention, rollback/corrupt/partial chains, invalid recovery, duplicate baselines and age/exposure circuits. | ||
|
|
||
| This is reference-mechanism validation, not production provenance or distributed durability. Hosted configured-Node checks and independent/maintainer review remain required. Revert auxiliary modules without altering canonical qualification; preserve any real ledger and historical revocations separately from code rollback. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,49 @@ | ||
| import { planContinuation } from './planner.mjs'; | ||
| import { assertProducer } from './evidence.mjs'; | ||
| import { canonical, hash, invariant, isGitId } from './primitives.mjs'; | ||
|
|
||
| /** | ||
| * Auxiliary admissibility check, not Taskdeck's canonical gate. All inputs/producer | ||
| * contracts and the verifyFresh callback must originate in a protected controller. | ||
| * The metadata-only GitHub observer cannot satisfy verifyFresh. | ||
| */ | ||
| export async function evaluateAdmission({ input, claimedPlan, executions = [], producerContracts = {}, verifyFresh = async () => false }) { | ||
| try { | ||
| invariant(input?.mode === 'enforce', 'explicit reviewed enforcement integration required'); | ||
| const plan = planContinuation(input); | ||
| invariant(!plan.planningError, 'planning failure requires full qualification'); | ||
| invariant(canonical(plan) === canonical(claimedPlan), 'plan differs from protected recomputation'); | ||
| invariant(Array.isArray(executions), 'execution inventory required'); | ||
|
Chris0Jeky marked this conversation as resolved.
|
||
| const records = new Map(); | ||
| for (const record of executions) { | ||
| invariant(record && !records.has(record.taskId) && plan.tasks.some(t => t.taskId === record.taskId), 'duplicate or unknown execution'); | ||
| records.set(record.taskId, record); | ||
| } | ||
| const satisfied = []; | ||
| for (const task of plan.tasks) { | ||
| const record = records.get(task.taskId); | ||
| // Contradictory observations cannot be hidden behind a reused/unaffected decision. | ||
| if (record) invariant(task.action === 'run', 'unexpected execution contradicts frozen decision'); | ||
| if (task.action === 'reuse') { satisfied.push({ taskId: task.taskId, kind: 'reuse', origin: task.origin }); continue; } | ||
|
Chris0Jeky marked this conversation as resolved.
|
||
| if (task.action === 'unaffected') { invariant(input.selectionQualified === true && input.graph.tasks[task.taskId].reviewed === true, 'unqualified input-based omission'); satisfied.push({ taskId: task.taskId, kind: 'policy-unaffected' }); continue; } | ||
| invariant(record, `missing required execution: ${task.taskId}`); | ||
| invariant(['runId', 'jobId'].every(k => typeof record[k] === 'string' && /^[1-9]\d*$/.test(record[k])), 'execution run/job identity required'); | ||
| const contract = producerContracts[task.taskId]; | ||
| invariant(contract && isGitId(contract.workflowRevision) && typeof contract.requiresTests === 'boolean', 'reviewed producer contract required'); | ||
| const spec = input.graph.tasks[task.taskId]; | ||
| const expected = { ...contract, repositoryId: input.repositoryId, commit: input.candidateState.commit, tree: input.candidateState.tree, | ||
| taskId: task.taskId, inputKey: task.inputKey, graphDigest: plan.binding.graphDigest, policyDigest: input.policyDigest, | ||
| platform: spec.platform, commandDigest: hash(spec.command), environmentDigest: hash(input.environments?.[task.taskId] ?? {}) }; | ||
| assertProducer(expected, record); | ||
| // Copy inputs: a verifier may inspect data but cannot mutate this verdict's bindings. | ||
| let verified = false; | ||
| try { verified = await verifyFresh(structuredClone(expected), structuredClone(record)); } | ||
| catch { throw new Error('fresh execution verifier unavailable'); } | ||
| invariant(verified === true, 'fresh execution provenance not established'); | ||
| satisfied.push({ taskId: task.taskId, kind: 'fresh' }); | ||
| } | ||
| return { format: 'ci.admission.v1', authority: 'none', admissible: true, binding: plan.binding, decisionDigest: plan.decisionDigest, satisfied }; | ||
| } catch (error) { | ||
| return { format: 'ci.admission.v1', authority: 'none', admissible: false, reason: error.message, satisfied: [] }; | ||
| } | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,95 @@ | ||
| import { mkdirSync, openSync, readFileSync, writeFileSync, closeSync, fsyncSync, lstatSync, rmSync } from 'node:fs'; | ||
| import { join, resolve } from 'node:path'; | ||
| import { canonical, hash, invariant, isDigest, isGitId } from './primitives.mjs'; | ||
|
|
||
| export const GENESIS = hash({ domain: 'ci.ledger.genesis.v1' }); | ||
| const ref = value => typeof value === 'string' && /^[a-z0-9-]+$/.test(value); | ||
| function validateEvent(e) { | ||
| invariant(e && Object.keys(e).every(k => ['kind', 'at', 'taskId', 'inputKey', 'reason', 'baselineId', 'policyDigest', 'tree', 'covers', 'complete', 'conclusion'].includes(k)), 'unknown ledger event field'); | ||
| invariant(Number.isSafeInteger(e.at) && e.at >= 0, 'invalid ledger timestamp'); | ||
| if (e.kind === 'revoke') invariant(isDigest(e.inputKey) && typeof e.reason === 'string' && e.reason.length > 0, 'invalid revocation'); | ||
| else if (e.kind === 'trip') invariant(ref(e.taskId) && typeof e.reason === 'string' && e.reason.length > 0, 'invalid circuit event'); | ||
| else if (e.kind === 'full') invariant(isDigest(e.baselineId) && isDigest(e.policyDigest) && isGitId(e.tree) && e.complete === true && e.conclusion === 'success' && Array.isArray(e.covers) && e.covers.length > 0 && new Set(e.covers).size === e.covers.length && e.covers.every(ref), 'invalid full baseline'); | ||
| else if (e.kind === 'recover') invariant(ref(e.taskId) && isDigest(e.baselineId), 'invalid recovery'); | ||
| else if (e.kind === 'merge') invariant(isGitId(e.tree) && isDigest(e.policyDigest), 'invalid merge exposure'); | ||
| else throw new Error('unknown ledger event kind'); | ||
| return e; | ||
| } | ||
|
|
||
| /** Hash chain detects corruption/rollback only against an independently protected anchor. */ | ||
| export function decodeLedger(text, expectedAnchor, { maxBytes = 8 * 1024 * 1024 } = {}) { | ||
| invariant(isDigest(expectedAnchor) && typeof text === 'string' && Buffer.byteLength(text) <= maxBytes, 'trusted anchor/bounded ledger required'); | ||
| invariant(text === '' || text.endsWith('\n'), 'partial ledger write'); | ||
| const records = [], lines = text === '' ? [] : text.slice(0, -1).split('\n'); | ||
| let anchor = GENESIS, previousAt = 0; | ||
| for (const line of lines) { | ||
| const r = JSON.parse(line); | ||
| invariant(r && Object.keys(r).sort().join(',') === 'digest,event,previous,sequence', 'invalid ledger record'); | ||
| validateEvent(r.event); | ||
| invariant(r.sequence === records.length + 1 && r.previous === anchor && r.event.at >= previousAt, 'ledger order/binding mismatch'); | ||
| invariant(r.digest === hash({ domain: 'ci.ledger.record.v1', sequence: r.sequence, previous: r.previous, event: r.event }), 'ledger corruption'); | ||
| records.push(r); anchor = r.digest; previousAt = r.event.at; | ||
| } | ||
| invariant(anchor === expectedAnchor, 'ledger rollback or stale anchor'); | ||
| return { records, anchor }; | ||
| } | ||
|
|
||
| /** Single-host reference store. Directory and external anchor are controller-owned, never PR writable. */ | ||
| export function appendLedger(directory, expectedAnchor, event) { | ||
| validateEvent(event); invariant(isDigest(expectedAnchor), 'trusted expected anchor required'); | ||
| const dir = resolve(directory); mkdirSync(dir, { recursive: true, mode: 0o700 }); | ||
| invariant(lstatSync(dir).isDirectory() && !lstatSync(dir).isSymbolicLink(), 'ledger directory must be ordinary trusted storage'); | ||
| const lock = join(dir, 'writer.lock'), path = join(dir, 'events.jsonl'); | ||
| let lockFd; | ||
| try { lockFd = openSync(lock, 'wx', 0o600); } | ||
| catch { throw new Error('ledger writer busy or storage unavailable'); } | ||
| try { | ||
| let text = ''; | ||
| try { | ||
| const stat = lstatSync(path); invariant(stat.isFile() && !stat.isSymbolicLink() && stat.size <= 8 * 1024 * 1024, 'invalid ledger file'); | ||
| text = readFileSync(path, 'utf8'); | ||
| } catch (error) { if (error.code !== 'ENOENT') throw error; } | ||
| const ledger = decodeLedger(text, expectedAnchor); | ||
| invariant(event.at >= (ledger.records.at(-1)?.event.at ?? 0), 'ledger clock moved backwards'); | ||
| const unsigned = { sequence: ledger.records.length + 1, previous: expectedAnchor, event }; | ||
| const record = { ...unsigned, digest: hash({ domain: 'ci.ledger.record.v1', ...unsigned }) }; | ||
| // Reject semantically invalid recovery before it can poison the durable chain. | ||
| ledgerState({ records: [...ledger.records, record], anchor: record.digest }); | ||
| const next = canonical(record) + '\n'; | ||
| invariant(Buffer.byteLength(text) + Buffer.byteLength(next) <= 8 * 1024 * 1024, 'ledger capacity reached'); | ||
| const fd = openSync(path, 'a', 0o600); | ||
| try { writeFileSync(fd, next); fsyncSync(fd); } finally { closeSync(fd); } | ||
| return record; // caller must publish digest through a protected compare-and-swap anchor store | ||
| } finally { closeSync(lockFd); rmSync(lock); } | ||
| } | ||
|
|
||
| /** Reducer does not erase historical revocations. Recovery requires a newer complete covering baseline. */ | ||
| export function ledgerState(ledger) { | ||
| const revoked = new Set(), disabled = new Map(), fulls = new Map(); let baseline = null, mergesSinceFull = 0; | ||
| for (const { event: e } of ledger.records) { | ||
| if (e.kind === 'revoke') revoked.add(e.inputKey); | ||
| else if (e.kind === 'trip') disabled.set(e.taskId, e.at); | ||
| else if (e.kind === 'full') { invariant(!fulls.has(e.baselineId), 'duplicate full baseline identity'); fulls.set(e.baselineId, e); baseline = e; mergesSinceFull = 0; } | ||
|
Chris0Jeky marked this conversation as resolved.
|
||
| else if (e.kind === 'merge') mergesSinceFull++; | ||
|
Chris0Jeky marked this conversation as resolved.
|
||
| else if (e.kind === 'recover') { | ||
| const full = fulls.get(e.baselineId), trip = disabled.get(e.taskId); | ||
| invariant(trip !== undefined && full && full.at > trip && full.at <= e.at && full.covers.includes(e.taskId), 'recovery lacks a newer covering full baseline'); | ||
|
Chris0Jeky marked this conversation as resolved.
|
||
| disabled.delete(e.taskId); | ||
| } | ||
| } | ||
| return { anchor: ledger.anchor, revokedInputKeys: [...revoked].sort(), disabledTasks: [...disabled.keys()].sort(), baseline, mergesSinceFull }; | ||
| } | ||
|
|
||
| /** Freshness plus merge exposure. Missing/partial/untrusted state always forces full qualification. */ | ||
| export function qualificationRequired({ state, anchorVerified, now, policyDigest, universe, maxAgeSeconds, maxMerges, mandatory = false }) { | ||
| try { | ||
| invariant(!mandatory && anchorVerified === true && state && isDigest(state.anchor), 'mandatory full or unverifiable state'); | ||
| invariant(Number.isSafeInteger(now) && Number.isSafeInteger(maxAgeSeconds) && maxAgeSeconds > 0 && Number.isSafeInteger(maxMerges) && maxMerges > 0, 'invalid qualification budget'); | ||
| invariant(Array.isArray(universe) && universe.length > 0 && new Set(universe).size === universe.length, 'complete qualification universe required'); | ||
| const b = state.baseline; | ||
| invariant(b?.complete === true && b.conclusion === 'success' && b.policyDigest === policyDigest && universe.every(t => b.covers.includes(t)), 'missing full baseline coverage'); | ||
| invariant(b.at <= now && now - b.at < maxAgeSeconds && Number.isSafeInteger(state.mergesSinceFull) && state.mergesSinceFull >= 0 && state.mergesSinceFull < maxMerges, 'qualification age/exposure exhausted'); | ||
| invariant(state.disabledTasks.length === 0, 'audit circuit is open'); | ||
| return { required: false, reason: 'verified baseline within age and exposure limits' }; | ||
| } catch (error) { return { required: true, reason: error.message }; } | ||
| } | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.