Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 43 additions & 0 deletions docs/ci/continuation/ADMISSION_AND_LEDGER.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# Admissibility, revocation and bounded exposure

Date: 2026-09-10. Related: #2327, #2336 and #2339. Parent: [engineering contract](README.md).

## Auxiliary decision, not a second required gate

`core/admission.mjs` recomputes the continuation plan from protected inputs rather than trusting an uploaded selected list or digest. The entire claimed plan must match. Every task is accounted for once: fresh verified execution, still-valid signed reuse, or explicitly qualified unaffected selection. Missing, duplicate, unknown, failed, skipped, cancelled, wrong-candidate, wrong-policy, empty-test and retry-erased outcomes reject admission.

Output is `authority:none`, not a ci-run.v1 replacement, and is never posted as a required GitHub check. Taskdeck's canonical gate remains authoritative; shadow mode is unchanged.

`verifyFresh` is a protected-controller callback, not a JSON field. Its default refuses proof. Expected command/environment/input identities are recomputed; producer contracts bind reviewed workflow revision/path/ID and run/job IDs must be present. The callback must authenticate actual execution independently. The fixture's callback returning true simulates that verifier; it is not production provenance.

Omission also requires explicit selection qualification and reviewed contracts. The package cannot grant those approvals itself. Taskdeck contracts remain unreviewed and production reuse disabled. Full qualification bypasses reuse; reused proof retains its original completion/expiry.

## Durable reference ledger

`core/ledger.mjs` supplies bounded single-host JSONL append storage, canonical hash chaining, exclusive writer lock, expected-anchor comparison, file fsync, revocation/circuit reduction and age-plus-merge-exposure decisions. Events record revocation, trip, successful complete baseline, explicit recovery or landed exposure. Invalid recovery is rejected before append; historical revocations are not silently cleared.

**Hash chaining is not authentication.** An independently protected latest anchor must be supplied and each returned digest published through a protected compare-and-swap store. Trusting a PR artifact's own final hash protects nothing. The directory, parent filesystem, tooling and anchor provider must be trusted and inaccessible to candidate writes. No production store/key/secret/admin setting is provisioned here.

Append validates the whole current chain against the supplied anchor, validates the new semantic transition, appends/fsyncs and returns the new digest. Concurrent stale-anchor writers fail. Partial writes, corruption, rollback/stale anchors, contention and the 8 MiB capacity ceiling fail closed. Archival/checkpointing requires explicit review; history is never silently dropped.

This is not a distributed database. Shared network filesystems, multi-host locking, power-loss guarantees beyond file fsync and transactional external-anchor publication are not claimed. A crash between append and anchor publication leaves an unanchored tail: disable optimisation and reconcile it against protected execution records before advancing the anchor. Do not auto-truncate tails or break stale locks merely because they are old.

## Circuit and exposure rules

The protected controller should append revocation/trip events after an omitted oracle failure. A trip remains until an explicitly referenced NEWER complete successful baseline covers the affected task. An older baseline or selective green retry cannot recover it. Duplicate baseline identities fail.

`qualificationRequired` checks authenticated state, current policy, full current task-universe coverage, maximum age, maximum landed merges since full baseline and open circuits. Missing/unverified/partial/expired/future state or exhausted limits requires full qualification. Release/R4/other mandatory policy passes mandatory:true; this utility cannot downgrade it.

Permission to use ordinary policy is not successful CI. Landed exposure must come from authenticated events. No event broker is installed; the metadata observer cannot issue full-baseline events. A protected controller must verify actual checkout, complete suite execution and bypassed reuse first.

## Activation and remaining integration

Validate immutable inputs and event/merge bindings, establish independent execution provenance, provide protected anchor/key/revocation storage, qualify selection with frozen-plan/full-oracle recall, rehearse corruption/cancellation/audit misses, then request maintainer review for one family. Merely changing mode to enforce is not activation approval.

Administrative and evidence gates remain open. This PR does not provide a production GitHub fresh-execution verifier or change canonical gate handling. Those integrations remain explicitly outstanding, not represented by a permissive fake verifier.

## Validation and rollback

Combined continuation/placement suite: **352 passed, zero failed/skipped/cancelled**, local Node 22.16.0/Linux. New regressions cover recomputation, stale/empty/skipped/cancelled/retried outcomes, missing verifier and exception redaction, unqualified omission, contention, rollback/corrupt/partial chains, invalid recovery, duplicate baselines and age/exposure circuits.
Comment thread
Chris0Jeky marked this conversation as resolved.

This is reference-mechanism validation, not production provenance or distributed durability. Hosted configured-Node checks and independent/maintainer review remain required. Revert auxiliary modules without altering canonical qualification; preserve any real ledger and historical revocations separately from code rollback.
1 change: 1 addition & 0 deletions scripts/ci/smart-ci/continuation.test.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -9,3 +9,4 @@ import './continuation/tests/repository.test.mjs';
import './continuation/tests/workflow.test.mjs';
import './continuation/tests/launcher-inputs.test.mjs';
import './continuation/tests/github.test.mjs';
import './continuation/tests/admission-ledger.test.mjs';
49 changes: 49 additions & 0 deletions scripts/ci/smart-ci/continuation/core/admission.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
import { planContinuation } from './planner.mjs';
import { assertProducer } from './evidence.mjs';
import { canonical, hash, invariant, isGitId } from './primitives.mjs';

/**
* Auxiliary admissibility check, not Taskdeck's canonical gate. All inputs/producer
* contracts and the verifyFresh callback must originate in a protected controller.
* The metadata-only GitHub observer cannot satisfy verifyFresh.
*/
export async function evaluateAdmission({ input, claimedPlan, executions = [], producerContracts = {}, verifyFresh = async () => false }) {
try {
invariant(input?.mode === 'enforce', 'explicit reviewed enforcement integration required');
const plan = planContinuation(input);
invariant(!plan.planningError, 'planning failure requires full qualification');
invariant(canonical(plan) === canonical(claimedPlan), 'plan differs from protected recomputation');
invariant(Array.isArray(executions), 'execution inventory required');
Comment thread
Chris0Jeky marked this conversation as resolved.
const records = new Map();
for (const record of executions) {
invariant(record && !records.has(record.taskId) && plan.tasks.some(t => t.taskId === record.taskId), 'duplicate or unknown execution');
records.set(record.taskId, record);
}
const satisfied = [];
for (const task of plan.tasks) {
const record = records.get(task.taskId);
// Contradictory observations cannot be hidden behind a reused/unaffected decision.
if (record) invariant(task.action === 'run', 'unexpected execution contradicts frozen decision');
if (task.action === 'reuse') { satisfied.push({ taskId: task.taskId, kind: 'reuse', origin: task.origin }); continue; }
Comment thread
Chris0Jeky marked this conversation as resolved.
if (task.action === 'unaffected') { invariant(input.selectionQualified === true && input.graph.tasks[task.taskId].reviewed === true, 'unqualified input-based omission'); satisfied.push({ taskId: task.taskId, kind: 'policy-unaffected' }); continue; }
invariant(record, `missing required execution: ${task.taskId}`);
invariant(['runId', 'jobId'].every(k => typeof record[k] === 'string' && /^[1-9]\d*$/.test(record[k])), 'execution run/job identity required');
const contract = producerContracts[task.taskId];
invariant(contract && isGitId(contract.workflowRevision) && typeof contract.requiresTests === 'boolean', 'reviewed producer contract required');
const spec = input.graph.tasks[task.taskId];
const expected = { ...contract, repositoryId: input.repositoryId, commit: input.candidateState.commit, tree: input.candidateState.tree,
taskId: task.taskId, inputKey: task.inputKey, graphDigest: plan.binding.graphDigest, policyDigest: input.policyDigest,
platform: spec.platform, commandDigest: hash(spec.command), environmentDigest: hash(input.environments?.[task.taskId] ?? {}) };
assertProducer(expected, record);
// Copy inputs: a verifier may inspect data but cannot mutate this verdict's bindings.
let verified = false;
try { verified = await verifyFresh(structuredClone(expected), structuredClone(record)); }
catch { throw new Error('fresh execution verifier unavailable'); }
invariant(verified === true, 'fresh execution provenance not established');
satisfied.push({ taskId: task.taskId, kind: 'fresh' });
}
return { format: 'ci.admission.v1', authority: 'none', admissible: true, binding: plan.binding, decisionDigest: plan.decisionDigest, satisfied };
} catch (error) {
return { format: 'ci.admission.v1', authority: 'none', admissible: false, reason: error.message, satisfied: [] };
}
}
95 changes: 95 additions & 0 deletions scripts/ci/smart-ci/continuation/core/ledger.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
import { mkdirSync, openSync, readFileSync, writeFileSync, closeSync, fsyncSync, lstatSync, rmSync } from 'node:fs';
import { join, resolve } from 'node:path';
import { canonical, hash, invariant, isDigest, isGitId } from './primitives.mjs';

export const GENESIS = hash({ domain: 'ci.ledger.genesis.v1' });
const ref = value => typeof value === 'string' && /^[a-z0-9-]+$/.test(value);
function validateEvent(e) {
invariant(e && Object.keys(e).every(k => ['kind', 'at', 'taskId', 'inputKey', 'reason', 'baselineId', 'policyDigest', 'tree', 'covers', 'complete', 'conclusion'].includes(k)), 'unknown ledger event field');
invariant(Number.isSafeInteger(e.at) && e.at >= 0, 'invalid ledger timestamp');
if (e.kind === 'revoke') invariant(isDigest(e.inputKey) && typeof e.reason === 'string' && e.reason.length > 0, 'invalid revocation');
else if (e.kind === 'trip') invariant(ref(e.taskId) && typeof e.reason === 'string' && e.reason.length > 0, 'invalid circuit event');
else if (e.kind === 'full') invariant(isDigest(e.baselineId) && isDigest(e.policyDigest) && isGitId(e.tree) && e.complete === true && e.conclusion === 'success' && Array.isArray(e.covers) && e.covers.length > 0 && new Set(e.covers).size === e.covers.length && e.covers.every(ref), 'invalid full baseline');
else if (e.kind === 'recover') invariant(ref(e.taskId) && isDigest(e.baselineId), 'invalid recovery');
else if (e.kind === 'merge') invariant(isGitId(e.tree) && isDigest(e.policyDigest), 'invalid merge exposure');
else throw new Error('unknown ledger event kind');
return e;
}

/** Hash chain detects corruption/rollback only against an independently protected anchor. */
export function decodeLedger(text, expectedAnchor, { maxBytes = 8 * 1024 * 1024 } = {}) {
invariant(isDigest(expectedAnchor) && typeof text === 'string' && Buffer.byteLength(text) <= maxBytes, 'trusted anchor/bounded ledger required');
invariant(text === '' || text.endsWith('\n'), 'partial ledger write');
const records = [], lines = text === '' ? [] : text.slice(0, -1).split('\n');
let anchor = GENESIS, previousAt = 0;
for (const line of lines) {
const r = JSON.parse(line);
invariant(r && Object.keys(r).sort().join(',') === 'digest,event,previous,sequence', 'invalid ledger record');
validateEvent(r.event);
invariant(r.sequence === records.length + 1 && r.previous === anchor && r.event.at >= previousAt, 'ledger order/binding mismatch');
invariant(r.digest === hash({ domain: 'ci.ledger.record.v1', sequence: r.sequence, previous: r.previous, event: r.event }), 'ledger corruption');
records.push(r); anchor = r.digest; previousAt = r.event.at;
}
invariant(anchor === expectedAnchor, 'ledger rollback or stale anchor');
return { records, anchor };
}

/** Single-host reference store. Directory and external anchor are controller-owned, never PR writable. */
export function appendLedger(directory, expectedAnchor, event) {
validateEvent(event); invariant(isDigest(expectedAnchor), 'trusted expected anchor required');
const dir = resolve(directory); mkdirSync(dir, { recursive: true, mode: 0o700 });
invariant(lstatSync(dir).isDirectory() && !lstatSync(dir).isSymbolicLink(), 'ledger directory must be ordinary trusted storage');
const lock = join(dir, 'writer.lock'), path = join(dir, 'events.jsonl');
let lockFd;
try { lockFd = openSync(lock, 'wx', 0o600); }
catch { throw new Error('ledger writer busy or storage unavailable'); }
try {
let text = '';
try {
const stat = lstatSync(path); invariant(stat.isFile() && !stat.isSymbolicLink() && stat.size <= 8 * 1024 * 1024, 'invalid ledger file');
text = readFileSync(path, 'utf8');
} catch (error) { if (error.code !== 'ENOENT') throw error; }
const ledger = decodeLedger(text, expectedAnchor);
invariant(event.at >= (ledger.records.at(-1)?.event.at ?? 0), 'ledger clock moved backwards');
const unsigned = { sequence: ledger.records.length + 1, previous: expectedAnchor, event };
const record = { ...unsigned, digest: hash({ domain: 'ci.ledger.record.v1', ...unsigned }) };
// Reject semantically invalid recovery before it can poison the durable chain.
ledgerState({ records: [...ledger.records, record], anchor: record.digest });
const next = canonical(record) + '\n';
invariant(Buffer.byteLength(text) + Buffer.byteLength(next) <= 8 * 1024 * 1024, 'ledger capacity reached');
const fd = openSync(path, 'a', 0o600);
try { writeFileSync(fd, next); fsyncSync(fd); } finally { closeSync(fd); }
return record; // caller must publish digest through a protected compare-and-swap anchor store
} finally { closeSync(lockFd); rmSync(lock); }
}

/** Reducer does not erase historical revocations. Recovery requires a newer complete covering baseline. */
export function ledgerState(ledger) {
const revoked = new Set(), disabled = new Map(), fulls = new Map(); let baseline = null, mergesSinceFull = 0;
for (const { event: e } of ledger.records) {
if (e.kind === 'revoke') revoked.add(e.inputKey);
else if (e.kind === 'trip') disabled.set(e.taskId, e.at);
else if (e.kind === 'full') { invariant(!fulls.has(e.baselineId), 'duplicate full baseline identity'); fulls.set(e.baselineId, e); baseline = e; mergesSinceFull = 0; }
Comment thread
Chris0Jeky marked this conversation as resolved.
else if (e.kind === 'merge') mergesSinceFull++;
Comment thread
Chris0Jeky marked this conversation as resolved.
else if (e.kind === 'recover') {
const full = fulls.get(e.baselineId), trip = disabled.get(e.taskId);
invariant(trip !== undefined && full && full.at > trip && full.at <= e.at && full.covers.includes(e.taskId), 'recovery lacks a newer covering full baseline');
Comment thread
Chris0Jeky marked this conversation as resolved.
disabled.delete(e.taskId);
}
}
return { anchor: ledger.anchor, revokedInputKeys: [...revoked].sort(), disabledTasks: [...disabled.keys()].sort(), baseline, mergesSinceFull };
}

/** Freshness plus merge exposure. Missing/partial/untrusted state always forces full qualification. */
export function qualificationRequired({ state, anchorVerified, now, policyDigest, universe, maxAgeSeconds, maxMerges, mandatory = false }) {
try {
invariant(!mandatory && anchorVerified === true && state && isDigest(state.anchor), 'mandatory full or unverifiable state');
invariant(Number.isSafeInteger(now) && Number.isSafeInteger(maxAgeSeconds) && maxAgeSeconds > 0 && Number.isSafeInteger(maxMerges) && maxMerges > 0, 'invalid qualification budget');
invariant(Array.isArray(universe) && universe.length > 0 && new Set(universe).size === universe.length, 'complete qualification universe required');
const b = state.baseline;
invariant(b?.complete === true && b.conclusion === 'success' && b.policyDigest === policyDigest && universe.every(t => b.covers.includes(t)), 'missing full baseline coverage');
invariant(b.at <= now && now - b.at < maxAgeSeconds && Number.isSafeInteger(state.mergesSinceFull) && state.mergesSinceFull >= 0 && state.mergesSinceFull < maxMerges, 'qualification age/exposure exhausted');
invariant(state.disabledTasks.length === 0, 'audit circuit is open');
return { required: false, reason: 'verified baseline within age and exposure limits' };
} catch (error) { return { required: true, reason: error.message }; }
}
Loading
Loading