[echo-fleet-builder]

Scope

Integrates the two independently reviewed open fixes into a draft, exact-main candidate:

• subscriber: do not mask a failed BEGIN with 'cannot rollback' (fleet #46912) #40: preserve the original SQLite BEGIN failure; roll back only an active transaction.
• fix(subscriber): terminalize runs whose dispatch gives up before execution starts (#49604) #41: terminalize dispatches that give up before execution, release reservations, and compensate the metered run exactly once.

No unrelated files are changed. The branch was based on current main 4f7d99e2dd4422f3d81cf5b709581be963f5818a. Commit tree 38542565dc2725132d44571ca4883afaa958ada0 matches the independent git write-tree from a clean clone of this exact canonical repository.

Verified local acceptance (2026-11-10)

• In an isolated clone of echoomegaprime/echo-certification-forge, both original patches applied cleanly in order.
• subscriber.py SHA-256 c8f088b2888c171e80b3ebdc6370752f35113fde288b2cfc894178c1bfe282a1.
• Patch-specific tests (test_subscriber_migration_rollback.py + test_dispatch_orphaned_runs.py): 7/7 passed.
• Subscriber governance (test_p7_subscriber_governance.py): 60/60 passed, four bounded groups of 15, with explicit exit-zero receipts.
• Combined candidate: 67 focused + governance tests passed. The previous complete-suite remote command lost its result, so the verified four-batch tests are the evidence of record.
• Exact Git blobs were reread and verified before creating the Git tree and this commit; no FORGE running service or existing branch was modified.

Required before merge or production promotion

• Exact-head hosted CI, security and static analysis on this combined commit.
• Independent review of run state transitions, compensation idempotency, migrations, and existing operational changes.
• Valid backup and rollback coverage for the subscriber database and dispatcher.
• Verified production end-to-end attestation for the exact immutable candidate; signed CertForge PRODUCTION_READY verdict.
• Authorized deployment with current-state readback and no forced merge.

Current CertForge signed verdict checks remain NOT_READY with production_e2e_attestation_missing. Do not treat local tests as authorization to bypass them.

This is a consolidation candidate, not an automatic replacement or merge of #40 or #41.

[echoomegaprime]

Exact-head CI readback — November 10, 2026

Draft PR #49 at commit 0ab617ab6ed97e1a40e5cb3d160215dc93dc86d5, from the canonical echoomegaprime/echo-certification-forge repository.

• GitHub Actions certification-forge-ci run #42933895397: completed / success.
• Latest-attempt job test (#130239055849): completed / success.
• Independently inspected job steps: checkout, Python setup, dependency installation, compilation, full deterministic test suite, P1 verification, P6 deployment enforcement acceptance, P7 subscriber governance acceptance, and evidence upload all completed successfully.
• Evidence artifact certification-forge-evidence-986b5019ef56cc46f5de9dfdbd68e47c747e6546 ID 13818176267, 4,739,539 bytes, not expired at inspection.
• Separate FORGE canonical-target isolated test proof: 7 focused and 60 subscriber-governance tests passed, with exact subscriber SHA-256 c8f088b2888c171e80b3ebdc6370752f35113fde288b2cfc894178c1bfe282a1 and Git tree 38542565dc2725132d44571ca4883afaa958ada0.

Release boundary: This is green hosted CI, not a signed CertForge PRODUCTION_READY release. The last inspected CertForge verdict was validly signed NOT_READY, with production_e2e_attestation_missing. Remain draft/unmerged until independent review, rollback proof, exact-target production-e2e attestation, and authorized release admission are satisfied. Do not force merge or silently replace the deployed subscriber.

[echoomegaprime]

Independent read-only PR #49 review — dispatch-recovery acceptance gaps (2026-11-11 01:02 UTC)

Reviewed exact commit 0ab617ab6ed97e1a40e5cb3d160215dc93dc86d5, specifically the subscriber.py patch and added focused tests. Hosted CI 42933895397 is green. This is additional review guidance, not a claim of a found exploitable defect or production approval.

1. Orphan sweep bound: _recover_expired_dispatches queries all FAILED dispatches joined to BOUND reservations and QUEUED runs, without an explicit page/limit. Establish a realistic orphan-volume performance bound and transaction-time behavior before production; consider bounded batches if necessary.
2. Crash/retry concurrency: Existing new tests prove sequential one-time reconciliation, but do not demonstrate two competing recovery workers or a retry interleaving. Verify transaction lock/claim semantics, run-state transition atomicity, reservation release and exactly-once compensating meter under concurrent recovery attempts.
3. Audit data minimization: certification.dispatch_orphan_reconciled stores last_error verbatim in structured audit details. Verify the upstream error field is constrained/redacted and cannot contain secrets, tokens or unbounded attacker-controlled content.
4. Negative-path contract: Keep tests for already-terminal runs, failed BEGIN IMMEDIATE, and old orphan state; add cases for missing usage-event rows and compensation failures with rollback if not already covered by the broader suite.

No source change, merge, database operation or protected deploy attempted. Required release gates still include backup/restore readback and signed exact-head production E2E attestation.

[echoomegaprime]

Supplemental concurrency evidence — draft test PR #53 (nonproduction)

Created PR #53 as a test-only companion to this combined PR. It targets the existing PR #49 feature branch (not main) and changes one new test file only, exact commit 3dcd40bd1db8694b5843e034c0333f1d1e9b0924.

The test launches two independent recovery workers behind a synchronization barrier against the same orphaned dispatch. It asserts one INFRASTRUCTURE_FAILURE terminal run, released reservation, zero outstanding metered run, and exactly one compensating usage row. FORGE isolated test: 1/1 passed in 11.69s. GitHub exact-blob readback verified, and the exact committed file passed independently in 2.45s. No production DB or worker touched; temporary test files removed.

Hosted CI run 42945718482 was in progress when last checked. This is a single two-worker scenario, not an exhaustive contention/rollback stress test. Keep PR49's exact-head release gates and signed production E2E attestation.

[echoomegaprime]

Independent combined-source concurrency acceptance — 2026-11-11 (nonproduction)

Refreshed release identities before this check:

• Draft PR fix(subscriber): combine transaction rollback and exhausted dispatch recovery (#40 #41) #49 still at exact head 0ab617ab6ed97e1a40e5cb3d160215dc93dc86d5 with completed successful certification-forge-ci; no independent approving review observed. Draft test companion PR test(subscriber): verify concurrent orphan reconciliation compensates once #53 still at 3dcd40bd1db8694b5843e034c0333f1d1e9b0924, CI successful. Draft governance PR docs: governed two-phase admission proposal for Swarm Meta OAuth (review required) #52 still at d195f84d781be417b7c8839c8e2d492447198241, CI successful, no approval.
• FORGE live services echo-certforge.service and echo-certforge-dispatcher.service both active, deployed immutable release marker d8ad9fd881365801354b78a758632d2a31ab6828; this is not PR fix(subscriber): combine transaction rollback and exhausted dispatch recovery (#40 #41) #49 or test(subscriber): verify concurrent orphan reconciliation compensates once #53.
• Fresh read-only echo.certforge.telemetry: 7 QUEUED, 7 active, 2502 terminal; 2 enrolled authenticated runners but 0 fresh heartbeats, effective enrolled capacity 0, last signed heartbeat 2026-07-26. This remains the real operational gate.

New isolated test, executed on FORGE: fetched tests/test_dispatch_orphan_concurrency.py directly from immutable PR #53 head; verified Git blob SHA 13fa345e78f6737ae880bd56260971970e5e1448. Ran it as a temporary test against the existing isolated PR #40 + #41 combined checkout (matched combined subscriber.py SHA-256 c8f088b2888c171e80b3ebdc6370752f35113fde288b2cfc894178c1bfe282a1). python3 -B -m pytest -q -p no:cacheprovider -o addopts= <temporary isolated test> 1 passed in 1.73s, exit 0. The temporary test was removed, and source SHA-256 was identical before/after. The test verifies two concurrent orphan sweepers yield exactly one meter compensation. It does not certify any active production database or live runner.

Unmet gates: review/approve PR #49 plus test companion, reconcile deployed-vs-main source ancestry, confirm real runner re-enrollment/heartbeat and queue reconciliation after a separately authorized release, and resolve the policy-owner two-phase admission circularity (#51 / PR #52). No database edits, artificial heartbeats, merges, production service changes or forged production attestations were performed; no PRODUCTION_READY claim. The Swarm PR #16 release worker remains separately owned.

[echoomegaprime]

Independent release review: global orphan sweep precedes tenant-specific dispatch selection — 2026-11-11 18:24 UTC

Source identities refreshed: combined draft PR #49 0ab617ab6ed97e1a40e5cb3d160215dc93dc86d5 (OPEN/DRAFT; hosted certification-forge-ci completed/success, no submitted independent approval observed); source PR #41 3adb7e0c4931a61254a6786cf40bc53416f29aa7; companion concurrency test PR #53 3dcd40bd1db8694b5843e034c0333f1d1e9b0924 (hosted CI successful); policy proposal PR #52 d195f84d781be417b7c8839c8e2d492447198241. A separate CodeQL success was not observed for these CertForge heads.

Fresh actual deployment: FORGE echo-certforge.service and echo-certforge-dispatcher.service active, resolved release directory /home/forge/echo-certification-forge-releases/d8ad9fd881365801354b78a758632d2a31ab6828-20260926T025556Z-2621724 (not a Git worktree). echo.certforge.telemetry(command=summary) at 18:23:37 UTC: 7 QUEUED/7 active, 2502 terminal; two enrolled Ed25519 runners, zero fresh authenticated heartbeats, signed capacity zero. Subscription quota slots remaining (93) do not indicate healthy enrolled execution capacity.

New specific review finding (static call-path; not a proven defect): In the deployed subscriber.py, claim_dispatch acquires its immediate DB transaction and calls _recover_expired_dispatches(connection, now_value) before filtering run_id or organization_id (deployed lines 4122–4132). PR #49 extends that recovery function with a FAILED dispatch + BOUND reservation + QUEUED run orphan query that has no tenant predicate and no batch bound, then terminalizes and compensates every matching row. Therefore, after deployment, even a dispatcher call scoped to one tenant can reconcile historical orphans for other tenants within the same transaction; a successful pending dispatch claim is not required to trigger the sweep. This may be an intentional privileged, platform-wide maintenance action, but its scope and transaction cost deserve explicit approval.

Suggested pre-release acceptance: in an isolated multi-tenant fixture, create an orphan for tenant B and invoke claim_dispatch for tenant A; assert B is terminalized only with its own audited state event, reservation release and one compensating usage row; assert A's metering and tenant boundaries remain unchanged. Add a bounded large-backlog/lock-contention timing case or explicitly accept the global unbounded sweep with measured latency. Preserve the existing exactly-once concurrency test. These are review/test requests, not an authorization to edit live SQLite rows, restart production, merge PRs or relax the signed PRODUCTION_READY gate.

[echoomegaprime]

Independent exact-tree multi-tenant regression — 2026-11-11 19:29 UTC (FORGE, isolated)

Immutable target: CertForge combined draft PR #49 head 0ab617ab6ed97e1a40e5cb3d160215dc93dc86d5. Independently fetched GitHub commit's tree 38542565dc2725132d44571ca4883afaa958ada0, then reconstructed the 457-file Git index tree from an existing isolated FORGE checkout, without writing to Git. Reconstructed SHA was identical, and git diff --quiet confirmed no unstaged source changes. Combined subscriber.py SHA-256: c8f088b2888c171e80b3ebdc6370752f35113fde288b2cfc894178c1bfe282a1.

New ephemeral pytest: 1 passed in 2.27s using FORGE system Python against the verified combined source. Test SHA-256 d2aec50ff331ccfb3e35d9512bd6fa30ce238e2e79c89da180483dfa02579f97. Three separate organizations, A/B/C, had independently metered queued runs. Tenant B's dispatch was deliberately set to historical FAILED with its run QUEUED and reservation BOUND in the temporary test-only database. A subsequent claim_dispatch(..., organization_id=A, run_id=A_run):

• Claimed only A; A and C remained QUEUED, metered at 1 each, with active reservations.
• Also reconciled B's unrelated orphan to INFRASTRUCTURE_FAILURE / INFRA_FAILED before A's organization filter (confirmed behavior, not a hypothetical).
• Released B's reservation, reduced only B's metered certification runs from 1 to 0, and wrote exactly one certification.dispatch_orphan_reconciled audit event under B's organization, actor certforge.dispatch-recovery.
• A repeat A-scoped call did not compensate B twice.
• Temporary sandbox was deleted; existing checkout, GitHub refs, production database, services and signed evidence were untouched.

Release review decision required: Is privileged dispatcher-wide orphan reconciliation intentional? If yes, document its maintenance authority and add this three-tenant ownership regression to the checked-in suite, plus bounded batching/transaction-duration or concurrent-load coverage. If no, constrain the orphan sweep to an explicitly authorized tenant/maintenance context while preserving exactly-once compensation. The tested behavior does not show cross-tenant audit misattribution, but does demonstrate global side effects from a tenant-scoped claim. This regression is separate from the already green PR #53 same-run concurrency test.

Gates unchanged: PR #49 still draft with CI success and zero submitted GitHub reviews; no separate CodeQL workflow was returned for this head. Deployed CertForge remains d8ad9fd881365801354b78a758632d2a31ab6828. No independent signed production-E2E attestation, approved release policy, or PRODUCTION_READY; no deployment or merge is requested here.

[echoomegaprime]

Independent combined-branch concurrency and tenant-isolation regression — 2026-11-11 20:29 UTC (FORGE)

Exact immutable inputs: CertForge PR #49 head 0ab617ab6ed97e1a40e5cb3d160215dc93dc86d5; PR #53 is based directly on #49 and has head 3dcd40bd1db8694b5843e034c0333f1d1e9b0924. Reacquired pull/53/head from GitHub into a new disposable FORGE checkout and verified Git tree 8cb9940eabc35faccec99dd62d1b9cd5ceeb6e82. All changes below were confined to the disposable checkout.

Executed, exit 0: python3 -B -m pytest -q -p no:cacheprovider tests/test_subscriber_migration_rollback.py tests/test_dispatch_orphaned_runs.py tests/test_dispatch_orphan_concurrency.py tests/test_independent_multitenant_concurrency.py → 9 passed in 5.54s, 2026-11-11 20:29:46 UTC.

The extra test (ephemeral source SHA-256 5840af4d36f925a5541014b1b65929a90780e01d328ede65bf8205eb57fbbf7d) independently exercised a three-tenant/two-worker race beyond the existing single-tenant #53 concurrency test:

• Tenant B had an old-style exhausted FAILED dispatch with its run still QUEUED and a BOUND, metered reservation.
• Tenant C had a legitimate separate QUEUED, BOUND, metered run that must remain pending.
• Two simultaneous workers scoped to tenant A called claim_dispatch. They independently triggered the existing global orphan sweep; neither claimed C.
• B transitioned once to INFRASTRUCTURE_FAILURE, with exactly one negative meter event and a RELEASED reservation; C remained QUEUED/BOUND with zero compensation events and its original meter; a third sweep did not compensate B again.

The temporary test file was removed and git status --porcelain was empty. The installed production CertForge virtualenv lacks pytest; the test executed using FORGE's existing python3 with pytest 9.0.3, without installing packages. An earlier attempt with the production virtualenv did not execute tests and is excluded from the pass count.

Limits: This is an isolated SQLite functional regression, not sustained contention/load proof, not a policy approval of global cross-tenant sweeps, and not hosted CI for the ephemeral test. Recommend checking the multi-tenant concurrent regression into the reviewed candidate suite (after maintainer approval), then obtaining hosted exact-head CI/security review. The deployed package remains a different source line (d8ad9fd881365801354b78a758632d2a31ab6828), while the Swarm candidate's signed production E2E and PRODUCTION_READY remain unavailable. No production restart, queue mutation, cert/attestation issuance, or gate bypass performed.

[echoomegaprime]

Independent 4/8-worker orphan-recovery stress probe — 2026-11-11 21:30 UTC (FORGE)

Immutable source verification: exact live GitHub heads before test: PR #49 0ab617ab6ed97e1a40e5cb3d160215dc93dc86d5 and dependent PR #53 3dcd40bd1db8694b5843e034c0333f1d1e9b0924 (PR #53 is based on #49). A fresh git ls-remote and disposable shallow Git checkout acquired refs/pull/53/head and verified the exact 40-character commit SHA. No existing checkout, production source or service was modified.

New bounded independent regression: in a temporary untracked pytest file, parameterized 4 and 8 concurrent dispatchers, each starting together and calling tenant-scoped claim_dispatch for the same historically orphaned FAILED dispatch + QUEUED run + BOUND reservation test-only SQLite fixture. The test asserted no erroneous claims; exactly one terminal transition to INFRASTRUCTURE_FAILURE; released reservation; certification-run meter returned to zero; exactly one negative dispatch-crash usage compensation event.

Command (in isolated exact-PR #53 checkout): python3 -B -m pytest -q -p no:cacheprovider tests/test_gate_parallel_8worker_probe.py
Result: 2 passed in 2.28s, exit code 0; complete checkout+test ~4.91s. No tracked changes. The disposable directory was removed by a context-managed cleanup. No production database, API worker or release key was touched.

Scope: this confirms the tested 4/8-thread races, not an unbounded concurrency guarantee. The broader global orphan sweep still runs inside claim_dispatch's SQLite immediate write transaction before tenant-specific filtering and has no explicit batch limit. It should receive a documented global-maintenance policy decision and a larger orphan-backlog/lock-duration test before release; do not infer production-scale lock safety from this short test.

Live status at 21:29 UTC: deployed FORGE API+dispatcher still point to d8ad9fd881365801354b78a758632d2a31ab6828. Authenticated CertForge SDK telemetry (read-only echo.certforge.telemetry) reported 7 QUEUED, 2 enrolled, 0 fresh authenticated Ed25519 runners and 0 signed execution capacity. Existing hosted CI for PRs #49/#52/#53 success, PR #49 ECHO Release Sentinel neutral, no separate CodeQL success observed; GitHub submitted review list empty. PR #52 remains a draft proposal, not an approved production canary exception.

Gate conclusion: no independent approval, deployed queue fix, signed runner recovery, exact-SHA PRODUCTION_READY or authorized reversible production window. Keep Swarm PR #16 production promotion blocked. No merge, deployment, cancellation, fabricated evidence, bypass, service restart or production signer action was attempted.

[echoomegaprime]

Independent production-ancestry reconciliation — 2026-11-11 22:23–22:27 UTC (NO PRODUCTION MUTATION)

Fresh exact source: this PR #49 0ab617ab6ed97e1a40e5cb3d160215dc93dc86d5; companion test PR #53 3dcd40bd1db8694b5843e034c0333f1d1e9b0924; GitHub main 4f7d99e2dd4422f3d81cf5b709581be963f5818a; live FORGE release-marker readback d8ad9fd881365801354b78a758632d2a31ab6828. Both CertForge services are running; CI success on exact PR heads, no independent approvals; ECHO Release Sentinel is neutral, with no separate CodeQL check reported.

New concrete release conflict: In an independent disposable canonical Git clone on FORGE, GitHub main and deployed history differ by 1/16 commits. Three exact git merge-tree --write-tree comparisons—deployed vs main, vs #49, and vs #53—each returned exit 1 with content conflict in deploy/deploy_forge.sh. File identities: deployed blob 7301e49cb3a3456b9409645aeecd4b7f3e70cbc3, main/#49 blob a266eba6f6c77d89bd8ce017deed7f65f5a52ba7, common-ancestor blob c28029783b2e613534b2063a9d471f0e708c4d06. The deployed script carries production-E2E trust/attestation path wiring, a longer Git-acquisition timeout and a different mandatory-manifest canonical digest; main changes readiness wait to 90 seconds. A naïve merge or choosing either whole-file side would risk dropping release safeguards or the newer cold-start fix. Require a separately reviewed script reconciliation.

Narrower integration path independently passed: In a second disposable clone, checkout exact deployed commit d8ad9fd8...; apply git cherry-pick --no-commit for #49 followed by #53. Both commands exit 0 without conflicts. The resulting staged changes touch ONLY src/echo_certification_forge/subscriber.py and tests/test_dispatch_orphaned_runs.py, tests/test_subscriber_migration_rollback.py, tests/test_dispatch_orphan_concurrency.py; git diff --cached --check and bash -n deploy/deploy_forge.sh both exit 0. Isolated /usr/bin/python3 -B -m pytest -q -p no:cacheprovider on those three tests, with checkout-local PYTHONPATH=src, yielded 8 passed in 5.40 s, exit 0. A subsequent broader test request failed at the remote execution boundary with no trustworthy result; do not count broad-suite pass/fail. Its abandoned scratch directories were checked for active processes and cleaned.

Safety/next gate: Treat this as a viable source-only integration approach, not a merge decision or production approval. Prepare an independently reviewed immutable deployed-baseline integration candidate; reconcile deploy_forge.sh (retaining production-E2E custody, exact-SHA/manifest checks, and verified readiness/rollback); run complete hosted CI/security + broader subscriber and production-verdict tests; prove backup/rollback, signed production evidence and policy-owner authorization before any promotion. Do not clear seven historical runs manually or treat old b069ab08 recertification as evidence for Swarm #16. No production services, SQLite records, signing keys, canonical OAuth routing or protected branches were changed.

[echoomegaprime]

2026-11-11 — independent deployed-baseline integration regression (NO PRODUCTION MUTATION)

Immutable inputs and provenance

• Current FORGE deployment marker and symlink: d8ad9fd881365801354b78a758632d2a31ab6828 (packaged release, not a Git checkout). Direct SHA-256 comparisons of installed subscriber.py, production_e2e.py, and deploy/deploy_forge.sh against the canonical Git blob at that exact deployed commit were 3/3 identical.
• Scratch checkout /tmp/certforge-gated-review-20261111-1726 cloned from echoomegaprime/echo-certification-forge and checked out at that deployed commit. git cherry-pick --no-commit applied PR fix(subscriber): combine transaction rollback and exhausted dispatch recovery (#40 #41) #49 head 0ab617ab6ed97e1a40e5cb3d160215dc93dc86d5, then its test companion PR test(subscriber): verify concurrent orphan reconciliation compensates once #53 head 3dcd40bd1db8694b5843e034c0333f1d1e9b0924 without conflicts. No branch or production tree was changed.
• Staged changes are exactly four files: src/echo_certification_forge/subscriber.py, tests/test_dispatch_orphaned_runs.py, tests/test_subscriber_migration_rollback.py, tests/test_dispatch_orphan_concurrency.py. git diff --cached --check exit 0; bash -n deploy/deploy_forge.sh exit 0. Patched subscriber.py SHA-256 c8f088b2888c171e80b3ebdc6370752f35113fde288b2cfc894178c1bfe282a1, unchanged by tests.

New independently executed coverage on this deployed-baseline integration, not PR #49 on stale main (all /usr/bin/python3 -B -m pytest -q -p no:cacheprovider with scratch-local PYTHONPATH=src, no training/production data):

• Patch and concurrency tests: test_dispatch_orphaned_runs.py, test_subscriber_migration_rollback.py, test_dispatch_orphan_concurrency.py — 8/8 passed, exit 0, 5.81 s elapsed (5.19 s pytest).
• test_p7_subscriber_governance.py — 60/60 passed, in six separate consecutive bounded ten-test batches (10/10 each; exits 0); times 11.87, 13.69, 10.97, 14.23, 10.16, 20.14 s. Each batch freshly collected the same 60 node IDs.
• test_production_e2e.py + test_evidence_and_verdict.py — 26/26 passed, exit 0, 7.40 s elapsed (7.10 s pytest).
• Total: 94/94 targeted tests passed, no source hash drift. An initial unsafely long test invocation failed at the remote-tool boundary and created duplicate test processes, which were identified by their exact scratch cwd/command and terminated; no result from that invocation is counted.

Production boundary/release blockers

• Fresh authorized CertForge tenant-scoped telemetry at 2026-11-11 23:35:03 UTC: queued 7, terminal 2,502, enrolled authenticated runners 2, fresh 0, signed capacity 0; latest authentic signed heartbeat was 2026-07-26 14:05:14 UTC. CertForge API and dispatcher remained running.
• GitHub exact PR heads fix(subscriber): combine transaction rollback and exhausted dispatch recovery (#40 #41) #49/docs: governed two-phase admission proposal for Swarm Meta OAuth (review required) #52/test(subscriber): verify concurrent orphan reconciliation compensates once #53 unchanged; hosted certification-forge-ci successful at each; 0 GitHub PR reviews on all three; no separate CodeQL workflow reported by exact-SHA workflow-run listing.
• This is candidate source-level test evidence only. It does not resolve the documented deploy/deploy_forge.sh main-versus-production merge conflict, authorize a global orphan sweep, approve PR docs: governed two-phase admission proposal for Swarm Meta OAuth (review required) #52's circular release admission, restore signed runner capacity, provide a genuine 40/40 external provider run, or constitute HAMMER-signed production E2E or CertForge PRODUCTION_READY.

Next: independent maintainer reviews the deployed-baseline integration and its governance; create a reviewed immutable PR including safe deployment-script reconciliation; run full hosted CI/security/certification on that exact integrated head, validate backup/rollback and real signed runner restoration. Do not merge or promote based solely on these 94 tests.

Cleanup readback: After the tests, one orphaned loopback-only uvicorn process (127.0.0.1:35001) was found with cwd exactly equal to this scratch checkout. Its PID, argv, and cwd were verified before a targeted SIGTERM. Exit was confirmed and the unique scratch directory was removed; no canonical CertForge or Swarm process, port, tunnel, or release directory was touched. Its presence followed the earlier aborted remote-tool invocation and is not counted as a failed source test.

[echoomegaprime]

Independent CertForge release-gate evidence — 2026-11-12 00:34 UTC (FORGE)

New: Verified the entire currently stacked orphan-recovery path against the actual deployed CertForge baseline, without altering the live release, database or working checkout.

• Deployed source baseline: d8ad9fd881365801354b78a758632d2a31ab6828.
• Exact candidate chain (unchanged GitHub heads): fix(subscriber): combine transaction rollback and exhausted dispatch recovery (#40 #41) #49 0ab617ab6ed97e1a40e5cb3d160215dc93dc86d5 → test(subscriber): verify concurrent orphan reconciliation compensates once #53 3dcd40bd1db8694b5843e034c0333f1d1e9b0924 → test(subscriber): prove atomic rollback when dispatch compensation fails #54 6cb0c9b0dfca33357cf7f8e7c6caa6671a383b78 → fix(subscriber): redact untrusted orphan-dispatch errors in immutable audit #55 24277990381d376ab427e42851c62a03a3110a34 → test(subscriber): reconcile multi-tenant orphan batches exactly once #56 9056dcae36a110e23e6b9721cc3524a65312620d.
• On FORGE, downloaded the immutable deployed-baseline source archive into a disposable TemporaryDirectory. Fetched each GitHub PR's corresponding .diff and required git apply --check and actual application, in order. 5/5 patch validations and applications passed with no conflicts.
• In that combined deployed-baseline sandbox: PYTHONPATH=src python3 -m pytest -q tests/test_dispatch_orphaned_runs.py tests/test_subscriber_migration_rollback.py tests/test_dispatch_orphan_concurrency.py tests/test_dispatch_compensation_atomicity.py tests/test_dispatch_orphan_audit_redaction.py tests/test_dispatch_orphan_batch_recovery.py → 14 passed in 13.05s, exit 0. This newly covers the stacked atomic rollback, redaction and five-tenant replay fixture beyond previous fix(subscriber): combine transaction rollback and exhausted dispatch recovery (#40 #41) #49/test(subscriber): verify concurrent orphan reconciliation compensates once #53-only integration checks. The disposable source directory was automatically deleted.
• Exact published patch SHA-256, in PR order: fix(subscriber): combine transaction rollback and exhausted dispatch recovery (#40 #41) #49 6d0f5e2a65ef3e41a92a9fef8673bef283d62d159b6d27045b06a20fe6e4cffa, test(subscriber): verify concurrent orphan reconciliation compensates once #53 084693bd1fc39153c712a2d129ae6c3c7afc7fd844b9ce4b1f23d0081fd7f8f7, test(subscriber): prove atomic rollback when dispatch compensation fails #54 95cdd4897edaec99e696acca3d5d3214accad53f89eb9e3e1e7fd7213ff36354, fix(subscriber): redact untrusted orphan-dispatch errors in immutable audit #55 bf5aad82c28835419819f76a22a305bfbfe852c0af9d387329661b9904a77bdd, test(subscriber): reconcile multi-tenant orphan batches exactly once #56 262744b00015fc7d3fe534321f5effec41eb8f0af6e99820f13a1531e2742b51.
• Post-test deployed installed blob readback equals GitHub d8ad9fd8 for all three compared files: subscriber.py = 3c69fb4267cb4086661cd68bd68c32bd1226a07c; production_e2e.py = 490f180a102795ed909fd6a2274301b2ddb53709; deploy/deploy_forge.sh = 7301e49cb3a3456b9409645aeecd4b7f3e70cbc3. CertForge API and dispatcher both remained active.
• Hosted certification-forge-ci concluded success at exact heads fix(subscriber): combine transaction rollback and exhausted dispatch recovery (#40 #41) #49, docs: governed two-phase admission proposal for Swarm Meta OAuth (review required) #52, test(subscriber): verify concurrent orphan reconciliation compensates once #53, test(subscriber): prove atomic rollback when dispatch compensation fails #54, fix(subscriber): redact untrusted orphan-dispatch errors in immutable audit #55, test(subscriber): reconcile multi-tenant orphan batches exactly once #56. No standalone CodeQL workflow was verified for these commits. Independent reviews on fix(subscriber): combine transaction rollback and exhausted dispatch recovery (#40 #41) #49 and docs: governed two-phase admission proposal for Swarm Meta OAuth (review required) #52–test(subscriber): reconcile multi-tenant orphan batches exactly once #56: none. The live queue/runner telemetry counts were not independently refreshed by this test; do not infer signed worker recovery.
• The deployed baseline and GitHub main still diverge (1 ahead / 16 behind in a deployed→main comparison, conflict in deploy/deploy_forge.sh). Successful patch application does not resolve that branch/release ancestry conflict.

Release decision: HOLD / NOT_READY. Required next steps: independently review the full stacked candidate and intended cross-tenant sweep semantics; preserve + reconcile deployed deploy_forge.sh controls before assembling a new immutable release head; run full exact-head CI/CodeQL/certification with signed rollback proof; restore genuinely signed enrolled-runner heartbeat; obtain policy-owner approval for #52's two-phase admission without redefining PRODUCTION_READY; and require authentic full-40 evidence and independently signed HAMMER production E2E. Nothing here licenses Swarm #16 promotion, production queue edits, self-approval, signature substitution or an unreviewed deployment.

[echoomegaprime]

Independent exact-head source-review finding — 2026-11-12 01:25 UTC (no production mutation)

Reviewed identities: combined PR #49 0ab617ab6ed97e1a40e5cb3d160215dc93dc86d5; dependent audit-redaction PR #55 24277990381d376ab427e42851c62a03a3110a34; stacked #56 9056dcae36a110e23e6b9721cc3524a65312620d. Hosted certification-forge-ci succeeded on those exact PR-triggered heads; no submitted independent reviews or separately verified CodeQL result were found in the available checks.

Additional, distinct error-retention path remains after #55: In exact #56 source, SubscriberGovernance.finish_dispatch(..., error=...) persists the caller-supplied error verbatim using UPDATE subscriber_run_dispatches ... last_error = ?. dispatch_status() then selects and returns that last_error. PR #55 redacts the immutable orphan-reconciliation audit details only; it does not sanitize the mutable dispatch-row field or the status response. A worker/runtime exception containing credential-shaped text could therefore still be retained and read back from the dispatch status. This is a source-level data-minimization concern, not proof of cross-tenant exposure, a real credential leak, or a signature bypass.

Bounded acceptance request before promoting the combined recovery: Add a synthetic credential-shaped error regression covering (a) final and retryable finish_dispatch writes, (b) dispatch_status after each, (c) legacy orphan reconciliation + immutable audit, and (d) exactly-once compensation and unchanged ownership/state transitions. Persist/report an allowlisted machine reason or redacted diagnostic (including safe readback for older rows), keep detailed secrets out of immutable history, and retain non-sensitive operational correlation. Do not rewrite historical audit records or edit production SQLite by hand. Any fix should receive its own reviewed exact-SHA CI/security checks.

Production gate remains closed for separate independent approval, deployment ancestry, signed-runner freshness, reviewed two-phase admission, and authentic full-40/HAMMER production evidence. No reviewer approval, deployment or production data change was attempted in this review.

[echoomegaprime]

Independent CertForge review update — Nov 12, 2026 02:35 UTC

The audit-redaction follow-up #55 does not by itself prevent finish_dispatch() from storing untrusted worker exception text in subscriber_run_dispatches.last_error or dispatch_status() from returning legacy raw values. Opened draft stacked PR #57 on exact #56 head 9056dcae36a110e23e6b9721cc3524a65312620d to close those two paths, with a synthetic-credential regression. Published #57 head 86a11e0b2ad0a72c7b9ecb0dde008b83240ffbb0 independently passed 16/16 focused tests on a clean, disposable FORGE checkout (14.57 s, exit 0). Hosted CI is running; independent review/CodeQL still missing. #49 itself remains at exact head 0ab617ab6ed97e1a40e5cb3d160215dc93dc86d5, draft, unapproved.

Read-only deployed release marker and git hash-object checks matched GitHub blobs for subscriber.py, production_e2e.py, and deploy_forge.sh at deployed d8ad9fd881365801354b78a758632d2a31ab6828. Main/deployed ancestry still diverges (1 ahead/16 behind); no merge, policy exception, queue edit, or production restart authorized. Explicit release decision: HOLD until independent approval, exact-head CI/security, production-source reconciliation, runner restoration and signed attestation.

[echoomegaprime]

Independent release ancestry / deployment safety review — 2026-11-12 03:34 UTC.

Fresh immutable source reconciliation: deployed CertForge marker and all three sampled deployed blobs match commit d8ad9fd881365801354b78a758632d2a31ab6828 (Git blob IDs: subscriber 3c69fb4267cb4086661cd68bd68c32bd1226a07c, production_e2e 490f180a102795ed909fd6a2274301b2ddb53709, deploy_forge.sh 7301e49cb3a3456b9409645aeecd4b7f3e70cbc3). Both systemd units are active and use /home/forge/echo-certification-forge-current. GitHub compare deployed→main still says diverged, ahead 1 / behind 16, and changed file deploy/deploy_forge.sh.

Concrete integration conflict to preserve: GitHub main commit 4f7d99e2dd4422f3d81cf5b709581be963f5818a contains cold-boot readiness timeout CERTFORGE_READY_TIMEOUT_S default 90s and matching READY_POLLS; deployed script uses hard-coded seq 1 40 (20s at 0.5s) for readiness, rollback restore and production health. Conversely, deployed script contains production-E2E key/attestation path wiring, corrected canonical manifest digest, effective production DB path safeguards, git-acquisition timeout and non-cascading dispatcher Wants= behavior missing from main. A blind selection of either side loses tested controls.

Requested least-change resolution: create an independently reviewed integration commit based on the deployed source with the queue-recovery stack and security tests, explicitly carry forward the 90s configurable readiness/restore probe from main, retain deployed production-E2E trust and DB safeguards, and add targeted rollback/readiness tests. Run hosted exact-head CI/CodeQL on that immutable integration and independently approve before any release; do not merge this draft, self-approve, deploy or mutate historical queue records now.

PR #49 HEAD remains 0ab617ab6ed97e1a40e5cb3d160215dc93dc86d5; source PR #41 and stacked #53–57 also remain open with hosted CI success but no independent approvals. Source-level reconciliation plan only, no production change.

[echoomegaprime]

Independent CertForge gate readback — 2026-11-12 04:36 UTC (FORGE, read-only SQLite URI mode=ro + PRAGMA query_only; no mutations).

Deployment identity: running /home/forge/echo-certification-forge-current resolves to the versioned directory whose .certforge-release-sha is d8ad9fd881365801354b78a758632d2a31ab6828. API and dispatcher active.

Direct queue state-machine evidence, 7/7: all seven historical Omnipresence runs.state=QUEUED records join to subscriber_run_dispatches.state=FAILED, attempts=3, and subscriber_run_reservations.state=BOUND. Each has exactly one matched positive certification_runs usage event and net metering +1 (no compensating negative event yet). This is a live reproduced instance of the exhausted-before-start orphan condition addressed by PR #49, not merely a stale aggregate count. These are unrelated to Swarm; no rows were modified or cancelled.

At this instant GitHub #49 remains draft at exact head 0ab617ab6ed97e1a40e5cb3d160215dc93dc86d5, hosted CI success, zero independent reviews, no separate exact-head CodeQL result established. New stacked security follow-up #58 is draft, current head 76b85df4185addbba0f8b700750c38c019e083ec, hosted CI still pending. No deployment authorization. Regression/release acceptance must prove tenant ownership, immutable audit consistency, exactly-once negative usage compensation, and subsequent idempotent sweeps without impacting legitimate reservations.

Decision: HOLD. This readback establishes the recovery target precisely but does not approve a production merge, run manual SQLite fixes, or exercise a release window.

[echoomegaprime]

CertForge independent gate checkpoint, 2026-11-12 05:24 UTC: exact PR #49 head remains 0ab617ab6ed97e1a40e5cb3d160215dc93dc86d5, hosted CI run 42933895397 success, independent review submissions none. Stacked security fix PR #58 at 76b85df4185addbba0f8b700750c38c019e083ec has now also completed hosted CI 43155802051 success, but is draft/unapproved. Fresh FORGE release marker /home/forge/echo-certification-forge-current/.certforge-release-sha is d8ad9fd881365801354b78a758632d2a31ab6828: installed subscriber.py, production_e2e.py, deploy_forge.sh each matched its immutable GitHub blob SHA (3/3). GitHub comparison d8ad...main: diverged, main ahead 1 / behind 16, with deploy/deploy_forge.sh conflict to reconcile. Fresh production DB read-only checks show 7 QUEUED -> FAILED dispatch attempts=3, BOUND reservations=7, matching the orphan recovery predicate, plus 3 global ACTIVE enrollments and zero fresh signed heartbeats. Gate remains HOLD until independent review/CodeQL evidence, safe deployed-baseline ancestry resolution, approved worker/recovery procedures, genuine signed-runner restoration, release policy and exact-SHA certification. Do not self-approve, alter historical SQLite, deploy or restart production, or treat successful CI as PRODUCTION_READY.

[echoomegaprime]

Independent exact-head release-gate verification — 2026-11-12 06:23–06:27 UTC (Chat mode, read-only):

1. Immutable PR heads: fix(subscriber): terminalize runs whose dispatch gives up before execution starts (#49604) #41 3adb7e0c4931a61254a6786cf40bc53416f29aa7; fix(subscriber): combine transaction rollback and exhausted dispatch recovery (#40 #41) #49 0ab617ab6ed97e1a40e5cb3d160215dc93dc86d5; two-phase docs: governed two-phase admission proposal for Swarm Meta OAuth (review required) #52 d195f84d781be417b7c8839c8e2d492447198241; stacked test(subscriber): verify concurrent orphan reconciliation compensates once #53 3dcd40bd1db8694b5843e034c0333f1d1e9b0924, test(subscriber): prove atomic rollback when dispatch compensation fails #54 6cb0c9b0dfca33357cf7f8e7c6caa6671a383b78, fix(subscriber): redact untrusted orphan-dispatch errors in immutable audit #55 24277990381d376ab427e42851c62a03a3110a34, test(subscriber): reconcile multi-tenant orphan batches exactly once #56 9056dcae36a110e23e6b9721cc3524a65312620d, fix: redact retained subscriber dispatch worker errors (stacked on #56) #57 86a11e0b2ad0a72c7b9ecb0dde008b83240ffbb0, fix(subscriber): redact worker-failure reason in audit, state history and dispatch #58 76b85df4185addbba0f8b700750c38c019e083ec.
2. For all those exact heads, the associated certification-forge-ci workflow runs completed successfully, including fix(subscriber): redact worker-failure reason in audit, state history and dispatch #58 run 43155802051; the GitHub connector returned no independent submitted reviews on any of fix(subscriber): terminalize runs whose dispatch gives up before execution starts (#49604) #41/fix(subscriber): combine transaction rollback and exhausted dispatch recovery (#40 #41) #49/docs: governed two-phase admission proposal for Swarm Meta OAuth (review required) #52/test(subscriber): verify concurrent orphan reconciliation compensates once #53–58. The current retrieved checks had no separate CodeQL success; do not represent this as a passing security gate.
3. Authenticated live production DB state remains 7 QUEUED + FAILED-dispatch / 3 attempts + BOUND reservation, 3 ACTIVE globally enrolled runners but zero fresh signed heartbeats (most recent worker heartbeat 2026-07-26). Full readback posted to issue CertForge: recover stale tenant queue and signed runner health blocking Swarm recertification #50, comment 6816462868.
4. Running installed source files match deployed CertForge commit d8ad9fd881365801354b78a758632d2a31ab6828 (source Git blob hashes documented on CertForge: recover stale tenant queue and signed runner health blocking Swarm recertification #50). GitHub main SHA 4f7d99e2dd4422f3d81cf5b709581be963f5818a still diverges: one ahead / 16 behind deployed line, with deploy/deploy_forge.sh differing. Deployed-baseline recovery integration requires an independently reviewed release-history reconciliation; no in-place merge or production change.
5. docs: governed two-phase admission proposal for Swarm Meta OAuth (review required) #52 remains a draft policy proposal, not authorization to canary/roll forward or relabel staging evidence as PRODUCTION_READY.

Release decision: HOLD pending independent review, CodeQL/security evidence, reconciled immutable integration head, genuine signed runner heartbeat, formally approved reversible admission, and exact-target signed production evidence. This check did not launch 40 paid seats, alter queued rows, restart service, deploy, or approve/merge a protected PR.

[echoomegaprime]

New independent deployed-baseline integration test — 2026-11-12 06:31–06:33 UTC, FORGE. This supersedes the test-coverage portion (not the release HOLD) of my earlier status comment.

• Reacquired exact source IDs from GitHub into an isolated, disposable checkout: actual deployed CertForge baseline d8ad9fd881365801354b78a758632d2a31ab6828; GitHub default main 4f7d99e2dd4422f3d81cf5b709581be963f5818a; combined stacked recovery/security head PR fix(subscriber): redact worker-failure reason in audit, state history and dispatch #58 76b85df4185addbba0f8b700750c38c019e083ec. Verified the PR fix(subscriber): redact worker-failure reason in audit, state history and dispatch #58 remote ref matches that SHA.
• Generated the full nine-file diff from main to PR fix(subscriber): redact worker-failure reason in audit, state history and dispatch #58, SHA-256 588f99adef9747af2164ca3e12d16e8d161180ccff4905d89a2b6b440e4e8939, 37,872 bytes. git apply --check and git apply both succeeded against the deployed baseline without conflicts; git diff --check found no whitespace errors. This is isolated candidate integration, not a branch merge or production release.
• Ran all eight changed regression-test files (orphan recovery, rollback, concurrent/batch recovery, audit redaction, error retention and worker failure): 20 passed, 1 Starlette/httpx deprecation warning, 0 failed, 23.03 seconds under a bounded Python/pytest run from the disposable checkout, with PYTHONPATH rooted to the candidate source.
• Post-test readback: both echo-certforge.service and echo-certforge-dispatcher.service stayed active; production still contains 7 QUEUED + FAILED dispatch + BOUND reservation records. No recovery was applied live. Every temporary /tmp/certforge-{review,tests,full-gate}-* directory was deleted.
• Caveat: these are focused isolated regression tests; they do not constitute a full production E2E, CodeQL, independent protected-PR review, signed runner heartbeat, approved two-phase admission, authentic full_40, or HAMMER attestation. Deployed/main deploy_forge.sh ancestry still requires a separately reviewed reconciliation.

Release decision stays HOLD. No service restart, SQLite write, queue cancellation, production signing operation, or Swarm PR #16 change.

[echoomegaprime]

Independent CertForge release-gate exact-head check — 2026-11-12 ~07:31 UTC (read-only; no production mutation).

Immutable source / operational proof

• PR fix(subscriber): combine transaction rollback and exhausted dispatch recovery (#40 #41) #49 head remains 0ab617ab6ed97e1a40e5cb3d160215dc93dc86d5; source fix(subscriber): terminalize runs whose dispatch gives up before execution starts (#49604) #41 3adb7e0c4931a61254a6786cf40bc53416f29aa7; policy PR docs: governed two-phase admission proposal for Swarm Meta OAuth (review required) #52 d195f84d781be417b7c8839c8e2d492447198241; stacked security PR fix(subscriber): redact worker-failure reason in audit, state history and dispatch #58 76b85df4185addbba0f8b700750c38c019e083ec. All are open; no submitted independent reviews.
• FORGE deployed .certforge-release-sha = d8ad9fd881365801354b78a758632d2a31ab6828. Installed blob SHA-1 matches GitHub at that exact commit: subscriber.py 3c69fb4267cb4086661cd68bd68c32bd1226a07c; production_e2e.py 490f180a102795ed909fd6a2274301b2ddb53709; deploy_forge.sh 7301e49cb3a3456b9409645aeecd4b7f3e70cbc3.
• Read-only connection to the actual production SQLite /mnt/documentation/echo/certforge/certforge.sqlite3: 7 QUEUED runs, all 7 with FAILED dispatch, 3 attempts, BOUND reservation. Both CertForge service and dispatcher remain active. Enrollments: 3 ACTIVE globally, latest authenticated heartbeat 2026-07-26T14:05:14.510257Z, so no fresh enrolled signed-runner capacity. The separate 2 ACTIVE subscriber_private_workers records must not be equated with the 3 operational enrollments.
• Git compare d8ad9fd8...main remains diverged (1 main-side ahead / 16 deployed-side behind), with conflict scope deploy/deploy_forge.sh; a reviewed deployed-baseline integration and full exact-SHA release tests are still necessary.

Fresh check-run evidence, not just workflow summary: GitHub Actions test succeeded on exact #49 head (run 42933895397); however GitHub API reports only two exact-head check runs, test=success and ECHO Release Sentinel=neutral. Sentinel explicitly states: "Required authentic checks not present: CI, CodeQL". The same pattern occurs on #41, #52 and #58. Default .github/workflows at main contains ci.yml only. This is not an established CodeQL pass and does not satisfy the protected release gate. In contrast, Swarm PR #16's separate exact head has real successful CodeQL checks; do not conflate repositories.

Decision: HOLD. No production queue/release edits, attempted cancellation, synthetic runner heartbeat, key access, 40-seat paid execution, merge or service restart. Required: independent reviewer; authentic CodeQL/release-check reconciliation; production-ancestry resolution; genuinely signed runner restoration; explicit #52 policy authorization; independently verified 40-live-seat exact-SHA production attestation. A separate new gate-coverage finding concerning generic-profile 2/40 acceptance at unsigned payload-validation layer only is recorded on issue #51 / PR #52; it does not demonstrate a signature bypass.

[echoomegaprime]

Independent CertForge release-gate update — 2026-11-12 08:33 UTC (source-only; NO production action).

NEW reviewed-ancestry candidate: Draft PR #60 #60 at immutable merge head 12d6fe72724039f149645be3b8cdadc668b1610a. Its two real parents are GitHub main 4f7d99e2dd4422f3d81cf5b709581be963f5818a and the currently deployed CertForge d8ad9fd881365801354b78a758632d2a31ab6828; merge tree 61eae085b9f3e1cbe00f000e22236e35b18b1654. It resolves the deploy/deploy_forge.sh conflict by retaining the deployed production E2E trust roots, canonical manifest digest, effective production DB, credential helper, bounded source-acquisition, and dispatcher resilience, plus main's configurable 90-second health/rollback readiness and staging failure logs. Bounded independent FORGE bash -n passed; merged script SHA-256 a7ee67610da822628e4904b8ceb2272d59083ca46c48c9d382f4267e6e091743.

Scope caution: GitHub main→merge compare spans 16 deployed-side commits / >60 changed files, not merely the deployment script; maintainer/security/release owner must review all changes. PR #60 is DRAFT, not approved, merged, certified or deployed. Exact-head hosted CI is running; at 08:33 UTC GitHub showed test in progress, ECHO Release Sentinel neutral, and no CodeQL check.

Queue recovery remains separate: #49 exact head 0ab617ab6ed97e1a40e5cb3d160215dc93dc86d5 and stacked #58 76b85df4185addbba0f8b700750c38c019e083ec are still draft, CI passed but no independent submitted approvals or separate exact-head CodeQL. Actual FORGE production DB remains 7 QUEUED / dispatch FAILED / reservation BOUND; 3 active enrolled runner identities globally, zero fresh authenticated heartbeats (latest 2026-07-26). No SQLite writes/cancellations, production restart or release attempted. Release gate HOLD until independent review, CodeQL, verified integration, runner authenticity, two-phase authorization and exact-SHA HAMMER full-40 attestation.

[echoomegaprime]

Independent CertForge gate update (2026-11-12 11:25 UTC); no merge/deployment authorization.

Exact head 0ab617ab6ed97e1a40e5cb3d160215dc93dc86d5 unchanged; GitHub workflow 42933895397 = SUCCESS, Release Sentinel = NEUTRAL; exact-head CodeQL check absent; reviews = 0.

Read-only FORGE production DB /mnt/documentation/echo/certforge/certforge.sqlite3: 7/7 QUEUED runs have FAILED dispatch (3 attempts) and BOUND reservation. All 7 show NULL execution_started_at, worker_id, worker_attestation_sha256, last_heartbeat_at, so the current live targets support the “unstarted” recovery classification. Separately, 8 historical CANCELLED/INCONCLUSIVE runs have PENDING dispatch (0 attempts), BOUND reservations, and zero execution starts. These 8 are not selected by PR #49's current FAILED/QUEUED/BOUND predicate and must not be reclassified or compensated by this fix.

Independent review requirement: the orphan SELECT in the PR tests only d.state='FAILED' AND r.state='BOUND' AND runs.state='QUEUED', without an explicit r.execution_started_at IS NULL or worker-ownership/evidence exclusion. Current production seven have no execution evidence, but add or document a negative fixture where a corrupted/inconsistent FAILED/QUEUED/BOUND record does carry execution/worker evidence: recovery must fail closed without erasing ownership or compensating a run that may have started. Verify baseline lease rules before choosing the precise guard. Follow-on PRs #55–58 also contain durable error-redaction protections; do not release only the earlier raw-error audit path without reviewing the full stack.

Deployed marker unchanged d8ad9fd881365801354b78a758632d2a31ab6828. No DB writes, forced compensation, service restart, signing or release action.

[echoomegaprime]

Independent CertForge release gate — 2026-11-12 12:42 UTC (read-only production + isolated FORGE test).

New draft follow-up #61, exact head 272bea3674f6cdecd4ca0423b30b745cbf755cab, stacked on #58 76b85df4185addbba0f8b700750c38c019e083ec: the historical QUEUED/FAILED/BOUND orphan query in #49 does not exclude contradictory execution/worker ownership evidence. #61 adds fail-closed predicates for reservation execution-start, worker/attestation/signing/lease/heartbeat fields and dispatch claim/owner fields, preserving state, bound reservations and metering for inconsistent records. 16/16 focused tests passed in a disposable FORGE checkout (35.96 s: 12 new negative cases + 4 existing); compiled Python and git diff --check passed. GitHub source/test blob identities were matched to the exact tested bytes before publishing.

The actual production DB still has seven historical QUEUED/FAILED/BOUND runs, each with three attempts; 0/7 have any of the 12 guarded execution/ownership fields, so guard does not exclude this cohort. Eight separate CANCELLED/PENDING/BOUND reservations with positive metering and no negative compensation remain outside #49 and require a separate accounting-policy review. No SQLite mutation.

#61 hosted test is running as of this note; neutral Sentinel, absent successful exact-head CodeQL and zero independent approvals mean no promotion or merge authorization. Integrate/review the safety patch, secure CI+CodeQL and independent approval, reconcile deployed ancestry via draft #60, and restore genuine signed runners before release. Existing #49 exact head remains 0ab617ab6ed97e1a40e5cb3d160215dc93dc86d5.

[echoomegaprime]

CertForge recovery coordination (2026-11-12): combined original PR #49 remains at exact 0ab617ab6ed97e1a40e5cb3d160215dc93dc86d5, CI successful but draft, no approvals/CodeQL. Its downstream hardening PR #61 now at b98d1cedd6532ac29fd7dfb33adb340a953975fc closes the previously raised consumed_at and final-attempt ownership guard gaps, with 23/23 isolated FORGE tests successful; hosted CI currently in progress. The seven original production orphans match the stricter legacy predicate without execution/consumption markers; the separate eight CANCELLED/PENDING/BOUND reservations are excluded pending independent accounting review. Treat PR #49 + reviewed hardening, CI/CodeQL, and separately reconciled deployed ancestry as prerequisites; do not promote the original PR #49 alone or alter production SQLite.

[echoomegaprime]

Release-gate coordination 2026-11-12: Combined recovery head remains 0ab617ab6ed97e1a40e5cb3d160215dc93dc86d5 (hosted test passed; no independent approval/CodeQL). The dependent draft PR #61 has now advanced to c95dd7d2a971ea26d827382a4f1d5ec96fa327d6 after an independently reproduced lease-expiry ownership defect. Its exact published tree matches the 30/30 passing FORGE regression checkout; new hosted CI is in progress. PR #49 is not itself updated and must not be promoted without reviewed integration/ancestry and the corrected dispatch safety, plus independent security/release gates. Details: #61 .

[echoomegaprime]

Independent CertForge gate readback (2026-11-12 ~17:30 UTC): #49 remains draft at exact head 0ab617ab6ed97e1a40e5cb3d160215dc93dc86d5 with successful hosted deterministic CI but no submitted independent review or successful exact-head CodeQL. Source #41 remains 3adb7e0c4931a61254a6786cf40bc53416f29aa7.

New source-only progress: Opened draft #62 (#62) at 9417dc5a06159291384d11f569a2ffdb5b15c108, based on ownership-guard #61 c95dd7d2a971ea26d827382a4f1d5ec96fa327d6, applying the exact independently hosted-tested CI+CodeQL workflow blob from #46. Only .github/workflows/ci.yml changes; existing deterministic, P1/P6/P7 tests are retained. New workflow #43270650624 is running, not yet certified. #62 does not retroactively provide CodeQL on #49, nor authorize #49 merge.

Live production remains d8ad9fd881365801354b78a758632d2a31ab6828. Read-only DB queries show 7 historical QUEUED/FAILED/BOUND dispatches (3 attempts each) and a separate 8 CANCELLED/PENDING/BOUND cohort. 3 active enrolled runner identities, last authentic heartbeat 2026-07-26 (0 fresh). Do not run raw SQLite recovery or arbitrary cancellations. Require independent approval, actual CodeQL per promoted exact SHA, safe reviewer disposition of the ownership-guard stack, reviewed #60 deployed ancestry and approved release plan before any production mutation.

[echoomegaprime]

Independent exact-head review update (2026-11-12 ~19:35 UTC): #49 remains 0ab617ab6ed97e1a40e5cb3d160215dc93dc86d5, hosted CI success but no independent approvals or exact-head CodeQL. The additional fail-closed ownership safety stack is draft #61 (c95dd7d2a971ea26d827382a4f1d5ec96fa327d6), exact-SHA CodeQL integration draft #62 (9417dc5a06159291384d11f569a2ffdb5b15c108), and claim-custody follow-up draft #63 (a0515d875c2a384965229eb5716331a776ab8190). #63 passed 32 focused isolated FORGE tests, CodeQL completed/success; hosted deterministic CI is still in progress. Production read-only SQLite still has seven QUEUED/FAILED/BOUND unrelated orphans with no selected execution/ownership markers, plus eight distinct CANCELLED/PENDING/BOUND reservations outside recovery scope. Do not merge or deploy on these test results alone. Require independent review, exact-head CI/CodeQL and secure release plan; preserve original run/custody evidence and account for rollback.

[echoomegaprime]

CI closure for dependent custody safety (2026-11-12 ~19:40 UTC): draft PR #63 head a0515d875c2a384965229eb5716331a776ab8190 now has both hosted CI SUCCESS and CodeQL SUCCESS at run #43291530983. Local focused tests also passed 32/32 and tested Git blobs matched GitHub readback. This does not certify #49's different exact head 0ab617ab6ed97e1a40e5cb3d160215dc93dc86d5 or authorize merging #49. #49 is still draft, review/CodeQL/production certification outstanding, with 7 unrelated queue orphans and zero fresh enrolled signed worker heartbeat. Preserve independent protected review and fail-closed release policy.

[echoomegaprime]

Independent CertForge gate evidence — 2026-11-12 (read-only production; disposable FORGE source tests)

• Verified exact heads: PR fix(subscriber): combine transaction rollback and exhausted dispatch recovery (#40 #41) #49 0ab617ab6ed97e1a40e5cb3d160215dc93dc86d5; ancestry reconciliation PR DRAFT: reconcile deployed CertForge ancestry with main; retain readiness + production safeguards #60 3648116c15d67616f0296a376696fc114b401708; reviewed recovery/CodeQL/custody stack PR fix(subscriber): preserve non-secret dispatcher claim custody during lease recovery #63 a0515d875c2a384965229eb5716331a776ab8190. PR fix(subscriber): terminalize runs whose dispatch gives up before execution starts (#49604) #41 3adb7e0c4931a61254a6786cf40bc53416f29aa7; PR docs: governed two-phase admission proposal for Swarm Meta OAuth (review required) #52 d195f84d781be417b7c8839c8e2d492447198241.
• New independent integration proof: On FORGE, fetched the exact immutable PR DRAFT: reconcile deployed CertForge ancestry with main; retain readiness + production safeguards #60 and fix(subscriber): preserve non-secret dispatcher claim custody during lease recovery #63 refs into an isolated disposable Git repository; git merge-tree --write-tree <PR60-SHA> <PR63-SHA> exited 0 (no conflicts), yielding tree 3745b34cb593134dccd99db46f34ff4892879a47. Made only a local, unpublished disposable two-parent test commit 323452e60eb96cc65760c206e0889b600517477c. Verified the resulting checkout contains deployed d8ad9fd881365801354b78a758632d2a31ab6828, main 4f7d99e2dd4422f3d81cf5b709581be963f5818a, PR fix(subscriber): combine transaction rollback and exhausted dispatch recovery (#40 #41) #49, PR DRAFT: reconcile deployed CertForge ancestry with main; retain readiness + production safeguards #60 and PR fix(subscriber): preserve non-secret dispatcher claim custody during lease recovery #63 as ancestors; git diff --check HEAD^1 HEAD clean. This temporary commit is not a release candidate or published PR.
• New exact merged-tree tests: python3 -m pytest -q tests/test_dispatch_orphaned_runs.py tests/test_subscriber_migration_rollback.py → 35 passed in 31.52s, exit 0; python3 -m pytest -q tests/test_production_e2e.py tests/test_evidence_and_verdict.py → 26 passed in 8.07s, exit 0. Total 61 passed on tree 3745b34.... These are isolated source tests, not canonical E2E or hosted CI/CodeQL on the merged tree. The separately published PR fix(subscriber): preserve non-secret dispatcher claim custody during lease recovery #63 SHA has successful hosted CI+CodeQL (workflow 43291530983).
• Production still unchanged: /home/forge/echo-certification-forge-current/.certforge-release-sha = d8ad9fd881365801354b78a758632d2a31ab6828. Read-only SQLite: 7 historical QUEUED/FAILED/BOUND orphans (0 tested ownership/consumption markers), 8 distinct CANCELLED/PENDING/BOUND reservations, 2 active target-tenant enrolled identities but 0 fresh signed heartbeats (newest 2026-07-26T14:05:14Z). No queue records were changed.
• Release gate: All inspected PRs fix(subscriber): terminalize runs whose dispatch gives up before execution starts (#49604) #41/fix(subscriber): combine transaction rollback and exhausted dispatch recovery (#40 #41) #49/docs: governed two-phase admission proposal for Swarm Meta OAuth (review required) #52/DRAFT: reconcile deployed CertForge ancestry with main; retain readiness + production safeguards #60/fix(subscriber): preserve execution ownership during historical orphan recovery #61/ci: exact-head CodeQL gate for reviewed PR61 recovery stack (draft) #62/fix(subscriber): preserve non-secret dispatcher claim custody during lease recovery #63 lack independent approvals; PR fix(subscriber): preserve non-secret dispatcher claim custody during lease recovery #63's CodeQL pass does not certify this disposable merged tree or older exact heads. The integration is source-mergeable and the focused tests pass; it is not admitted. An independently reviewed/published exact merged head needs full hosted CI, CodeQL, custody/regression review, formal two-phase policy authorization, and reversible window before any promotion. Do not merge/deploy on the strength of this comment; protect original run evidence and ownership. Do not hand-edit SQLite or queue states.

Disposition: HOLD / NOT_READY. No protected merge, production restart, mutation, simulated heartbeat, paid 40-seat journey, or HAMMER attestation was performed.

[echoomegaprime]

Independent recovery-stack coordination (2026-11-12 UTC): new draft integration PR #64, exact head ccf59365cb2987f2e2bc2da5e33f4fbd30eb985b, combines #60 deployed-ancestry reconciliation and #63 guarded recovery/custody stack (including #49/#41) as a two-parent commit. The resulting GitHub tree 3745b34cb593134dccd99db46f34ff4892879a47 exactly matches the earlier locally tested 61-case integration tree. Hosted exact-head CodeQL passed; full CI pending; no approvals. The original #49 head 0ab617ab6ed97e1a40e5cb3d160215dc93dc86d5 remains unchanged. Do not merge the source #49 alone or terminalize production jobs absent independently approved recovery release and evidence custody. Read-only FORGE evidence still finds 7 historical QUEUED/FAILED/BOUND orphans, 8 separate CANCELLED/PENDING/BOUND; no automated recovery has run.

Corrective exact-head update, 2026-11-12 UTC: Integration parent ccf59365cb2987f2e2bc2da5e33f4fbd30eb985b finished hosted CI successfully, but its separate GitHub Advanced Security CodeQL analysis check FAILED (HIGH) despite a passing GitHub Actions CodeQL job. The draft PR #64 head has advanced, by a bounded two-file HMAC custody follow-up, to e6b51c683c0027cab3ed446a26fdd614476a32c6 (tree dc7e69a253d586eee578d2560bf12eabb51ea63f); 2 focused tests passed on the exact changed blob bytes. At this update, new-head GitHub Actions CodeQL job passed, full CI was still in progress, and the separate GHAS CodeQL check remained FAILED, alert #1, py/weak-sensitive-data-hashing at the pre-existing API-key HMAC helper (subscriber.py:889). Generated API-key secrets are randomly created with a 24-byte minimum and server pepper is 32-byte minimum; independent security review must determine whether this alert is a false positive. Do not suppress, dismiss, or bypass it. No independent approvals or production release authorization. This note supersedes any impression above that all CodeQL gates had passed.

Final current-head check reconciliation (2026-11-12 UTC): CertForge draft PR #64 head e6b51c683c0027cab3ed446a26fdd614476a32c6: hosted CI SUCCESS (job 131508526846), GitHub Actions CodeQL job SUCCESS (131508527314), but independent GitHub Advanced Security CodeQL check FAILURE (131509086490, one HIGH alert py/weak-sensitive-data-hashing at subscriber.py:889). ECHO Release Sentinel reports SUCCESS (131511834665) on the same exact head despite the failed security-analysis check. Acceptance regression required: Actions CodeQL succeeded && GHAS CodeQL failed MUST block release, regardless of Sentinel status. The flagged pre-existing helper uses HMAC-SHA256 with a >=32-byte pepper and random >=24-byte API-key secret; potential false positive requires independent security-owner adjudication, not self-dismissal. CertForge signed staging result remains NOT_READY without production E2E; zero independent approvals. No promotion or Swarm deployment.

[echoomegaprime]

Independent CertForge release-gate read (2026-11-13 00:28 UTC; no production mutation): current PR #49 head is 0ab617ab6ed97e1a40e5cb3d160215dc93dc86d5, hosted CI success, zero review submissions, no exact-head GHAS CodeQL success (the check inventory contains only Actions test:success and neutral Sentinel). It is not production release-authorized.

Actual FORGE /home/forge/echo-certification-forge-current was independently dereferenced to release directory d8ad9fd881365801354b78a758632d2a31ab6828-20260926T025556Z-2621724; .certforge-release-sha content is d8ad9fd881365801354b78a758632d2a31ab6828. Both echo-certforge and echo-certforge-dispatcher are active at that deployment path. Read-only SQLite confirms 7 QUEUED runs / FAILED dispatches / BOUND reservations, 3 attempts each, all with subscriber_worker_claim_denied, none with consumed/execution-started/worker/claim markers. Another 8 CANCELLED / PENDING / BOUND reservations are a separate cohort and must never be swept by this fix. 33 completed certifications updated since Nov 12 UTC, so ordinary certification is not uniformly stalled.

PR #64 at e6b51c683c0027cab3ed446a26fdd614476a32c6 includes PR #49 and deployed ancestry (GitHub compare: both zero behind), but GHAS CodeQL fails HIGH on its exact SHA despite CI success; independent review remains zero. Do not merge/release #49 or #64 without reviewing the protected recovery predicates, claim evidence and compensation boundaries, independent GHAS adjudication, exact-head certification, and rollback acceptance. No queued/cancelled records were modified.

[echoomegaprime]

Independent recovery-gate update (2026-11-13 UTC): A bounded, least-change guard for the combined #49/#64 recovery is now proposed in draft PR #65, exact head 1d15bdf4cb5db6ec120f2b8c05928f29361d8593, tested Git tree 03980ad24ebdd00985b1a60d555fdd6abf17e975 (stacked directly on #64 e6b51c683c0027cab3ed446a26fdd614476a32c6). The historical compensation query now requires exhausted dispatch attempts, known pre-execution errors, >=24h stable age, deterministic ordering and maximum four recoveries per transaction while retaining all prior execution/ownership exclusions. 43 focused disposable-FORGE tests passed; exact published source/test Git blobs match the tested tree. Fresh production RO sample: 7 FAILED/BOUND/QUEUED historical orphans (3 attempts each, subscriber_worker_claim_denied) match the narrow shape; separate 8 cancelled reservations excluded; target tenant has 2 enrolled and zero fresh signed heartbeats. PR #65's hosted CI/security/reviews and all integration gate approvals remain required. No production records or services changed. See #65.

Same-run correction: Initial PR #65 head 3fccacd... failed hosted CI with 3 outdated legacy test expectations (736 tests passed). New exact head 1d15bdf4cb5db6ec120f2b8c05928f29361d8593 updates the audit/batch fixture acceptance without weakening production guards. All 43 focused tests passed on the final tested tree 346c295485ed9b08458d25c946b6b75a2b671d95; new hosted CI/security and independent review must still be checked at that head. No deployment or production state change.

[echoomegaprime]

Independent CertForge gate coordination (2026-11-13 ~04:00 UTC): PR #49 remains draft at exact head 0ab617ab6ed97e1a40e5cb3d160215dc93dc86d5, hosted CI passed, no independent approval and no exact-head GHAS CodeQL result. The integrated ancestry/recovery candidate #64 remains e6b51c683c0027cab3ed446a26fdd614476a32c6 with failing GHAS CodeQL and neutral CertForge. I opened draft stacked PR #66 (#66) at dd54ff11f7ed5a6d6169c79170834809aefecf1f, adding a bounded-parameter exhausted-dispatch-attempt guard and a negative fixture regression. Independent read-only execution of its exact SQL at policy attempts=3 selects all seven old orphans (all with three failed attempts) and excludes incomplete retries by predicate; no DB mutation. PR #66 GHAS CodeQL succeeded; hosted CI still in progress and review outstanding. Unknown legacy failure-reason scope and batch lock budget still require independent acceptance. FORGE deployed d8ad9fd881365801354b78a758632d2a31ab6828, 2 target enrolled runners, 0 fresh authenticated heartbeats. Do not compensate or deploy until protected reviews, exact-head checks, security gate, and reversible release window genuinely pass.