Skip to content

Pipeline resume infers an Article 14 approval from writable files (directory rename or state status), ignoring audit approve/reject events (CR-A04-002) #463

Open

Description

@cemililik

Wave 2 · theme D2-1: Security and integrity: medium/large effort · highest severity High · 1 finding(s) · effort medium

Summary

  1. Run A (ungated) leaves out/s1/final_model. 2. Run B (require_human_approval) stages s1 and exits 4. 3. The reviewer runs forgelm reject and later deletes the preserved staging dir. 4. --resume-from s2 logs 'gated stage s1_sft was approved', trains s2 from run A's final_model (which never passed the gate), exits 0, and records s1 as completed. 5. verify-annex-iv --pipeline with FORGELM_AUDIT_SECRET returns exit 0, success true, audit corroboration 'corroborated'. In a second variant, a plain mv staging final_model with no approve command does the same, and forgelm approvals --pending still lists the run as pending.

Primary finding

CR-A04-002 — Pipeline resume infers an Article 14 approval from writable files (directory rename or state status), ignoring audit approve/reject events

severity High · reachability default · intent contract-violation · confidence confirmed · independently confirmed · reproduced end to end

What happens. _resolve_resume_skiplist treats a gated_pending_approval stage as approved when the staging dir is absent and the derived promoted path exists (forgelm/cli/_pipeline.py:527-549). It skips any 'completed' stage whose output_model is an existing directory (569-574, 943-945), trusting unvalidated pipeline_state.json fields, and never consults the stage audit log for human_approval.granted or .rejected. The inferred approval flips the status to completed with no audit event, and verify-annex-iv --pipeline then passes. A rejected run (reject keeps staging) or a plain mv of the staging dir is accepted.

Where.

Evidence.

  • exp5 (compliance block + FORGELM_AUDIT_SECRET): A EXIT=0; B EXIT=4; REJECT EXIT=0; rm staging; RESUME EXIT=0 ('Resuming: gated stage 's1_sft' was approved'); VERIFY EXIT=0 {'success': True, 'violations': [], 'audit_corroboration': {'outcome': 'corroborated', ...}}. The s1 audit log has human_approval.rejected and no human_approval.granted. exp6 (plain mv, no approve): RESUME EXIT=0, VERIFY EXIT=0 corroborated; s1 audit has only human_approval.required; approvals --pending lists fg-54309d60421…
  • dyn/gate_bypass.py (copied to outputs/a review run/a1-W05/gate_bypass.py). It uses the real orchestrator, compliance export and verifier, stubs only _invoke_trainer, and sets FORGELM_AUDIT_SECRET. baseline: resume exit 1 (refused). tamper_state: resume exit 0, trainer inputs [('sft_stage','org/base'), ('dpo_stage','.../stage1/final_model.staging.abc123')], verify-annex-iv --pipeline EXIT=0 'OK: pipeline manifest ... (hash verified, 2 stage artefact(s))'. manual_mv: resume exit 0, chained from s…

Recommended fix. Before treating a gated stage as approved, require a human_approval.granted event for the staged run_id in the stage's audit log (HMAC-verified when FORGELM_AUDIT_SECRET is set), and refuse on human_approval.rejected. Emit a pipeline.* audit event that records the approval taken over from the stage log. Reset exit_code and set gate_decision when flipping the status. Optionally have forgelm approve record the decision where the orchestrator reads it.

Acceptance criteria

  • A regression test reproduces the failure scenario and asserts the corrected behaviour.
  • Documentation listed above is corrected in English and in its Turkish mirror in the same PR.
  • Every finding in this issue is resolved or explicitly closed with a reason.

Found by the October 2026 ForgeLM code and documentation review against commit cab2947. Code links point at that commit; line numbers may have moved since. Finding IDs (CR-…, DOC-…, TECH-…) are stable references for this backlog.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area: pipelineCode area: pipelinebugSomething isn't workingseverity: highWrong behaviour or a false claim users rely onsource: review-2026-10Found by the October 2026 code and documentation reviewwave: 2Roadmap wave 2: high severity, larger effort; next-release tech

    Type

    No type

    Projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions