Repository navigation
Description
Wave 1 · theme D1-10: Documentation-only fixes · highest severity High · 1 finding(s) · effort docs fix
Summary
Operators expect a baseline-relative regression check and a guarantee that nothing is written. In fact the run is gated on an absolute threshold, the model is written and then deleted, and the whole mechanism is off unless evaluation.auto_revert is set.
Primary finding
DOC-D2B09-001 — docs/product_strategy.md: false claim — Auto-revert: if safety regression is detected against baseline, training output is automatically rejected before artifacts are written
severity High · already tracked: KNOWN-OPEN DOCS-03/DOCS-04 (S15 auto-revert) — docs/roadmap/phase-16-trust-surface-hardening.md:760-765
Where. docs/product_strategy.md:22 (Turkish mirror: docs/product_strategy-tr.md:22) · type: false claim
The document says. Auto-revert: if safety regression is detected against baseline, training output is automatically rejected before artifacts are written
What the code does. There is no baseline comparison. max_safety_regression is an absolute ceiling on the current run's unsafe ratio. Auto-revert is opt-in (default false) and runs after the model has been saved: it deletes the saved final_model directory with shutil.rmtree and emits model.reverted.
Code references. forgelm/safety/_gates.py:255-262, forgelm/config.py:940-945, forgelm/config.py:1191-1194, forgelm/trainer.py:960-1005
User impact. Operators expect a baseline-relative regression check and a guarantee that nothing is written. In fact the run is gated on an absolute threshold, the model is written and then deleted, and the whole mechanism is off unless evaluation.auto_revert is set.
Proposed correction. Rewrite as: "Auto-revert (opt-in, evaluation.auto_revert: true): when the safety gate fails (unsafe-response ratio above the absolute max_safety_regression ceiling, or another configured gate), the saved model directory is deleted and the revert is recorded in the audit log."
Turkish mirror. the Turkish page has the same error; fix both in the same PR
Acceptance criteria
- Each statement is re-checked against the code at fix time.
- Documentation listed above is corrected in English and in its Turkish mirror in the same PR.
- Every finding in this issue is resolved or explicitly closed with a reason.
Found by the October 2026 ForgeLM code and documentation review against commit cab2947. Code links point at that commit; line numbers may have moved since. Finding IDs (CR-…, DOC-…, TECH-…) are stable references for this backlog.
Metadata
Metadata
Assignees
Labels
Type
Projects
- StatusShow more project fieldsTodo
Activity