Repository navigation
Description
Summary
forgelm ingest writes {"text": "<chunk>"} rows with no id field, and forgelm purge --row-id matches only a row's id or row_id (it deliberately refuses line numbers). A corpus produced by ForgeLM's own ingestion therefore cannot be purged row by row until the operator adds ids with a script of their own, which the GDPR guide and manual now provide as a Python snippet.
Evidence
forgelm/ingestion.pywrites each chunk withdumps_strict({"text": payload}, ...).forgelm/cli/subcommands/_purge.pylocates a row byid/row_id(_ROW_ID_KEYS); a corpus without them yieldsNoMatchingRow.docs/guides/gdpr_erasure.mdanddocs/reference/purge_subcommand.md(and their Turkish mirrors) say ForgeLM provides no id-population helper and that aforgelm audit --add-row-idsflag is "on the Phase 28 backlog". No Phase 28 and no such backlog item exist in the roadmap; this issue is the tracking item.docs/usermanuals/{en,tr}/compliance/gdpr.mdcarries a standalone script.
Why it matters
An erasure request is the case where the operator is under a deadline (one month under GDPR Art. 12(3)). The documented path is: find the line with forgelm reverse-pii, then read that row's id and pass it to forgelm purge. For ForgeLM-ingested corpora that path needs a separate preparatory step that is easy to miss until the first request arrives. Re-assigning ids after training also breaks the link between the audit log and the corpus, so the ids have to exist from the start.
Acceptance criteria
-
forgelm ingestcan write a stableidper chunk (an opt-in flag first; the default can follow once the data loaders are confirmed to ignore the extra key). The id is deterministic for the same source file and chunk position so a re-ingest does not orphan earlier audit events. - The id carries no personal data in clear: not the raw source file name (file names often hold a person's name) and not the chunk text. Ids are copied into datasets, splits and downstream artefacts.
-
forgelm purge --row-id <id> --corpus <ingest output>erases such a row without any helper script; a test covers ingest -> purge end to end. - The GDPR guide, the purge reference and the user manual (EN and TR) replace the snippet with the flag; the "ForgeLM does not provide a helper" sentences and the "Phase 28 backlog" pointer are removed. Whether corpora that ForgeLM did not ingest also need a helper (the documented
audit --add-row-idsidea) is decided in the pull request; if it is wanted, it becomes its own issue. - Data loading and the audit are unaffected by the extra key (existing ingestion and data tests stay green).
Context
Found while correcting the GDPR erasure documentation for #101. Related: #102 (memorisation warning), #554 (CR-A10-036: --row-id matches on str() of the JSON value, so the id type matters), #555 (module sweep: purge / reverse-pii).
Metadata
Metadata
Assignees
Labels
Type
Projects
- StatusShow more project fieldsTodo
Activity
fix(docs): align GDPR, erasure, retention and PII documentation with …
cemililik commentedon Oct 22, 2026
Triage (2026-10-22): accepted, scope extended. Medium · Wave 3 · parent #870 (Wave 3 · Module sweep).
Verified on
developmentat 735a02f:forgelm/ingestion.pywrites only{"text": ...}and_purge.pymatchesid/row_idonly.Two additions to the description:
docs/reference/purge_subcommand.md(EN/TR) carries the same "no helper" sentence as the guide. Both pages sayforgelm audit --add-row-idsis "on the Phase 28 backlog", but no Phase 28 or such item exists in the roadmap. This issue is now the tracking item, and the criteria cover removing that pointer.For the implementer: write the id as a JSON string. #554 (CR-A10-036) records that
--row-idcomparesstr()of the stored value, so null, bool and float ids misbehave.Not blocked. Neither Phase 16 S9 (#895, purge and erasure) nor S10 (#896, data layer) owns ingest row ids.