fix: preserve entity evidence fields in source event CSV

docs/free_source_setup.md

@@ -12,4 +12,22 @@
 - `data/live/political_events.csv`: stable Advisor input, refreshed by RSS/source pipeline or maintained after manual review.
 - `data/live/source_tracker.csv`: merged watchlist and event tracker.
 
+## Entity-field preservation stage
+
+The RSS/source extractor exports the eight existing event columns followed by
+`entity_match_type`, `match_evidence`, and `relationship_type`. This stage only
+preserves the normalized fields in the CSV; it does not verify company
+relationships. Extracted mentions retain `unverified`, an empty evidence string,
+and `unverified`, respectively. Source `confidence=high` describes the source
+type, not verified company evidence. Generic terms such as "strategy",
+"cybersecurity", or "crypto assets", and incidental references to WhatsApp do
+not establish an issuer or direct-beneficiary relationship.
+
+With `commit_outputs=true`, the source workflows stage generated live files and
+upload a patch plus a `HUMAN_REQUIRED` receipt. An authorized maintainer reviews
+the patch and submits it through a normal protected-branch PR; the workflows do
+not directly publish to `main`. Preserving these fields does not collect new
+evidence, regenerate live data, or change downstream acceptance requirements.
+Real company evidence and its data PR remain a separate reviewed step.
+
 See the Chinese setup note for cron wiring, refresh commands, and operator checks.

docs/free_source_setup.zh-CN.md

@@ -52,8 +52,8 @@ gh workflow run "RSS Source Pipeline" \
 
 ## 推荐运营流程
 
-1. RSS pipeline 定时生成并提交 `data/live/source_items.csv`、`source_events.csv`、`political_events.csv`、`source_tracker.csv`、`source_fetch_status.json` 和 `source_manifest.json`。
-2. 人工来源仍可以通过 `Source Event Pipeline` 输入 `source_items.csv`，再选择 `commit_outputs=true` 写回 live CSV。
+1. RSS pipeline 定时生成并暂存 `data/live/source_items.csv`、`source_events.csv`、`political_events.csv`、`source_tracker.csv`、`source_fetch_status.json` 和 `source_manifest.json`，上传 patch 与 `HUMAN_REQUIRED` receipt；由已授权维护者审核并通过正常受保护分支 PR 提交，不直接发布到 `main`。
+2. 人工来源仍可以通过 `Source Event Pipeline` 输入 `source_items.csv`，选择 `commit_outputs=true` 后也只生成上述人工 PR 交接，不会自动写回远程 live CSV。
 3. 如果单个 RSS 源失败，workflow 会继续处理其他源，并把失败写入 `source_fetch_status.json`；连续失败的源再单独移除或替换。
 4. 如果 RSS 拉到了文章但事件为空，优先检查 alias 覆盖；很多官方政策只写主题词，不写公司名。
 5. 发布 Advisor 时使用：
@@ -69,6 +69,12 @@ gh workflow run "Publish Model Recommendations Site" \
 
 如果 `political_events_path` / `political_watchlist_path` 不在 `examples/` 下，Advisor 输出会标记为 `source_mode=operator_supplied`。
 
+## 实体字段保留阶段
+
+RSS/source extractor 在原有八列后输出 `entity_match_type`、`match_evidence` 和 `relationship_type`。本阶段只把规范化结果已有的字段保留到 CSV，不核实公司关系；自动抽取记录的三个字段仍分别为 `unverified`、空字符串和 `unverified`。
+
+来源的 `confidence=high` 只表示来源类型，不代表公司实体证据已核实。普通词语 strategy、cybersecurity、crypto assets，以及附带提及 WhatsApp，都不能据此认定发行人或直接受益关系。本阶段不采集新证据、不重生成 live 数据、不改变下游接受门槛；真实公司证据及其数据 PR 另行审核。
+
 ## 长线证据边界
 
 本仓库可以给长线推荐提供官方来源加分，但不要求每个长线候选都必须有近期政策/新闻事件。长线主题归属和语义背景由 `ResearchSignalContextPipelines` 维护；本仓库只在存在 SEC 文件、发行人公告、政府采购、Federal Register、官方拨款或政策文件时提供点时事实证据。

src/political_event_tracking_research/source_mention_extract.py

@@ -150,7 +150,19 @@ def extract_source_records(raw_items_path: str | Path, aliases_path: str | Path,
     rows = normalize_records(records)
     write_csv_rows(
         output_path,
-        ["event_id", "event_date", "symbol", "event_type", "direction", "confidence", "source_url", "notes"],
+        [
+            "event_id",
+            "event_date",
+            "symbol",
+            "event_type",
+            "direction",
+            "confidence",
+            "source_url",
+            "notes",
+            "entity_match_type",
+            "match_evidence",
+            "relationship_type",
+        ],
         rows,
     )
     return rows

tests/test_source_mention_extract.py

@@ -1,10 +1,16 @@
 from __future__ import annotations
 
+import csv
 from pathlib import Path
 
-from political_event_tracking_research.source_mention_extract import extract_source_records, infer_direction, match_symbols
-from political_event_tracking_research.source_mention_extract import MentionAlias
+import pytest
 
+from political_event_tracking_research.source_mention_extract import (
+    MentionAlias,
+    extract_source_records,
+    infer_direction,
+    match_symbols,
+)
 
 ROOT = Path(__file__).resolve().parents[1]
 
@@ -51,3 +57,69 @@ def test_extract_source_records_outputs_confidence_by_source_type(tmp_path: Path
     assert by_symbol["EVT3"]["confidence"] == "low"
     assert by_symbol["EVT4"]["event_type"] == "procurement"
     assert output.exists()
+
+
+def test_extract_source_records_preserves_entity_fields_in_csv(tmp_path: Path) -> None:
+    output = tmp_path / "source_events.csv"
+
+    rows = extract_source_records(
+        ROOT / "examples/source_items.example.csv",
+        ROOT / "examples/symbol_aliases.example.csv",
+        output,
+    )
+
+    with output.open(newline="", encoding="utf-8") as handle:
+        reader = csv.DictReader(handle)
+        assert reader.fieldnames == [
+            "event_id",
+            "event_date",
+            "symbol",
+            "event_type",
+            "direction",
+            "confidence",
+            "source_url",
+            "notes",
+            "entity_match_type",
+            "match_evidence",
+            "relationship_type",
+        ]
+        exported_rows = list(reader)
+
+    assert exported_rows == rows
+    for row in exported_rows:
+        assert row["entity_match_type"] == "unverified"
+        assert row["match_evidence"] == ""
+        assert row["relationship_type"] == "unverified"
+
+
+@pytest.mark.parametrize(
+    ("text", "symbols"),
+    [
+        ("National standards strategy creates a shared framework.", {"MSTR"}),
+        ("Cybersecurity policy supports workforce development.", {"PANW"}),
+        ("New policy for crypto assets custody.", {"COIN", "MSTR"}),
+        ("Fraudsters used WhatsApp to contact investors.", {"META"}),
+    ],
+)
+def test_high_source_confidence_does_not_verify_company_mentions(
+    tmp_path: Path, text: str, symbols: set[str]
+) -> None:
+    raw_items = tmp_path / "source_items.csv"
+    with raw_items.open("w", newline="", encoding="utf-8") as handle:
+        writer = csv.writer(handle)
+        writer.writerow(["item_id", "published_at", "source_type", "source_url", "author", "text"])
+        writer.writerow(
+            ["synthetic-1", "2026-01-10", "government_policy", "https://www.sec.gov/example", "Example", text]
+        )
+    output = tmp_path / "source_events.csv"
+
+    extract_source_records(raw_items, ROOT / "config/core_us_equity_aliases.csv", output)
+
+    with output.open(newline="", encoding="utf-8") as handle:
+        rows = list(csv.DictReader(handle))
+    assert {row["symbol"] for row in rows} == symbols
+    for row in rows:
+        assert row["confidence"] == "high"
+        assert row["entity_match_type"] == "unverified"
+        assert row["match_evidence"] == ""
+        assert row["relationship_type"] == "unverified"