Skip to content

SW-01 transcript 持久化:ModelRouter prompt/response 落 SQLite - #8

Merged
randypanding merged 2 commits into
mainfrom
sw/sw-01-transcripts
Aug 22, 2026
Merged

SW-01 transcript 持久化:ModelRouter prompt/response 落 SQLite#8
randypanding merged 2 commits into
mainfrom
sw/sw-01-transcripts

Conversation

@randypanding

Copy link
Copy Markdown
Owner

这个 PR 改的是哪一层?(必选其一)

  • A 资产层spec/ profiles/ brands/ cases/export/)→ 必须打标签 asset-change 并附 ADR
  • B 生成物层src/ tests/)→ 无需 ADR
  • 文档 / CI

工单

Closes # · 工单号:SW-01(上游依赖卡,Lab 仓 docs/WORK_ORDERS.md §上游依赖卡)

改动

src/nsc/runtime/models.py 单点改动:ModelRouter.complete 返回前把每次 LLM 调用的 prompt/response 写入 SQLite transcripts 表。表结构对齐 Lab 仓 ADR-0001 §接口:(ts, caller, model, prompt, response, tokens_in, tokens_out, cost_usd, experiment_id)

  • 库路径:构造参数 transcript_db > 环境变量 NSC_TRANSCRIPT_DB > 默认 out/transcripts.db(out/ 已 gitignore;Lab 经 subprocess 调 SW 时用环境变量接线)
  • 实验号:NSC_EXPERIMENT_ID 或构造参数
  • best-effort:建库/写库失败静默降级,绝不影响路由本身(与既有 _trace Langfuse 模式一致)
  • 不触碰 cases/cases.db(真相层走 db-export 流程)

验收命令(贴出你本地跑通的输出)

uv run pytest tests/test_transcripts.py -q   # 3 passed
uv run ruff format --check . && uv run ruff check .   # All checks passed
uv run pyright   # 0 errors
uv run pytest -m "not llm" -n auto   # 除 test_db_export_fresh 外全绿

注:test_db_export_fresh 在本机失败源于工作树里一份未提交的 cases/cases.db 本地改动(禁止提交,见运行纪律);在 origin/main 的干净 worktree 上该测试通过,本 PR 未触碰该文件,远端 CI 不受影响。

检查表

  • 测试先红后绿(先见 TypeError: unexpected keyword argument 'experiment_id',实现后转绿)
  • 没有在 Python 里写业务规则
  • 未新增 check 规则(不适用)
  • 未新增 check 规则(不适用)
  • 没有手改 prompts/
  • 手写行数仍在 spec/BUDGETS.yaml 预算内(runtime+checker 增 ~60 行,远低于 1600 上限)
  • 未改 IR
  • 未改 rubric

影响生成结果吗?

  • 否(只加观测台账,不改任何生成路径)

- transcripts 表对齐 Lab 仓 ADR-0001 §接口 (ts, caller, model, prompt,
  response, tokens_in, tokens_out, cost_usd, experiment_id)
- 库路径默认 out/transcripts.db,NSC_TRANSCRIPT_DB / 构造参数可改道,
  实验号 NSC_EXPERIMENT_ID(Lab subprocess 接线口)
- best-effort:写库失败静默降级,不影响路由(同 _trace 模式)
- 测试:tests/test_transcripts.py(3 例,stub litellm,无网络)
Copilot AI lite review requested due to automatic review settings August 21, 2026 19:40

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

@randypanding

Copy link
Copy Markdown
Owner Author

Copilot review 已重试两次均返回 "encountered an error"(动态 reviewer 侧故障,非 PR 内容问题)。本地等价审查要点:改动仅 models.py 单点 + 3 例无网络测试;best-effort 写库(失败静默不影响路由);transcripts 独立库 out/transcripts.db,不触碰 cases.db 真相层。如需人工/重放 review 请再触发。

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The transcript write path can still impact routing latency and robustness (SQLite lock timeout + missing rollback/connection reset on failures), which conflicts with the stated best-effort guarantees.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details
  • Files reviewed: 2/2 changed files
  • Comments generated: 3
  • Review effort level: Lite

Comment thread src/nsc/runtime/models.py
Comment on lines +82 to +86
try:
self.transcript_db.parent.mkdir(parents=True, exist_ok=True)
self._tconn = sqlite3.connect(str(self.transcript_db))
self._tconn.execute(_TRANSCRIPT_SCHEMA)
self._tconn.commit()
Comment thread src/nsc/runtime/models.py Outdated
Comment on lines +122 to +123
except Exception:
pass
Comment thread tests/test_transcripts.py Outdated
assert len(rows) == 1
row = rows[0]
assert row["caller"] == "tier_bulk"
assert row["model"] == "openai/LongCat-2.0"
@randypanding

Copy link
Copy Markdown
Owner Author

Copilot 复审意见处理(3/3,commit 0c533f7)

  1. busy timeout 与 best-effort 矛盾sqlite3.connect(..., timeout=0.0):transcripts 库被并发写锁住时立刻失败走静默路径,绝不为记账阻塞路由。
  2. 失败后连接半开 ✅ except 分支改为 rollback + 弃置连接(_tconn = None,下次懒重建),防半开事务长期持锁拖垮后续写入。
  3. 测试硬编码模型 ID ✅ 改为断言落库值 == router.resolve('tier_bulk')['model'](与路由实际选中一致)。

本地 pytest tests/test_transcripts.py 3/3 过;pytest -m 'not llm' 仅 test_db_export_fresh 失败(工作树 cases/cases.db 未提交的既有环境问题,CI 绿,与前述记录一致)。

@randypanding
randypanding merged commit fb8c99b into main Aug 22, 2026
10 checks passed
@randypanding
randypanding deleted the sw/sw-01-transcripts branch August 22, 2026 03:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants