Repository navigation
Conversation
四个手动离线脚本(校准工作流,人在环、按需跑,不接入调度器): - selectbench_matrix:拉 AIHOT 精选 feed 与库内全池重放对齐成四象限 (首份结果:一致率 26.6%,TopicEye 对该源选出 77% vs 其自选 ~4%, 精选率偏高在单一信源上的集中实证);AIHOT 信号仅内部对照、不导出 不分发不训练(用途边界与所有者决定记录于脚本 docstring) - selectbench_sample:分歧加权分层抽样导出标注候选 CSV - selectbench_import_labels:人工标注 CSV 批量导入 content_labels - selectbench_eval:金标集 1×4 percentile 扫描,P/R/F1 + FP/FN 错例清单 仅存于本分支(所有者决定),不合并 main。
同一套精选评分口径曾分散在三处各写一份且互相矛盾:prompt 教 LLM 五项 权重 30/25/20/15/10 加门槛 60,引擎 CONFIG 六维加门槛 55,picks SQL 硬编码 55/8 不读 CONFIG。CONFIG 全量迁入 scoring_caliber.py 作为唯一声明处, 三套 prompt 的口径文本从它渲染,picks 默认值改为运行时解析,today_picks fallback 常量同源;新增漂移回归测试锁死三处一致(先红后绿验证过)。 有意的行为变化仅一处:prompt 改教六维真实权重与门槛 55。
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
同一套精选评分口径曾分散在三个存储位置各写一份且互相矛盾(实测):prompt 教 LLM 五项权重 30/25/20/15/10(缺时效维)+门槛 60;引擎 CONFIG 六维 0.25/0.20/0.18/0.15/0.12/0.10+门槛 55(实际跑 percentile P70);picks SQL 签名硬编码 55/8 不读 CONFIG;today_picks fallback 又复制了一份 8。
curation_score占混合分 60%,核心链路被系统性教错权重且无测试能抓。本 PR 新建
app/services/scoring_caliber.py作为唯一声明处(CONFIG 迁入 + 收编picks_weight_bonus/curation_blend_weight两个散落数字),三套 prompt 的口径文本从它渲染,picks 默认值改运行时解析,并加「漂移即红」回归测试。Fixes #110
Area
Verification
tests/test_scoring_caliber.py→ 6 个失败(引擎 CONFIG 非同源/三套 prompt 权重与门槛/picks 硬编码),正是 issue 描述的矛盾集;改造后 10/10 通过python -m pytest tests/test_scoring_engine.py tests/test_today_picks.py tests/test_paper_analysis.py tests/test_scoring_flow.py tests/test_scoring_caliber.py -q→ 65 passedmake test-backend(一次性 PG 容器全量)→ exit 0,全绿至 100%python scripts/check_layering.py→ 50 个 api/v1 文件无违规ruff check+ruff format --check(六个改动文件)→ 通过py_compile六个改动文件 → 通过Verifier verdict
pass(PASS-with-concerns)——独立复核员(非本实现过程)五条声明全部 confirmed:CONFIG 字面量全仓唯一且引擎为同一对象再导出(
is实证)、blend 重构 10 万随机样本浮点 diff=0、picks 默认解析数值等价(55/8)、漂移测试四种变异模拟全部变红且非自比较、分层无违反。concerns 均为既有现状、不阻塞:① 前端recommendation.ts:46-48镜像了 RISK_HARD_EXCLUDE=82/QUALITY_GATE_MIN=45(建议另立 issue,可由接口下发);②content_repo.py:870/912存在 repositories→services 逆向 import(既有违例,本次未触碰)。Checklist
fix(scoring): …)issue-110-scoring-caliber-single-source,正文写Fixes #110Notes for reviewer
analyzed_at分界在 admin/scoring-dashboard 观察迁移幅度(issue 已声明)。引擎数值与 picks 行为零变化。