Add topic 06 benchmark test suite

[2813183274-cloud]

变更概述

将课题 06 编译器性能测试套件迁移并集成到 ScratchV 项目目录中，用于自动验证 DSL 编译结果、TinyFive 模拟结果和编译性能。

主要改动

• 增加 23 个 DSL 测试用例，覆盖 activation、elementwise、loop、branch、reduction 和 tensor 六类计算模式。
• 增加 scripts/run_topic06_benchmarks.py 测试入口。
• 使用真实 TinyFive 执行编译生成的 RISC-V 汇编。
• 支持编译超时和模拟超时，避免单个用例卡住整套测试。
• 支持普通测试、Benchmark、多次运行取平均值和置信区间。
• 支持性能基线保存和退化检测。
• 支持按类别和名称筛选测试用例。
• 生成统一结构的 Markdown、JSON 和 HTML 测试报告。
• 增加寄存器映射输出，用于向 TinyFive 注入标量输入。
• 增加 pytest 集成测试和 GitHub Actions 工作流。
• 将原独立交付目录迁移到项目标准的 scripts/、tests/、benchmarks/ 和 docs/ 目录。

测试结果

• Topic 06 相关测试：55 passed
• 项目测试（排除一个既有的 Windows 临时文件清理问题）：430 passed，4 skipped
• TinyFive 稳定 CI 类别：
  • activation：4/4 passed
  • elementwise：5/5 passed
  • loop：3/3 passed

已知限制

当前真实 TinyFive 全量测试为 13/23 passed。

未通过用例主要集中在：

• branch：编译后控制流在 TinyFive 中超时。
• reduction 和 tensor：数组、矩阵输入尚未建立完整的内存传递与结果读取约定。

这些用例保留为诊断用例，不作为当前稳定 CI 门禁。稳定 CI 当前只启用 activation、elementwise 和 loop 类别。

使用方法

python scripts/run_topic06_benchmarks.py
python scripts/run_topic06_benchmarks.py --benchmark 3
make bench-topic06

[github-actions]

🤖 AI Code Review

共审查 10 个变更文件
⚠️ 另有 18 个文件超过上限（最多 10 个）未审查

📁 Makefile

🟡 Magic number in bench-topic06 — --benchmark 3 has no explanation of what 3 represents (3 iterations? 3 test cases? a benchmark ID?). If the semantics change or a new reader encounters this, it's opaque. Consider adding a comment (e.g., # 3 = number of runs for statistical confidence) or, if scripts/run_topic06_benchmarks.py supports a named option (e.g., --runs or --mode), use the more descriptive flag.

💭 No test dependency — bench-topic06 is documented as "DSL correctness + TinyFive benchmark" but doesn't run test first. The existing bench target also doesn't enforce this, so it's consistent, but worth noting in case a stale build gives misleading benchmark numbers.

💭 The target is clean and follows the existing style. No issues with .PHONY coverage.

---

📁 README.md

🟡 Suggestion: make bench-topic06 命名不一致 — 其他 benchmark 命令都是 make bench-cnn / make bench-ci / make bench-reports，按模块或用途命名。bench-topic06 混入了项目内部术语（"课题 06"），与 bench-cnn（按模型命名）风格不一致。考虑改为 make bench-tinyfive 或 make bench-dsl，既与 bench-cnn 对齐，又对未参与课题分配的人更友好。

🟡 Suggestion: 目录注释范围重叠 — benchmarks/ 和 tests/ 现在都提到"课题 06"，且 tests/ 同时承担"单元测试"和"课题 06 的 23 个 DSL 用例"两个职责。读者难以区分哪些用例属于通用测试、哪些属于课题专用。建议在注释中明确边界，或加目录结构子注释。

💭 Nit: 链接目标路径 — docs/topics/06-性能测试套件使用说明.md 使用了文件名而非 slug，如果后续文件名变动需同步改 README。可考虑在 docs/INDEX.md 中维护统一索引，README 仅链接索引页。

---

📁 benchmarks/topic06/baseline.json

🟡 Missing benchmarks/topic06/ directory context — This is a new baseline file with no companion code (e.g., benchmark definitions or collection script). If other files in this PR define these 20 cases, ensure their names are identical to avoid silent mismatches.

🟡 No metadata for baseline provenance — There's no indication of the hardware, compiler version, or environment used to produce these avg_instr_count values. Future reviewers won't know whether drift in avg_instr_count is meaningful or expected. Consider adding a "meta" field or a sibling README.

🟡 avg_instr_count stored as float but always integer-valued — All values are .0. If these are true averages over runs: 3, consider rounding explicitly and documenting that, or store as int if runs always yields whole-number averages. The float type suggests the expectation of non-integer results that never materializes.

💭 No trailing newline — \ No newline at end of file. Most POSIX tools and git diff behave better with a trailing newline.

💭 No "runs" variance captured — With runs: 3, the standard deviation or min/max could help distinguish stable baselines (e.g., vector_add: 3.0) from noisy ones. A future regression alert could benefit from knowing the spread.

---

📁 docs/topics/06-性能基准套件.md

🔴 Bug: 死链风险 — 新增了两条相对链接 06-性能测试套件使用说明.md 和 06-性能测试套件设计文档.md。如果这两个文件尚未提交或未在 PR 中创建，文档将出现 404 死链。需确认这些文件在同一变更中已存在。

🔴 Bug: 信息丢失 — 常见坑全部移除 — 原文档中的"常见坑"表（无 const 操作符、.expected 文件格式、超时保护）是用户实际踩坑频率最高的内容。迁移到说明文档后需确认新文件包含同等内容，否则新用户将失去关键指引。

🟡 Suggestion: 缺少导航上下文 — 原文档头部有 > **难度**：中 | **类型**：项目实战 | **源文件**：... | **行数**：... 元信息行和 > **状态**：✅ 已完成 状态标记。新文档完全移除，如果这是系列课题索引的一部分，丢失的元数据可能影响其他文档的交叉引用。

🟡 Suggestion: 代码块缺少语言标记 — ```text 代码块内列了 4 条路径，这些是文件/目录引用而非命令输出，建议直接用纯文本列表或 - 无序列表代替，语义更清晰：

- `scripts/run_topic06_benchmarks.py`
- `tests/topic06/cases/`
- `benchmarks/topic06/baseline.json`
- `.github/workflows/topic06-benchmark.yml`

🟡 Suggestion: 缺少"如何添加新用例"指引 — 原文档有完整的"添加新测试用例"步骤（cat > ..., echo > ..., 运行确认），新文档仅有安装和运行命令。新贡献者无从得知如何扩展测试集。

💭 Nit — "课题 06" 的编号格式在旧文档中是"课题6"，若系列中其他文档使用 课题06 格式则无问题，否则存在编号风格不一致。

---

📁 docs/topics/06-性能测试套件使用说明.md

🔴 数字易过时 — "23 个 DSL 测试用例"、"13 个通过、10 个失败" 在首次新增/删除用例或修复失败用例后即为错误信息，读者无法区分文档是否已同步。建议用 tests/topic06/cases/ 下的 .dsl 文件数量动态表述（如"当前包含的 DSL 测试用例覆盖以下类别"），或明确标注生成日期/commit 哈希。

🟡 --filter 匹配对象不明确 — "按用例名称进行大小写不敏感的包含匹配"，未说明是匹配文件名、.meta.json 中的 description 还是 DSL 内部标识符。建议明确：如"匹配 .dsl 文件名（不含路径和扩展名）"。

🟡 Benchmark 全失败场景未说明 — "失败运行不会作为 0 加入平均值"，但如果所有 --benchmark N 次都 timeout/失败，报告中的 avg_instruction_count 是 null、报错还是空列表？建议在报告字段说明或此处补充边界行为。

🟡 course_report_instructions.png 用途不清晰 — 文件名暗示是课程报告指导图，但上下文完全没有解释它包含什么内容、何时需要。建议加一句话说明生成条件或用途，否则读者会疑惑是否需要它。

🟡 JSON schema 未展示 — 文中强调"统一 JSON schema"作为设计亮点，但全文没有展示 report.json 的字段结构。考虑至少给出一个精简的 JSON 示例或指向 report.json 的 README。

💭 time.perf_counter() 属于实现细节 — 在"运行测试"的面向用户步骤列表中，第 11 条暴露了具体计时函数，与其余面向行为描述的条目粒度不一致。可简化为"记录编译、模拟和总耗时"。

💭 标题与正文存在重复 — "运行测试"段落的 bullet 列表与"测试报告"段落有内容重叠（如报告生成逻辑）。考虑在 bullet 中精简，将报告详情完全归入"测试报告"段。

---

📁 docs/topics/INDEX.md

🟡 文件重命名语义变更 — 标题从「性能基准套件」改为「性能测试套件使用说明」，不只是文件名调整，主题含义也变了（从 benchmark 套件变为 usage guide）。

• 如果原文件确实是重命名，需确认仓库中 06-性能测试套件使用说明.md 已同步推送，否则链接会 404。
• 如果原文档内容仍是性能基准相关，建议文件名和标题保持一致，避免读者预期与实际内容不符。

💭 其余行无问题，变更清晰、范围小。

---

📁 pyproject.toml

🟡 Suggestion: pytest already included in existing groups? — If pytest is already listed elsewhere in [project.optional-dependencies] (not shown in this diff), consider whether topic06 should just reference it rather than duplicating the dependency. Duplicated version pins across groups can cause drift.

🟡 Suggestion: Version pinning — tinyfive, jinja2, and matplotlib have no version constraints. If these are third-party packages, consider pinning (e.g., matplotlib>=3.8,<4) to prevent unexpected breaking changes when pip install .[topic06-report] is run in the future.

💭 Nit: Naming inconsistency — Other groups use descriptive verbs (verify, llvm) while these use topic/course numbering (topic06, topic06-report). If these are permanent project groups rather than temporary course assignments, a more descriptive name like simulation-report would be clearer long-term.

---

📁 scratchv/backend/register_alloc.py

🟡 No guard on allocation state — register_map returns an empty {} if called before run() completes allocation. Consider raising RuntimeError or returning None if self._output is None, so callers get a clear signal rather than silently getting an empty map.

🟡 Shallow copy on every access — dict(self._vreg_map) allocates a new dict each call. If this property is accessed frequently (e.g., in a loop during instruction emission), consider returning self._vreg_map.copy() only when mutation is needed, or documenting that callers should not mutate the result.

💭 Documentation gap — The docstring doesn't mention that the result is a snapshot (immutable copy), which is a useful contract for consumers.

---

📁 scratchv/compiler.py

🟡 Inconsistent deep copy of register map — Lines 419, 446: self._last_register_map = alloc.register_map assigns a direct reference, while line 420 uses dict(lsa.alloc_map) (a copy). If alloc.register_map is mutated later (e.g., allocator reuse across calls), the stored map silently changes. Apply dict() consistently, or document the ownership contract.

💭 Defensive reset placement — Line 238: self._last_register_map = {} resets at the top of compile(), which is good. However, if _generate_riscv_linear/_generate_riscv_dag raises before reaching the stats dict construction, the returned result (via raise CompileError) would not include stale stats, so this is fine in practice — just noting the dependency on early reset.

💭 Naming — _last_register_map reads as an internal implementation detail, but it's now exposed to callers via stats["register_map"]. Consider whether a clearer public-facing key name (e.g., "alloc_map") would reduce confusion between temporary spill registers and architectural registers.

---

📁 scratchv/main.py

🟡 Uncaught TypeError in JSON serialization — Line ~267: json.dumps will raise TypeError if any value in register_map isn't JSON-serializable (e.g., int vs np.int64, custom objects). This isn't covered by the OSError catch, so it would propagate as an unhandled exception.
Suggestion: Either catch (OSError, TypeError) or add a default=str fallback in json.dumps.

🟡 No confirmation on success — The register map is written silently. If something went wrong with the data (e.g., empty register map written), the user gets no feedback. Consider printing a confirmation message like "Register map written to {path}" to stderr after a successful write.

🟡 Potential issue with result.stats.get(...) — If result.stats is ever None or a non-dict type, this will raise an uncaught AttributeError. Depends on how stats is always populated, but a defensive check (result.stats and result.stats.get(...)) or a comment asserting it's always a dict would improve robustness.

💭 OSError is broad — Catches IsADirectoryError, PermissionError, FileExistsError, etc. Consider catching more specific exceptions or at least adding a comment explaining the intent. OSError isn't wrong here, just wide.

💭 Indentation consistency — The print(f"OK ...") line appears pre-existing but uses a trailing comma + line break for the second argument. Minor, but worth noting if this is the style for this codebase.

---

---

⚠️ 未审查的文件

• scratchv/simulator/tinyfive.py
• scripts/run_topic06_benchmarks.py
• tests/test_simulator.py
• tests/topic06/cases/activation/add_relu_relu.dsl
• tests/topic06/cases/activation/add_relu_relu.meta.json
• tests/topic06/cases/activation/relu_add.dsl
• tests/topic06/cases/activation/relu_add.meta.json
• tests/topic06/cases/activation/relu_only.dsl
• tests/topic06/cases/activation/relu_only.meta.json
• tests/topic06/cases/activation/relu_twice.dsl
• tests/topic06/cases/activation/relu_twice.meta.json
• tests/topic06/cases/branch/if_else.dsl
• tests/topic06/cases/branch/if_else.meta.json
• tests/topic06/cases/branch/if_relu.dsl
• tests/topic06/cases/branch/if_relu.meta.json
• tests/topic06/cases/branch/if_then.dsl
• tests/topic06/cases/branch/if_then.meta.json
• tests/topic06/cases/elementwise/add_chain.dsl