Engine, TechEngineBot and automation (workstream)

[Seungpyo1007]

Purpose

Tracker for the engine, TechEngineBot and CI/automation that check and publish the dataset. It is one of the workstreams split out of the #1 roadmap on 2026-09-27.

TechAPI PRs labelled ci/app link here with Closes #<this>. TechEngine PRs can't close issues across repos, so they reference this issue as Refs GetTechAPI/TechAPI#<this>; a bare #1 points to TechEngine's own issue.

Engine-side trackers

Engine work is tracked in TechEngine: GetTechAPI/TechEngine#97 (roadmap), GetTechAPI/TechEngine#98 (verification & TechEngineBot), GetTechAPI/TechEngine#99 (ingest & sources), GetTechAPI/TechEngine#100 (dump, API & CI). This issue collects the TechAPI side: relay workflows, pr-metadata and board hygiene.

Scope

• TechEngineBot PR comments (validation, Tier 0 verify, integrity gate, dump check)
• Workflows in TechAPI and TechEngine: relays, weekly refresh/ingest, dump refresh, verify-network, pr-metadata
• Verify layer scoring rules and promotion
• Repo hygiene: Project feat(data,site): CPU catalog + CPU/GPU benchmark scores + new homepage #2 fields, labels, milestones

Status (2026-09-27)

• Coverage: since TechEngine data(verify): verified promotions (2026-06-29) #95 + TechAPI ci: send the PR base to TechEngine checks and stop tracking scores.jsonl #294, bot comments cover 12/12 categories · 191,797 records and diff against the PR's real base. Before that it was 8 categories, and the integrity gate checked only CPU/GPU (3.4%).
• Weekly refresh: PRs now target develop (TechEngine feat(data/cpu): weekly ingest #96). Previously chore(data): weekly refresh 2026-09-21 #195 landed on main directly.
• Project feat(data,site): CPU catalog + CPU/GPU benchmark scores + new homepage #2: all 362 items have Status, Priority, Start/Target date and Effort. It is kept tidy with a board-sync script after PRs.

Open items

• Laptop and monitor show red in Tier 0 (laptop 1,888/1,995, monitor 882/882) since the new structural gate. Find out whether this is a gate false positive or real record defects, and fix whichever it is.
• Scoring rules for laptop, monitor, software and website (currently structural checks only; they can't reach green).
• TechEngine weekly-refresh: 5 consecutive failures/cancellations; re-check after the 2026-09-28 run.
• 18 pre-existing CPU anomalies reported by the full integrity scan.
• pr-metadata: route PRs to the workstream trackers (roadmap TechAPI dataset roadmap and status: all categories (1989-2026) #1 / collection / accuracy / automation / site Track homepage and site improvements #19).

Reporting

One comment per change: what changed, the checks' scope and coverage, and CI status.

[Seungpyo1007]

2026-09-27 batch: verification coverage and promotion fixes

• feat(verify): score software and websites with batched Wikidata liveness TechEngine#103: software and website have domain scoring and a batched Wikidata liveness check
• fix(verify): skip fully cached records before capping the check-urls frontier TechEngine#104: check-urls no longer lets already-checked records fill --max every run
• fix(verify): keep the query in URL dedupe and cache every citation of a checked page TechEngine#105: URL dedupe keeps the query (cpu.php?cpu=X pages are distinct), and a verdict is cached for every #fragment citation of a checked page
• feat(ingest): fill absent software and website fields from Wikidata TechEngine#106: app.ingest.wikidata_fill fills absent software and website fields from the cited Wikidata item (applied in feat(data): fill missing software and website fields from their cited Wikidata items #312)

Promotions: #309 (7,812), #310 (51), #311 (772), #313 (204).
Verified is now 24,249 / 109,237 device records (22.2%) and 32,242 / 191,814 across all 12 categories (16.8%), up from 21.4% and 12.2% this morning.