Skip to content

feat(data): backfill 47 real Wikipedia Commons images for flagship smartphones - #246

Merged
Seungpyo1007 merged 5 commits into
developfrom
data/wikipedia-image-backfill
Sep 24, 2026
Merged

Seungpyo1007 merged 5 commits into
developfrom
data/wikipedia-image-backfill

Conversation

@Seungpyo1007

@Seungpyo1007 Seungpyo1007 commented Sep 24, 2026 •

Copy link
Copy Markdown
Member

Summary

Real image backfill (via TechEngine PR #85's new tool) for the flagship records that PR #244 nulled out. 47/354 eligible records got a real, correctly-licensed image (image_url + image_license + image_attribution): CC-BY-SA-4.0 ×29, CC-BY-2.0 ×6, CC-BY-4.0 ×5, CC0-1.0 ×4, CC-BY-SA-2.0/3.0 ×2. The rest (113 no usable image, 1 non-free-licensed, 193 logo-like/identity mismatches) were left null rather than guessed — see TechEngine PR #85 for the gating details.

Closes #1

Test plan

  • python -m app.validate passed

@TechEngineBot

Copy link
Copy Markdown
Member

🔎 Data verification — Tier 0 (offline existence/trust)

Scored by app.verify; posted by TechEngineBot. Informational only — the structural gate (app.validate) is separate and authoritative for merge.

Changed records in this PR

47 record(s) scored.

%%{init: {"theme":"base","themeVariables":{"pie1":"#3fb950","pie2":"#d29922","pie3":"#f85149","pieStrokeWidth":"0px","pieOpacity":"1"}}}%%
pie showData
    title Verification bands — all records
    "Green" : 47
    "Yellow" : 0
    "Red" : 0
Loading
Category Distribution Total 🟢 🟡 🔴 🟢 %
smartphone 🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩 47 47 0 0 100.0%
All 🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩 47 47 0 0 100.0%

Full-dataset baseline

105800 record(s) scored.

%%{init: {"theme":"base","themeVariables":{"pie1":"#3fb950","pie2":"#d29922","pie3":"#f85149","pieStrokeWidth":"0px","pieOpacity":"1"}}}%%
pie showData
    title Verification bands — all records
    "Green" : 23435
    "Yellow" : 81244
    "Red" : 1121
Loading
Category Distribution Total 🟢 🟡 🔴 🟢 %
brand 🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨 207 24 183 0 11.6%
soc 🟩🟨🟨🟨🟨🟨🟥🟥🟥🟥🟥🟥 2104 231 825 1048 11.0%
smartphone 🟩🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟥 93396 17909 75420 67 19.2%
tablet 🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟥 3455 277 3177 1 8.0%
watch 🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟥 433 39 390 4 9.0%
pda 🟩🟩🟩🟩🟩🟩🟩🟨🟨🟨🟨🟨 140 83 57 0 59.3%
gpu 🟩🟩🟩🟩🟩🟩🟩🟨🟨🟨🟨🟨 2030 1211 819 0 59.7%
cpu 🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟨🟥 4035 3661 373 1 90.7%
All 🟩🟩🟩🟨🟨🟨🟨🟨🟨🟨🟨🟥 105800 23435 81244 1121 22.2%

Hard violations (forced red):

Count Check
1 !boost_ge_base

green = authoritative source + complete + consistent · yellow = plausible, needs confirmation · red = sparse/weak source or a hard contradiction. Promotion to verified runs in the scheduled verify-network workflow.

@TechEngineBot

Copy link
Copy Markdown
Member

TechEngine change review: PASS

Check Result
python -m app.validate PASS
New hard integrity anomalies vs PR base PASS

Changed data

Category Added Modified Deleted Added verified Added unverified Added Kaggle-sourced
brand 0 0 0 0 0 0
soc 0 0 0 0 0 0
smartphone 0 47 0 0 0 0
tablet 0 0 0 0 0 0
watch 0 0 0 0 0 0
pda 0 0 0 0 0 0
gpu 0 0 0 0 0 0
cpu 0 0 0 0 0 0

Changed record examples

smartphone modified

  • smartphone/honor/2024/honor-magic-6-pro.json - HONOR Magic6 Pro
  • smartphone/honor/2024/honor-x9b.json - HONOR X9b
  • smartphone/huawei/2018/huawei-mate-20-pro.json - Huawei Mate 20 Pro
  • smartphone/huawei/2018/huawei-p20-pro.json - Huawei P20 Pro
  • smartphone/huawei/2019/huawei-p30-pro.json - Huawei P30 Pro
  • smartphone/huawei/2021/huawei-p50-pro.json - Huawei P50 Pro
  • smartphone/huawei/2023/huawei-mate-x3.json - Huawei Mate X3
  • smartphone/huawei/2023/huawei-mate-x5.json - Huawei Mate X5
  • smartphone/huawei/2024/huawei-mate-xt-ultimate.json - Huawei Mate XT Ultimate
  • smartphone/lg/2015/lg-g4.json - LG G4
  • smartphone/lg/2016/lg-g5.json - LG G5
  • smartphone/lg/2016/lg-v20.json - LG V20
  • smartphone/lg/2017/lg-v30.json - LG V30
  • smartphone/motorola/2025/motorola-edge-60-pro.json - Motorola Edge 60 Pro
  • smartphone/oneplus/2016/oneplus-3.json - OnePlus 3
  • ... 32 more

Heuristic review

  • Heuristic warnings: none found.

@TechEngineBot

Copy link
Copy Markdown
Member

TechEngine validation stats: PASS

Data summary

Category Total Verified Unverified Missing verified Tracked Verified % of tracked
brand 207 24 183 0 207 11.6%
soc 2104 184 1920 0 2104 8.7%
smartphone 93396 17495 75901 0 93396 18.7%
tablet 3455 248 3207 0 3455 7.2%
watch 433 39 394 0 433 9.0%
pda 140 74 66 0 140 52.9%
gpu 2030 1021 1009 0 2030 50.3%
cpu 4035 2868 1167 0 4035 71.1%
all 105800 21953 83847 0 105800 20.7%

Warning

Tracked verified coverage is below 50% for tablet 7.2% (248/3455), soc 8.7% (184/2104), watch 9.0% (39/433), brand 11.6% (24/207), smartphone 18.7% (17495/93396), all 20.7% (21953/105800).
Tracked coverage excludes records missing the verified field; see the Missing verified column for those records.
This does not fail validation. Keep imported records verified: false until manual audit, but treat this as follow-up verification work before relying on the affected categories as curated data.

Validation notes

  • The integrity gate blocks only hard anomalies introduced by this PR relative to its merge base.
  • Full advisory outlier listings are suppressed on successful runs because they are dataset-wide and mostly stable between PRs.
  • Failure runs still include a detailed log excerpt for debugging.

Key output:

## app.validate
## integrity_check.py (PR head compared with PR base)
loaded CPU=4035 GPU=2030
Integrity section Flagged lines
structural 18
CPU name/tier consistency (desktop mainstream only) 0
CPU single>multi (cinebench/geekbench — should be multi>=single) 0
CPU era-vs-score outliers 8
CPU cross-source ratio outliers (possible wrong-variant) 155
GPU cross-source ratio outliers + sanity 18

@Seungpyo1007
Seungpyo1007 merged commit 6d89c20 into develop Sep 24, 2026
5 checks passed
@Seungpyo1007
Seungpyo1007 deleted the data/wikipedia-image-backfill branch September 24, 2026 22:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

data Dataset changes enhancement New feature or request

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

Massive dataset rebuild: CPU + brand + GPU + smartphone + SoC (1989-2026)

2 participants