Skip to content

fix(data): null out 367 broken placeholder image_url values - #244

Merged
Seungpyo1007 merged 2 commits into
developfrom
fix/broken-placeholder-image-urls
Sep 24, 2026
Merged

Seungpyo1007 merged 2 commits into
developfrom
fix/broken-placeholder-image-urls

Conversation

@Seungpyo1007

@Seungpyo1007 Seungpyo1007 commented Sep 24, 2026 •

Copy link
Copy Markdown
Member

Summary

TechPicks (reference-consumer app) reported all image_url 404s on its canonical device set. Root cause: 367 hand-curated flagship smartphone records (366 of them verified: true) had image_url literally set to https://cdn.jsdelivr.net/gh/GetTechAPI/images/smartphones/{slug}.webp — copied from SPEC.md's old (also-wrong) response example. That repo (GetTechAPI/images) has never existed; every one of those 367 URLs 404s. Older bulk-imported records are unaffected (they hotlink real hosts like gsmarena.com/aitoolbuzz.com and work fine).

  • Nulled out image_url on the 367 affected records (surgical single-line diff, no reformatting) rather than leaving an actively-wrong value in place.
  • 354/367 already cite Wikipedia in source_urls — real image backfill from Wikipedia infoboxes is a viable follow-up, tracked separately (not in this PR).
  • Last commit refreshes the public dump — this also picks up the model_3d/body schema fields added in TechEngine PR test: post-migration /verify e2e (throwaway) #83 for the first time (schema-wide, touches every smartphone dump page) plus catch-up from PR feat(data): Wikipedia URL backfill for smartphones/tablets + dump refresh #240 which had gone unmerged since yesterday.

Closes #1

Test plan

  • python -m app.validate passed

@TechEngineBot

Copy link
Copy Markdown
Member

🔎 Data verification — Tier 0 (offline existence/trust)

Scored by app.verify; posted by TechEngineBot. Informational only — the structural gate (app.validate) is separate and authoritative for merge.

Changed records in this PR

406 record(s) scored.

%%{init: {"theme":"base","themeVariables":{"pie1":"#3fb950","pie2":"#d29922","pie3":"#f85149","pieStrokeWidth":"0px","pieOpacity":"1"}}}%%
pie showData
    title Verification bands — all records
    "Green" : 404
    "Yellow" : 2
    "Red" : 0
Loading
Category Distribution Total 🟢 🟡 🔴 🟢 %
smartphone 🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟨 380 378 2 0 99.5%
tablet 🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩 26 26 0 0 100.0%
All 🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟨 406 404 2 0 99.5%

Full-dataset baseline

105800 record(s) scored.

%%{init: {"theme":"base","themeVariables":{"pie1":"#3fb950","pie2":"#d29922","pie3":"#f85149","pieStrokeWidth":"0px","pieOpacity":"1"}}}%%
pie showData
    title Verification bands — all records
    "Green" : 23435
    "Yellow" : 81244
    "Red" : 1121
Loading
Category Distribution Total 🟢 🟡 🔴 🟢 %
brand 🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨 207 24 183 0 11.6%
soc 🟩🟨🟨🟨🟨🟨🟥🟥🟥🟥🟥🟥 2104 231 825 1048 11.0%
smartphone 🟩🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟥 93396 17909 75420 67 19.2%
tablet 🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟥 3455 277 3177 1 8.0%
watch 🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟥 433 39 390 4 9.0%
pda 🟩🟩🟩🟩🟩🟩🟩🟨🟨🟨🟨🟨 140 83 57 0 59.3%
gpu 🟩🟩🟩🟩🟩🟩🟩🟨🟨🟨🟨🟨 2030 1211 819 0 59.7%
cpu 🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟨🟥 4035 3661 373 1 90.7%
All 🟩🟩🟩🟨🟨🟨🟨🟨🟨🟨🟨🟥 105800 23435 81244 1121 22.2%

Hard violations (forced red):

Count Check
1 !boost_ge_base

green = authoritative source + complete + consistent · yellow = plausible, needs confirmation · red = sparse/weak source or a hard contradiction. Promotion to verified runs in the scheduled verify-network workflow.

@Seungpyo1007
Seungpyo1007 merged commit 0784536 into develop Sep 24, 2026
5 checks passed
@Seungpyo1007
Seungpyo1007 deleted the fix/broken-placeholder-image-urls branch September 24, 2026 17:31
@TechEngineBot

Copy link
Copy Markdown
Member

TechEngine change review: PASS

Check Result
python -m app.validate PASS
New hard integrity anomalies vs PR base PASS

Changed data

Category Added Modified Deleted Added verified Added unverified Added Kaggle-sourced
brand 0 0 0 0 0 0
soc 0 0 0 0 0 0
smartphone 0 367 0 0 0 0
tablet 0 0 0 0 0 0
watch 0 0 0 0 0 0
pda 0 0 0 0 0 0
gpu 0 0 0 0 0 0
cpu 0 0 0 0 0 0

Changed record examples

smartphone modified

  • smartphone/apple/2012/iphone-5.json - iPhone 5
  • smartphone/apple/2013/iphone-5c.json - iPhone 5c
  • smartphone/apple/2013/iphone-5s.json - iPhone 5s
  • smartphone/apple/2014/iphone-6-plus.json - iPhone 6 Plus
  • smartphone/apple/2014/iphone-6.json - iPhone 6
  • smartphone/apple/2015/iphone-6s-plus.json - iPhone 6s Plus
  • smartphone/apple/2015/iphone-6s.json - iPhone 6s
  • smartphone/apple/2016/iphone-7-plus.json - iPhone 7 Plus
  • smartphone/apple/2016/iphone-7.json - iPhone 7
  • smartphone/apple/2016/iphone-se-1st-gen.json - iPhone SE (1st generation)
  • smartphone/apple/2017/iphone-8-plus.json - iPhone 8 Plus
  • smartphone/apple/2017/iphone-8.json - iPhone 8
  • smartphone/apple/2017/iphone-x.json - iPhone X
  • smartphone/apple/2018/iphone-xr.json - iPhone XR
  • smartphone/apple/2018/iphone-xs-max.json - iPhone XS Max
  • ... 352 more

Heuristic review

  • Heuristic warnings: none found.

@TechEngineBot

Copy link
Copy Markdown
Member

TechEngine validation stats: PASS

Data summary

Category Total Verified Unverified Missing verified Tracked Verified % of tracked
brand 207 24 183 0 207 11.6%
soc 2104 184 1920 0 2104 8.7%
smartphone 93396 17495 75901 0 93396 18.7%
tablet 3455 248 3207 0 3455 7.2%
watch 433 39 394 0 433 9.0%
pda 140 74 66 0 140 52.9%
gpu 2030 1021 1009 0 2030 50.3%
cpu 4035 2868 1167 0 4035 71.1%
all 105800 21953 83847 0 105800 20.7%

Warning

Tracked verified coverage is below 50% for tablet 7.2% (248/3455), soc 8.7% (184/2104), watch 9.0% (39/433), brand 11.6% (24/207), smartphone 18.7% (17495/93396), all 20.7% (21953/105800).
Tracked coverage excludes records missing the verified field; see the Missing verified column for those records.
This does not fail validation. Keep imported records verified: false until manual audit, but treat this as follow-up verification work before relying on the affected categories as curated data.

Validation notes

  • The integrity gate blocks only hard anomalies introduced by this PR relative to its merge base.
  • Full advisory outlier listings are suppressed on successful runs because they are dataset-wide and mostly stable between PRs.
  • Failure runs still include a detailed log excerpt for debugging.

Key output:

## app.validate
## integrity_check.py (PR head compared with PR base)
loaded CPU=4035 GPU=2030
Integrity section Flagged lines
structural 18
CPU name/tier consistency (desktop mainstream only) 0
CPU single>multi (cinebench/geekbench — should be multi>=single) 0
CPU era-vs-score outliers 8
CPU cross-source ratio outliers (possible wrong-variant) 155
GPU cross-source ratio outliers + sanity 18

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working data Dataset changes enhancement New feature or request

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

Massive dataset rebuild: CPU + brand + GPU + smartphone + SoC (1989-2026)

2 participants