Skip to content

feat(verify): cross-reference smartphone records against Wikipedia - #75

Merged
Seungpyo1007 merged 5 commits into
mainfrom
Seungpyo1007/feat-smartphone-wikipedia-backfill
Sep 22, 2026
Merged

Seungpyo1007 merged 5 commits into
mainfrom
Seungpyo1007/feat-smartphone-wikipedia-backfill

Conversation

@Seungpyo1007

Copy link
Copy Markdown
Member

Summary

Cross-reference tool to harvest verified Wikipedia URLs for unverified smartphone records (gsmarena-kaggle dataset).

  • Reuses strict safeguards from crossref.py:
    • Exact heading matching (_heading_matches)
    • Requirement of >= 2 independent agreeing specifications (launch year alone is blocked)
    • Hard-gated variant confusion filters (Pro, Plus/+, Max, Ultra, Mini, Lite, Fold/Flip, 4G vs 5G, Tablet/Watch)
    • Rejection of multiple competing candidates (multiple-rows)
    • Brand isolation on WikiRow preventing cross-brand matching
  • Supports fast sampling across brand directories without full filesystem scans
  • Respects Wikipedia robot policy and rate limits
  • Includes 11 comprehensive unit tests covering variant conflict, year-only block, spec rank, spec conflict, and HTML table extraction

@Seungpyo1007
Seungpyo1007 merged commit 8c25e5d into main Sep 22, 2026
1 check passed
@Seungpyo1007
Seungpyo1007 deleted the Seungpyo1007/feat-smartphone-wikipedia-backfill branch September 23, 2026 12:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant