feat(verify): GSMArena source_urls backfill tool (dry-run validated) - #70
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
검증률 병목 해결: 스마트폰 레코드 중 Tier-3(kaggle.com)만 source_urls로 갖고 있어 완성도와 무관하게 영원히 yellow에 갇힌 케이스(실제 출처는 이미 Tier-2로 신뢰되는 gsmarena.com)를 위한 백필 도구.
구성
app/verify/gsmarena_backfill.py— sitemap 기반 slug→URL 인덱스,crossref.py의 Fetcher 프로토콜/정확매칭 재사용,http_check.pyliveness 재사용, CONFIRM/AMBIGUOUS/NOTFOUND/CONTRADICT 2단 검증 게이트.tests/verify/test_gsmarena_backfill.pyDry-run 결과 (47건, gsmarena.com 429로 중단)
실행 전 보완 (이번 PR에 포함)
상태
도구만 병합 대상. 실제 TechAPI source_urls 갱신(대상 약 2,224개 gsmarena 페이지, 관련 레코드 다건)은 gsmarena.com 요청 제한이 풀린 뒤(오늘 03:00 UTC 기준 약 10시간 후) 별도로 실행.