fix(verify): skip fully cached records before capping the check-urls frontier - #104
Merged
Merged
Conversation
…feat/software-website-scoring
…feat/software-website-scoring
…frontier The frontier took the top --max unverified records by score before dropping fresh cached URLs, so the same already-checked records filled the quota on every run and lower-ranked green records were never checked. 848 green records (763 CPUs) had never had a citation checked and could not be promoted. Refs #98 Refs GetTechAPI/TechAPI#297
Merged
3 tasks done
Seungpyo1007
added a commit
to GetTechAPI/TechAPI
that referenced
this pull request
Sep 27, 2026
…ource These records were green but never had their own citation checked (check-urls frontier and dedupe bugs, fixed in GetTechAPI/TechEngine#104 and #105). Each now has a live Tier-1/Tier-2 citation; cpubenchmark.net returns 404 for unknown or near-miss CPU names, so a live page confirms the exact SKU. Refs #296
5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
check-urlscapped the frontier (the top--maxunverified records by score) before removing records whose citations were already freshly cached. The same high-scoring, already-checked records (dead links or non-promotable) filled the quota on every run, so lower-ranked records were never checked.Measured on TechAPI develop: 848 green records had never had a T1/T2 citation checked (763 CPUs, 50 GPUs, 29 SoCs, and a few tablets, watches and laptops), so promotion could never reach them.
The fix drops fully cached records before capping (
--recheckkeeps the old behavior). A regression test covers the starvation case and fails on the old code.Validation: ruff, mypy, full pytest.
Refs #98
Refs GetTechAPI/TechAPI#297