Repository navigation
Trending: daily term rollup and a public /api/trending - #271
Merged
Merged
Conversation
Two weeks of feed_items cannot be grouped on a request, so the poller rolls each finished UTC day into trend_days and trend_terms: a one in four sample of English blogs and news, counting feeds per term so a firehose is one voice. The API ranks this week against last by rate, and folds overlapping terms into one subject. The job sits outside the catch-up gate because prod runs CRAWL_CATCHUP=1. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Crawlproof's trend source (SameBrain) returns no topics, so this gives it a real one. Grouping two weeks of items on a request does not finish, so the poller rolls each finished day into
trend_days/trend_terms, andGET /api/trendingranks this week against last by rate.A 1 in 4 sample of English blog and news titles, counted per feed so a firehose is one voice. Overlapping terms fold into one subject. The job is outside the catch-up gate because prod runs
CRAWL_CATCHUP=1. The migration applies at poller boot.🤖 Generated with Claude Code