Find a year of buyers from the words they use, and keep the threads that rank - #1
Open
kev1n wants to merge 15 commits into
Open
Find a year of buyers from the words they use, and keep the threads that rank#1kev1n wants to merge 15 commits into
kev1n wants to merge 15 commits into
Conversation
The judgement call spends most of its tokens rejecting posts no product would ever qualify: sellers announcing what they built, people answering others, and threads where nobody wants anything. Nothing about that reading is product-specific, so every project watching a post was paying to reach the same conclusion again. One small model call now reads each post before any product is considered, saying who is speaking and whether they are looking for something, and only a buyer who is asking reaches the judge. The reading is stored beside the post rather than beside a verdict, so the first project to see a post pays for all of them, and it is read again only when the post's own words change. The reading claims nothing it did not see: it becomes an assessment with every score left null, and the existing gates settle the decision from it, so a rejection here follows exactly the rules a rejection by the judge follows. A post the model could not read goes to the judge as it always did, because an outage upstream must cost money and never a lead. Measured over the 540 judged posts replayed in the embed test, requiring a buyer who is asking cut 55 to 78% of the rejections on all five products and lost no lead on any of them, and it agreed with the judge on every post the judge called a seller and every post it qualified. SCORER_VERSION moves, so each project's next scan judges its candidates again under the gate. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…hat rank Backfill was a 30-day poll of a subreddit list, which is the wrong tool for a demand that appears once per city subreddit per quarter. It is now a one-time year-window keyword sweep at project creation, and the threads it finds survive long enough to show. - reddit.search already returns selftext; store it instead of re-buying the body through reddit.post. The clean run made zero reddit.post calls. - Walk one search per constraint and ask each inferred phrasing in plain words. The combined query stopped at Reddit's bounded listing and lost most buyers. - Route instead of only rejecting: a thread where nobody asks but the product plainly fits becomes a "Worth a comment" lead, beside the unchanged buyer axis. - Pin any post a lead or an SEO row references, so retention keeps the ranking threads it used to delete at 30 days. - Infer phrasings by situation, not by rewording one sentence. - A failed backfill retries one scan interval later, and re-persists what it found before crediting it, so a concurrent retention run cannot orphan it. Measured on a fresh HotelsAllow project: 86 walks, 2,203 posts, 37 leads, 41 Google-sourced ranking threads on the SEO tab, gate green at 278 tests. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…rd a need Three defects the HotelsAllow quality check proved, each with a test that fails without the fix. The walk stopped on the first page that carried nothing new. Reddit's relevance sort returns sparse pages in the middle of a listing, so one of them truncated a whole query: `(hotel OR hotels) AND "under 21"` returned pages of 2, 6, 6, 1, 0 and stopped at 16 posts, where the same call live returns six full pages and 150 posts holding 29 of the 48 buyers that run missed. A listing is over when Reddit stops handing out a cursor, or hands back the one just followed. A backfill now reuses no cached search. A retry exists because the first attempt was truncated or died, and a cached page from that attempt hands the retry the same truncated listing: the 2026-09-10 run served all 532 of its searches from the cache of earlier failures and never reached Reddit. The judge treated a word from the product's own vocabulary as a need. The only false positive in 37 leads was a 20-year-old offering to cover a stranger's concert ticket and hotel, "looking for 18+ M or F", read as an under-21 check-in problem. It now has to name the job the matching words actually describe. A query is also walked once however many compiled keywords produce it, because without the cache four near-identical keywords would each buy the same listing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A backfill's walks are independent of each other and are nearly all waiting on Reddit, but they ran strictly one at a time. Walking the full listings that way put the first sweep of a project at over an hour before the feed had anything in it, which is the whole first experience. They now run the same number at a time the reading pass does. That number moves to constants.ts as its one owner, with the measurement it rests on, instead of living privately in reading.ts. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A transient OpenRouter disconnection on triage batch 48 of 51 threw away every batch before it and killed the run, for the third time in two days. A batch the model never answered now means what a batch it answered with nothing already meant: those candidates are unread, never rejected, so the sweep finishes and the next run judges them. A failed scoring batch loses its own ten and no more. Both model phases also run their batches at once rather than one after another, under the same concurrency the reads and the walks use. That constant and its worker pool now live in constants.ts as their one owner. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Kevin, on the re-run: "There's so many false positives ... and even now, taking 22 minutes is kinda unacceptable from a first user perspective." The post he quoted was never a lead. Its evaluation is `review`, fit 2, and the judge's own reason says no check-in need is stated. The feed put it in front of him anyway, because the stream merged the held pile with the leads and sorted the lot by date: four held cards, warm-badged "Maybe a buyer", opened the all-time feed above the first real lead. - The stream carries leads only, in the feed query's own order: score, then how recently the need was posted. Freshness is already half of that score, so re-sorting by date on top of it threw fit and intent away and opened the feed with whatever was newest. - Held candidates get their own shut section under the feed, labelled for what they are. - Model calls get their own concurrency, separate from Reddit's. Measured against OpenRouter with the real prompt: 30 batches take 279s ten at a time, 136.5s thirty at a time, and 60 batches take 147.8s sixty at a time, with no failure and no per-call slowdown at any arm. - Reasoning effort is low. The endpoint refuses to turn reasoning off at all, but over 100 already-judged posts low effort agrees with the shipped run on 96 of 100, exactly as the default does, in a third of the wall clock and well under half the output tokens. - Triage's free-text reason is gone. Nothing read it. - The backfill commits each batch of verdicts as it lands, so the feed fills while the sweep is still running, and writes anything uncommitted at the end. - Posts are upserted in id order. Two concurrent walks landing on the same post in different orders deadlocked and killed a sweep. Same project, nothing cached: 22 minutes becomes 3.5, 4,295 posts found becomes 4,786, and the all-time feed opens on a buyer.
- `main` had no `min-w-0`, so it could not shrink below its widest content: one long Reddit body sized the whole app to 5,246px inside a 1,440px window, every card ran off the right edge with its score and actions off screen, and the people strip's own horizontal scroll never engaged because its parent had already grown to fit it. Measured after: 1,440px on all seven app pages, and the strip scrolls itself. - The rail and the header are sticky. The rail is how you leave the page you are on, and it has to be reachable from the bottom of a feed of eighty leads. The header stays with it, because a rail pinned to the top edge of a header that had scrolled away would float. - A header height token, so the rail's offset and the header's own height are one value. - A long project name no longer sits under the switcher's own select arrow.
An SEO refresh on a project with no problem phrasings returned before it booked its successor, so the one job a new project got was also its last. Nothing queued insights except the button on its own page, and nothing queued a competitor scan except the button on its own page, so a project whose owner never pressed either had no themes and no rivals, ever. A scan and a year sweep now book the grouping when they wrote leads, boot seeds both missing kinds for projects made before this, and a test reads every source file for an em or en dash.
deleteExpiredPosts drops every post past the retention window across the whole database, so a 40-day-old fixture post vanished mid-test whenever tests/retention.test.ts ran beside this file. The report counts on judged_at and first_seen_at and never reads the post's own age.
…rose The leads page gave every lead a full card: the judge's reason, three 0-4 meters, the reason codes, and a 320-character slice of the post. Eighty of those is a debug log you scroll, and the one thing you came for - what the person actually asked - was the part that got cut. It is a workspace now. The list column carries only what picks a row out of eighty: face, ask, community, age, score. The detail column is pinned under the header and holds the whole post, one labelled line saying why it is a lead, and a ledger rail of who asked, where, what the thread did and how it scored. Held candidates are the second group in the same list, still shut until asked for, and open into the same pane with their verdict and their reason codes in plain words. Nothing in the rail is estimated: a karma, an account age, a weekly-active count or a judgement the scan never made reads as a dash.
The Leads header read "84 cleared the bar" above a list of 8, because scanReport bounded on lead_evaluations.judged_at and candidate_sources.first_seen_at - when we looked - while the feed bounds on the need date, coalesce(reddit_comments.created_at, reddit_posts.created_at). A backfill judges old threads today, so the two populations diverge by hundreds of posts and nothing on the page says so. scanReport now reuses the feed's own newerThan, joining reddit_posts and left-joining reddit_comments so a comment verdict is dated by its comment and not by the thread it sits in. The candidate count has no comment dimension, so it bounds on the post's own date. The fixture posts are written at today's date, given a lead row, and only then aged, because deleteExpiredPosts drops every unreferenced post past the retention window across the whole database.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Four commits: one product-agnostic reading of every post before judging, the backfill rebuild, the recall and precision fixes the quality check proved, and the parallelism.
Why. Backfill was a 30-day poll of a subreddit list. For HotelsAllow the demand is one post per city subreddit per quarter across 38 city subs, so a 30-day poll of 10 subreddits saw 3 buyers where a year-window keyword sweep sees dozens.
What changed
reddit.searchalready returnsselftext; store it instead of re-buying the body throughreddit.post. The clean run made zeroreddit.postcalls.(hotel OR hotels) AND "under 21"to 16 posts where the listing holds 150.Proof, same project run before and after the fixes
Hand-judged precision on the 79: one clear false positive, two kind mislabels, three crosspost duplicates. Gate green at 286 tests.
Still open. The title triage rejects the largest context thread in the corpus (1,121 upvotes, 146 comments), and the all-time feed opens on held review cards because the stream sorts by date.
🤖 Generated with Claude Code