switchcheck

A price/spec aggregator for mechanical keyboards and mice sold by gaming-gear vendors in Nepal. Search once, it checks 10 vendors in parallel, and shows you real prices, real stock status, real listing images, and (where the data supports it) switch type and spec comparisons — no invented or approximated data anywhere in the pipeline.

Built as an incremental, session-by-session project. Every adapter and every extraction heuristic below is either confirmed against a real live fetch or explicitly flagged as unverified/a stub — see the comments in each file for the actual state, since that's more reliable than this README going stale.

What it does

• Search a product name (e.g. "aula f75"), and it fans out to every vendor adapter in parallel (Promise.allSettled, so one vendor failing doesn't break the others).
• Overview — how many vendors were checked, how many listings came back, which vendors currently have it in stock.
• Price Compare — every listing as a card (image, name, vendor, price, stock), sortable by price/vendor, with a vendor filter to exclude sellers you don't want to see. Cheapest listing is flagged; unverified 3rd-party Daraz sellers are flagged separately.
• Compare Mode — pick 2–3 specific listings and view them side by side: image, price, stock, switch type, and recognized features. Same vendor filter + sort as Price Compare.
• Feature Comparison — a grid of every listing against spec keywords recognized in its title (wireless/BT/2.4GHz, hot-swap, RGB, hall-effect, layout size, keycap material, mount style, NKRO, etc).
• Sound Test — pick a switch name (auto-detected from listing titles) and:
  • Look up a real switch-sound comparison video (YouTube, via a couple of known channels), embedded inline.
  • Try to resolve a real, playable Mechvibes sound pack from GitHub and render a virtual keyboard — click a key, hear the actual mapped sound file, streamed live from raw.githubusercontent.com. Personal-use playback only; nothing is downloaded, cached, or re-hosted. See Known limitations — this one depends on GitHub's free API quota.
• Autocomplete — as you type, suggests real product names seen in past searches (own lightweight in-memory/on-disk history, not a vendor API).

Architecture

├── server.js           # plain Node http server, no framework — routes below
├── search.js           # searchAll(): fan out to every adapter, cache, aggregate
├── adapters/            # one file per vendor — each exports { name, search(query) }
├── lib/
│   ├── fetchHtml.js          # plain fetch + UA header
│   ├── fetchRendered.js      # headless Puppeteer fetch, for JS-rendered vendors (Daraz)
│   ├── paginate.js           # walks a vendor's ?page=N until it stops matching
│   ├── parsePrice.js         # normalizes vendor price text -> a comparable number
│   ├── extractImage.js       # shared lazy-load-aware image URL extraction
│   ├── cache.js              # on-disk cache, wiped on every server boot
│   ├── autocomplete.js       # records/suggests from real past search results
│   ├── crossVendorSwitch.js  # mines switch names out of listing titles (all vendors)
│   ├── featureExtract.js     # mines spec keywords out of listing titles
│   ├── switchExtract.js      # (legacy) per-page switch-option scrape — see below
│   ├── soundLookup.js        # finds a real switch-sound-test video
│   └── mechvibesLookup.js    # resolves a real, playable Mechvibes pack from GitHub
└── public/
    └── index.html       # entire frontend — vanilla JS, no build step, no framework

No React/Vue/bundler — public/index.html is a single static file with inline <style>/<script>, served as-is by server.js.

Vendors

Vendor	Platform	Status
Backseat Gaming	WooCommerce	✅ live, category-page scrape
Exortstore	custom "Stackma" platform	✅ live
RapidoTech	WooCommerce	✅ live
BigByte	WooCommerce	✅ live, largest catalog
Mudita	Magento	✅ live
GadgetHouse	WooCommerce/Woodmart	✅ live
PCModNepal	WooCommerce	✅ live (uses native search, category was empty)
S3Tech	WooCommerce	✅ live
Daraz	Next.js/CSR marketplace	✅ live, needs a headless browser (see below)
iiTi (ITTI Computer World)	Next.js SPA	❌ stub — product data loads via a client-side API call this adapter can't see yet; needs the real endpoint identified via devtools

Every adapter's file header documents exactly what was confirmed live vs. assumed, and what would need re-checking if a vendor changes their site.

How switch/feature detection works

There's no per-product detail-page scrape for this — no vendor site reliably exposes switch/spec data in a scrapeable dropdown or spec table across all 10 of them (confirmed broken on Daraz specifically). Instead, lib/crossVendorSwitch.js and lib/featureExtract.js mine the listing titles already fetched for every vendor during the search — titles like "Aula F75 Reaper Switch Hot-swap RGB Keyboard" routinely contain the real switch name and specs directly, and mining them costs zero extra requests.

Rules that matter if you're extending this:

• Word-boundary matched, so a brand name that happens to contain a vocab word as a substring (e.g. "Redragon" contains "red") doesn't false-hit.
• Plain color words (red, black, brown, ...) are ambiguous — a title might mean the switch color or the case/keycap color. They only count as a switch hit if the word "switch" actually appears near them; otherwise they're skipped rather than guessed.
• Adjacent model numbers (F75, K617) are stripped out of the captured switch name.
• If nothing in a title matches, the listing just gets an empty switch/ feature list — nothing is ever invented.

lib/switchExtract.js (the older per-detail-page approach) is left in the codebase as a standalone diagnostic via /api/switches?url=..., but nothing in the current UI calls it.

Sound Test internals

• Video lookup (lib/soundLookup.js): checks a couple of known switch-sound-test YouTube channels for a real matching video; embeds it if found, says so plainly if not.
• Mechvibes pack lookup (lib/mechvibesLookup.js): searches GitHub for a dedicated pack repo first (rarely hits — most real packs are one folder inside a bigger community collection repo, not their own repo), then falls back to matching the switch name against folder names inside a short curated list of known collection repos (dusklinux/wayclick_soundpacks confirmed live). If a real config.json resolves, the frontend renders a virtual keyboard that streams the actual mapped audio file per key from raw.githubusercontent.com — no proxy, no re-hosting, no synthetic fallback sound for unmapped keys.

Known limitations

• GitHub API rate limit: the Mechvibes pack lookup uses GitHub's unauthenticated search + tree APIs, capped at 60 requests/hour and shared across whatever IP the server runs from. This gets exhausted fast in normal use (confirmed live, not a guess). Set a GITHUB_TOKEN env var (a plain personal access token, no scopes needed) before running — this raises the quota substantially and the Sound Test tab starts resolving packs reliably instead of coming back "rate limited."
• Daraz requires Puppeteer: Daraz renders products client-side, so its adapter launches a real headless Chromium via lib/fetchRendered.js. Run npx puppeteer browsers install chrome if it's missing, or set PUPPETEER_SKIP_DOWNLOAD=true during npm install and install Chrome separately if your environment blocks the automatic download.
• iiTi isn't implemented — see the vendor table above.
• Vendor image loading: some vendor-hosted images may not load in the browser (broken icon) depending on that vendor's hotlink/referrer protection, separately from whether the URL was extracted correctly. Not yet root-caused — needs checking against a real broken image's network response, vendor by vendor.
• Cache is wiped on every server boot by design (see server.js) — it's meant to avoid repeat requests within one running session, not to serve stale data across restarts.

Running it

cd scraper
npm install                      # or: PUPPETEER_SKIP_DOWNLOAD=true npm install
export GITHUB_TOKEN=ghp_xxxxxx   # optional but strongly recommended, see above
node server.js

Then open http://localhost:3000.

CLI search (no server, prints raw JSON):

node search.js "aula f75"

Design principles this project holds itself to

• Real data or an honest "not available" — never a fabricated or approximated value. No invented specs, no synthetic sound fallback, no guessed prices.
• Confirmed live, or flagged as unverified. Every adapter and heuristic says which one it is, in its own file, so nothing is silently assumed to work.
• Fail loud enough to fix, not so loud it breaks the page. One vendor failing shows a note; it doesn't take down the whole search.