BOMGuard — self-healing component change intelligence

Dependabot for electronic components. A hardware repo commits its BOM; BOMGuard watches the manufacturers' PCN/End-of-Life feeds, and when a chip in your shipping product gets discontinued it opens an evidence-backed GitHub issue before the last-time-buy deadline passes — and repairs its own scrapers when the manufacturer site changes under it.

Built for the Into the Scrape-Verse hackathon with Bright Data Scraper Studio (custom collectors, same-ID self-healing).

---

The problem

A missed discontinuation notice can strand a physical product on an unobtainable component. Enterprise BOM-monitoring SaaS charges ~$250+/mo and hides the source. Small hardware teams get nothing. Manufacturer pages are heterogeneous, frequently redesigned, and break scrapers quietly — the worst kind of failure, because nobody notices until the order fails.

BOMGuard is open-source, GitHub-native, and treats the manufacturer page as the source of truth — permanently frozen at the moment of collection via content-hashed, WARC-referenced evidence.

The product loop

Bright Data Collector (custom, c_* ID)
        │
        ▼
Raw immutable observation ──────────► WARC / content-hash evidence
        │
        ▼
Typed source adapter (NXP / onsemi / Microchip)
        │
        ▼
Data-contract validator
   │ healthy               │ unhealthy
   ▼                       ▼
Normalize              Diagnose failure ("affected_mpn null in 92%…")
   │                       │
   │             Same-ID heal (strategy battery / bdata scraper heal)
   │                       │
   │                  Preview + revalidate
   │                       │
   ▼                       ▼
Last-known-good store (frozen until a passing run)
        │
        ▼
 Domain matching (conservative MPN + packaging aliases)
        │
        ▼
 Change detection → GitHub issue (evidence-backed) / dashboard

Quickstart (offline, uses real captured manufacturer pages)

uv sync
bomguard init .                    # scaffold config + example boards + fixtures
bomguard fetch                     # collect + validate NXP & onsemi (fixtures)
bomguard scan --issues             # match BOMs, open deduped issues
bomguard serve                     # http://127.0.0.1:8000 dashboard

Full demo including the break-and-heal story:

scripts/demo.sh

The self-healing loop (the engineering core)

if not rows: heal() is not enough — a redesign can keep the row count while every affected_mpn goes null. BOMGuard validates collector output against a per-source data contract:

min row count  ·  required-field null rate  ·  valid notice-ID pattern
valid MPN rate · date-parse success rate     ·  duplicate notice/MPN rate
source-URL validity · historical row-count deviation · unexpected schema fields

When the contract fails:

1. Diagnose — one precise sentence, e.g. affected_mpn null/invalid in 100% of records after the table moved under a details accordion.
2. Heal on the same Collector ID — locally by walking the extraction strategy battery (documented selector → structural fallback → regex row scrape) over the same broken bytes; in production by invoking bdata scraper heal <c_*id> "<diagnosis>" (Bright Data's AI refactor keeps the ID) then approve.
3. Preview + revalidate the proposed revision before promotion.
4. Promote only a passing revision; bump collector_version.
5. Preserve last-known-good — the old dataset is frozen (never deleted) so consumers keep working until a passing run.
6. Record everything in an append-only healing ledger (old/new revision, diagnosis, content hashes, preview result, timestamps).

State machine: HEALTHY → CONTRACT_FAILED → DIAGNOSING → HEALING → PREVIEWING → PROMOTED (any failure → UNHEALTHY, LKG preserved). Covered by tests (tests/test_heal.py).

The 90-second demo story

1. bomguard fetch — dashboards go green (37 components monitored, 0 alerts).
2. bomguard scan --issues — one BOM line is a real NXP part about to be EOL'd: MCIMX31LCVKN5D — Last-time-buy 2026-10-09; another has a manufacturer replacement (NTS0304EPWJ → NTS0302JKZ, flagged as requiring engineering qualification). Issues open with the full evidence.
3. The site changes (?broken=1) — contract fails, strategy battery heals, same collector, promoted 1.0 → 1.1, ledger shows the preview.
4. Worse break (?broken=2) — collectible data gone; collector goes UNHEALTHY with the famous diagnosis, last-known-good still served.
5. Site recovers — green again, no consumer code changed.
6. "The website changed. The product did not break."

How Bright Data Scraper Studio is used

• Custom collectors created with the bdata CLI: bdata scraper create <url> "<field description>" → c_* Collector ID (exact prompts for NXP / onsemi / Microchip in scraper-studio/README.md).
• Run: bdata scraper run <c_*id> <url> → structured rows, validated by the same contracts as our local twins.
• Self-healing: bdata scraper heal <c_*id> "<diagnosis>" + approve (or --auto-approve) — the repair keeps the same Collector ID.
• The adapter lives in bomguard/adapters/brightdata.py; the runner path in bomguard/runner.py. bdata login (OAuth once) or BRIGHTDATA_API_KEY (headless CI). WARC snapshots back every alert as evidence.

Evidence matrix

Requirement	Proof in this repo
Custom Scraper Studio collector	scraper-studio/README.md — prompts + CLI flow; adapter tested against real CLI output shapes
Real run (offline twin)	examples/structured-output/sample.json — real NXP + onsemi captures
Live Scraper Studio run	✓ NXP collector c_mt3z272b1qdvxauw57 ran the real 202606009DN EOL (105 parts, LTB 2026-10-09) + 202409026DN, contract passed, 4 alerts → issues (examples/structured-output/nxp-live-run.json)
Live same-ID heal	✓ real bdata scraper heal + --auto-approve --auto-save completed on the same collector (template saved; envelope in examples/structured-output/nxp-heal-envelope.json) — surfaced in the ledger via bomguard heal-live
Downstream use	examples/control-board/.issues/*.md + dashboard — EOL with 2026-10-09 LTB (live countdown)
Failure detected	tests/test_validate.py, tests/test_heal.py; ledger diagnosis lines
Self-heal	strategy battery → PROMOTED 1.0 → 1.1, same collector ID (ledger)
Recovery	re-run passes, downstream schema unchanged
Evidence (WARC)	content-addressed WARC snapshots of every collected page under evidence/, referenced by all notices and alerts
Public-data compliance	public manufacturer pages only; no login/paywall/personal/government data
Reproducibility	uv sync && bomguard init . … one-command local demo; 82 tests
Live Bright Data run (pending creds)	bdata login once → set collector_id in config → bomguard fetch

Repository layout

bomguard/            core package
  models.py          typed schema (notice, bom, contract, healing ledger)
  normalize.py       conservative MPN canonicalization + packaging aliases
  validate.py        per-source data contracts (the failure detector)
  heal.py            healing state machine + strategy battery
  store.py           SQLite: last-known-good, observations, ledger, alerts
  intercept.py       fixture / httpx / browser / brightdata transports
  collectors/        NXP (grounded), onsemi (grounded), Microchip, chaos
  adapters/          Bright Data bdata CLI adapter
  runner.py          transport dispatch (local HTML vs Scraper Studio JSON)
  rows.py            Scraper Studio rows -> typed notices (field-map aware)
  issue.py           GitHub issue providers (API + offline mock)
  server.py          dashboard (health, alerts, notices, ledger)
  chaos_server.py    controlled breakable NXP-shaped page for the demo
  cli.py             bomguard fetch/scan/…/ledger/serve/chaos
fixtures/raw/        real pages captured from nxp.com and onsemi.com (evidence)
scraper-studio/      collector prompts + integration notes
examples/            control-board demo + structured output sample
.github/             nightly GitHub Action (fetch → self-heal → issues)
tests/               82 tests (normalize, validate, heal, match, parsers, service, collectors)

Design decisions worth defending

• Conservative matching, no LLM part equivalence. Canonical MPN equality plus documented packaging variants (tape/reel) and explicit config aliases. Manufacturer-listed replacements are candidates requiring qualification — NXP itself disclaims form/fit/function equivalence.
• The page is the source of truth. Every alert carries source_url, source_content_hash, warc_reference, collector id/version — reproducible evidence, not a JSON-only claim.
• Local twins of the collectors make the whole pipeline testable without the platform, and make the trip-wire (contract) and the repair (battery / heal API) independently verifiable.

Roadmap / honest caveats

• Live NXP collector runs are proven; onsemi and Microchip collectors are defined (prompts + local twins + Bright Data reachability verified for Microchip) — their AI generation is slower against heavy/JS pages, so the on-repo collector_ids for those two are set once generation completes.
• Every collected page is frozen to a content-addressed WARC snapshot (evidence/, BOMGUARD_EVIDENCE_DIR) and every notice references it; a live remote WARC fetch remains a one-flag change on bdata scraper run.
• bomguard heal-live drives a real same-ID Bright Data heal on demand and records it in the ledger; the automatic contract-failure heal does the same (the runner already self-heals without any flag).

AI-use disclosure

Built with an AI coding assistant — DeepSeek V4 Flash (model id 0731) — pair-programming the implementation (scraper collectors, orchestration, tests). The architecture, data contracts, and every test expectation are specified, reviewed and verified by a human; the author can explain all of it. Original work completed during the hackathon window per the event rules.

License

MIT