Skip to content

isManufacturerOwnDomain misses subdomains, flagging a maker's own page as an aggregator #644

Description

@jeffreylouden

What

isManufacturerOwnDomain in scripts/lib/aggregator-domains.ts builds its comparison label from every host part except the TLD, so a subdomain is concatenated in:

const label = host.split(".").slice(0, -1).join("");

tools.splice.com reduces to toolssplice, which never equals the slug splice. The maker's own page reads as somebody else's.

Multi-part TLDs have the same shape (bestservice.co.ukbestserviceco), though no entry currently hits that.

Measured impact

Checked against the whole dataset on 27c6cdda:

Entries with a url 11,258
Flagged aggregator-url 371
Would be exempted by a subdomain-tolerant check 1

The one real case:

data/software/splice-astra.yaml
  manufacturer: splice
  url: https://tools.splice.com/astra

Splice's own product page for Astra, on Splice's own subdomain, is currently treated as an aggregator link.

What it costs today, and what it could cost

Today this is audit noise plus a spurious research entry, not data corruption. splice-astra.yaml has no links: array, so promote-canonical-urls finds no candidate and routes it to researchPool rather than rewriting it.

The rewrite path is latent rather than absent. promote-canonical-urls.ts does:

if (!doc?.url || !isAggregatorUrl(doc.url)) continue;
if (isManufacturerOwnDomain(manufacturer, doc.url)) continue;   // ← the exemption that misses
// ...pickCandidate(...) → promotions.push({ oldUrl, newUrl })

If that entry ever gains a links: array with a title-matching product link, the correct tools.splice.com URL gets rewritten away. Same for any future entry on a maker's subdomain.

Suggested direction

Compare the slug against each host label rather than the concatenation of all of them, ignoring the TLD. That fixes both the subdomain and multi-part-TLD shapes without pulling in a public-suffix list.

Worth deciding deliberately: it makes the exemption more permissive, so a manufacturer whose slug happens to match a label of a genuine aggregator host would newly be exempted. With 371 flagged entries the blast radius is inspectable before merging.

Current state

Both limitations are pinned by tests in scripts/__tests__/aggregator-domains.test.ts, named as known limitations, so a fix will show up there as tests that need updating rather than as a silent behaviour change:

it("does not match across a subdomain (known limitation)")
it("does not match a multi-part TLD (known limitation)")

Found while adding that suite in #638; documented rather than changed at the time, because both dataset:audit and promote-canonical-urls consume this and the call is yours.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugIncorrect or missing data in the catalogscriptsChanges to build/validation scripts

    Type

    No type

    Projects

    Status
    Todo

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions