fix: match repo urls case-insensitively when linking packages for impact (IN-1420)

[joanagmaia]

Summary

• Repo-to-package linking in the impact pipes joined repos on an exact, case-sensitive URL, so projects whose repo URL differs only by casing (e.g. BerriAI/litellm) got no packages and no impact data.
• Both project_insights_impact_breakdown_copy.pipe and health_score_v2_impact.pipe now join on lower(url).

Impact

• ~2,189 projects gain linked packages. Verified on the live query: berriai-litellm goes from NULL to 2,330 direct dependents / impact 40.6; k8s is unchanged.
• Go/NuGet transitive inflation (cycles in the requirements graph) is out of scope.

Jira: https://linuxfoundation.atlassian.net/browse/IN-1420

[cursor]

PR Summary

Medium Risk
Changes nightly Tinybird COPY logic that drives health impact scores and project impact breakdowns for thousands of projects; scoped to GitHub/GitLab casing but still alters published analytics values.

Overview
Impact materialization pipes were failing to link packages when a project’s repositories.url and repos.url differed only by casing on GitHub or GitLab, leaving impact metrics NULL for those projects.

health_score_v2_impact.pipe and project_insights_impact_breakdown_copy.pipe now match repositories → repos with lower(url) only when domain(url) is github.com or gitlab.com; other hosts still use exact URL equality.

The project impact breakdown pipe also filters out deleted repos (rep.deletedAt IS NULL) and rolls up metrics via a project_packages subquery so the LEFT JOIN chain still emits every project (including those with no linked packages) before joining package metrics.

^{Reviewed by Cursor Bugbot for commit 7ede3d0. Bugbot is set up for automated code reviews on this repo. Configure here.}

[Copilot]

Copilot review overview

🟡 Changes recommended

Unconditional lowercasing can incorrectly merge distinct repositories on case-sensitive hosts.

Review effort: Balanced
Findings: 2

Open (2)

• Avoid case-folding repository paths on case-sensitive hosts · New
• Prevent joining case-distinct repositories on unknown hosts · New

What changed in this PR

Updates Tinybird impact pipelines to match repository URLs case-insensitively when linking packages.

Changes:

• Applies lowercase URL matching to project and health-score impact calculations.

File	Description
project_insights_impact_breakdown_copy.pipe	Makes package-impact URL matching case-insensitive.
health_score_v2_impact.pipe	Makes health-score package linking case-insensitive.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

Copilot review overview

🟡 Changes recommended

Both host checks remain case-sensitive, so mixed-case GitHub/GitLab hostnames still fail to link.

Review effort: Balanced
Findings: 2

Open (2)

• Normalize hostname casing before allowlist matching · New
• Normalize hostname casing before allowlist matching · New

Resolved since last review (2)

• Prevent joining case-distinct repositories on unknown hosts
• Avoid case-folding repository paths on case-sensitive hosts

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit b2e8b05. Configure here.}

[gaspergrom]

LGTM. I checked both join sites against the packageRepos path. The GROUP BY with max() keeps the score stable even if two rows match, and the host restriction keeps other hosts on exact matching. health_score_v2_security, lifecycle and development still join repos on the exact URL, so a follow-up there would keep the scores consistent. lower(domain(...)) would also cover uppercase hosts.

[Copilot]

Copilot review overview

🟢 Approval recommended

The host-scoped normalization correctly addresses the reported casing mismatch without changing other-host semantics.

Review effort: Balanced
Findings: None

Resolved since last review (2)

• Normalize hostname casing before allowlist matching
• Normalize hostname casing before allowlist matching

[Copilot]

Copilot review overview

🟡 Changes recommended

The join incorrectly lowercases self-hosted GitLab repository URLs on only one side, breaking existing exact matches.

Review effort: Balanced
Findings: 2

Open (2)

• Avoid lowercasing self-hosted GitLab paths asymmetrically · New
• Preserve case for self-hosted GitLab URL matching · New

[Copilot]

Copilot review overview

🟡 Changes recommended

The project breakdown query can include soft-deleted case variants and double-count package downloads.

Review effort: Balanced
Findings: 1

Open (1)

• Exclude soft-deleted repositories to prevent duplicated download sums · New

Resolved since last review (2)

• Preserve case for self-hosted GitLab URL matching
• Avoid lowercasing self-hosted GitLab paths asymmetrically

[Copilot]

Copilot review overview

🟢 Approval recommended

The host-scoped normalization and deletion filter address the identified correctness issues without changing matching semantics for other hosts.

Review effort: Balanced
Findings: None

Resolved since last review (1)

• Exclude soft-deleted repositories to prevent duplicated download sums

[Copilot]

Copilot review overview

🟡 Changes recommended

GitLab case-duplicate repo rows can fan out package joins and inflate summed downloads.

Review effort: Balanced
Findings: 1

Open (1)

• Case-folded GitLab join can duplicate repos and inflate download totals · New

[Copilot]

Copilot review overview

🟢 Approval recommended

The URL matching fix is correctly scoped and all prior correctness concerns are addressed.

Review effort: Balanced
Findings: None

Resolved since last review (1)

• Case-folded GitLab join can duplicate repos and inflate download totals

[Copilot]

Copilot review overview

🟢 Approval recommended

The URL matching is correctly host-scoped, and all previous correctness concerns are addressed.

Review effort: Balanced
Findings: None