Skip to content

feat(domain): add the hourly per-service AI vendor discovery rollup - #436

Closed
JeremyFunk wants to merge 1 commit into
ai/06-rust-sql-equivalencefrom
ai/07-service-ai-vendors-rollup
Closed

feat(domain): add the hourly per-service AI vendor discovery rollup#436
JeremyFunk wants to merge 1 commit into
ai/06-rust-sql-equivalencefrom
ai/07-service-ai-vendors-rollup

Conversation

@JeremyFunk

@JeremyFunk JeremyFunk commented Aug 11, 2026

Copy link
Copy Markdown
Collaborator

What this layer contains

The read-path rollup and the last remaining pieces of the write-side v1:

  • service_ai_vendors_hourly and its materialized view — ClickHouse migration 0016, the serviceAiVendorsHourly datasource, materializations.ts, ai-vendors-rollup-sql.ts, and local-store migration v5 → v6 (local schema version 6, frozen local-schema-v6.sql, history entry, manifest check).
  • ai-registry/rollup-enablement.ts and its index.ts export; the 400-day retention tier.
  • ai-vendors-rollup.clickhouse.e2e.test.ts, plus the package.json ch:test entry and the CI step that run it.
  • Regenerated schema artifacts at project revision 097d8372….

Why it is shaped this way

One row per (org, service, vendor, hour), carrying span/eligibility/session-key-state counters plus uniqCombined(12) trace and session states — enough for read-path service pruning, vendor-per-service lookup, session-key health and sampling-exemption suggestions. The MV filters AiVendor != '', which is simultaneously the cost model and the semantics: the HLL states never run on ordinary HTTP or DB traffic, and post-enablement "no rows" therefore means "no AI spans" rather than "not computed". Because it writes synchronously inside the INSERT into traces, this target's part-count and merge-lag alerts are trace-ingestion health, not side-table health.

There is no POPULATE and no backfill: correctness depends on the source rows having been classified, not on the view having existed, so the enablement hour is recorded instead and earlier hours simply do not exist for readers. 0016 is requiredForIngest: false — it touches nothing the gateway inserts, so the ingest gate stays at 15 and no BYO-ClickHouse org is un-readied by a read-path table.

Gates run at this level

Full battery, all green:

  • bun typecheck — 37/37 tasks.
  • bun run clickhouse:schema:check — up to date at revision 097d8372…, lint OK (38 tables, 40 MVs), local manifest at schema v6.
  • packages/domain vitest under CLICKHOUSE_E2E=1 — 554 passed.
  • apps/cli vitest — 430 passed.
  • bun run ch:test — 4 files, 194 passed (including the new rollup reader-contract and parity suite).
  • cargo test --locked (+ TRACE_CAPTURE_DIR) — 136 lib + 75 bin, 0 failures.

Stack integrity gate: at this commit git diff against the verified pre-split implementation is empty — the seven layers reassemble that tree exactly. Re-running bun run clickhouse:schema and bun run tinybird:manifest here reproduces every generated file byte-for-byte.

Dependencies

Top of the stack; requires #435 and everything below it.

🤖 Generated with Claude Code


Stack created with GitHub Stacks CLIGive Feedback 💬


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.

Adds `service_ai_vendors_hourly` and its materialized view (ClickHouse
migration 0016, local-store migration v5 -> v6): one row per (org, service,
vendor, hour) carrying span/eligibility/session-key-state counters plus
uniqCombined(12) trace and session states, feeding read-path service
pruning, vendor-per-service lookup, session-key health and
sampling-exemption suggestions.

The MV filters `AiVendor != ''`, so the HLL states never run on ordinary
HTTP or DB traffic — that filter is both the cost model and the semantics,
since post-enablement "no rows" then means "no AI spans". It writes
synchronously inside the INSERT into `traces`, which makes this target's
part-count and merge-lag trace-ingestion health rather than side-table
health. There is no POPULATE and no backfill: correctness depends on the
source rows having been classified, not on the view having existed, so the
enablement hour is recorded instead and hours before it do not exist for
readers. 0016 is `requiredForIngest: false` — it touches nothing the
gateway inserts, so the ingest gate stays at 15.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@JeremyFunk
JeremyFunk force-pushed the ai/07-service-ai-vendors-rollup branch from 75a72d7 to b37aef0 Compare August 12, 2026 21:05
@JeremyFunk

Copy link
Copy Markdown
Collaborator Author

Superseded by the v2 stack: #451#452#453. The registry-artifact + SQL-equivalence design this stack implemented was dropped in the write-side plan's v2 revision; vendor rules are now plain Rust with the vendored classification fixture as the CI gate.

@github-actions

Copy link
Copy Markdown

🍁 Maple PR preview

Note

Preview resources were removed when this pull request closed.

Final commit b37aef0 · View workflow run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant