Skip to content

fix: declare ai_product in the v2 gateway properties blob - #1217

Draft
posthog[bot] wants to merge 1 commit into
mainfrom
posthog-self-driving/fixgateway-declare-ai_product-so-wizard-602386
Draft

fix: declare ai_product in the v2 gateway properties blob#1217
posthog[bot] wants to merge 1 commit into
mainfrom
posthog-self-driving/fixgateway-declare-ai_product-so-wizard-602386

Conversation

@posthog

@posthog posthog Bot commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

Problem

  • Every wizard AI generation lost its cost and error label when the v2 gateway cutover finished. About 24k generations a day are now untagged for anything that reads the unprefixed ai_product key: the AI Errors insight and its alert, usage_report.py, personal_spend.py, and the AI observability billing markup calculation.
  • An anomaly alert fired on the resulting fake collapse. Wizard usage never dropped: wizard: started held at 748 runs on the day the series fell.
  • Cause: buildWizardPropertiesBlob never declared ai_product. On the v2 path the gateway owns $ai_product only (values like wizard:posthog-integration, source pinned), and it refuses a legacy product override on a scoped phe_ token, so the plain key nobody sent is now absent.
key who sets it before cutover after cutover after this PR
$ai_product gateway wizard:<program> wizard:<program>
ai_product wizard blob present (legacy path) absent wizard

Changes

  • The v2 properties blob now declares ai_product: 'wizard', so the plain key reaches every consumer again. This is the same fix warehouse-sources-admin and SherlockHog applied after the same cutover.
  • One line, one seed value in the blob builder. Flags, team_id, and metadata keys are untouched.

Note

This restores the label at the source. The consumer side is a separate repository: usage_report.py, personal_spend.py, usePosthogAIBillingCalculations.ts, and the AI Errors insight should still coalesce $ai_product with ai_product and match on a prefix, because the pinned value carries a program suffix. Until they do, historical rows from the untagged window stay unattributed.

Test plan

  • src/lib/__tests__/gateway-session.test.ts asserts the exact blob shape, so it now pins ai_product.
  • Full suite green: 2638 tests, 162 files. pnpm lint reports 0 errors.
  • Verified the legacy header path is unchanged: buildAgentEnv and buildGatewayHeaders only read the blob on edition === 'v2'.

LLM context

  • Written by PostHog Desktop from the linked inbox report. Checked the other places the wizard emits $ai_generation: src/lib/agent/aio-capture.ts posts into the user's own project behind the --capture-aio dev flag, so it is out of scope and was left alone.

Created with PostHog Desktop from this inbox report.

The v2 gateway pins only `$ai_product` (as `wizard:<program>`) and refuses a
legacy product override on a scoped `phe_` token. The blob never declared the
unprefixed `ai_product`, so after the cutover every wizard generation lost the
label that cost and error attribution reads.

Generated-By: PostHog Desktop
Task-Id: 98a6a2c9-a3a9-45ef-9cdb-b0702060daaa
@posthog

posthog Bot commented Sep 7, 2026

Copy link
Copy Markdown
Contributor Author

🦔 PostHog Review reviewed this pull request

Nothing worth raising this time, so here's a calming picture instead:

A happy dog on a sunny path

@github-actions

github-actions Bot commented Sep 7, 2026

Copy link
Copy Markdown

🧙 Wizard CI

Run the Wizard CI and test your changes against wizard-workbench example apps by replying with a GitHub comment using one of the following commands:

Test all apps:

  • /wizard-ci all

Test all apps in a directory:

  • /wizard-ci ai-observability
  • /wizard-ci basic-integration
  • /wizard-ci mcp-analytics
  • /wizard-ci replay-vision
  • /wizard-ci revenue
  • /wizard-ci self-driving
  • /wizard-ci warehouse
  • /wizard-ci warehouse-seeded

Test an individual app:

  • /wizard-ci ai-observability/anthropic
  • /wizard-ci ai-observability/google-adk
  • /wizard-ci ai-observability/groq
Show more apps
  • /wizard-ci ai-observability/manual-capture
  • /wizard-ci ai-observability/openai
  • /wizard-ci ai-observability/openai-agents
  • /wizard-ci ai-observability/opentelemetry
  • /wizard-ci ai-observability/vercel-ai
  • /wizard-ci basic-integration/android
  • /wizard-ci basic-integration/angular
  • /wizard-ci basic-integration/astro
  • /wizard-ci basic-integration/django
  • /wizard-ci basic-integration/fastapi
  • /wizard-ci basic-integration/flask
  • /wizard-ci basic-integration/flutter
  • /wizard-ci basic-integration/javascript-node
  • /wizard-ci basic-integration/javascript-web
  • /wizard-ci basic-integration/laravel
  • /wizard-ci basic-integration/next-js
  • /wizard-ci basic-integration/nuxt
  • /wizard-ci basic-integration/python
  • /wizard-ci basic-integration/rails
  • /wizard-ci basic-integration/react-native
  • /wizard-ci basic-integration/react-router
  • /wizard-ci basic-integration/sveltekit
  • /wizard-ci basic-integration/swift
  • /wizard-ci basic-integration/tanstack-router
  • /wizard-ci basic-integration/tanstack-start
  • /wizard-ci basic-integration/vue
  • /wizard-ci mcp-analytics/custom-dispatcher
  • /wizard-ci mcp-analytics/typescript-sdk
  • /wizard-ci replay-vision/javascript-node
  • /wizard-ci replay-vision/next-js
  • /wizard-ci replay-vision/react-vite
  • /wizard-ci revenue/stripe
  • /wizard-ci self-driving/astro
  • /wizard-ci self-driving/fastapi
  • /wizard-ci self-driving/nuxt
  • /wizard-ci self-driving/react-router
  • /wizard-ci self-driving/sveltekit
  • /wizard-ci warehouse/monorepo-env
  • /wizard-ci warehouse/multi-source-next
  • /wizard-ci warehouse/stripe-node
  • /wizard-ci warehouse/zero-source
  • /wizard-ci warehouse-seeded/next-stripe
  • /wizard-ci warehouse-seeded/next-stripe-declined

Test against a Context Mill branch:

  • /wizard-ci all context-mill:my-branch

Add context-mill:<branch> to any command above to pin the Context Mill branch. It defaults to main.

Results will be posted here when complete.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants