feat(services): derive the Apdex threshold from a detected app kind - #439
Open
Makisuo wants to merge 3 commits into
Open
feat(services): derive the Apdex threshold from a detected app kind#439Makisuo wants to merge 3 commits into
Makisuo wants to merge 3 commits into
Conversation
Apdex on the service detail pane was scored against a fixed T = 500 ms for
every service. That is a target for a backend API and meaningless for a
browser app, where an entry span is a request from a device on someone's
home wifi — such a service reads as permanently frustrated and the chart
stops carrying signal.
Detection. `service_platforms_hourly` already answered "where does this
service run"; it could not answer "what kind of app is this", because the
only signal it carried for that was `maple.sdk.type`, present solely on
services using a Maple SDK. Migration 0015 adds the vendor-neutral markers
(`telemetry.sdk.language`, `browser.platform`, `device.type`) so a customer
on vanilla OTel browser JS is classified too. `classifyServiceAppKind`
resolves browser | mobile | backend | unknown, checking browser first: a
browser app can carry `cloud.provider` from a CDN or a `k8s.*` leak from a
gateway it was proxied through, while a server never reports
`browser.platform`.
Two things this surfaced that were already broken:
- Maple's own browser SDK writes `maple.sdk.type = "browser"` but the
platform classifier only matched `"client"`, so every browser service
was classifying as `unknown`.
- The service detail page never fetched platform data at all; the
standalone `servicePlatforms` handler had no caller.
Threshold. browser 2500 ms (the Core Web Vitals "good" LCP boundary, which
puts the frustrated line at 4T = 10 s), mobile 1000 ms, everything else
unchanged at 500 ms.
`canUseAnnualServiceOverview` requires `apdexThresholdMs === 500`, and the
Overview chart is a single `allMetrics` request — threading a browser
target through it would drop throughput, latency AND error rate onto the
30-day raw path for the sake of one series. So `payload.timeseries` is
forwarded untouched and Apdex is re-scored by a second, narrower query,
only when T differs from the default.
That override reads `service_overview_spans` (30-day TTL) while the rest of
the chart reaches a year back, so on a longer range its early buckets have
no score. `apdexScore` widens to `number | null` and those buckets render
as gaps: carrying the 500 ms number through would silently mix two
thresholds in one series, and zero would draw a crater reading as "every
user was frustrated".
Migration 0015 is `requiredForIngest: false` — `service_platforms_hourly`
is filled by a materialized view, never by a native INSERT, so a BYO
cluster that has not applied it keeps ingesting correctly and simply
classifies its services exactly as it does today. There is no backfill:
`max()` over the viewed window means one hour of fresh telemetry classifies
a service, and the table's one `sum` column would double-count if
re-inserted.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Apdex on the service detail pane is scored against a fixed
T = 500 msfor every service. That's a reasonable target for a backend API and a meaningless one for a browser app, where an entry span is a request made from a device on someone's home wifi — a "web page" service reads as permanently frustrated and the chart stops carrying signal.What changed
Detection.
service_platforms_hourlyalready answered where a service runs (k8s / cloudflare / lambda). It could not answer what kind of app it is, because the only signal it carried for that wasmaple.sdk.type— present solely on services instrumented with a Maple SDK. Migration0015adds the vendor-neutral markers so a customer on vanilla OTel browser JS is covered too:TelemetrySdkLanguagetelemetry.sdk.language(webjs= OTel browser SDK)BrowserPlatformbrowser.platform(per semconv, only ever set in a browser)DeviceTypedevice.typeclassifyServiceAppKindresolvesbrowser | mobile | backend | unknown. Browser is checked first on purpose: a browser app can carrycloud.providerfrom a CDN or ak8s.*leak from an OTel gateway it was proxied through, and neither makes it a backend — while the reverse mistake is impossible, since a server never reportsbrowser.platform.Two pre-existing bugs this surfaced:
maple.sdk.type = "browser"(packages/browser/src/tracing.ts:86) buttoServicePlatformRowonly matched"client"— every browser service was classifying asunknown.servicePlatformshandler had no caller.Threshold.
browser2500 ms (the Core Web Vitals "good" LCP boundary, so the frustrated line at4Tlands at 10 s),mobile1000 ms, everything else unchanged at 500 ms. One table in@maple/domain/service-app-kind.UI. An app-kind badge next to the service title (renders nothing for
unknown— a badge saying the product can't tell is worse than no badge), and the Apdex card header states the active target (Target < 2.5s) via the metrics grid's existingheaderValueslot. A score is uninterpretable without theTthat produced it.Reviewer notes
The load-bearing decision.
canUseAnnualServiceOverviewrequiresapdexThresholdMs === 500, and the Overview chart is a singleallMetricsrequest. Threading a browser target through it would knock throughput, latency, and error rate off the one-year rollup and onto the 30-day raw path for the sake of one series. Sopayload.timeseriesis forwarded untouched and Apdex is re-scored by a second, narrower query — and only whenTdiffers from the default.ch.test.tsasserts that fork directly.Known limitation, surfaced rather than papered over. The override reads
service_overview_spans(30-day TTL) while the rest of the chart reaches a year back, so on a longer range a browser service's early Apdex buckets have no score.apdexScorewidens tonumber | nulland those buckets render as gaps. Carrying the 500 ms number through would silently mix two thresholds in one series; zero would draw a crater reading as "every user was frustrated".Migration safety.
0015isrequiredForIngest: false:service_platforms_hourlyis filled by a materialized view, never by a native INSERT, so a BYO-ClickHouse cluster that hasn't applied it keeps ingesting correctly and simply classifies its services exactly as it does today. Gating ingest here would route every BYO org back to managed over a display-only classification. There is deliberately no backfill —max()over the viewed window means one hour of fresh telemetry classifies a service, and the table's singlesumcolumn (SpanCount) would double-count if re-inserted.The local-store schema gate required the full v4 → v5 ceremony: frozen
local-schema-v5.sqlsnapshot, appended identity, and av4-to-v5-service-app-kindmigration module. Unlike v3 → v4 the bootstrap pass alone isn't enough — the table already exists, so itsCREATE TABLE IF NOT EXISTSis a no-op and the ALTERs have to run first.Out of scope:
top-operations.ts(MCP/CLI path) and the alert-rule Apdex default keep their hardcoded 500 ms. Worth a follow-up now thatappKindexists, but changing an alert'sTsilently re-scores live rules.Verification
bun typecheckclean across all 37 workspace tasks.bun run clickhouse:schema:checkandbun run tinybird:manifest:checkgreen, including the local-schema manifest gate at v5.webjs, mobile, k8s, cloudflare, all-empty →unknown, plus the browser-beats-infrastructure precedence), the migration's shape, the rollup-guard fork, andmergeApdexOverride's null semantics.🤖 Generated with Claude Code
Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is disabled.