Skip to content

feat(metrics)!: drop session.id and project.id from metric labels - #124

Open
ShawnZhang31 wants to merge 2 commits into
DEVtheOPS:mainfrom
ShawnZhang31:feat/drop-metric-high-cardinality-labels
Open

ShawnZhang31 wants to merge 2 commits into
DEVtheOPS:mainfrom
ShawnZhang31:feat/drop-metric-high-cardinality-labels

Conversation

@ShawnZhang31

Copy link
Copy Markdown

Description

Two identifiers currently sit on every metric data point: session.id (on all 15 instruments) and project.id (spread into each one via the shared attribute set). Both are unbounded as Prometheus labels, and the failure mode is quiet:

  • Every distinct label combination is its own time series, and a histogram multiplies it by its bucket count — 20 series per combination at the SDK's default boundaries (16 buckets + _sum + _count + _min + _max). At 8 tools × 2 success values, tool.duration alone reaches the SDK's 2000 attribute-set ceiling after ~125 sessions, i.e. ~40,000 series.
  • Past that ceiling the SDK silently merges further combinations into a single otel.metric.overflow series, so the metric degrades with no error anywhere.
  • Cumulative aggregation never evicts, so a long-lived process re-exports every session and project it has ever seen on every export, indefinitely. project.id is derived from the project directory, so it looks bounded interactively but grows without bound in CI, where every checkout can be a fresh path.

After this change the remaining metric labels are bounded dimensions only: model, provider, agent, agent.type, type, tool_name, success, is_subagent, plus whatever the user adds via OPENCODE_SPAN_ATTRIBUTES.

Both identifiers are unchanged on spans and OTLP log events, where per-session and per-project drill-down belongs and high cardinality is acceptable.

The Gauge could not simply lose its label

lines_of_code.total was a Gauge carrying session.id. Dropping the label from a Gauge does not produce a low-cardinality metric — with LastValue aggregation the SDK collapses every session into one attribute set and exports whichever session wrote last, so it would have become silently wrong rather than merely coarse.

It is replaced by an opencode.session.lines_of_code.total Histogram. A histogram of a per-session value also cannot be recorded on session.idle: idle fires once per turn, not once per session, while opencode's session.diff is cumulative for the whole session — so recording there added the running session total once per turn. A two-turn session reaching a cumulative of 25 reported _sum 35 across two observations. The histogram is therefore recorded once when the session ends, via the session.deleted event, which the plugin previously ignored.

Fixing that also fixes the same defect in the gross lines_of_code.count counter, which discarded its per-session diff baseline on every idle and so treated the whole session cumulative as new churn on each later turn.

Net values are additionally attached to the run and session spans as session.total_lines_added / session.total_lines_removed, on both the idle and error paths.

project.id is removed by splitting the shared attribute set: commonAttrs keeps it for spans and log events, and a new metricAttrs — the configured span attributes only — is what every metric call site now spreads. Adding project.id explicitly at each span/log site was the alternative, but that means editing 14 logic-heavy sites instead of 22 mechanical { ...ctx.metricAttrs, ... } spreads.

Type of change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update
  • Refactoring (no functional changes)
  • Chore (dependency updates, etc.)

Checklist

  • I have read the CONTRIBUTING.md document
  • My code follows the style guidelines of this project
  • bun run lint passes with no errors
  • bun run check:jsdoc-coverage passes with no errors
  • bun run typecheck passes with no errors
  • bun test passes with no errors
  • I have added tests that prove my fix is effective or that my feature works
  • I have updated the documentation accordingly
  • My commits follow the Conventional Commits specification

Related issues

None — no related issue was referenced in the repository.

Additional context

Verified against the real SDK, not only the unit-test doubles. Driving a full session lifecycle through a real MeterProvider with an InMemoryMetricExporter yields 15 instruments and 0 data points carrying session.id or project.id. The complete set of metric label keys is now:

agent, is_subagent, model, provider, team, type

session.id and project.id both remain on the emitted log events and spans.

The regression guards are mutation-verified. tests/handlers/metric-cardinality.test.ts asserts that no metric data point carries either identifier, that every handler-reachable log event and every span still does, and — importantly — that each instrument recorded something, so the guard cannot pass by observing nothing. Injecting session.id or project.id back into a single counter fails it with instrument "commit" leaked project.id. Two pre-existing tests that pinned project.id onto counters are inverted to pin it off.

The instrument names and kinds are now pinned by a test. These names are also the OPENCODE_DISABLE_METRICS keys and the dashboards' contract, and had no coverage at all. createInstruments gained an optional meter parameter to make this testable — the OTel API accepts only one global provider registration per process, so a recording meter cannot be injected through the global.

Known gap. user_prompt is emitted from the plugin's chat.message hook in src/index.ts, which needs a full plugin harness to reach. It is the one emitLog site not covered by the log-side guard, and the test names that exclusion explicitly rather than implying full coverage.

BREAKING CHANGE

  • session.id and project.id are no longer present on any metric data point. Dashboards and alerts that group or filter metrics by either must move to traces or log events, where both remain, or use the model / agent attributes that stay on metrics.
  • The opencode.lines_of_code.total Gauge is removed and replaced by the opencode.session.lines_of_code.total Histogram, which exports as opencode_session_lines_of_code_total_bucket / _sum / _count (adjusting for OPENCODE_METRIC_PREFIX).
  • Its OPENCODE_DISABLE_METRICS suffix is now session.lines_of_code.total. The old suffix lines_of_code.total no longer matches anything and is silently ignored.
  • The new histogram is emitted only when a session ends, so sessions that are never deleted or errored report no line totals at all.
  • Cost and token attribution per project is no longer possible from metrics alone — it requires traces or logs. This is the main capability removed.

🤖 Generated with Claude Code

ShawnZhang31 and others added 2 commits September 19, 2026 00:47
…sion end

Remove the `session.id` attribute from all 15 metric instruments. It was the
dominant source of Prometheus series growth: every label combination is a
separate time series and a histogram multiplies it by its bucket count (20 at
the SDK's default boundaries), the SDK caps a metric at 2000 attribute sets and
silently collapses the rest into `otel.metric.overflow`, and cumulative
aggregation never evicts — so a long-lived process re-exports every session it
has ever seen on every export, indefinitely.

`session.id` is unchanged on spans and OTLP log events, where per-session
drill-down belongs and high cardinality is acceptable.

Replace the `lines_of_code.total` Gauge with a `session.lines_of_code.total`
Histogram. A Gauge cannot carry a per-session dimension: with LastValue
aggregation the SDK collapses every session into one attribute set and exports
whichever session wrote last, so simply dropping `session.id` from the Gauge
would have made it silently wrong.

The histogram is recorded once when the session ends (`session.deleted`, or
`session.error`), not on `session.idle`. `session.idle` fires once per *turn*
while opencode's `session.diff` is cumulative for the whole session, so
recording there added the running session total once per turn — a four-turn
session reported its LOC four times over. This adds handling for the
`session.deleted` event, which the plugin previously ignored.

Fixing that also fixes the same defect in the gross counter: the per-session
diff baseline is no longer discarded on every idle, so a later turn now emits
the true delta instead of treating the whole session cumulative as new churn.

The net values are also attached to the run and session spans as
`session.total_lines_added` / `session.total_lines_removed`, on both the idle
and error paths, so the information survives with traces enabled.

Tests: add a cardinality guard asserting no metric data point carries
`session.id` and that every handler-reachable log event still does, a
non-vacuity check that fails if an instrument is missing from the test double,
a multi-turn regression test for the over-count, and a test pinning the
instrument names and kinds, which are the disable-metrics keys.

BREAKING CHANGE: `session.id` is no longer present on any metric data point.
Dashboards and alerts that group or filter metrics by `session.id` must move to
traces or logs, or to the `project.id` / `model` / `agent` attributes that
remain. The `opencode.lines_of_code.total` Gauge is removed and replaced by the
`opencode.session.lines_of_code.total` Histogram, which exports as
`opencode_session_lines_of_code_total_bucket` / `_sum` / `_count` (adjusting for
`OPENCODE_METRIC_PREFIX`). Its `OPENCODE_DISABLE_METRICS` suffix is now
`session.lines_of_code.total`; the old suffix `lines_of_code.total` no longer
matches anything and is silently ignored. The new histogram is emitted only
when a session ends, so sessions that are never deleted or errored report no
line totals at all.

Co-Authored-By: Claude Code <noreply@anthropic.com>
`project.id` was spread into every metric data point through `commonAttrs`. It
is derived from the project directory, so it stays small for interactive use,
but it has the same two properties that made `session.id` a problem: it is
unbounded in CI, where every checkout can be a fresh path, and cumulative
aggregation never evicts, so a long-lived process keeps re-exporting every
project it has ever seen on every export.

Split the shared attribute set in two. `commonAttrs` keeps `project.id` and is
used by spans and log events, where per-project drill-down belongs. The new
`metricAttrs` carries only the configured `OPENCODE_SPAN_ATTRIBUTES` pairs and
is what every metric call site now spreads. With both identifiers off metrics,
the remaining labels are bounded dimensions only: `model`, `provider`, `agent`,
`agent.type`, `type`, `tool_name`, `success`, `is_subagent`.

`project.id` is unchanged on spans and OTLP log events.

Tests: the cardinality guard now asserts `project.id` is absent from every
metric data point as well as `session.id`, and asserts the reverse for logs and
spans so the split cannot silently drop the attribute from both sides. Two
existing tests that pinned `project.id` onto counters are inverted to pin it
off. Mutation-verified: re-injecting `project.id` into a single counter fails
the guard with `instrument "commit" leaked project.id`.

BREAKING CHANGE: `project.id` is no longer present on any metric data point.
Dashboards and alerts that group metrics by project must move to traces or log
events, where `project.id` remains, or use the `model` / `agent` attributes that
stay on metrics. Cost and token attribution per project is no longer possible
from metrics alone.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant