feat(metrics): add posthog.metrics count/gauge/histogram API (alpha)#739
feat(metrics): add posthog.metrics count/gauge/histogram API (alpha)#739DanielVisca wants to merge 4 commits into
Conversation
Port of the posthog-js core metrics client: statsd-style pre-aggregation (counts sum, gauges keep last, histograms bucket with the OTel default bounds), one OTLP/JSON data point per series per flush window, delta temporality, posted to /i/v1/metrics with the project token. Thread-safe; a daemon timer flushes every 10s and shutdown() drains the window. Includes the cardinality guardrail (max_series_per_flush, default 1000), the type-collision warning, transient-failure window merge-back, and the same resource-attribute layering (SDK keys over user keys) as posthog-js. Generated-By: PostHog Code Task-Id: 0d65d94c-c18a-4d84-93f6-18eb54876551
posthog-python Compliance ReportDate: 2026-07-14 18:05:17 UTC ✅ All Tests Passed!111/111 tests passed Capture_V1 Tests✅ 94/94 tests passed View Details
Feature_Flags Tests✅ 17/17 tests passed View Details
|
Crash safety (a telemetry SDK must never raise into the host app): the series key is now JSON-based like the JS core's seriesKey, so list/dict attribute values and mixed-type keys aggregate instead of raising TypeError; before_send returns are validated (non-dict dropped, unknown metric types dropped instead of folding as histograms); and a catch-all guard drops-with-warning anything that still escapes. Fork safety: a forked child inherits the parent's window and a timer handle whose thread doesn't exist in the child — captures now detect the PID change, drop the inherited window (avoiding duplication), and re-arm. Failure handling: retry-later flushes log a warning and are bounded by a 3-consecutive-failure budget (then dropped loudly), merge-back respects the series cap so an outage with attribute churn can't grow memory without bound, and the send goes through the shared pooled session with a gzipped body and a URL-encoded token. Encoding parity with posthog-js: non-finite floats emit proto3 literals (Infinity/-Infinity/NaN), integral floats emit intValue, None-valued attributes are stripped before keying, and bool/int values stay distinct series via the JSON key. Generated-By: PostHog Code Task-Id: 0d65d94c-c18a-4d84-93f6-18eb54876551
|
Reviews (1): Last reviewed commit: "fix(metrics): harden the capture path an..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
|
|
||
| def _to_otlp_key_value_list(attributes: dict) -> list: | ||
| return [ | ||
| {"key": key, "value": _to_otlp_any_value(value)} |
There was a problem hiding this comment.
When callers pass non-string attribute keys, _series_key() accepts them by converting keys with str(k), but the OTLP payload still emits the original key object here. The tested input attributes={"a": 1, 2: "x"} can therefore send key: 2 even though OTLP KeyValue.key is a string field, causing strict decoding to reject the metric or drop the attribute.
| {"key": key, "value": _to_otlp_any_value(value)} | |
| {"key": str(key), "value": _to_otlp_any_value(value)} |
There was a problem hiding this comment.
Fixed in be5e767 — keys are now str()d at the wire boundary too, matching the series identity. Regression test added (test_non_string_attribute_keys_stringify_on_the_wire) asserting {"a": 1, 2: "x"} emits keys {"a", "2"} as strings.
Typed extraction after the before_send isinstance narrow (dict[str, Any] + closed defaults the validation below drops), RequestException added to the vendored requests stub as the base the real hierarchy has, and the public API snapshot regenerated for the new Client.metrics surface. Generated-By: PostHog Code Task-Id: 0d65d94c-c18a-4d84-93f6-18eb54876551
Series identity already str()s keys, but the payload emitted the raw object — a numeric key violates OTLP's string KeyValue.key and strict decoders reject or drop the attribute. Generated-By: PostHog Code Task-Id: 0d65d94c-c18a-4d84-93f6-18eb54876551
| unit: Optional[str], | ||
| attributes: Optional[dict], | ||
| ) -> None: | ||
| if getattr(self._client, "disabled", False): |
There was a problem hiding this comment.
[P1] send=False still transmits metrics
Client.__init__ documents send=False as queueing captures without sending them (posthog/client.py:287), and the event path returns before transport when self.send is false. The metrics gate here checks only disabled, so the new capture surface bypasses that established transport switch.
For example:
client = Client("phc_test", send=False, thread=0)
client.metrics.count("jobs.processed")
client.metrics.flush()The current code produces:
HTTP POST calls: 1
Expected:
HTTP POST calls: 0
This causes unexpected telemetry egress in environments and tests that deliberately construct a no-send client.
Please make metrics capture/flush honor client.send, or explicitly document why metrics intentionally differ from event capture.
A regression test should cover both manual flush and timer flush with send=False and assert that _get_session().post is never called.
| self.disable_geoip = disable_geoip | ||
| self._metrics_config = metrics | ||
| self._metrics: Optional[PostHogMetrics] = None | ||
| self._metrics_lock = threading.Lock() |
There was a problem hiding this comment.
[P1] Reinitialize the new metrics locks after fork() to avoid a child deadlock
Client._reinit_after_fork() already replaces thread-owned queue/session/cache state because locks held by vanished parent threads remain locked in the child. The new _metrics_lock here, plus PostHogMetrics._lock and _flush_lock, are not replaced. The metrics PID guard cannot recover because _capture() acquires _lock before calling _reset_after_fork_locked().
A concrete timeline is:
- A parent thread holds
client.metrics._lock. - The process forks.
- The child calls
client.metrics.count("after.fork"). - The child blocks forever on the inherited lock.
The current reproduction exited via a two-second alarm:
fork_child_exit=42
Expected: the child records the metric without blocking and uses fresh metrics state.
Please replace _metrics_lock and any instantiated metrics locks/timer/window in the existing at-fork callback, without acquiring the inherited locks.
A regression test should hold each metrics lock from another thread, call os.fork(), invoke the public metrics.count() path in the child, and assert that the child exits normally within a short deadline.
| self.flush(timeout_seconds=None) | ||
| if self._metrics is not None: | ||
| try: | ||
| self._metrics.flush() |
There was a problem hiding this comment.
[P1] The public module-level posthog.shutdown() leaves metrics buffered
This adds metrics flushing to direct Client.shutdown(), but the documented module helper in posthog/__init__.py:1122 still calls only _proxy("flush") and _proxy("join"). A default client obtained through posthog.setup() therefore bypasses this new lifecycle path.
For example:
client = posthog.setup()
client.metrics.count("lost.on.global.shutdown")
posthog.shutdown()The current code produces:
metric HTTP calls: 0
buffered series after shutdown: 1
Expected:
metric HTTP calls: 1
buffered series after shutdown: 0
This loses the final metrics window for applications using the global SDK lifecycle.
Please make module-level shutdown() delegate to the client's shutdown() implementation rather than separately calling flush() and join().
A regression test should record through posthog.setup().metrics, call the public posthog.shutdown(), and assert that the metrics transport ran and the window was cleared.
2 participants
What this is
The Python leg of the
posthog.metricsSDK surface (Metrics docs) — a faithful port of the posthog-js core client (posthog-js #4115, node wiring in #4117), so backend Python services can record metrics with zero OpenTelemetry setup:Design (mirrors posthog-js exactly, adapted to Python)
count()calls is one data point on the wire. Delta temporality throughout, so restarts need no cross-window state.ExportMetricsServiceRequestto{host}/i/v1/metrics?token=...— nano timestamps as decimal strings but histogramcount/bucketCountsas plain JSON numbers (string-encoded u64s are silently dropped upstream, opentelemetry-rust#3328). SameDEFAULT_HISTOGRAM_BOUNDSas JS.threading.Timerarmed on first capture (10s default).shutdown()flushes and resets.max_series_per_flushcardinality cap (default 1000, warn once per window, existing series keep folding), type-collision warning, monotonic-counter validation, finite-value validation,before_sendhook.posthog/metrics_capture.py(notmetrics.py— that name would shadow a future module-levelposthog.metricsaccessor and matches theexception_capture.pyhouse naming).How did you test this code?
TDD — the test file was written first and observed failing at import, then all green:
posthog/test/test_metrics.py— 14 tests, each pinning a distinct contract: count burst → one delta+monotonic data point; gauge keeps last (and has nostartTimeUnixNano, matching JS); the full histogram wire shape (the opentelemetry-rust#3328 encoding pin); series identity split by attributes but not by attribute order; OTLP value encoding (including the Python-specific bool-before-int trap —boolis anintsubclass and would silently encodeTrueasintValue: 1); cardinality cap drops new series but keeps folding existing ones; parameterized invalid-sample drops; disabled client no-ops; resource-attribute layering (user keys can't spooftelemetry.sdk.*); transient-failure merge-back (a 503'd window's counts sum into the next flush); shutdown flush.posthog/test/test_client.py— 145 passed (I touchedClient.__init__andshutdown).ruff check/ruff formatclean.check_public_api.py --write(griffe isn't installable in my env — PEP 668); CI should regenerate/check the snapshot, and the newClient.metricsproperty will need it. Also no live end-to-end send from this environment — the wire shape is byte-identical to posthog-js's, which was validated end-to-end against ingestion during the metrics alpha.🤖 Agent context
Autonomy: Human-driven (agent-assisted) — Daniel directed this; assigned as DRI.
I (Claude) wrote this by porting
packages/core/src/metrics/from posthog-js (read in full: index.ts, metrics-utils.ts, config.ts, the value encoder in logs-utils.ts, and the_sendMetricsBatchtransport for the URL/auth/outcome contract). Deliberate deviations from the JS original:flush_intervalin seconds (Python client idiom, vsflushIntervalMs), thread-safety via locks (JS is single-threaded), nodrainWindow()(browser-unload concern), and plain-requeststransport with the same ok/too-large/retry-later/fatal outcome classification instead of the fetch retry stack. Part of the metrics-SDK track: js ✅ merged, node in review, python (this PR).