feat(cli): add PostHog usage analytics with a persistent opt-out - #235
feat(cli): add PostHog usage analytics with a persistent opt-out#235designcode wants to merge 1 commit into
Conversation
Greptile SummaryAdds PostHog command analytics to the CLI, a shared persistent opt-out for analytics and Sentry, and telemetry status/enable/disable commands.
Confidence Score: 4/5The persistent opt-out and value-redaction failures should be fixed before merging because they can resume telemetry after an asserted disable and transmit a positional value. Filesystem errors are suppressed before the opt-out handler prints success, and raw argv scanning does not stop at the end-of-options separator before collecting flag names. Files Needing Attention: packages/cli/src/utils/telemetry-config.ts, packages/cli/src/lib/telemetry/disable.ts, packages/cli/src/utils/analytics.ts
|
| Filename | Overview |
|---|---|
| packages/cli/src/utils/analytics.ts | Implements PostHog capture and identity linking, but raw argv flag extraction can disclose dash-prefixed positional values. |
| packages/cli/src/utils/telemetry-config.ts | Adds shared persistent telemetry state, but suppresses write failures needed to guarantee a durable opt-out. |
| packages/cli/src/lib/telemetry/disable.ts | Adds the opt-out command but reports success without knowing whether the preference was persisted. |
| packages/cli/src/cli-core.ts | Starts analytics before command execution without otherwise changing command dispatch behavior. |
| packages/cli/src/utils/install-method.ts | Centralizes npm, Homebrew, and standalone-binary install detection with regression coverage. |
| packages/cli/src/utils/telemetry.ts | Routes existing Sentry reporting through the new shared telemetry gate. |
| packages/cli/src/specs.yaml | Declares the telemetry command group and its status, disable, and enable subcommands. |
Reviews (1): Last reviewed commit: "feat(cli): add PostHog usage analytics w..." | Re-trigger Greptile
The CLI reported errors to Sentry but nothing about usage, so there was no way to see which commands matter or how adoption follows a console signup. Report one `cli_command` event per invocation carrying the canonical command path, the scrubbed command arguments, flag names, CLI/runtime/OS versions, install method, auth method, and CI/TTY shape. Credentials are never collected — access keys, secrets, tokens, passwords and authorization headers are stripped whether they appear as a flag value or a positional, as are third-party email addresses. The hostname and working directory are never sent. Both telemetry surfaces now share one scrubber (utils/redact.ts, extracted from utils/telemetry.ts) so the guarantee cannot drift apart between usage analytics and error reports; tests assert against the whole serialized payload so a property added later cannot reintroduce a leak. - Send with raw `node:https` rather than `posthog-node`. The SDK batches behind its own timers, which fights the ~370 synchronous `exitWithError` call sites, and this package also compiles to standalone binaries. - Fire at command *start*, not completion. `program.parse()` is not awaited and most commands exit synchronously, so there is no reliable place to flush; firing early overlaps the request with the command's own work. The trade is invocations, not outcomes — failures stay Sentry's job. A command that hard-exits faster than the ~200ms request still loses its event. - Identify signed-in runs by account email, which is what the console aliases to, so CLI and web activity land on one person record. Unauthenticated runs use a stable anonymous id that `$identify` merges into the account on the next login, making the install → first command → logged in funnel work. - Keep telemetry state in ~/.tigris/telemetry.json rather than config.json, because `logout` clears the latter wholesale and neither the opt-out nor the anonymous id should come back because someone logged out. - Fail loudly when that state cannot be written. A swallowed error would let `telemetry disable` report success and then resume sending on the next run, which is the one outcome the opt-out exists to prevent. - Size the request timeout to 1s against a measured ~200ms round trip. A blackholed host otherwise costs the full timeout on every command. `socket.unref()` does not help: for a still-connecting socket it does not release the event loop. - Extract install-method detection out of update-check.ts so the update notifier and analytics share one source of truth. - Stop `updatedocs` deleting hand-written trailing README sections: the generated block now ends at the first of them, not at `## License`. Add `tigris telemetry status | disable | enable` plus a one-time disclosure notice, and route both telemetry surfaces through a single opt-out gate. Note: `TIGRIS_NO_TELEMETRY` and `DO_NOT_TRACK` were compared against the literal '1', so the common `DO_NOT_TRACK=true` silently did nothing. Both now honor any truthy value, which also newly gates error reporting. Assisted-by: Claude Opus 5 via Claude Code Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
a35c14a to
6e259de
Compare
What
Adds PostHog usage analytics to the CLI. It already reported errors to Sentry but nothing about usage, so there was no way to see which commands matter or how adoption follows a console signup.
One
cli_commandevent per invocation, reported to the same PostHog project as the console so CLI and web activity join on one person record.Privacy model
Credentials are never collected. Access keys, secret keys, tokens, passwords, and authorization headers are stripped whether they appear as a flag value or a bare positional. Third-party email addresses are redacted too. The machine hostname and working directory are never sent.
Both telemetry surfaces now share one scrubber (
src/utils/redact.ts, extracted fromutils/telemetry.ts), so the guarantee can't drift apart between usage analytics and error reports — a redaction fix protects both or neither.Command arguments are collected, scrubbed: bucket names, object keys, paths and flag values are what make a usage trend actionable. Verified end-to-end against the built CLI:
Tests assert against the whole serialized payload rather than individual fields, so a property added later can't reintroduce a leak. Note one consequence of collecting paths:
tigris cp ~/x.pdf …sends/Users/alice/x.pdf, so OS usernames will appear in path arguments.Decisions worth a look
Email as
distinct_id. Signed-in runs are identified by account email — what the console'sposthog.alias()already resolves to, so the two surfaces land on one person. Flagging it as the one personal value we deliberately send.Fired at command start, not completion.
program.parse()isn't awaited and ~370 call sites exit synchronously throughexitWithError, so there's no reliable place to flush at exit. Firing early overlaps the request with the command's own work. The trade: we record invocations, not outcomes — failure rates stay Sentry's job.No
posthog-nodedependency. The SDK batches behind its own timers, which fights those synchronous exits, and this package also compiles to standalone binaries. Rawnode:httpsinstead.State in
~/.tigris/telemetry.json, notconfig.json.logoutclears the latter wholesale, and neither the opt-out nor the anonymous id should come back to life because someone logged out. A write failure is a hard error, not a silent no-op — otherwisetelemetry disablewould report success and resume sending on the next run.Opt-out
Plus
TIGRIS_NO_TELEMETRY/DO_NOT_TRACK, which take precedence over the stored setting, and a one-time disclosure notice on first interactive use. One gate covers both analytics and error reports.'1', so the commonDO_NOT_TRACK=truesilently did nothing. Both now honor any truthy value per the DO_NOT_TRACK convention. This newly gates error reporting for anyone who set a non-1value expecting it to work.Measured cost
Round trip to the ingest host is ~170–220ms.
The timeout is sized to 1s against that measurement. An earlier 2s value cost 2.35s on every command when the host was blackholed.
socket.unref()was tried to cap it and does not work: for a socket still connecting, unref doesn't release the event loop, even called synchronously.Known gaps
A command that hard-exits faster than the ~200ms request loses its event —
process.exit()kills an in-flight socket regardless of refs. Verified both ways: acpfailing at 480ms delivered, one failing at 220ms didn't. Local validation failures are under-counted; anything doing real network work reports reliably. Closing it needs aparseAsyncrefactor, which would also fix the identical latent flush gap in Sentry.Separately, unverified:
update-check.tsrelies on the same ineffectivesocket.unref()with a 10s timeout, so a blackholed npm registry may hold the CLI longer than intended. Not touched here — flagging for follow-up.Also in here
pnpm updatedocswas silently deleting any hand-written README section between## Usageand## License— it wiped the Telemetry section on the first regenerate. The generated block now ends at the first hand-written trailing section instead of hardcoding## License.Verification
tsc --noEmitclean on both the npm and binary targets;biome checkclean--jsonoutput, env-var precedence, opt-out write failure exiting non-zero, and the first-run notice firing exactly once in a real TTYChangeset included (
minorfor@tigrisdata/cli).🤖 Generated with Claude Code