Skip to content

feat(cli): add PostHog usage analytics with a persistent opt-out - #235

Open
designcode wants to merge 1 commit into
mainfrom
feat/cli-posthog-integration
Open

feat(cli): add PostHog usage analytics with a persistent opt-out#235
designcode wants to merge 1 commit into
mainfrom
feat/cli-posthog-integration

Conversation

@designcode

@designcode designcode commented Aug 5, 2026

Copy link
Copy Markdown
Collaborator

What

Adds PostHog usage analytics to the CLI. It already reported errors to Sentry but nothing about usage, so there was no way to see which commands matter or how adoption follows a console signup.

One cli_command event per invocation, reported to the same PostHog project as the console so CLI and web activity join on one person record.

Privacy model

Credentials are never collected. Access keys, secret keys, tokens, passwords, and authorization headers are stripped whether they appear as a flag value or a bare positional. Third-party email addresses are redacted too. The machine hostname and working directory are never sent.

Both telemetry surfaces now share one scrubber (src/utils/redact.ts, extracted from utils/telemetry.ts), so the guarantee can't drift apart between usage analytics and error reports — a redaction fix protects both or neither.

Command arguments are collected, scrubbed: bucket names, object keys, paths and flag values are what make a usage trend actionable. Verified end-to-end against the built CLI:

configure  args="configure --access-key [redacted] --access-secret [redacted] --endpoint https://t3.storage.dev"
ls         args="ls [redacted]"                     # AKIA… positional
ls         args="ls my-public-bucket/photos/2026/"  # bucket + prefix kept

Tests assert against the whole serialized payload rather than individual fields, so a property added later can't reintroduce a leak. Note one consequence of collecting paths: tigris cp ~/x.pdf … sends /Users/alice/x.pdf, so OS usernames will appear in path arguments.

Decisions worth a look

Email as distinct_id. Signed-in runs are identified by account email — what the console's posthog.alias() already resolves to, so the two surfaces land on one person. Flagging it as the one personal value we deliberately send.

Fired at command start, not completion. program.parse() isn't awaited and ~370 call sites exit synchronously through exitWithError, so there's no reliable place to flush at exit. Firing early overlaps the request with the command's own work. The trade: we record invocations, not outcomes — failure rates stay Sentry's job.

No posthog-node dependency. The SDK batches behind its own timers, which fights those synchronous exits, and this package also compiles to standalone binaries. Raw node:https instead.

State in ~/.tigris/telemetry.json, not config.json. logout clears the latter wholesale, and neither the opt-out nor the anonymous id should come back to life because someone logged out. A write failure is a hard error, not a silent no-op — otherwise telemetry disable would report success and resume sending on the next run.

Opt-out

tigris telemetry status    # shows which switch is in effect, and what is collected
tigris telemetry disable   # persists
tigris telemetry enable

Plus TIGRIS_NO_TELEMETRY / DO_NOT_TRACK, which take precedence over the stored setting, and a one-time disclosure notice on first interactive use. One gate covers both analytics and error reports.

⚠️ Behavior change to existing telemetry: both env vars were compared against the literal '1', so the common DO_NOT_TRACK=true silently did nothing. Both now honor any truthy value per the DO_NOT_TRACK convention. This newly gates error reporting for anyone who set a non-1 value expecting it to work.

Measured cost

Round trip to the ingest host is ~170–220ms.

Scenario Overhead
Typical command (real network work) ~0ms — request overlaps the command
Fastest local commands ~100ms
Blackholed host (dropped packets, no RST) ~1.1s worst case

The timeout is sized to 1s against that measurement. An earlier 2s value cost 2.35s on every command when the host was blackholed. socket.unref() was tried to cap it and does not work: for a socket still connecting, unref doesn't release the event loop, even called synchronously.

Known gaps

A command that hard-exits faster than the ~200ms request loses its event — process.exit() kills an in-flight socket regardless of refs. Verified both ways: a cp failing at 480ms delivered, one failing at 220ms didn't. Local validation failures are under-counted; anything doing real network work reports reliably. Closing it needs a parseAsync refactor, which would also fix the identical latent flush gap in Sentry.

Separately, unverified: update-check.ts relies on the same ineffective socket.unref() with a 10s timeout, so a blackholed npm registry may hold the CLI longer than intended. Not touched here — flagging for follow-up.

Also in here

pnpm updatedocs was silently deleting any hand-written README section between ## Usage and ## License — it wiped the Telemetry section on the first regenerate. The generated block now ends at the first hand-written trailing section instead of hardcoding ## License.

Verification

  • tsc --noEmit clean on both the npm and binary targets; biome check clean
  • 910 tests pass, up from 840 (70 new)
  • Drove the built CLI end-to-end against a local capture server: credential redaction in flag and positional form, argument retention, disable/enable round trip, --json output, env-var precedence, opt-out write failure exiting non-zero, and the first-run notice firing exactly once in a real TTY

Changeset included (minor for @tigrisdata/cli).

🤖 Generated with Claude Code

@greptile-apps

greptile-apps Bot commented Aug 5, 2026

Copy link
Copy Markdown

Greptile Summary

Adds PostHog command analytics to the CLI, a shared persistent opt-out for analytics and Sentry, and telemetry status/enable/disable commands.

  • Records canonical command paths, flag names, runtime/install metadata, and authenticated or anonymous identity.
  • Persists telemetry state separately from login configuration and displays a one-time interactive disclosure.
  • Refactors install-method detection for reuse by analytics and update guidance.
  • Adds focused tests and a minor package changeset.

Confidence Score: 4/5

The persistent opt-out and value-redaction failures should be fixed before merging because they can resume telemetry after an asserted disable and transmit a positional value.

Filesystem errors are suppressed before the opt-out handler prints success, and raw argv scanning does not stop at the end-of-options separator before collecting flag names.

Files Needing Attention: packages/cli/src/utils/telemetry-config.ts, packages/cli/src/lib/telemetry/disable.ts, packages/cli/src/utils/analytics.ts

Security Review

A dash-prefixed positional value supplied after -- can be misclassified as a flag name and transmitted despite the stated guarantee that values are never collected.

Important Files Changed

Filename Overview
packages/cli/src/utils/analytics.ts Implements PostHog capture and identity linking, but raw argv flag extraction can disclose dash-prefixed positional values.
packages/cli/src/utils/telemetry-config.ts Adds shared persistent telemetry state, but suppresses write failures needed to guarantee a durable opt-out.
packages/cli/src/lib/telemetry/disable.ts Adds the opt-out command but reports success without knowing whether the preference was persisted.
packages/cli/src/cli-core.ts Starts analytics before command execution without otherwise changing command dispatch behavior.
packages/cli/src/utils/install-method.ts Centralizes npm, Homebrew, and standalone-binary install detection with regression coverage.
packages/cli/src/utils/telemetry.ts Routes existing Sentry reporting through the new shared telemetry gate.
packages/cli/src/specs.yaml Declares the telemetry command group and its status, disable, and enable subcommands.

Reviews (1): Last reviewed commit: "feat(cli): add PostHog usage analytics w..." | Re-trigger Greptile

Comment thread packages/cli/src/utils/telemetry-config.ts
Comment thread packages/cli/src/utils/analytics.ts Outdated
The CLI reported errors to Sentry but nothing about usage, so there was no way
to see which commands matter or how adoption follows a console signup.

Report one `cli_command` event per invocation carrying the canonical command
path, the scrubbed command arguments, flag names, CLI/runtime/OS versions,
install method, auth method, and CI/TTY shape.

Credentials are never collected — access keys, secrets, tokens, passwords and
authorization headers are stripped whether they appear as a flag value or a
positional, as are third-party email addresses. The hostname and working
directory are never sent. Both telemetry surfaces now share one scrubber
(utils/redact.ts, extracted from utils/telemetry.ts) so the guarantee cannot
drift apart between usage analytics and error reports; tests assert against
the whole serialized payload so a property added later cannot reintroduce a
leak.

- Send with raw `node:https` rather than `posthog-node`. The SDK batches behind
  its own timers, which fights the ~370 synchronous `exitWithError` call sites,
  and this package also compiles to standalone binaries.
- Fire at command *start*, not completion. `program.parse()` is not awaited and
  most commands exit synchronously, so there is no reliable place to flush;
  firing early overlaps the request with the command's own work. The trade is
  invocations, not outcomes — failures stay Sentry's job. A command that
  hard-exits faster than the ~200ms request still loses its event.
- Identify signed-in runs by account email, which is what the console aliases
  to, so CLI and web activity land on one person record. Unauthenticated runs
  use a stable anonymous id that `$identify` merges into the account on the
  next login, making the install → first command → logged in funnel work.
- Keep telemetry state in ~/.tigris/telemetry.json rather than config.json,
  because `logout` clears the latter wholesale and neither the opt-out nor the
  anonymous id should come back because someone logged out.
- Fail loudly when that state cannot be written. A swallowed error would let
  `telemetry disable` report success and then resume sending on the next run,
  which is the one outcome the opt-out exists to prevent.
- Size the request timeout to 1s against a measured ~200ms round trip. A
  blackholed host otherwise costs the full timeout on every command.
  `socket.unref()` does not help: for a still-connecting socket it does not
  release the event loop.
- Extract install-method detection out of update-check.ts so the update
  notifier and analytics share one source of truth.
- Stop `updatedocs` deleting hand-written trailing README sections: the
  generated block now ends at the first of them, not at `## License`.

Add `tigris telemetry status | disable | enable` plus a one-time disclosure
notice, and route both telemetry surfaces through a single opt-out gate.

Note: `TIGRIS_NO_TELEMETRY` and `DO_NOT_TRACK` were compared against the
literal '1', so the common `DO_NOT_TRACK=true` silently did nothing. Both now
honor any truthy value, which also newly gates error reporting.

Assisted-by: Claude Opus 5 via Claude Code

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
@designcode
designcode force-pushed the feat/cli-posthog-integration branch from a35c14a to 6e259de Compare August 6, 2026 08:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants