Skip to content

docs: plain README, and fix the code where it didn't match - #12

Merged
abrichr merged 12 commits into
mainfrom
docs/plain-readme
Oct 14, 2026
Merged

abrichr merged 12 commits into
mainfrom
docs/plain-readme

Conversation

@abrichr

@abrichr abrichr commented Oct 14, 2026 •

Copy link
Copy Markdown
Member

Summary

This PR rewrites the README for a developer who finds openadapt-telemetry on GitHub or PyPI. It also fixes the places where the code didn't do what the old README promised.

README

  • The first screen says what the package sends, who it is for, and how to turn it off. It then gives a quick start (install, set a DSN, report an exception) and a doctest that shows the filter working without a DSN.
  • "Turn it off" lists the four opt-outs and their order of precedence. The pyproject.toml opt-out from 0.3.0 was missing from the old README.
  • "What is sent" is a single table covering error reports and usage events. It now includes what the Sentry SDK collects on its own (log lines, HTTP request URLs, installed packages).
  • Limits are in one short section: the pattern scrubbing misses names and free text, URL paths are kept, and the servers see the IP address of each connection.
  • The failure-signal example and its boundary statement, internal-usage tagging, and the full filter rules moved to docs/COMPLIANCE.md, which the README links once.
  • The first screen addresses people whose OpenAdapt packages depend on this one (pip show lists them), since the openadapt launcher and engine don't. It defines DSN and says that usage events go to OpenAdapt's PostHog project unless another is named.
  • The decorator section says that track_performance sends 1% of calls by default, track_feature sends a message event per call, and a TelemetrySpan outside a transaction sends nothing.
  • tests/test_readme.py runs the README's examples. The quick start runs against an in-memory Sentry transport, and the message GlitchTip would receive must match the one the README quotes. The test also runs the no-DSN case, the doctest, the usage event, the decorators and the failure-signal example in the docs. It fails if process terms such as "admitted" or "signed ledger" come back.

Code fixes

I pointed a real client at a local HTTP sink and compared what it sent with what the README claimed:

  • One opt-out rule. Before this PR, the pyproject.toml opt-out stopped usage events but not error reports. "enabled": false in the config file stopped error reports but not usage events. DO_NOT_TRACK=yes stopped only usage events. Both paths now call is_telemetry_allowed().
  • Error reports. The host name (server_name), the full command line (sys.argv), local variable values and source lines were all sent. Transactions skipped the privacy filter entirely. Log-call arguments (logentry.params) were sent raw, and so were HTTP query strings. All of these are now removed or scrubbed. Before, the filter also replaced trace_id with [REDACTED], which broke the link between errors and traces.
  • Usage events. Property values are now scrubbed, not only their keys.
  • Dependencies. The code needs sentry-sdk 2.15: track_errors raised AttributeError on older SDKs, and so did every TelemetrySpan. The floor is now 2.15.0, and CI also runs the tests against 2.15.0. On Python 3.10, the pyproject.toml opt-out was ignored unless tomli happened to be installed. tomli is now a dependency for Python versions before 3.11.
  • Deprecated calls. push_scope, set_measurement, the explicit send_default_pii and datetime.utcnow are replaced. The suite passes with DeprecationWarning treated as an error.
  • Config file. "internal": true in the config file now tags reports as internal. It was read but ignored.
  • Repository. Twelve tracked __pycache__/*.pyc files are removed and now ignored.
  • Release. initialize() didn't set a release, so the SDK sent the commit hash of whatever git checkout the process ran in. The release is now <package_name>@<package_version>, and a caller can still pass release=.

Testing

  • pytest -W error::DeprecationWarning: 168 passed on Python 3.12 with sentry-sdk 2.72.0, and on Python 3.10 with sentry-sdk 2.15.0.
  • I simulated the CI steps (uv sync --extra dev, ruff, pytest, then pytest again with sentry-sdk 2.15.0) in a clean copy. All passed.
  • I built a wheel and sdist with uv build, and twine check passes. In a fresh Python 3.11 venv with the wheel installed, I ran the README quick start against a local HTTP sink. It received FileNotFoundError: [Errno 2] No such file or directory: '/Users/<user>/notes.txt' with no server_name, no sys.argv, and frames that hold only file, function, line, module and in_app. Without a DSN, initialize() returned False.
  • In a fresh Python 3.10 venv with the wheel installed, is_telemetry_allowed() returns False under a pyproject.toml that opts out. It returned True before the tomli dependency was added.
  • I followed the quick start from a fresh venv against a local HTTP server, from a plain directory and from inside a git checkout. Both sent one request with the quoted message and release mypackage@0.1.0. The four opt-outs each sent nothing. The usage-event example, run as written, sent nothing because the process exits before the background thread posts; the README now says so.
  • The prospect-name guard is clean.

Notes

  • Merging to main runs the semantic-release workflow. The fix: commits make that a patch release, which is also what updates the README on PyPI. The README describes the behavior after this PR, so the published 0.3.1 still has the gaps listed above until that release.
  • Not changed: the config keys error_tracking, performance_tracking and feature_usage are saved and loaded but nothing reads them. The README doesn't mention them.

🤖 Generated with Claude Code

abrichr and others added 10 commits October 14, 2026 03:19
Twelve __pycache__/*.pyc files were committed, so they showed in the
GitHub tree and changed on every local test run. Remove them from the
index and ignore Python build and environment artifacts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Error reports (Sentry/GlitchTip) and usage events (PostHog) each had
their own opt-out check, and they disagreed:

- `[tool.openadapt] telemetry = false` in pyproject.toml stopped usage
  events but not error reports.
- `"enabled": false` in ~/.config/openadapt/telemetry.json stopped error
  reports but not usage events.
- DO_NOT_TRACK=yes or DO_NOT_TRACK=on stopped usage events but not error
  reports.

Both paths now call config.is_telemetry_allowed(), which applies
DO_NOT_TRACK, then OPENADAPT_TELEMETRY_ENABLED, then pyproject.toml,
then the config file. Usage events still stay off in CI unless
OPENADAPT_TELEMETRY_IN_CI or OPENADAPT_TELEMETRY_ENABLED turns them on.
An empty OPENADAPT_TELEMETRY_ENABLED now counts as unset on both paths.

Tests no longer read or write the real ~/.config/openadapt and
~/.openadapt files.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The README says the package never sends names, file contents, text
content or paths with user names. A real Sentry client pointed at a
local sink showed four gaps:

- server_name carried the host name, which often contains the user's
  name (for example "alices-macbook").
- extra["sys.argv"] carried the full command line.
- Stack frames carried local variable values and the source lines
  around each frame. Values were only pattern-scrubbed, so free text
  such as a patient note passed through.
- Transactions from track_performance and TelemetrySpan skipped the
  privacy filter entirely, because only before_send was set.

The client now turns off include_local_variables and
include_source_context, and the filter drops server_name, sys.argv,
frame values and source lines on every event. The same filter runs as
before_send_transaction and scrubs span descriptions, data and tags.
A caller's before_send_transaction runs first, like before_send.
data_collection, which would supersede send_default_pii in newer SDKs,
is refused like send_default_pii.

scrub_string now also removes user names from home-directory paths, so
a path inside an exception message is cleaned the same way as a frame
path.

The filter also replaced trace_id with [REDACTED], because a 32-character
hex ID matches the key-like pattern. That broke the link between errors
and traces and put an invalid trace header on the envelope. Trace IDs
and the sampling context are now kept.

tests/test_sentry_payload.py checks the payloads a real client builds,
using an in-memory transport.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
pyproject.toml allowed sentry-sdk 2.0.0, but the code needs 2.15.0:

- track_errors calls sentry_sdk.get_current_scope(), added in 2.12.0.
  On older SDKs the decorator raised AttributeError in place of the
  caller's exception.
- TelemetrySpan passes name= to start_span, accepted from 2.15.0. On
  older SDKs every span raised TypeError.

Raise the floor to 2.15.0 and run the tests against it in CI.

Also replace calls that newer SDKs or Python deprecate:

- push_scope() becomes new_scope().
- set_measurement() becomes set_data() for duration_ms.
- send_default_pii is no longer passed to sentry_sdk.init. The SDK
  default is off, a caller still cannot override it, and a test checks
  should_send_default_pii() is False on a real client.
- TelemetryEvent timestamps use datetime.now(timezone.utc) in place of
  datetime.utcnow().

The test suite now passes with DeprecationWarning treated as an error.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
capture_usage_event dropped properties whose key looked sensitive
(password, token, email) but sent every value as given, so
{"owner": "alice@example.com"} or a path under /Users/alice reached
PostHog unchanged.

Property values now go through the same string scrubbing as error
reports. A value that is fully redacted is dropped, as before; a value
that is partly redacted, such as a path, is sent with the sensitive
part replaced.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Once initialize() runs, the SDK's logging integration turns ERROR log
records into events and keeps recent log lines and HTTP requests as
breadcrumbs. Two parts of those reached the server unscrubbed:

- logentry.params, the arguments of a call such as
  logging.error("save failed for %s", path). The formatted message was
  scrubbed, but the raw arguments, including emails and home paths,
  were sent next to it.
- The http.query and http.fragment fields of HTTP breadcrumbs, which
  often carry tokens or record IDs (for example ?token=...).

Log arguments are now scrubbed like other strings, whether the SDK
gives them as a tuple or a list. Query strings and fragments are
dropped from breadcrumbs, and query_string and cookies from request
data. The URL path is kept.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The README's config file example listed "internal", and load_config
read it, but the client tagged reports as internal only from
environment variables and CI detection. Setting "internal": true in
~/.config/openadapt/telemetry.json did nothing.

The client now tags reports as internal when the config file says so,
in addition to the existing signals.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The README opened with a feature list, and its examples called
placeholder functions (risky_operation, transform, save_demo) that
don't exist. It now says on the first screen what the package sends,
who it is for, and how to turn it off, then gives a quick start
(install, set a DSN, report an exception) and a doctest that shows the
filter without a DSN.

- "Turn it off" lists the four opt-outs and their precedence. The
  pyproject.toml opt-out, added in 0.3.0, was missing from the README.
- "What is sent" is one table for error reports and usage events:
  destination, trigger, contents and what is removed. It now mentions
  what the SDK collects on its own (log lines, HTTP request URLs,
  installed packages), which the old "What We Collect" table left out.
- Limits are in one short section: pattern scrubbing misses names and
  free text, URL paths are kept, and servers see connection IPs.
- The failure-signal example and its boundary statement, internal-usage
  tagging, and the full filter rules moved to docs/COMPLIANCE.md, linked
  once.
- The GlitchTip query example is gone. docs/COMPLIANCE.md describes the
  internal and ci tag values instead.

tests/test_readme.py now runs the README's examples: the quick start
against an in-memory Sentry transport (the received message must match
the one the README quotes), the no-DSN case, the doctest, the usage
event, the decorators, and the failure-signal example in
docs/COMPLIANCE.md. It also fails if process terms such as "admitted"
or "signed ledger" appear in the README. The in-memory transport moved
to conftest.py so the payload tests share it.

CLAUDE.md's internal-usage list and package layout matched an older
version of the code; both are updated. The PyPI summary is plainer.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The `[tool.openadapt] telemetry = false` opt-out parses pyproject.toml
with tomllib, which Python 3.10 lacks, and falls back to tomli. tomli
was not a dependency, so on Python 3.10 without it the opt-out was
silently ignored: a fresh 3.10 install in a project that opted out
still reported is_telemetry_allowed() as True.

Depend on tomli for Python versions before 3.11. The test suite didn't
catch this because pytest itself pulls in tomli on 3.10, so the new
test checks the declared dependency.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
include_local_variables and include_source_context are off by default,
but a caller's initialize() kwargs can turn them on. The filter still
removes local variables and source lines, so the doc now says that
instead of implying the options themselves are locked.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@abrichr abrichr changed the title docs: plain README, with the privacy gaps it hid fixed in code docs: plain README, and fix the code where it didn't match Oct 14, 2026
abrichr and others added 2 commits October 14, 2026 03:51
initialize() didn't set a release, so the Sentry SDK filled it in with
the output of `git rev-parse HEAD` in the current directory. A report
from a process started inside any git checkout carried that
repository's commit hash, which the README's list of what is sent
doesn't include and which says nothing about the code that raised.

The release is now `<package_name>@<package_version>`. A caller can
still pass release= to initialize().

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Running the README examples against a local HTTP server showed these
gaps between the text and the code:

- The first screen told people who run OpenAdapt to read this page,
  but the launcher and engine don't depend on this package. It now
  addresses people whose OpenAdapt packages depend on it, and says how
  to check with pip show.
- "Other Python projects can use it with their own DSN" left out that
  usage events go to OpenAdapt's PostHog project by default.
- A TelemetrySpan outside a track_performance transaction sends
  nothing, and track_performance sends 1% of calls by default.
  track_feature sends a message event to GlitchTip on every call.
- The usage-event example, run as a script, sends nothing because the
  process exits before the background thread posts the event.
- Usage events carry the OS name, not its version.
- Stack frames keep each file's full path apart from the user name,
  which the Limits section now says alongside URL paths.

It also defines DSN on first use, says "usage events" instead of
"usage counts" in the README and package description, and drops the
term "closed-schema". A new test runs the decorator example after
initialize() and checks what it sends.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@abrichr
abrichr merged commit 19d16ef into main Oct 14, 2026
1 check passed
@abrichr
abrichr deleted the docs/plain-readme branch October 14, 2026 09:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant