Skip to content

fix(cli): stop recording expected outcomes as errors, enable checkpoints by default - #455

Merged
Makisuo merged 2 commits into
mainfrom
fix/cli-telemetry-hygiene
Aug 13, 2026
Merged

fix(cli): stop recording expected outcomes as errors, enable checkpoints by default#455
Makisuo merged 2 commits into
mainfrom
fix/cli-telemetry-hygiene

Conversation

@Makisuo

@Makisuo Makisuo commented Aug 13, 2026

Copy link
Copy Markdown
Collaborator

Why

Triaging the maple-cli error stream via the Maple MCP turned up two structural problems before any individual bug:

  1. We can't tell customers from CI. Every maple-cli error span carries deployment.environment=development — the SDK's fallback, since none of MAPLE_ENVIRONMENT / RAILWAY_ENVIRONMENT_NAME / DEPLOYMENT_ENV exist on a laptop. Separating CI runners from humans meant reading filesystem paths out of error strings.
  2. Expected outcomes drowned out real failures. The top error sources were the CLI behaving correctly: the already-running guard (~24k events), /health readiness polling (9k), and maple --help.

The three loudest remaining issues turned out to be already fixed at HEAD, with old binaries still reporting them (Error {} on POST /v1/traces from failing a span with a bare Response, fixed in v0.0.13; the maintenance-lock ENOENT, fixed 2026-08-07; max_query_size, fixed 2026-07-20). Those need binaries to age out, not code.

What changed

Telemetry hygiene

  • Split the start/stop/reset/restore precondition guards out of ServerError into a new ServerStateError, and recover them — plus --help and mode-resolution failures — inside the root maple span, the same placement ArchiveError already used. The outcome is annotated as maple.cli.outcome rather than dropped, so "how often do people hit this?" is still answerable. Genuine failures (bind failure, dirty store, incompatible store) stay uncaught and still close the span Error.
  • Mode resolution arrives as a WarehouseConfigError (remapped in core/warehouse.ts); pipeName === "mode" discriminates the expected case so real query failures are re-raised.
  • TracerDisabledWhen on the /health probes. orElseSucceed could never have fixed these — it runs after the client span has already closed. Scoped per-request, not layer-wide, so other /health calls stay traced.
  • environment is now cli, or ci when CI is set.

Checkpoints work out of the box

BACKUP … TO Disk('default', …) needs <backups> in the running connection's config. chDB allows one connection per process, held for the process lifetime, and maple checkpoint is a separate process talking over HTTP — it can never supply that config after the fact. Without --chdb-config-file, checkpoints were simply impossible, which also made the dirty-store recovery advice (maple restore --yes) point at a checkpoint that could not exist.

maple start now generates the config when the flag is absent. writeBackupConfig() already existed and was only used for a throwaway scratch file during restore.

Verification

  • bun run test in apps/cli (Bun runner): 437 pass, incl. 5 new.
  • tsc --noEmit clean; oxfmt + oxlint clean.
  • End to end, against a real local server:
    • maple start then maple checkpoint with no flags → checkpoint created (impossible before this change).
    • maple --help → exit 0, empty stderr.
    • maple start while running → maple is already running (PID …), exit 1, message unchanged.
    • maple services --remote --local → single clean line, exit 1 (previously a full Effect error render).

Deliberately not included

  • Gating deployment.commit_sha on isCommitSha. The CLI stamps its semver (0.0.18) into a commit-SHA field, which is wrong — but the fix sits in shared effect-sdk code used by the browser/client SDKs, and dropping the attribute for non-SHA values could break customer dashboards that group by it. Worth doing deliberately, not as a side effect of CLI triage.
  • Local raw-SQL failures tagged as WarehouseSchemaDriftError/WarehouseConfigError, indistinguishable from real cloud warehouse drift (44 events).
  • trace_resource_attribute_values_mv Unknown identifier 'key' (4 events) — a chDB analyzer difference on ARRAY JOIN mapKeys/mapValues, not local/remote SQL divergence; the definition is byte-identical to the canonical one.

🤖 Generated with Claude Code


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.

…nts by default

Triaging the CLI's error stream was impossible: expected outcomes dominated it,
and every install reported `deployment.environment=development`, so customers,
local dev and CI runners were indistinguishable.

Telemetry hygiene:

- Split the `maple start`/`stop`/`reset`/`restore` precondition guards out of
  `ServerError` into `ServerStateError`, and recover them — plus `--help` and
  mode-resolution failures — inside the root `maple` span. They were the top
  error sources by volume (~24k events for the already-running guard alone).
  The outcome is annotated as `maple.cli.outcome` rather than dropped, so the
  question "how often do people hit this?" is still answerable. Genuine
  failures (bind failure, dirty store, incompatible store) stay uncaught and
  still close the span `Error`.
- Mode resolution reaches `bin.ts` as a `WarehouseConfigError` (remapped in
  core/warehouse.ts); `pipeName === "mode"` discriminates the expected case, so
  real query failures are re-raised.
- Skip span creation for the `/health` readiness probes via
  `TracerDisabledWhen`. Polling until the server binds emitted ~10 `Error`
  spans per `maple start` inside an otherwise-`Ok` root span. `orElseSucceed`
  could not help — it runs after the client span has already closed.
- Report `environment` as `cli`, or `ci` when `CI` is set, instead of
  defaulting to `development`.

Checkpoints:

- `maple start` now generates a backups-enabled chDB config when
  `--chdb-config-file` is absent. `BACKUP … TO Disk('default', …)` needs
  `<backups>` in the *running* connection's config, and chDB allows one
  connection per process, so `maple checkpoint` — a separate process — could
  never supply it. Checkpoints were unusable out of the box, which also made
  the dirty-store recovery advice (`maple restore --yes`) point at a checkpoint
  that could not exist. `writeBackupConfig` already existed and was only used
  for a throwaway scratch file during restore.
- Reword the missing-backups-config error, now only reachable with a custom
  config lacking the stanza.

Verified end to end: `maple start` + `maple checkpoint` with no flags now
succeeds; `--help` exits 0; the guards keep their messages and exit 1.
The probe asserted the behaviour this branch deliberately removes: that a
server started without `--chdb-config-file` cannot checkpoint. With the default
now generated at `maple start`, that case succeeds.

Replaced with two assertions covering both halves of the new contract:

- A server started with no config flag generates a backups-enabled chDB config
  beside the data dir and can take a checkpoint that publishes state. This is
  the guarantee that makes the dirty-store recovery advice (`maple restore
  --yes`) reachable at all.
- The narrow, actionable missing-backups error still fires — now only through a
  custom config that omits the `<backups>` stanza, which is the one way left to
  reach it.

Both use their own data dir so the main scenario still starts from a fresh
store. Verified locally against a compiled bundle: probe passes end to end,
shellcheck clean.
@Makisuo
Makisuo merged commit fc0bd8e into main Aug 13, 2026
30 checks passed
@Makisuo
Makisuo deleted the fix/cli-telemetry-hygiene branch August 13, 2026 15:56
@Makisuo
Makisuo deployed to pr-preview August 13, 2026 15:56 — with GitHub Actions Active
@github-actions

Copy link
Copy Markdown

🍁 Maple PR preview

Note

Preview resources were removed when this pull request closed.

Final commit 56651ca · View workflow run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant