Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,21 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).

### Added

- **Command timing.** Every Bash command Claude runs is now a row in
`command_runs` (migration 058): its signature (the program and up to two
plain words, such as `bun run build`; never the command itself), whether it
ran in the foreground, the background, or was moved there after its
timeout, how long the session waited on it, its start-to-finish duration,
and the outcome. Background commands are settled by the new
`command.background_complete` event from plugin 0.38.0, which reports when
each one finished and its exit code. `/api/commands/summary`, `/trend` and
`/projects` aggregate by command and project (team or own, never per
developer; a teammate's session that is not shared counts under its program
only), and the new **Commands** page and a section on each project page show
which commands the sessions wait on most and whether they are getting
slower. `scripts/command-runs-backfill.ts` fills the table from stored
events (program only, since older plugins sent nothing more).

- **Voiced audio ends cleanly.** `/api/ai/voice-audio` rewrites the speech
service's WAV into a plain 44-byte-header PCM file with its real sizes
(Kokoro streams its WAV with `0xFFFFFFFF` sizes and a LIST chunk, so players
Expand Down
33 changes: 32 additions & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -93,14 +93,17 @@ WebSocket message types: `event.new`, `session.update`, `developer.update`.
| `/api/ai/voice-summary` | POST | Spoken text for the plugin's voice features (`trigger`, `project`, optional `tool`/`detail`/`last_message`/`session_id`/`label`), starting with the session's label (`project, topic` from the caller's own session title, else branch; a sent `label` is reused as is, so the name never changes) and returning it as `label`: one sentence for the announcer, or with `trigger: "reply"` a summary of Claude's reply for `/devscope:voice auto`, as detailed as the optional `length` asks (`short` one sentence ≤ 20 words, `normal` 2-3 sentences ≤ 50 words, `long` 4-6 sentences ≤ 110 words; `REPLY_LENGTHS`); stateless, only token usage is recorded as `voice_summary`. The plugin never calls it for `private` sessions |
| `/api/ai/voice-audio` | POST | That text (or one piece of a `/devscope:voice explain` explanation) as WAV (`text`, optional `model` = a `TTS_SERVICES` name, Kokoro `voice`, `speed`, `volume`, `pad_ms` trailing silence, default 400), rewritten as a canonical PCM WAV with real sizes (`services/wav.ts`), voiced by the homelab speech services; 503 when none is configured or all fail, so the plugin falls back to a local voice. Own 20/min bucket, no Gemini budget |
| `/api/sessions/usage/backfill` | POST | Exact token usage per transcript and model from `/devscope:backfill-usage` (`{items: [{transcriptId, sessionId?, byModel}]}`, max 500); only the caller's own sessions are updated, the rest are counted as `skipped` |
| `/api/commands/summary?days=&scope=team\|me&group=signature\|program&project=&limit=` | GET | Bash commands by time the sessions waited on them, with p50/p95/max duration, failures, background/moved counts and the period before for comparison (see Command timing) |
| `/api/commands/trend?days=&scope=&group=&project=&command=&bucket=day\|week` | GET | One command's (or all commands') runs, p50/p95 and time waited per day or week |
| `/api/commands/projects?days=` | GET | Projects with command runs the caller may see, for the project filter |
| `/api/health` | GET | Health check + WS client count |
| `/ws` | WS | Real-time event stream |

## Local dev: making API calls

Two flavors of auth — pick one per route:

- **API key (`x-api-key` header)** — the plugin path. Required by `/api/events` and the plugin-accessible routes (`/api/ai`, `/api/insights`, `/api/patterns`, `/api/playbooks`, `/api/skills`, `/api/topology`, `/api/workflow-profiles`, `/api/friction`, `/api/sessions`, `/api/similar`, `/api/live`). Keys are minted via better-auth's `auth.api.createApiKey({ body: { userId, name } })`; the dashboard exposes this at Settings → API Keys, but there's no UI before you have a session.
- **API key (`x-api-key` header)** — the plugin path. Required by `/api/events` and the plugin-accessible routes (`/api/ai`, `/api/insights`, `/api/patterns`, `/api/playbooks`, `/api/skills`, `/api/topology`, `/api/workflow-profiles`, `/api/friction`, `/api/sessions`, `/api/similar`, `/api/live`, `/api/commands`). Keys are minted via better-auth's `auth.api.createApiKey({ body: { userId, name } })`; the dashboard exposes this at Settings → API Keys, but there's no UI before you have a session.
- **Session cookie** — what the dashboard uses. Sign-in via `/api/auth/sign-in/email` returns a session cookie. Most dashboard routes go through `orgScopeMiddleware`, which reads `session.activeOrganizationId` to scope queries.

Two traps that bite manual testers:
Expand Down Expand Up @@ -218,6 +221,34 @@ Session tokens and cost (migration 055, `db/tokenUsageQueries.ts`). Costs are **
- **Backfill from disk:** `/devscope:backfill-usage` uploads exact totals for transcripts still on the developer's machine. Claude Code deletes transcripts after `cleanupPeriodDays` (30 by default), so older sessions keep the estimate.
- **Pricing:** `token_cost_usd()` and the rows in 055; runbook in `packages/backend/docs/token-pricing.md`. Sessions with no `model` (pre-037) are priced at the `*` fallback (Sonnet-4 rates), which understates Opus-era usage.

## Command timing

How long the Bash commands Claude runs take, per project and over time (`command_runs`, migration 058,
`db/commandQueries.ts`, `routes/commands.ts`, dashboard `components/commands/`).

- **Rows** come from Bash `tool.complete` / `tool.fail` at ingestion (`recordCommandRun`, in
`runPostInsertHandlers`, so it never blocks or fails ingestion). Key: Claude Code's `tool_use_id` (plugin
0.38.0+), else `event:<event id>` with `legacy_timing` set. `wait_ms` is the event's `duration` (Claude
Code's `duration_ms` since plugin 0.38.0; before that the plugin's own timer, which mixed up parallel
calls); the start is the event timestamp minus it.
- **Signature, never the command.** The plugin sends `commandSignature` (`_ds_command_signature`: quotes and
heredocs removed; program + up to two plain words only for programs that take subcommands, the third only
after a group word like `run`/`compose`/`pr`; any other program alone; private mode the program only).
`cleanSignature` re-checks the shape and the same subcommand list (`MULTI`, keep in sync with the
plugin's `multi`) and falls back to the program, and private sessions store the program whatever was sent. Older plugins only have `toolSubcommand` (the first word).
- **Background commands.** PostToolUse fires when a command is backgrounded (`run_in_background` →
`background`; a timeout or the user → `moved`), so the row starts as `running` with `duration_ms` NULL. The
plugin finds the settling in the transcript (`queue-operation` enqueue of a `<task-notification>`) at Stop
and SessionEnd and sends `command.background_complete`; `settleBackgroundCommands` updates only the
sender's own running rows, by `tool_use_id` (not session).
- **Privacy.** No developer or project column: both come from `sessions` at read time, so retention
anonymization applies, and the row cascades away with its event. Reads aggregate by command and project
only; a signature or project is named only for sessions the viewer may see (`visibleSessionSql`), the rest
count under their program, and a `project` filter matches only visible sessions. `scope=me` is the
caller's own ids in the org (no `developerId` parameter).
- **Backfill:** `scripts/command-runs-backfill.ts` (dry run by default, `--write`) runs stored Bash events
through `recordCommandRun`; idempotent.

## Server voice

The plugin's voice announcer speaks through `/api/ai/voice-audio`, so users get a natural voice without installing a model. Both voice endpoints live in `routes/voice.ts` (mounted under `/api/ai`; the Gemini guard and rate limiter are shared with `routes/ai.ts` via `routes/aiGuards.ts`).
Expand Down
107 changes: 107 additions & 0 deletions packages/backend/scripts/command-runs-backfill.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
/**
* Fill command_runs (migration 058) from the Bash tool.complete / tool.fail
* events already stored, so command trends start with history rather than
* from the day of the deploy.
*
* Each event goes through the same recordCommandRun as live ingestion: the
* signature is sanitized the same way and private sessions keep only the
* program. History predates plugin 0.38.0, so rows carry the program only
* (toolSubcommand) and legacy_timing (the old plugin timer, which mixed up
* parallel calls of one session). Background commands from that time were
* recorded at the moment they were backgrounded and count as finished there.
*
* DATA SAFETY. Only inserts into command_runs; ON CONFLICT DO NOTHING, so it
* is idempotent and never touches rows ingestion already wrote. Events that
* retention has purged are gone, and so are their rows.
*
* Usage:
* DATABASE_URL=... bun run scripts/command-runs-backfill.ts # dry run: report only
* DATABASE_URL=... bun run scripts/command-runs-backfill.ts --write # insert the rows
*/

import { SQL } from "bun";
import type { DevscopeEvent } from "@devscope/shared";
import { recordCommandRun } from "../src/db/commandQueries";

const BATCH = 500;

function parseArgs(argv: string[]) {
const out = { write: false };
for (const a of argv) {
if (a === "--write") out.write = true;
else {
console.error(`Unknown argument: ${a}`);
process.exit(2);
}
}
return out;
}

async function main() {
const { write } = parseArgs(process.argv.slice(2));
const url = process.env.DATABASE_URL;
if (!url) {
console.error("DATABASE_URL is required");
process.exit(2);
}
const sql = new SQL(url);

const [counts] = (await sql`
SELECT
COUNT(*)::INT AS events,
COUNT(cr.event_id)::INT AS already,
MIN(e.created_at) AS first_at
FROM events e
LEFT JOIN command_runs cr ON cr.event_id = e.id
WHERE e.event_type IN ('tool.complete', 'tool.fail')
AND e.payload->>'toolName' = 'Bash'
`) as Array<{ events: number; already: number; first_at: Date | null }>;
console.log(
`Bash tool events: ${counts!.events} (since ${counts!.first_at?.toISOString?.() ?? "-"}), ` +
`already in command_runs: ${counts!.already}, to add: ${counts!.events - counts!.already}`,
);
if (!write) {
console.log("Dry run. Re-run with --write to insert.");
await sql.close();
return;
}

let cursor: { at: Date; id: string } | null = null;
let added = 0;
for (;;) {
const rows = (await sql`
SELECT e.id, e.session_id, e.event_type, e.payload, e.created_at
FROM events e
WHERE e.event_type IN ('tool.complete', 'tool.fail')
AND e.payload->>'toolName' = 'Bash'
AND NOT EXISTS (SELECT 1 FROM command_runs cr WHERE cr.event_id = e.id)
AND (${cursor?.at ?? null}::timestamptz IS NULL
OR (e.created_at, e.id) > (${cursor?.at ?? null}::timestamptz, ${cursor?.id ?? ""}))
ORDER BY e.created_at, e.id
LIMIT ${BATCH}
`) as Array<{ id: string; session_id: string; event_type: string; payload: unknown; created_at: Date }>;
if (rows.length === 0) break;
for (const r of rows) {
const payload = typeof r.payload === "string" ? JSON.parse(r.payload) : r.payload;
await recordCommandRun(sql, {
id: r.id,
sessionId: r.session_id,
eventType: r.event_type,
timestamp: new Date(r.created_at).toISOString(),
payload,
} as unknown as DevscopeEvent);
}
added += rows.length;
const last = rows[rows.length - 1]!;
cursor = { at: last.created_at, id: last.id };
console.log(` ${added} processed`);
}
const [after] = (await sql`SELECT COUNT(*)::INT AS n FROM command_runs`) as Array<{ n: number }>;
console.log(`Done. command_runs now has ${after!.n} rows.`);
await sql.close();
}

main().catch((e) => {
console.error(e);
process.exit(1);
});
Loading
Loading