Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
name: Token usage tests
on:
pull_request:
push:
branches: [main]
jobs:
tests:
runs-on: ubuntu-latest
strategy:
matrix:
python: ['3.11', '3.14']
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python }}
- run: python3 -m unittest discover -s tests -v
- run: node --check dashboard/static/app.js
10 changes: 8 additions & 2 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ This file is auto-loaded by Claude Code into its context whenever you open this

## What is this project

`token-usage` is a Claude Code skill that does local-first analytics on your own Claude Code transcripts (`~/.claude/projects/**/*.jsonl`). Three interfaces:
`token-usage` is a shared skill for local-first analytics on Claude Code transcripts (`~/.claude/projects/**/*.jsonl`) and Codex rollouts (`CODEX_HOME` or `~/.codex`, sessions and archived_sessions). Three interfaces:

- **CLI** — `scripts/count_tokens.py`, `scripts/analyze.py`
- **Browser dashboard** — `dashboard/server.py` + static frontend
Expand All @@ -26,7 +26,7 @@ These are non-negotiable. Don't suggest changes that violate them without explic

1. **Pure stdlib only.** No `numpy`, `pandas`, `scipy`, `requests`, `flask`, `fastapi`, `pyyaml`, etc. on the Python side. The dashboard frontend uses Chart.js loaded from a CDN — that's the only external runtime dependency in the whole project.
2. **Zero install steps.** A user who clones the repo into `~/.claude/skills/token-usage/` should be able to run any script with `python3 scripts/foo.py` immediately. No `pip install -r requirements.txt`. No `npm install`.
3. **Don't touch `~/.claude/projects/**/*.jsonl`.** Read-only, ever. Those are the user's transcripts.
3. **Don't touch Claude or Codex transcript files.** Read-only, ever. Those are the user's transcripts.
4. **Chinese UI strings stay Chinese.** Don't "translate" 用法画像 / 突发型 / 晚高峰 etc. to English. The user-facing language is Chinese; code/comments are English.
5. **No telemetry, no analytics, no outbound calls.** Only outbound call permitted: the local `claude` CLI subprocess from the AI 解读 button (server.py `h_interpret`).

Expand All @@ -40,6 +40,8 @@ README.md English landing page (GitHub default).
README.zh-CN.md Chinese landing page.

scripts/
codex_records.py Codex request/counter parsing, mirrors and fork-history dedup.
transcript_index.py Disposable per-file SQLite/memory parsing cache for dashboard.
token_stats.py CORE — JSONL parsing, dedup-by-message-id (field-wise max),
time-window filtering, bucket aggregation, billing-equiv
(5m/1h ephemeral cache split). Imported by everything else.
Expand Down Expand Up @@ -74,6 +76,10 @@ docs/

**The single source of truth for token math is `scripts/token_stats.py`.** Don't reinvent dedup or billing-equiv logic anywhere else — import from there.

Codex cache is included in input and reasoning is included in output. Never add these subsets again. Pricing/billing-equivalent values cover Claude only; expose the unpriced Codex scope. See [Codex accounting](docs/codex-usage.md). Run `python3 -m unittest discover -s tests -v` for regression tests.

Dashboard persistence is a rebuildable statistics cache, not a transcript store. Keep per-file identities and raw counter candidates until global dedup; caching only final per-file totals breaks copied/forked/mirrored records. See [index lifecycle](docs/performance.md) and [ADR 001](docs/decisions/001-transcript-index.md).

---

## How the analysis modules talk to each other
Expand Down
13 changes: 10 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
# token-usage

> Local-first analytics for your Claude Code token consumption.
> Local-first analytics for your Claude Code and Codex token consumption.

**English** · [简体中文](./README.zh-CN.md)

A [Claude Code](https://claude.com/claude-code) skill that reads JSONL transcripts under `~/.claude/projects/`, aggregates per model / project / day, and surfaces patterns and AI-generated interpretations through four interfaces:
A local skill for Claude Code and Codex that reads JSONL transcripts under `~/.claude/projects/` and `${CODEX_HOME:-~/.codex}/sessions/` / `archived_sessions/`, aggregates per source / model / project / day, and surfaces patterns and optional AI-generated interpretations through four interfaces:

- **CLI** — `count_tokens.py` for one-shot reports in any time range
- **Browser dashboard** — charts + a **Work-efficiency panel** (throughput per active minute, CDF + KDE) + a **Patterns panel** (Markov / ACF / change-point) + an in-page **🤖 AI 解读** button
Expand All @@ -13,6 +13,8 @@ A [Claude Code](https://claude.com/claude-code) skill that reads JSONL transcrip

Zero dependencies (pure stdlib, no `numpy` / `pandas` / `chart.js` server-side). Chart.js is loaded from CDN by the browser only.

By default both sources are counted. Use `--source claude|codex|all` on the CLIs, or the dashboard's source selector. Codex cache/reasoning subsets are never counted twice; cost and billing-equivalent estimates currently cover Claude only. See [Codex accounting and tests](docs/codex-usage.md).

---

## Screenshots
Expand Down Expand Up @@ -47,6 +49,8 @@ Claude Code's built-in `~/.claude/stats-cache.json` lags by one day and gives yo
git clone https://github.com/WholeNightCoding/token-usage ~/.claude/skills/token-usage
```

For shared discovery by Codex, clone into `~/.agents/skills/token-usage` instead; invoke scripts from that actual folder. Codex's data directory can be overridden with `--codex-home` or `CODEX_HOME`.

The skill is auto-discovered by Claude Code. Just open Claude Code in any project and ask:

> *"How many tokens did I use today?"*
Expand Down Expand Up @@ -98,9 +102,12 @@ python3 ~/.claude/skills/token-usage/dashboard/server.py
# → opens http://127.0.0.1:8787/ in your default browser
```

The dashboard keeps a rebuildable local statistics index across restarts and reparses only changed files. The first build reads all history. Use `--cache-dir PATH` or `--no-cache` to control disk caching; see [performance and cache lifecycle](docs/performance.md).

What you get:

- **4 KPI cards** — total / billing-equiv / USD estimate / last-1h rate
- **4 overview cards** — total / Claude Code / Codex / last-1h rate, with the selected scope and each source's share. Excluded sources display “—” instead of zero.
- **Claude cost reference** — expandable API cost and billing-equivalent estimates; excludes Codex and is not a subscription bill.
- **Daily trend** — stacked bar chart by model, with totals labeled
- **By model / Top 10 projects** — doughnut + bar
- **Realtime** — last-Nh line chart, auto-refreshes every 10s, configurable bucket size (1 min → 4 hour)
Expand Down
16 changes: 13 additions & 3 deletions README.zh-CN.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
# token-usage

> Claude Code token 用量本地化分析工具。
> Claude Code 与 Codex token 用量本地化分析工具。

[English](./README.md) · 简体中文

一个 [Claude Code](https://claude.com/claude-code) skill —— 直接读 `~/.claude/projects/` 下的 JSONL transcript,按 model / project / day 聚合,并通过四个界面把数据变成可读的洞察:
一个同时支持 Claude Code 与 Codex 的本地 skill —— 读取 `~/.claude/projects/` 和 `${CODEX_HOME:-~/.codex}/sessions/`、`archived_sessions/` 下的 JSONL,按来源 / model / project / day 聚合,并通过四个界面把数据变成可读的洞察:

- **CLI** — `count_tokens.py` 任意时间区间一次性出报告
- **浏览器仪表板** — 图表 + **工作效率面板**(按活跃分钟算吞吐 + CDF/KDE 分布图)+ **Patterns 面板**(Markov / ACF / 变化点检测)+ 内嵌 **🤖 AI 解读** 按钮
Expand All @@ -13,6 +13,8 @@

零 pip 依赖(纯 stdlib,连 numpy/pandas 都没有)。Chart.js 只在浏览器端通过 CDN 加载。

默认统计两者;命令行可用 `--source claude|codex|all`,面板可选“全部 / Claude Code / Codex”。Codex 缓存和推理子集只记一次,金额与计费折算目前只包含 Claude。详见 [Codex 统计口径与验证](docs/codex-usage.md)。

---

## 截图
Expand Down Expand Up @@ -47,6 +49,8 @@ Claude Code 自带的 `~/.claude/stats-cache.json` 有两个不爽的地方:**
git clone https://github.com/WholeNightCoding/token-usage ~/.claude/skills/token-usage
```

希望 Codex 发现共享 skill 时,可安装到 `~/.agents/skills/token-usage`;脚本从实际安装目录执行。Codex 数据目录可通过 `--codex-home` 或 `CODEX_HOME` 指定。

Claude Code 会自动发现这个 skill。打开任意项目,问它:

> *「我今天用了多少 token?」*
Expand Down Expand Up @@ -87,6 +91,9 @@ python3 ~/.claude/skills/token-usage/scripts/count_tokens.py \

# JSON 输出(管道用)
python3 ~/.claude/skills/token-usage/scripts/count_tokens.py --json

# 只看 Codex
python3 ~/.claude/skills/token-usage/scripts/count_tokens.py --source codex --today
```

输出列:`MODEL | MSGS | INPUT | OUTPUT | CACHE_READ | CACHE_CREATE | TOTAL`,外加总计和按权重 `input=1×, cache_read=0.1×, cache_create_5m=1.25×, cache_create_1h=2×, output=5×` 算的「计费等效输入 token」估算。
Expand All @@ -98,9 +105,12 @@ python3 ~/.claude/skills/token-usage/dashboard/server.py
# → 自动打开 http://127.0.0.1:8787/
```

仪表板用本机统计索引复用历史解析结果,重启后也保留,只重新解析变化的文件。首次建立索引仍需读取历史。可以用 `--cache-dir PATH` 或 `--no-cache` 控制磁盘缓存,详见[性能与索引生命周期](docs/performance.md)。

包含:

- **4 张 KPI 卡** — 总量 / 计费等效 / 美元估算 / 最近 1 小时速率
- **4 张概览卡** — 总用量 / Claude Code / Codex / 最近一小时速率,标明筛选范围和来源占比;未纳入筛选的来源显示“—”,不误显示为零。
- **Claude 费用参考** — 展开查看 API 金额和计费等效 token 的估算,不含 Codex,也不是订阅账单。
- **Daily trend** — 按 model 堆叠的柱状图,柱顶标总计
- **By model / Top 10 projects** — 甜甜圈 + 柱状
- **Realtime** — 最近 N 小时折线图,每 10 秒自动刷新,桶大小可选(1 min → 4 hour)
Expand Down
90 changes: 32 additions & 58 deletions SKILL.md
Original file line number Diff line number Diff line change
@@ -1,84 +1,58 @@
---
name: token-usage
description: Use when the user asks how many Claude Code tokens were consumed — today, yesterday, this week, this month, on a specific date, in a rolling window (last Nh/Nd), or across a custom date/time range — or wants a per-model breakdown with cache-read vs. cache-create split. Also applies when `~/.claude/stats-cache.json` is stale (it only updates through the previous day) or when the user wants a billing-equivalent estimate from local transcripts. Also triggers for visual/browser-based token usage monitoring — "dashboard", "监控板", "面板", "show me in a browser", live rate, per-project breakdown — via the local web UI at `dashboard/server.py`.
description: Report local Claude Code and Codex token usage for today, a date, a rolling window, or a custom range, with model, project, day, and source breakdowns. Use the local dashboard for visual monitoring and work-efficiency analysis; use the CLI for one-shot questions.
---

# Token Usage Reporter

## Overview
Read local transcripts without modifying them. Default scope is both Claude Code and Codex; use `--source claude` or `--source codex` when the user names one. These are local recorded tokens, not account quota percentages or subscription charges.

Aggregates Claude Code token `usage` records from local JSONL transcripts under `~/.claude/projects/**/*.jsonl` for any time range. More accurate than `~/.claude/stats-cache.json`, which lags by one day.
## One-shot reports

**Core principle:** parse raw transcripts → filter by `timestamp` → dedupe by `message.id` → sum per model.

## When to Use

Trigger symptoms:
- "How many tokens did I use today / yesterday / this week / last 6 hours?"
- "Token usage between 2026-04-01 and 2026-04-15"
- "Breakdown by model" or "how much was cache vs. new input?"
- Questions about cost / billing-equivalent tokens for a period
- `stats-cache.json` doesn't include today's data
- User wants a **visual dashboard / monitoring page** (e.g. "token 监控板", "打开 token 面板", "in a browser", per-project top-N, live rate) → use the dashboard, not a one-shot CLI table

**Do NOT use for:** message/session counts (those are in `stats-cache.json` — read it directly).

## Quick Reference

Script: `~/.claude/skills/token-usage/scripts/count_tokens.py` (invoke with `python3`).

| Range flag | Scope |
|---|---|
| `--today` (default) | Today, local tz |
| `--yesterday` | Yesterday, local tz |
| `--this-week` | Mon–Sun of current week |
| `--this-month` | Current calendar month |
| `--all` | Entire history (all transcripts) |
| `--date YYYY-MM-DD` | One specific day |
| `--last 7d` / `12h` / `30m` | Rolling window ending now |
| `--from X --to Y` | Explicit range, `YYYY-MM-DD` or `YYYY-MM-DD HH:MM[:SS]` |

Modifiers: `--by-day` (per-day table for multi-day ranges), `--json` (machine-readable), `--projects-dir PATH` (override transcript root).

## Dashboard (visual, browser-based)

Use when the user wants ongoing monitoring, live rate, per-project breakdown, or just a more navigable view than a CLI table.
Run from this skill's actual directory; do not assume it lives in `~/.claude/skills` or `~/.codex/skills`.

```bash
python3 ~/.claude/skills/token-usage/dashboard/server.py
# default: 127.0.0.1:8787, auto-opens browser
# flags: --port N, --host ADDR (keep 127.0.0.1 unless asked), --no-open
python3 scripts/count_tokens.py --today
python3 scripts/count_tokens.py --source codex --yesterday
python3 scripts/count_tokens.py --source claude --this-week
python3 scripts/count_tokens.py --from 2026-10-01 --to 2026-10-07 --by-day
python3 scripts/count_tokens.py --source codex --last 12h --json
```

Shows: total + billing-equiv + USD estimate (pay-as-you-go) + last-1h rate cards, stacked daily-trend line, per-model doughnut, top-10 projects bar, realtime 1h line (auto 10s refresh), a **work-efficiency panel** (active hours, % of wall-clock, throughput while actually working, 30-min slot rate distribution, per-day breakdown — answers "how productive am I during the hours I'm at the keyboard?"), and a model×project detail table. Range selector: Today / Yesterday / This week / This month / 7d / 30d / All time / Custom… (from–to date pickers; All time auto-clamps to the first day with activity).
Range flags: `--today` (default), `--yesterday`, `--this-week`, `--this-month`, `--all`, `--date YYYY-MM-DD`, `--last 7d|12h|30m`, or `--from X --to Y`. Dates/times use local timezone. JSON retains the existing `by_model` and `total` fields, and adds `by_source`, reasoning-output and pricing-scope metadata.

The efficiency panel reports rates against **active minutes** (minutes that consumed tokens), not wall clock — so a 20-min work block isn't averaged against the 40 idle minutes around it. Also available as a CLI: `python3 ~/.claude/skills/token-usage/scripts/work_efficiency.py [days]`.
Sources:

Server: same data source as CLI (numbers match), in-memory cached + mtime-invalidated, binds `127.0.0.1` only, no auth.
- Claude Code: `~/.claude/projects/**/*.jsonl`; override with `--projects-dir`.
- Codex CLI/Desktop: `${CODEX_HOME:-~/.codex}/sessions/**/*.jsonl` and `archived_sessions/**/*.jsonl`; override with `--codex-home` (the home containing both folders).

**Do NOT launch the dashboard just to answer a one-shot CLI-style question.** Use `count_tokens.py` instead for "how many tokens yesterday" — it's faster and fits better in chat.
## Dashboard

## Canonical Example
Use when the user asks for a monitoring page, charts, live rates or navigable analysis. Do not launch it for a one-shot token-count question.

```bash
python3 ~/.claude/skills/token-usage/scripts/count_tokens.py --from 2026-04-01 --to 2026-04-21 --by-day
python3 dashboard/server.py
# 127.0.0.1:8787; --no-open, --port N, --projects-dir PATH, --codex-home PATH
```

Output columns: `MODEL | MSGS | INPUT | OUTPUT | CACHE_READ | CACHE_CREATE | TOTAL`, plus grand total and a **billing-equivalent input tokens** estimate using weights `input=1×, cache_read=0.1×, cache_create_5m=1.25×, cache_create_1h=2×, output=5×`. CACHE_CREATE in the table is the 5m+1h sum; the billing-equiv splits them out via the per-record `cache_creation.ephemeral_1h_input_tokens` field.
The source selector (全部 / Claude Code / Codex) applies to totals, trends, models, projects, realtime, efficiency and Patterns. The dashboard reuses a local per-file statistics index across refreshes and restarts; only changed transcripts are reparsed. The first index build still reads all history. Use `--cache-dir PATH` to choose its directory or `--no-cache` for memory only; see [index lifecycle](docs/performance.md). Keep it bound to loopback unless the user requests another binding.

## Reading the output
Other reports use the same scanner:

- **Scope**: covers Claude Code only. Anthropic API direct calls, Claude.ai web/desktop usage are NOT in these transcripts — totals will be lower than your actual Anthropic billing.
- **Sub-agent dedup is automatic**: same `message.id` appearing in parent + sub-agent transcripts is merged via field-wise max. The numbers you see are post-dedup; no double-counting.
- `<synthetic>` model rows are cache-only compaction events; always sum to zero, safe to ignore.
- `cache_create_5m` is priced at 1.25× input, `cache_create_1h` at 2×; billing-equiv splits them via `cache_creation.ephemeral_1h_input_tokens` per record.
```bash
python3 scripts/work_efficiency.py 7 --source codex
python3 scripts/analyze.py --days 7 --source codex --html report.html
```

(Internals — dedup-by-max, UTC handling, subagent scanning — see CLAUDE.md.)
The existing optional AI interpretation button uses the local `claude` CLI, regardless of the selected statistics source; ordinary statistics require only Python's standard library.

## Common Mistakes
## Interpret correctly

- **Reporting `cache_read` as new-input cost.** It's 0.1× in billing; cite billing-equiv when cost is the question.
- **Using `stats-cache.json` for "today".** It only updates through the previous day.
- **Pre-converting dates to UTC.** The script handles local-tz; pass the user's date as-is.
- **Filtering by file mtime.** Transcripts append throughout a session; always filter by the JSONL `timestamp` field.
- Claude Code: dedupe parent/subagent and streaming snapshots by message ID with field-wise maxima; cache creation retains the 5-minute/1-hour split.
- Codex: prefer per-response `token_usage_record` when present, dedupe response IDs, and suppress mirrored `token_count` events. Older rollouts use cumulative-counter differences, reading preceding counters before applying a time window. Repeated snapshots, counter resets and inherited fork history are handled separately.
- Codex cached input is a subset of input; the INPUT column shows the non-cached remainder. Reasoning output is a subset of OUTPUT. Never add either subset again to total tokens.
- Amounts and billing-equivalent values use the existing Claude pricing configuration only. Codex tokens are marked unpriced. Never apply Claude rates to Codex, or present the API estimate as the user's subscription bill. Mixed-source efficiency disables the billing view and retains raw throughput.
- Missing/deleted/local-unrecorded history cannot be reconstructed. Cloud conversations without local rollout files are outside the report.
- Filter by event timestamp, not file modification date. Do not use `stats-cache.json` to answer today's usage.

For format/dedup details and validation commands, see [docs/codex-usage.md](docs/codex-usage.md).
Loading
Loading