Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,11 +15,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- **Global Caches**: Browser Temp detection (macOS) — zombie Chromium-family code-sign clones under `/private/var/folders/*/*/X/*.code_sign_clone` left behind by force-killed browsers (headless automation like lighthouse/puppeteer), labeled with browser name and copy count. `safe` when the browser is not running; `caution` while it runs (newest copy may be in use) or for unrecognized bundle IDs.
- **LLM Model Stores** scanner (`llm`) covering local model weights at fixed home paths: LM Studio (`~/.lmstudio/models`, per model) and Hugging Face hub (`~/.cache/huggingface/hub`, per model, `models--org--name` decoded to `org/name`), plus the Ollama (`~/.ollama/models`) and llamafile (`~/.llamafile`) stores as a whole. All `safe` with re-download notes; the Ollama note points to `ollama rm <model>` for removing individual models. Results carry a new `last_used_at` JSON field (model directory mtime; omitted when unknown), shown in the table as a dim "last used …" hint.

#### Sizing
- Sparse-aware sizing: a sparse artifact shows its real on-disk size next to the size it reports — `8.6 GB (appears as 494.4 GB)` for a `Docker.raw` image. JSON gains an `apparent_size` field alongside the disk-based `size`.

### Changed

- Project scanners (Node, Rust, Ruby, Python, Go) are now declarative rule tables executed by a single shared filesystem walk instead of five independent traversals — one pass over the scan root regardless of how many project ecosystems are active. Stat-based scanners (xcode, global, llm) are unchanged.
- The scan spinner shows a single "Scanning projects..." stage for all project scanners (previously "Scanning node...", "Scanning rust...", … in sequence). Stat scanners still report under their own names.
- The project walk now sizes matched artifacts concurrently (bounded worker pool) instead of one `du` call at a time — up to ~4× faster warm-cache sizing on a many-artifact tree, with cold-scan and few-large-artifact gains varying; disk-usage numbers are unchanged.
- Artifact sizing runs in-process (`scanner.Measure`) instead of forking `du` per artifact, collecting disk (allocated blocks) and apparent (logical) size in one pass. Disk stays the primary figure for sorting, filtering and totals.
- Totals count hard-linked blocks shared across artifacts once (e.g. a pnpm store blob linked into `node_modules`), so `total_size` no longer double-counts them. Per-artifact sizes are unchanged.
- The walk engine now reads each directory once and reuses those entries for both project-marker detection and recursion, instead of reading every directory twice (a `filepath.WalkDir` read plus a second `os.ReadDir`). Directory traversal — the dominant cost of scanning a large tree — is roughly 1.8× faster (~29% faster end-to-end on a workspace with tens of thousands of directories); scan results are unchanged.
- Git classification now forks `git` across a bounded worker pool (`min(NumCPU, 8)`) instead of one repo at a time — root resolution, `status`/`log`, and `check-ignore` all run concurrently across repos. On a workspace spanning dozens of repos this cut the classify phase roughly in half (~1.24s → ~0.54s warm cache, ~19% faster end-to-end); protection and activity results are unchanged.
- The status-badge, safety-icon, and relative-time display formatters, previously duplicated between the table output and the interactive tree selector, are now single shared functions in `internal/ui` (`StatusBadge`, `SafetyIcon`, `RelativeTime`). Table and selector rendering are unchanged; the formatters gained direct unit tests.
Expand Down
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,10 +38,11 @@ See [Ecosystems](docs/ecosystems.md) for full detection and safety details.
- Activity classification — active, recent, stale, dormant (based on git + filesystem)
- Gitignore-aware protection — only git-tracked artifacts are protected
- `--min-size` filter to suppress small artifacts and focus on real targets
- Accurate sizing — sparse-aware (a 460 GB `Docker.raw` using 8.6 GB shows as such) and hard-link-aware (shared blocks counted once)
- Interactive tree selector for clean — select by project or individual artifact
- Soft delete (Trash) by default, with force delete option
- Vendor-native cleanup hooks (`xcrun simctl delete unavailable`, etc.) via `--vendor-cleanup`
- JSON output for scripting and AI agent integration
- JSON output (`--json`) for scripting and AI agents, with `apparent_size` and a deduped `total_size`
- Colored terminal output with ecosystem grouping

## Install
Expand Down
6 changes: 4 additions & 2 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ type Scanner interface {
}
```

Scanners report progress via context-attached callbacks for real-time UI updates; the walk batch reports under a single "projects" label, stat scanners under their own names. Size is calculated with `du -sk` for accurate disk usage. The walk engine does not size artifacts inline — it collects every matched artifact during the single pass, then sizes them across a bounded worker pool (`min(NumCPU, 8)`) so the per-artifact `du` calls and their I/O overlap. `du` is faster per artifact than an in-process traversal (macOS uses bulk attribute syscalls), so the win comes from concurrency, not from replacing `du`.
Scanners report progress via context-attached callbacks for real-time UI updates; the walk batch reports under a single "projects" label, stat scanners under their own names. Sizing collects two figures per artifact via an in-process walk (`scanner.Measure`): **disk** (allocated blocks, `st_blocks×512` — sparse-accurate and matching `du`) and **apparent** (sum of logical file sizes). Directories contribute their own blocks to disk (real on ext4, ~0 on APFS); symlinks are never followed. Hard-linked inodes (`Nlink>1`) are counted once per artifact and recorded (keyed by `(dev, ino)`) so shared blocks can be netted out across artifacts. The walk engine does not size inline — it collects every matched artifact during the single pass, then sizes them across a bounded worker pool (`min(NumCPU, 8)`) so the traversals and their I/O overlap. Disk is the primary figure (sorting, `--min-size`, totals); apparent surfaces only when a file is materially sparse.

### Walk engine

Expand All @@ -72,7 +72,9 @@ Results are sorted by (table order, path) before returning, keeping output order

### Display Units

`model.HumanSize` formats sizes with **decimal SI units** (1 KB = 1000 B). The CLI's `--min-size` flag uses the same convention by default (humanize.ParseBytes), so the threshold a user types and the size they see in output agree on the same arithmetic. Internally, `du -sk` returns binary kilobytes, but the formatting layer is decimal — so a 1 GiB directory renders as `1.1 GB` and `--min-size 1GB` will include it. Binary suffixes (`KiB`, `MiB`, `GiB`) are still accepted by `--min-size` for users who want explicit binary thresholds.
`model.HumanSize` formats sizes with **decimal SI units** (1 KB = 1000 B). The CLI's `--min-size` flag uses the same convention by default (humanize.ParseBytes), so the threshold a user types and the size they see in output agree on the same arithmetic. Internally sizes come from `st_blocks×512` (binary 512-byte units), but the formatting layer is decimal — so a 1 GiB directory renders as `1.1 GB` and `--min-size 1GB` will include it. Binary suffixes (`KiB`, `MiB`, `GiB`) are still accepted by `--min-size` for users who want explicit binary thresholds.

**Sparse-aware display**: the table shows an artifact's real on-disk size, annotating it with the larger size the file nominally reports when that apparent size exceeds double the disk figure by more than 1 GiB — e.g. a `Docker.raw` image renders `24.0 GB (appears as 460.0 GB)`, making clear it only uses 24 GB on disk though it presents as 460 GB. The JSON output always carries `apparent_size` (`omitzero`, so dropped when zero) so agents can detect sparse files. Ordinary directories, where block-rounding leaves apparent ≤ disk, are never annotated.

### Metadata Enrichment

Expand Down
30 changes: 30 additions & 0 deletions docs/commands.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,36 @@ Legend: ✔ safe ⚠ caution ✖ protected ● Active ● Recent ● Stale
Run 'devclean list' for details
```

A sparse artifact is shown as `8.6 GB (appears as 494.4 GB)` — real disk size, then the size it reports. When hard-linked blocks are shared across artifacts, the total counts them once and says so.

### JSON (`--json`)

```json
{
"total_size": 5583457484,
"total_count": 5,
"results": [
{
"path": "/Users/you/workspace/my-app/node_modules",
"ecosystem": "node",
"category": "deps",
"size": 1782579200,
"apparent_size": 1690123456,
"last_modified": "2026-07-14T09:12:00Z",
"activity": "active",
"safety": "safe",
"protected": false
}
]
}
```

- `size` — disk usage (allocated blocks); sparse-aware, used for sorting and `--min-size`.
- `apparent_size` — logical size; omitted when zero. Much larger than `size` for sparse files.
- `total_size` — sum of `size` with hard-linked blocks counted once.

Also present when known: `reason`, `project_root`, `label`, `recommendation`, `last_used_at`.

## clean

Clean reclaimable disk space. See `devclean clean --help` for all flags.
Expand Down
4 changes: 3 additions & 1 deletion internal/integration_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -258,7 +258,9 @@ func TestGolden_JSONOutput(t *testing.T) {
// Normalize for golden file: zero out volatile fields
for i := range results {
results[i].Path = filepath.Base(results[i].Path)
results[i].Size = 0 // du returns block-aligned sizes
results[i].Size = 0 // block-aligned, machine-specific
results[i].ApparentSize = 0 // block-rounding slack varies by filesystem
results[i].Links = nil // inode-keyed; also keeps DedupedTotal at 0 with Size zeroed
results[i].LastMod = time.Time{} // zero out to make golden file stable
results[i].ProjectRoot = "" // holds the absolute temp path — machine-specific
}
Expand Down
48 changes: 44 additions & 4 deletions internal/model/types.go
Original file line number Diff line number Diff line change
Expand Up @@ -74,12 +74,22 @@ type ArtifactDef struct {
AlwaysSafe bool `json:"always_safe"`
}

// InodeKey identifies a physical inode uniquely. Inode numbers are only unique
// per device, so any cross-artifact dedup key must include Dev — a home scan can
// span multiple filesystems (external volumes, network mounts, Docker/APFS
// containers), where two unrelated files may share an inode number.
type InodeKey struct {
Dev uint64
Ino uint64
}

// ScanResult represents a single scannable item on disk.
type ScanResult struct {
Path string `json:"path"`
Ecosystem Ecosystem `json:"ecosystem"`
Category Category `json:"category"`
Size int64 `json:"size"`
Size int64 `json:"size"` // disk usage: allocated blocks (st_blocks×512), sparse-aware
ApparentSize int64 `json:"apparent_size,omitzero"` // sum of logical file sizes; exceeds Size for sparse files
LastMod time.Time `json:"last_modified"`
Activity ActivityStatus `json:"activity"`
Safety SafetyLevel `json:"safety"`
Expand All @@ -89,6 +99,11 @@ type ScanResult struct {
Label string `json:"label,omitempty"` // human-readable display name (e.g. "iPhone 17 Pro · iOS 26.3")
Recommendation string `json:"recommendation,omitempty"` // hint for the user (e.g. "old build", "unavailable runtime")
LastUsedAt time.Time `json:"last_used_at,omitzero"` // when the item itself was last used, if the scanner can tell (omitzero: zero time is dropped from JSON)

// Links maps each hard-linked inode (Nlink>1) found in this artifact to its
// disk blocks, so a caller can dedup blocks shared across artifacts (e.g.
// pnpm store ↔ node_modules) when computing a grand total. Not serialized.
Links map[InodeKey]int64 `json:"-"`
}

// HumanSize returns a human-readable size string.
Expand All @@ -99,9 +114,10 @@ func (r ScanResult) HumanSize() string {
// HumanSize formats bytes into a human-readable string using decimal SI
// units (1 KB = 1000 B). This matches the macOS Finder convention and keeps
// the display aligned with the `--min-size` parser, which treats "MB" as
// 10^6 per humanize/SI convention. Internally we still derive sizes from
// `du -sk` (binary kilobytes), but the formatting layer is decimal so the
// number a user types and the number they see agree on the same threshold.
// 10^6 per humanize/SI convention. Sizes are derived internally from
// st_blocks×512 (binary 512-byte units), but the formatting layer is decimal
// so the number a user types and the number they see agree on the same
// threshold.
func HumanSize(size int64) string {
const (
KB = 1000
Expand All @@ -124,6 +140,30 @@ func HumanSize(size int64) string {
}
}

// DedupedTotal returns the total disk usage across results with blocks shared
// via hard links counted once. Each result's Size already counts its own
// hard-linked inodes once (intra-artifact); this nets out inodes that recur
// across artifacts — e.g. a pnpm store blob also hard-linked into a project's
// node_modules — so the total reflects the space actually freed by deleting
// everything shown, not an inflated sum.
func DedupedTotal(results []ScanResult) int64 {
var total int64
seen := make(map[InodeKey]struct{})
for _, r := range results {
total += r.Size
for key, blocks := range r.Links {
// First artifact to hold this inode keeps it (already in r.Size);
// every later artifact double-counted it, so subtract it back out.
if _, dup := seen[key]; dup {
total -= blocks
continue
}
seen[key] = struct{}{}
}
}
return total
}

// ProtectionResult holds the result of a protection analysis.
type ProtectionResult struct {
IsProtected bool `json:"is_protected"`
Expand Down
7 changes: 1 addition & 6 deletions internal/output/json.go
Original file line number Diff line number Diff line change
Expand Up @@ -16,13 +16,8 @@ type ScanOutput struct {

// WriteJSON writes scan results as formatted JSON.
func WriteJSON(w io.Writer, results []model.ScanResult) error {
var totalSize int64
for _, r := range results {
totalSize += r.Size
}

out := ScanOutput{
TotalSize: totalSize,
TotalSize: model.DedupedTotal(results), // nets out blocks shared via hard links
TotalCount: len(results),
Results: results,
}
Expand Down
80 changes: 80 additions & 0 deletions internal/output/output_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,86 @@ func sampleResults() []model.ScanResult {
}
}

// TestWriteTableSparseAnnotation verifies a materially sparse artifact renders
// its apparent size alongside disk — the point of A1. Full render-path test:
// it pins that the artifact size cell actually routes through sizeCell.
func TestWriteTableSparseAnnotation(t *testing.T) {
sparse := model.ScanResult{
Path: "/Users/dev/proj/node_modules",
Ecosystem: model.EcoNode,
Category: model.CatDeps,
ProjectRoot: "/Users/dev/proj",
Size: 4096, // disk: sparse, almost nothing allocated
ApparentSize: 2147483649, // apparent: ~2 GiB logical
Activity: model.StatusDormant,
Safety: model.SafetySafe,
}
var buf bytes.Buffer
output.WriteTableWithOptions(&buf, []model.ScanResult{sparse}, output.TableOptions{Verbose: true})
if out := buf.String(); !strings.Contains(out, "appears as") {
t.Errorf("sparse artifact must show its nominal size; got:\n%s", out)
}
}

// TestWriteTableNoSparseAnnotationForDense verifies a normal artifact (apparent
// below disk from block slack) is not annotated — the threshold must not fire
// on ordinary trees.
func TestWriteTableNoSparseAnnotationForDense(t *testing.T) {
dense := model.ScanResult{
Path: "/Users/dev/proj/node_modules",
Ecosystem: model.EcoNode,
Category: model.CatDeps,
ProjectRoot: "/Users/dev/proj",
Size: 150 * 1000 * 1000, // 150 MB disk
ApparentSize: 122 * 1000 * 1000, // apparent below disk (rounding slack)
Activity: model.StatusDormant,
Safety: model.SafetySafe,
}
var buf bytes.Buffer
output.WriteTableWithOptions(&buf, []model.ScanResult{dense}, output.TableOptions{Verbose: true})
if out := buf.String(); strings.Contains(out, "appears as") {
t.Errorf("dense artifact must not be annotated; got:\n%s", out)
}
}

// TestWriteJSONDedupsHardlinkedTotal pins that the JSON total_size nets out
// blocks shared across artifacts via hard links (the pnpm case), rather than
// summing the overlapping per-artifact sizes.
func TestWriteJSONDedupsHardlinkedTotal(t *testing.T) {
shared := map[model.InodeKey]int64{{Dev: 1, Ino: 7}: 400}
results := []model.ScanResult{
{Path: "/a", Ecosystem: model.EcoNode, Category: model.CatDeps, Size: 500, Links: shared},
{Path: "/b", Ecosystem: model.EcoNode, Category: model.CatDeps, Size: 500, Links: shared},
}
var buf bytes.Buffer
if err := output.WriteJSON(&buf, results); err != nil {
t.Fatalf("WriteJSON error: %v", err)
}
var out output.ScanOutput
if err := json.Unmarshal(buf.Bytes(), &out); err != nil {
t.Fatalf("JSON parse error: %v", err)
}
// 500 + 500 − 400 (shared inode counted once) = 600, not 1000.
if out.TotalSize != 600 {
t.Errorf("TotalSize = %d, want 600 (deduped)", out.TotalSize)
}
}

// TestWriteTableDedupNote verifies the table annotates its grand total when
// hard-link dedup made it smaller than the naive sum.
func TestWriteTableDedupNote(t *testing.T) {
shared := map[model.InodeKey]int64{{Dev: 1, Ino: 7}: 400}
results := []model.ScanResult{
{Path: "/a/node_modules", Ecosystem: model.EcoNode, Category: model.CatDeps, ProjectRoot: "/a", Size: 500, Links: shared, Safety: model.SafetySafe, Activity: model.StatusDormant},
{Path: "/b/node_modules", Ecosystem: model.EcoNode, Category: model.CatDeps, ProjectRoot: "/b", Size: 500, Links: shared, Safety: model.SafetySafe, Activity: model.StatusDormant},
}
var buf bytes.Buffer
output.WriteTableWithOptions(&buf, results, output.TableOptions{Verbose: true})
if out := buf.String(); !strings.Contains(out, "excludes hard-linked") {
t.Errorf("expected hard-link dedup note in total; got:\n%s", out)
}
}

func TestWriteJSON(t *testing.T) {
results := sampleResults()
var buf bytes.Buffer
Expand Down
Loading