formatools · stepango · Aug 3, 2026 · Aug 3, 2026
diff --git a/docs/CONFIGURATION-PERFORMANCE.md b/docs/CONFIGURATION-PERFORMANCE.md
@@ -28,6 +28,23 @@ Warm / configuration-cache path:
 Optional: `./gradlew help --scan` when a Build Scan account is available
 (GH #42). Local `--profile` is enough for regression checks on this host.
 
+### NiA migration pair (F-115, 2026-08-03)
+
+Same-host upstream NiA vs Forma spike harness:
+
+```bash
+export JAVA_HOME=/usr/local/opt/openjdk@21/libexec/openjdk.jdk/Contents/Home
+export ANDROID_HOME=/usr/local/share/android-commandlinetools
+cd /Users/claw/work/forma   # or any checkout with the script
+BENCH_ASSEMBLE=1 bash scripts/bench-nia-migration.sh
+```
+
+Results index: [`docs/benchmarks/README.md`](benchmarks/README.md).  
+Locked pair: cold configuring projects **12.51 s → 5.14 s (−59%)**; warm
+**1.58 s → 0.71 s (−55%)**; CC hit `help` ~parity. Full table + caveats in
+[`docs/benchmarks/nia-migration-20260803T174730Z/RESULTS.md`](benchmarks/nia-migration-20260803T174730Z/RESULTS.md)
+and case study §3.5.
+
 ### Profile anchors on worker host (2026-07-12)
 
 Absolute numbers move with machine load, daemon state, and composite plugin

diff --git a/docs/DOGFOOD-NIA-CASE-STUDY.md b/docs/DOGFOOD-NIA-CASE-STUDY.md
@@ -113,19 +113,46 @@ Plugins published: `bash scripts/publish-local.sh 0.1.3-NIA` from the Forma repo
 
 ### 3.5 Configuration-time performance
 
-No same-machine cold `--profile` pair was locked for upstream NiA vs spike in this Phase D pass (different wrappers, SDKs, and composite `forma-defs` confound absolute seconds).
-
-**Recipe** (from [`CONFIGURATION-PERFORMANCE.md`](CONFIGURATION-PERFORMANCE.md)) if a future worker wants a pair:
+**Same-host pair locked 2026-08-03** (OpenJDK 21.0.11, worker Mac). Harness:
+[`scripts/bench-nia-migration.sh`](../scripts/bench-nia-migration.sh) →
+[`docs/benchmarks/nia-migration-20260803T174730Z/RESULTS.md`](benchmarks/nia-migration-20260803T174730Z/RESULTS.md).
+
+| Metric (Gradle `--profile`) | Upstream NiA | Forma spike | Δ |
+|-----------------------------|--------------|-------------|---|
+| Cold `help` **configuring projects** | **12.51 s** | **5.14 s** | **−59%** |
+| Warm `help` configuring projects | 1.58 s | 0.71 s | −55% |
+| CC miss `help` configuring projects | 2.51 s | 0.98 s | −61% |
+| CC hit `help` wall clock | 1.24 s | 1.34 s | ~parity |
+| Cold `help` wall clock | 22.8 s | 19.0 s | −17% |
+
+**Read as:** on this graph size, Forma’s validators / attrs-only apply path are
+**not** the dominant cost versus NiA convention plugins + AGP — configuration is
+**substantially cheaper** on the spike for cold and warm runs. Configuration
+cache hit path is parity (~1.3 s) once both trees store a CC entry.
+
+**Caveats (do not over-claim):**
+
+- Module graphs are similar but not identical (spike omits Roborazzi / baseline /
+  lint-catalog tooling modules).
+- Wrappers differ (upstream Gradle **9.4.0** pin vs spike **9.6.1**).
+- Spike cold startup can be slightly higher (composite `forma-defs` + mavenLocal
+  plugin resolution) while **configuring projects** still wins.
+- `assembleDemoDebug` wall clock also favored spike in this run (~279 s → ~65 s)
+  but is **confounded** by tooling graph + cache warmth; quote **configuring
+  projects** from `help` for migration perf, not assemble alone.
+
+Re-run:
 
 ```bash
-cd /Users/claw/work/nowinandroid-forma/forma-spike
-./gradlew --stop
-rm -rf .gradle/configuration-cache
-./gradlew help --no-configuration-cache --profile --offline
-# build/reports/profile/profile-*.html → "Configuring Projects"
+export JAVA_HOME=/usr/local/opt/openjdk@21/libexec/openjdk.jdk/Contents/Home
+export ANDROID_HOME=/usr/local/share/android-commandlinetools
+cd /Users/claw/work/forma
+BENCH_ASSEMBLE=1 bash scripts/bench-nia-migration.sh
 ```
 
-Qualitative: spike configuration is dominated by AGP per Android module + Hilt/KSP/Room on annotated types — the same class of work NiA pays — not by Forma validator overhead on this graph size. Prefer same-host before/after if quoting wall-clock.
+Qualitative hold: configuration remains AGP/Hilt/KSP/Room-class work per Android
+module — Forma is not free, but the closed matrix + type-owned plugins did not
+regress configuration vs convention-plugin NiA on this host.
 
 ---
 

diff --git a/docs/PROGRESS.md b/docs/PROGRESS.md
@@ -2,6 +2,19 @@
 
 Newest entries first.
 
+## 2026-08-03 — NiA migration perf benchmark (post F-115)
+
+- **Context:** F-115 done; user asked to run tests + understand perf implications of the migration and create a benchmark
+- **Plugins unit tests:** `plugins/` `./gradlew test jacocoHappyPathCoverageVerification` → **BUILD SUCCESSFUL** (~1m 5s, happy-path gate green)
+- **Harness:** `scripts/bench-nia-migration.sh` — same-host upstream NiA vs `forma-spike` (cold/warm `help`, CC miss/hit, optional assemble, LOC)
+- **Locked results:** `docs/benchmarks/nia-migration-20260803T174730Z/` (+ `docs/benchmarks/README.md`)
+  - Cold **configuring projects** 12.51s → **5.14s (−59%)**
+  - Warm configuring 1.58s → **0.71s (−55%)**
+  - CC hit `help` ~parity (~1.2–1.3s)
+  - Assemble wall recorded but **confounded** (tooling graph / wrapper / cache) — quote config metrics
+- **Docs:** case study §3.5 filled; `CONFIGURATION-PERFORMANCE.md` pointer
+- **Next:** empty gated coding board still; F-094 human-blocked
+
 ## 2026-08-02 — F-115: Phase D case study (NiA dogfood complete)
 
 - **Ticket:** F-115 → **`done`** (was `in_progress` · `priority: now` · `cron may continue`)

diff --git a/docs/benchmarks/README.md b/docs/benchmarks/README.md
@@ -0,0 +1,33 @@
+# Forma benchmarks
+
+Reproducible host measurements for product claims (configuration cost, migration
+surface). Not a substitute for CI unit tests.
+
+| Suite | Script | Latest results |
+|-------|--------|----------------|
+| **NiA → Forma migration** (F-115 perf pair) | [`scripts/bench-nia-migration.sh`](../../scripts/bench-nia-migration.sh) | [`nia-migration-20260803T174730Z/RESULTS.md`](nia-migration-20260803T174730Z/RESULTS.md) |
+
+## NiA migration bench
+
+Same-host pair:
+
+- Upstream: `/Users/claw/work/nowinandroid` (convention-plugin NiA)
+- Spike: `/Users/claw/work/nowinandroid-forma/forma-spike` (external Forma consumer, `0.1.3-NIA`)
+
+Measures cold/warm `help` (no configuration cache), configuration-cache miss/hit,
+optional `assembleDemoDebug`, plus LOC surface metrics.
+
+```bash
+export JAVA_HOME=/usr/local/opt/openjdk@21/libexec/openjdk.jdk/Contents/Home
+export ANDROID_HOME=/usr/local/share/android-commandlinetools
+cd /Users/claw/work/forma
+BENCH_ASSEMBLE=1 bash scripts/bench-nia-migration.sh
+# writes docs/benchmarks/nia-migration-<UTC stamp>/
+```
+
+**Primary metric to quote:** Gradle profile **Configuring Projects** on cold/warm
+`help`. Assemble wall clock is recorded but confounded (tooling graph, wrapper
+pin, cache warmth) — see RESULTS caveats.
+
+Also see [`docs/CONFIGURATION-PERFORMANCE.md`](../CONFIGURATION-PERFORMANCE.md)
+and case study §3.5 in [`docs/DOGFOOD-NIA-CASE-STUDY.md`](../DOGFOOD-NIA-CASE-STUDY.md).
diff --git a/docs/benchmarks/nia-migration-20260803T174730Z/RESULTS.md b/docs/benchmarks/nia-migration-20260803T174730Z/RESULTS.md
@@ -0,0 +1,97 @@
+# NiA → Forma migration benchmark results
+
+Generated by `scripts/bench-nia-migration.sh` on worker host **2026-08-03**.
+
+## Host
+
+```
+stamp=20260803T174730Z
+host=Claws-MacBook-Pro.local
+date=Mon Aug  3 10:47:30 PDT 2026
+java=openjdk version "21.0.11" 2026-04-21
+JAVA_HOME=/usr/local/opt/openjdk@21/libexec/openjdk.jdk/Contents/Home
+ANDROID_HOME=/usr/local/share/android-commandlinetools
+UPSTREAM_DIR=/Users/claw/work/nowinandroid
+SPIKE_DIR=/Users/claw/work/nowinandroid-forma/forma-spike
+BENCH_ASSEMBLE=1
+BENCH_OFFLINE=0
+```
+
+## LOC surface (re-measured)
+
+| Metric | Upstream | Spike | Δ |
+|--------|----------|-------|---|
+| Module `build.gradle.kts` LOC | 1476 | 600 | -59% |
+| Org build logic Kotlin LOC | 1917 | 401 | -79% |
+| Spike `.withPlugin` | — | 0 | |
+
+## Timing (same host, corrected profile parse)
+
+| Phase | Metric | Upstream NiA | Forma spike | Δ |
+|-------|--------|--------------|-------------|---|
+| Cold `help` (no CC, profile) | Wall clock | 22.84s (22844 ms) | 19.00s (18996 ms) | -3.85s (-17%) |
+| Cold `help` (no CC, profile) | Profile: total build | 20.75s (20755 ms) | 17.12s (17121 ms) | -3.63s (-18%) |
+| Cold `help` (no CC, profile) | Profile: configuring projects | 12.51s (12510 ms) | 5.14s (5137 ms) | -7.37s (-59%) |
+| Cold `help` (no CC, profile) | Profile: loading projects | 1.25s (1253 ms) | 149 ms | -1.10s (-88%) |
+| Cold `help` (no CC, profile) | Profile: task execution | 302 ms | 220 ms | -82 ms (-27%) |
+| Cold `help` (no CC, profile) | Profile: startup | 5.12s (5123 ms) | 6.57s (6569 ms) | +1.45s (+28%) |
+| Warm `help` (no CC, profile) | Wall clock | 4.09s (4092 ms) | 3.28s (3279 ms) | -813 ms (-20%) |
+| Warm `help` (no CC, profile) | Profile: total build | 3.82s (3823 ms) | 3.06s (3056 ms) | -767 ms (-20%) |
+| Warm `help` (no CC, profile) | Profile: configuring projects | 1.58s (1581 ms) | 714 ms | -867 ms (-55%) |
+| Warm `help` (no CC, profile) | Profile: loading projects | 106 ms | 29 ms | -77 ms (-73%) |
+| Warm `help` (no CC, profile) | Profile: task execution | 63 ms | 27 ms | -36 ms (-57%) |
+| Warm `help` (no CC, profile) | Profile: startup | 675 ms | 1.03s (1030 ms) | +355 ms (+53%) |
+| CC miss `help` | Wall clock | 5.23s (5232 ms) | 4.00s (3995 ms) | -1.24s (-24%) |
+| CC miss `help` | Profile: total build | 5.00s (5005 ms) | 3.80s (3798 ms) | -1.21s (-24%) |
+| CC miss `help` | Profile: configuring projects | 2.51s (2509 ms) | 979 ms | -1.53s (-61%) |
+| CC miss `help` | Profile: loading projects | 252 ms | 71 ms | -181 ms (-72%) |
+| CC miss `help` | Profile: task execution | 77 ms | 32 ms | -45 ms (-58%) |
+| CC miss `help` | Profile: startup | 1.27s (1272 ms) | 1.42s (1416 ms) | +144 ms (+11%) |
+| CC hit `help` | Wall clock | 1.24s (1244 ms) | 1.34s (1336 ms) | +92 ms (+7%) |
+| CC hit `help` | Profile: total build | 990 ms | 1.09s (1095 ms) | +105 ms (+11%) |
+| CC hit `help` | Profile: configuring projects | 0 ms | 0 ms | +0 ms |
+| CC hit `help` | Profile: loading projects | 0 ms | 0 ms | +0 ms |
+| CC hit `help` | Profile: task execution | 1 ms | 2 ms | +1 ms (+100%) |
+| CC hit `help` | Profile: startup | 0 ms | 0 ms | +0 ms |
+| assembleDemoDebug (profile) | Wall clock | 278.82s (278820 ms) | 64.70s (64698 ms) | -214.12s (-77%) |
+| assembleDemoDebug (profile) | Profile: total build | 276.61s (276610 ms) | 62.38s (62380 ms) | -214.23s (-77%) |
+| assembleDemoDebug (profile) | Profile: configuring projects | 12.31s (12309 ms) | 4.21s (4209 ms) | -8.10s (-66%) |
+| assembleDemoDebug (profile) | Profile: loading projects | 1.36s (1359 ms) | 191 ms | -1.17s (-86%) |
+| assembleDemoDebug (profile) | Profile: task execution | 937.65s (937650 ms) | 180.91s (180910 ms) | -756.74s (-81%) |
+| assembleDemoDebug (profile) | Profile: startup | 5.45s (5448 ms) | 6.32s (6318 ms) | +870 ms (+16%) |
+
+## Headline (config-time)
+
+| Metric | Upstream | Spike | Takeaway |
+|--------|----------|-------|----------|
+| Cold configuring projects | 12.51s (12510 ms) | 5.14s (5137 ms) | **~59% less** config work on spike |
+| Warm configuring projects | 1.58s (1581 ms) | 714 ms | **~55% less** |
+| CC hit `help` wall | 1.24s (1244 ms) | 1.34s (1336 ms) | parity (~1.2–1.3s) |
+
+## Assemble note (confounded)
+
+Wall clock assembleDemoDebug: upstream **278.82s (278820 ms)** vs spike **64.70s (64698 ms)**.
+
+Do **not** quote the large assemble wall-clock delta as pure Forma gain without caveats:
+
+- Upstream NiA graph includes more tooling modules (Roborazzi, baseline profiles, lint/catalog paths).
+- First full task graph after cold caches vs partially warm artifact cache differs by tree.
+- Gradle wrappers differ (**9.4.0** upstream pin vs **9.6.1** spike).
+- Config slice of assemble still favors spike (**~12.3s → ~4.2s** configuring projects), matching the `help` pair.
+- Profile **Task Execution** on assemble is the **sum of task durations** (can exceed wall clock under parallel execution) — use wall clock for end-to-end, configuring projects for config cost.
+
+## How to reproduce
+
+```bash
+export JAVA_HOME=/usr/local/opt/openjdk@21/libexec/openjdk.jdk/Contents/Home
+export ANDROID_HOME=/usr/local/share/android-commandlinetools
+cd /Users/claw/work/forma
+BENCH_ASSEMBLE=1 bash scripts/bench-nia-migration.sh
+```
+
+## Raw
+
+- `results.tsv`
+- `profiles/*.html`
+- `logs/`
+
diff --git a/docs/benchmarks/nia-migration-20260803T174730Z/loc.json b/docs/benchmarks/nia-migration-20260803T174730Z/loc.json
@@ -0,0 +1,13 @@
+{
+  "upstream_module_build_files": 36,
+  "upstream_module_build_loc": 1476,
+  "spike_module_build_files": 37,
+  "spike_module_build_loc": 600,
+  "upstream_build_logic_kt_files": 28,
+  "upstream_build_logic_kt_loc": 1917,
+  "spike_forma_defs_kt_files": 5,
+  "spike_forma_defs_kt_loc": 401,
+  "spike_raw_android_blocks": 1,
+  "spike_withPlugin": 0,
+  "spike_withPlugin_tree": 0
+}
diff --git a/docs/benchmarks/nia-migration-20260803T174730Z/logs/bench.log b/docs/benchmarks/nia-migration-20260803T174730Z/logs/bench.log
@@ -0,0 +1,37 @@
+[bench] LOC metrics
+[bench] === tree=upstream dir=/Users/claw/work/nowinandroid ===
+[bench] pre-resolve help (online) for upstream
+[bench] RUN upstream/cold_help_1 in /Users/claw/work/nowinandroid — help --no-configuration-cache --profile
+[bench]   wall_ms=22844 exit=0 log=/Users/claw/work/forma/docs/benchmarks/nia-migration-20260803T174730Z/logs/upstream-cold_help_1.log
+[bench]   profile → /Users/claw/work/forma/docs/benchmarks/nia-migration-20260803T174730Z/profiles/upstream-cold_help_1.html
+[bench] RUN upstream/warm_help_1 in /Users/claw/work/nowinandroid — help --no-configuration-cache --profile
+[bench]   wall_ms=4092 exit=0 log=/Users/claw/work/forma/docs/benchmarks/nia-migration-20260803T174730Z/logs/upstream-warm_help_1.log
+[bench]   profile → /Users/claw/work/forma/docs/benchmarks/nia-migration-20260803T174730Z/profiles/upstream-warm_help_1.html
+[bench] RUN upstream/cc_miss_help in /Users/claw/work/nowinandroid — help --configuration-cache --profile
+[bench]   wall_ms=5232 exit=0 log=/Users/claw/work/forma/docs/benchmarks/nia-migration-20260803T174730Z/logs/upstream-cc_miss_help.log
+[bench]   profile → /Users/claw/work/forma/docs/benchmarks/nia-migration-20260803T174730Z/profiles/upstream-cc_miss_help.html
+[bench] RUN upstream/cc_hit_help in /Users/claw/work/nowinandroid — help --configuration-cache --profile
+[bench]   wall_ms=1244 exit=0 log=/Users/claw/work/forma/docs/benchmarks/nia-migration-20260803T174730Z/logs/upstream-cc_hit_help.log
+[bench]   profile → /Users/claw/work/forma/docs/benchmarks/nia-migration-20260803T174730Z/profiles/upstream-cc_hit_help.html
+[bench] RUN upstream/assemble_demo_debug in /Users/claw/work/nowinandroid — :app:assembleDemoDebug --profile
+[bench]   wall_ms=278820 exit=0 log=/Users/claw/work/forma/docs/benchmarks/nia-migration-20260803T174730Z/logs/upstream-assemble_demo_debug.log
+[bench]   profile → /Users/claw/work/forma/docs/benchmarks/nia-migration-20260803T174730Z/profiles/upstream-assemble_demo_debug.html
+[bench] === tree=spike dir=/Users/claw/work/nowinandroid-forma/forma-spike ===
+[bench] pre-resolve help (online) for spike
+[bench] RUN spike/cold_help_1 in /Users/claw/work/nowinandroid-forma/forma-spike — help --no-configuration-cache --profile
+[bench]   wall_ms=18996 exit=0 log=/Users/claw/work/forma/docs/benchmarks/nia-migration-20260803T174730Z/logs/spike-cold_help_1.log
+[bench]   profile → /Users/claw/work/forma/docs/benchmarks/nia-migration-20260803T174730Z/profiles/spike-cold_help_1.html
+[bench] RUN spike/warm_help_1 in /Users/claw/work/nowinandroid-forma/forma-spike — help --no-configuration-cache --profile
+[bench]   wall_ms=3279 exit=0 log=/Users/claw/work/forma/docs/benchmarks/nia-migration-20260803T174730Z/logs/spike-warm_help_1.log
+[bench]   profile → /Users/claw/work/forma/docs/benchmarks/nia-migration-20260803T174730Z/profiles/spike-warm_help_1.html
+[bench] RUN spike/cc_miss_help in /Users/claw/work/nowinandroid-forma/forma-spike — help --configuration-cache --profile
+[bench]   wall_ms=3995 exit=0 log=/Users/claw/work/forma/docs/benchmarks/nia-migration-20260803T174730Z/logs/spike-cc_miss_help.log
+[bench]   profile → /Users/claw/work/forma/docs/benchmarks/nia-migration-20260803T174730Z/profiles/spike-cc_miss_help.html
+[bench] RUN spike/cc_hit_help in /Users/claw/work/nowinandroid-forma/forma-spike — help --configuration-cache --profile
+[bench]   wall_ms=1336 exit=0 log=/Users/claw/work/forma/docs/benchmarks/nia-migration-20260803T174730Z/logs/spike-cc_hit_help.log
+[bench]   profile → /Users/claw/work/forma/docs/benchmarks/nia-migration-20260803T174730Z/profiles/spike-cc_hit_help.html
+[bench] RUN spike/assemble_demo_debug in /Users/claw/work/nowinandroid-forma/forma-spike — :binary:assembleDemoDebug --profile
+[bench]   wall_ms=64698 exit=0 log=/Users/claw/work/forma/docs/benchmarks/nia-migration-20260803T174730Z/logs/spike-assemble_demo_debug.log
+[bench]   profile → /Users/claw/work/forma/docs/benchmarks/nia-migration-20260803T174730Z/profiles/spike-assemble_demo_debug.html
+[bench] DONE out=/Users/claw/work/forma/docs/benchmarks/nia-migration-20260803T174730Z
+[bench] See /Users/claw/work/forma/docs/benchmarks/nia-migration-20260803T174730Z/RESULTS.md
diff --git a/docs/benchmarks/nia-migration-20260803T174730Z/logs/loc.txt b/docs/benchmarks/nia-migration-20260803T174730Z/logs/loc.txt
@@ -0,0 +1,13 @@
+{
+  "upstream_module_build_files": 36,
+  "upstream_module_build_loc": 1476,
+  "spike_module_build_files": 37,
+  "spike_module_build_loc": 600,
+  "upstream_build_logic_kt_files": 28,
+  "upstream_build_logic_kt_loc": 1917,
+  "spike_forma_defs_kt_files": 5,
+  "spike_forma_defs_kt_loc": 401,
+  "spike_raw_android_blocks": 1,
+  "spike_withPlugin": 0,
+  "spike_withPlugin_tree": 0
+}