run3-speed: row groups, module-close gate, lint diff, hot reload

CHANGELOG.md

@@ -19,6 +19,10 @@ Sections dated before 2026-09-19 predate the cycle and stay as they are. Named m
 
 ## Unreleased
 
+- new(exec, row-groups): **row-group gate**: `bin/row-groups.sh` plans scripts from the build plan (consecutive low-risk BUILD rows of one module, at most 4, in one script; security, workflow, agents, REST, Java, Marketplace and changes to built documents stay one row per script), and `exec.sh` refuses a split low-risk group, a mixed high-risk script, or the first script past a module with no `docs/progress/module-close/<Module>.md`. The gate checks a script that names its rows in a `-- rows:` header, or, when PROJECT.md says `Row groups: on`, its file name; otherwise it is skipped, because `brd-to-build-plan.md` numbers scripts on their own sequence (on the real SampleDashboards plan `16-cutover.mdl` is row 26 and was refused as row 16; `tests/wave2/test-row-groups.sh`). A row counts as built only when its State says built / done / complete / verified / proven / passed / applied (`planned`, `partial`, `**blocked**` and an empty State do not), or when an applied script's `-- rows:` header names it. Override: `MXTK_ROWS_OVERRIDE="<why>"` (ProcureFlow run 3 arm D: 0.93 → 0.48 scripts per row, 0 overrides; build to Test 2 12/12 in 322 min vs arm C stopped at row 121 after 650) — cook-off ProcureFlow run 2/3
+- fix(exec, lint): **lint prints and counts only the findings THIS script added, plus any earlier one still unfixed**, diffed against the previous exec (`.claude/loop/lint-prev-exec.json`); a finding that was reported and is neither fixed nor accepted into the baseline is carried (`.claude/loop/lint-unacked.json`) and stays "lint rose" on every later exec, and a diff that cannot run or cannot read `lint-last.json` (missing, unparseable, no `new_findings`), or itemises nothing with no previous exec to explain the rise, keeps the rise as lint-gate returned it (and does not advance `lint-prev-exec.json`, so the next exec still reads it as a rise), never "0 new" (`tests/wave2/test-exec-lint-diff.sh`); the baseline cannot be refreshed unattended, so "rose vs baseline" stayed true for every later exec (run 2 arm C: ROSE on 123 of 147 rows, ~60 lines each, 6 real SEC010 unread) — cook-off ProcureFlow run 2
+- new(reload): **`bin/reload.sh` hot-reloads instead of restarting** and finds the app URL from `APP_URL`, then `.docker/.env` `APP_PORT`, then CLAUDE.md (run 2 arm C: 176 app starts; run 3 arm D: 33, 1 failed); it and `bin/module-close.sh` find the `.mpr` and mxcli through `_common.sh`, so a two-tree checkout (`.mpr` under `app/`) works, and `reload.sh` reads the `.docker/` folder beside the `.mpr`, where mxcli writes it (`test-module-close.sh` C2, R1); `bin/module-close.sh` runs the module-close checks and lists pages still owed a LOOK; its report carries `<!-- mxtk-module-close: lint=N vm=RC -->`, and the row-group gate counts a module as closed only at `lint=0` or with a filled row in the report's Page | Role LOOK table, so a report with findings and no page looked at does not open the next module (`tests/wave2/test-row-groups.sh` M1, `test-module-close.sh` with a captured mxcli v0.28.0 lint) — cook-off ProcureFlow run 2/3
+- process(iterative-build-loop): **a script may carry several low-risk rows of one module; high-risk rows stay one per script**, and the module-close report is owed before the next module's scripts — cook-off ProcureFlow run 2/3
 - learn(page-patterns, file-upload-widget): **the last two unpromoted traps from the 2026-10-12 retro**: a snippet's top-level text has no context object, so wrap it in a DataView on the snippet parameter (`learned-page-patterns.md`); and on the Nova theme the File Uploader's three icon variables point at `/resources/images/` files nothing ships, so the icons are blank with three 404s while every gate stays green (`learned-file-upload-widget.md` traps table). The other retro facts were already covered: watch-mode and calculated attributes (`ui-loop.md`), Datagrid 2 / Atlas input traps (lint UI001), licence session cap. On-demand files, 0 baseline words. — retro 2026-10-12, Nova trap from AlvinnYeo (#282)
 - learn(microflow-patterns): **an object saved only through `commitInSeparateDatabaseTransaction` is still new in the user's request**; after browser Back the runtime checks Create on it, and a role without Create gets "Attempt to illegally add an object to the state" (HTTP 560) on every server-backed widget of the page. Fix: `rollback $X refresh` before the microflow ends; never grant Create or `commit` the request's copy. Added beside "Commit / Change / Rollback" (on demand; `rules-microflow.md` re-rendered). The proposed `learned-detection-gaps.md` register row is baseline text and waits for its A/B. — AlvinnYeo (#565)
 - learn(mendix-agent-ui): **new section "Stock popup variant (`layout-chat-popup`)"**: the popup header text is the popup page's Title; its `h4` is invisible by default (same colour as the header background) and is fixed with a token-scoped rule in `main.scss`, never a literal or `!important`; the width is a shared `!default` variable set in `custom-variables.scss`, and it also widens the sidebar chat. One checklist line added. The routing trigger is unchanged (that would add words to the mdl agent stub). On-demand file, 0 baseline words. — joshuamoesa (#574)

bin/lib/install-manifest.sh

@@ -128,7 +128,7 @@ MXTK_AGENTS_STAGE_BUILD="mdl-agent.md gate-agent.md test-agent.md review-agent.m
 # runs it per module (informational; see project-bin/mf-canvas-facts.py header).
 # model-scope.py joined on 2026-10-21: it imports mxpatch.py (and through it mxunit_bson.py) from
 # its own directory. It puts back units a write changed only in XPath whitespace (mxcli#1596).
-MXTK_PROJECT_BIN="process-map.js process-map.template.html app-restart.sh test-scratch.sh script-owners.sh describe-baseline.sh assemble-prototype.js prototype-route.js check-prototype-links.js wf-add-path-terminators.py wf-set-call-captions.py model-scope.py mf-canvas-facts.py mxunit_bson.py mxpatch.py _common.sh app-facts.sh _claims.sh snapshot-mpr.sh restore-mpr.sh exec.sh affected-tests.sh look-ledger.sh save-sp.sh restart-sp.sh check-sp-health.sh verify-module.sh harness-scan.sh test-stack-up.sh fixture-manifest.sh check-root-clean.sh lint-gate.sh lint-trend.sh metrics-ledger.sh metrics-trend.sh close-task.sh conformance-check.sh coverage-preflight.sh graph-sweep.sh review-module.sh coherence-cadence.sh build-plan-status.sh progress-report.py done-drift-check.sh page-scope.sh render-improvement-register.sh check-design-portability.sh check-design-reaches-app.sh check-page-shell.sh page-fidelity.js model-stamp.sh verify-model.sh install-project-hooks.sh mx-metadata-note.py session-check.sh constants-audit.sh context-pack.sh check-staged-secrets.sh"
+MXTK_PROJECT_BIN="reload.sh module-close.sh row-groups.sh process-map.js process-map.template.html app-restart.sh test-scratch.sh script-owners.sh describe-baseline.sh assemble-prototype.js prototype-route.js check-prototype-links.js wf-add-path-terminators.py wf-set-call-captions.py model-scope.py mf-canvas-facts.py mxunit_bson.py mxpatch.py _common.sh app-facts.sh _claims.sh snapshot-mpr.sh restore-mpr.sh exec.sh affected-tests.sh look-ledger.sh save-sp.sh restart-sp.sh check-sp-health.sh verify-module.sh harness-scan.sh test-stack-up.sh fixture-manifest.sh check-root-clean.sh lint-gate.sh lint-trend.sh metrics-ledger.sh metrics-trend.sh close-task.sh conformance-check.sh coverage-preflight.sh graph-sweep.sh review-module.sh coherence-cadence.sh build-plan-status.sh progress-report.py done-drift-check.sh page-scope.sh render-improvement-register.sh check-design-portability.sh check-design-reaches-app.sh check-page-shell.sh page-fidelity.js model-stamp.sh verify-model.sh install-project-hooks.sh mx-metadata-note.py session-check.sh constants-audit.sh context-pack.sh check-staged-secrets.sh"
 
 # Files in project-bin/ that are deliberately NOT installed into projects. The reverse check
 # below flags anything named by NEITHER list, so a new file in project-bin/ has to be either

project-bin/exec.sh

@@ -514,7 +514,7 @@ HDR2
   # The gate cell is read from GATE_STATE rather than passed in, so no caller
   # can omit it — a blank cell in this table reads as "fine", which is the exact
   # false-green the gate exists to prevent.
-  _detail="$2${CHANGE_CLASS:+ · change: $CHANGE_CLASS}${CR_ID:+ · cr: $CR_ID}${SCAN_NOTE:+ · $SCAN_NOTE}${RESTATE_NOTE:+ · $RESTATE_NOTE}${GATE_NOTE:+ · $GATE_NOTE}${MXB_LOG_NOTE:+ · $MXB_LOG_NOTE}${HV_NOTE:+ · $HV_NOTE}$_EXEC_APPROVAL_SUFFIX${STEP_TIMES:+ · steps: $STEP_TIMES}"
+  _detail="$2${CHANGE_CLASS:+ · change: $CHANGE_CLASS}${CR_ID:+ · cr: $CR_ID}${SCAN_NOTE:+ · $SCAN_NOTE}${ROWS_NOTE:+ · $ROWS_NOTE}${RESTATE_NOTE:+ · $RESTATE_NOTE}${GATE_NOTE:+ · $GATE_NOTE}${MXB_LOG_NOTE:+ · $MXB_LOG_NOTE}${HV_NOTE:+ · $HV_NOTE}$_EXEC_APPROVAL_SUFFIX${STEP_TIMES:+ · steps: $STEP_TIMES}"
   printf '| %s | `%s` | %s | %s | %s |\n' \
     "$(date '+%Y-%m-%dT%H:%M:%S%z')" "$LOG_NAME" \
     "${GATE_STATE:-not-run}" "$1" "$_detail" >> "$BUILD_LOG"
@@ -607,6 +607,28 @@ if [ "$PATCH_MODE" != "1" ]; then
   fi
 fi
 
+# ── Row-group gate (run 3, 2026-10-27) ──────────────────────────────────────
+# bin/row-groups.sh check: low-risk build-plan rows of one group go in ONE script; a high-risk row
+# (security, agents, workflow, REST, Java, changes to built documents) goes alone. Runs here, before
+# the lock and the snapshot, for every caller of exec.sh, agent or person. Run 2 arm C paid ~2 min of
+# instruments per exec on 147 rows. No build plan, or a script that names no row: skipped.
+# Override, with a reason that lands in the BUILD-LOG row: MXTK_ROWS_OVERRIDE="<why>".
+ROWS_NOTE=""
+if [ "$PATCH_MODE" != "1" ] && [ -x "$PROJECT_ROOT/bin/row-groups.sh" ]; then
+  _rg_rc=0; _rg_out="$("$PROJECT_ROOT/bin/row-groups.sh" check "$SCRIPT" 2>&1)" || _rg_rc=$?
+  if [ "$_rg_rc" = 1 ]; then
+    echo "$_rg_out"
+    if [ -z "${MXTK_ROWS_OVERRIDE:-}" ]; then
+      echo "✗ Row-group gate: refusing exec. NOTHING was locked, snapshotted or written."
+      echo "  See the groups: bin/row-groups.sh. Override with a reason: MXTK_ROWS_OVERRIDE=\"<why>\" ./bin/exec.sh $SCRIPT"
+      log_build "🚫 blocked" "row-group gate: $(printf '%s' "$_rg_out" | head -1 | tr '|' '/' | cut -c1-200) — nothing written"
+      exit 1
+    fi
+    echo "⚠ Row-group gate OVERRIDDEN: ${MXTK_ROWS_OVERRIDE}" >&2
+    ROWS_NOTE="rows override: $(printf '%s' "$MXTK_ROWS_OVERRIDE" | tr '|' '/')"
+  fi
+fi
+
 # ── Concurrent-writer guard ──────────────────────────────────────────────────
 # The .mpr must have exactly ONE writer. Two mxcli/SP writers on the same file
 # silently clobber each other and have caused near-total module loss.
@@ -972,6 +994,7 @@ fi
 # to stop. ~13 s measured on a 10k-finding project.
 LINT_RC=0; LINT_DETAIL="lint not-run"
 run_lint_gate() {
+  _noadv=""
   if [ -n "${SKIP_LINT:-}" ]; then
     LINT_DETAIL="lint SKIPPED (SKIP_LINT=$SKIP_LINT)"
     echo "  ⚠ lint ratchet skipped: SKIP_LINT=$SKIP_LINT"; return
@@ -1050,12 +1073,65 @@ run_lint_gate() {
          sed -n '/^!! A RULE COULD NOT READ/,/^$/p' "$_lo" | head -12 | sed 's/^/    /'
          echo "    Configure or fix the rule (see the message above); do not re-baseline over it."
        else
-         LINT_DETAIL="lint ROSE: ${_rules:-see output}${_blind:+; BLIND: $_blind}"
-         echo "  ✗ lint: rule(s) rose vs baseline — ${_rules:-see below}"
-         sed -n '/^FAIL/,$p' "$_lo" | head -60 | sed 's/^/    /'
+         # New THIS script (run 3, 2026-10-27). The baseline cannot be refreshed unattended, so
+         # "rose vs baseline" stays true for every later exec once one script adds a finding:
+         # run 2 arm C logged ROSE on 123 of 147 rows and printed ~60 lines each time, and the 6
+         # real SEC010 findings in it went unread. Compare with the previous exec's findings
+         # (.claude/loop/lint-prev-exec.json); only what this script added is printed and counts.
+         IFS= read -r -d '' _thispy <<'PY' || true
+import json, os, sys
+d = sys.argv[1]
+def load(n):
+    try: return {(r, f) for r, fs in (json.load(open(os.path.join(d, n))).get("new_findings") or {}).items() for f in fs}
+    except (OSError, ValueError): return set()
+# Unreadable lint-last.json, or one without a new_findings object: nothing to itemise, so print
+# nothing and let the shell keep the rise as lint-gate returned it (never downgrade to "0 new").
+try:
+    if not isinstance(json.load(open(os.path.join(d, "lint-last.json"))).get("new_findings"), dict): sys.exit(0)
+except (OSError, ValueError, AttributeError): sys.exit(0)
+cur = load("lint-last.json")
+added = cur - load("lint-prev-exec.json")
+# Carried: reported before, still in lint-last (so neither fixed nor accepted into the baseline).
+# Without it a finding an agent ignored once read "0 new" on every later exec (PR #581 review).
+carried = (load("lint-unacked.json") & cur) - added
+new = sorted(added | carried)
+# Rise reported but nothing itemised and no previous exec to explain it: not provably old debt.
+if not new and not os.path.exists(os.path.join(d, "lint-prev-exec.json")): sys.exit(0)
+try:
+    by_r = {}
+    for r, f in new: by_r.setdefault(r, []).append(f)
+    json.dump({"new_findings": by_r}, open(os.path.join(d, "lint-unacked.json"), "w"))
+except OSError:
+    pass
+by = {}
+for r, _ in new: by[r] = by.get(r, 0) + 1
+print(len(new), len(carried), " ".join(f"{r}:{n}" for r, n in sorted(by.items())))
+for r, f in new[:10]: print(f"    {r}  {f[:180]}" + ("   (carried: still unfixed)" if (r, f) in carried else ""))
+if len(new) > 10: print(f"    … {len(new) - 10} more in .claude/loop/lint-last.json")
+PY
+         _thispyexe="${PY:-$(resolve_py 2>/dev/null || true)}"
+         _thisnew=$([ -n "$_thispyexe" ] && "$_thispyexe" -c "$_thispy" "$PROJECT_ROOT/.claude/loop")
+         _thisn=$(printf '%s\n' "$_thisnew" | head -1 | cut -d' ' -f1)
+         _thisc=$(printf '%s\n' "$_thisnew" | head -1 | cut -d' ' -f2)
+         if [ -z "$_thisnew" ]; then
+           _noadv=1   # undiffed rise: keep prev-exec as is, or the next exec reads these as old debt
+           # No Python, or the diff failed: never read that as "0 new". Report the rise vs baseline.
+           LINT_DETAIL="lint rose: ${_rules:-see output} (per-script diff unavailable)"
+           echo "  ✗ lint: rule(s) rose vs baseline — ${_rules:-see below} (per-script diff unavailable)"
+           sed -n '/^FAIL/,$p' "$_lo" | head -60 | sed 's/^/    /'
+         elif [ "${_thisn:-0}" = 0 ]; then
+           LINT_RC=0
+           LINT_DETAIL="lint: 0 new this script (baseline debt: ${_rules:-see lint-last.json})"
+           echo "  ✓ lint: this script added no finding (older debt vs baseline unchanged)"
+         else
+         # "lint rose" stays the wording, as before this change (metrics and readers key on it).
+         LINT_DETAIL="lint rose: +${_thisn} new or unfixed (${_thisc:-0} carried from earlier execs): $(printf '%s\n' "$_thisnew" | head -1 | cut -d' ' -f3-)"
+         echo "  ✗ lint: ${_thisn} finding(s) new this script or still unfixed from an earlier one — read each one:"
+         printf '%s\n' "$_thisnew" | tail -n +2
          echo "    Fix the script and re-run. Accepting the debt is the user's call, made in chat:"
          echo "    bin/lint-gate.sh --update-baseline --reason \"<why>\" (a security-tier rise also"
          echo "    needs --approved-in-chat). The acceptance is logged to .claude/lint-acceptances.tsv."
+         fi
        fi ;;
     *) LINT_DETAIL="lint could not run (lint-gate exit $LINT_RC)"; echo "  ⚠ $LINT_DETAIL"
        tail -5 "$_lo" | sed 's/^/    /' ;;
@@ -1064,6 +1140,8 @@ run_lint_gate() {
   # should know the files changed under them.
   [ -n "$_restored" ] && [ "$LINT_RC" -ne 3 ] && LINT_DETAIL="$LINT_DETAIL; toolkit lint rules auto-restored after a stock re-seed (${_restored})"
   rm -f "$_lo"
+  # The next exec compares against what lint saw now.
+  case "$LINT_RC" in 0|1) [ -n "${_noadv:-}" ] || cp "$PROJECT_ROOT/.claude/loop/lint-last.json" "$PROJECT_ROOT/.claude/loop/lint-prev-exec.json" 2>/dev/null || true ;; esac
 }
 # One place decides the applied-row wording, so both mxbuild-clean branches agree.
 log_applied() {  # $1 = mxbuild detail