From 15daa7c1592302d2b8da295c8561b3d8e6a4b818 Mon Sep 17 00:00:00 2001 From: MendixMau Date: Tue, 27 Oct 2026 07:41:25 +0100 Subject: [PATCH 01/13] run3-speed: per-module checks, row groups, diff on overlap, hot reload via --direct Local experiment branch for cook-off run 3 (arm D). Not for the shared toolkit unless promoted. - project-bin/reload.sh: mxcli docker reload --direct (no curl in the JRE image), --css, --restart; falls back to a restart and says so. - project-bin/module-close.sh : lint (module's warnings), score, mx check, verify-module --quick, LOOK owed, into docs/progress/module-close/.md. - iterative-build-loop.md: low-risk rows of one module may share a script; diff only when a landed apply wrote a document the draft also writes. - install-manifest: both new scripts. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01XM8AzTxUsYcriPXQ58ZPEV --- bin/lib/install-manifest.sh | 2 +- project-bin/module-close.sh | 53 ++++++++++++++++++++++++++++++++++ project-bin/reload.sh | 46 +++++++++++++++++++++++++++++ skills/iterative-build-loop.md | 17 +++++++++-- 4 files changed, 114 insertions(+), 4 deletions(-) create mode 100755 project-bin/module-close.sh create mode 100755 project-bin/reload.sh diff --git a/bin/lib/install-manifest.sh b/bin/lib/install-manifest.sh index 4e3c2da0..9dd941c8 100644 --- a/bin/lib/install-manifest.sh +++ b/bin/lib/install-manifest.sh @@ -128,7 +128,7 @@ MXTK_AGENTS_STAGE_BUILD="mdl-agent.md gate-agent.md test-agent.md review-agent.m # runs it per module (informational; see project-bin/mf-canvas-facts.py header). # model-scope.py joined on 2026-10-21: it imports mxpatch.py (and through it mxunit_bson.py) from # its own directory. It puts back units a write changed only in XPath whitespace (mxcli#1596). -MXTK_PROJECT_BIN="process-map.js process-map.template.html app-restart.sh test-scratch.sh script-owners.sh describe-baseline.sh assemble-prototype.js prototype-route.js check-prototype-links.js wf-add-path-terminators.py wf-set-call-captions.py model-scope.py mf-canvas-facts.py mxunit_bson.py mxpatch.py _common.sh app-facts.sh _claims.sh snapshot-mpr.sh restore-mpr.sh exec.sh affected-tests.sh look-ledger.sh save-sp.sh restart-sp.sh check-sp-health.sh verify-module.sh harness-scan.sh test-stack-up.sh fixture-manifest.sh check-root-clean.sh lint-gate.sh lint-trend.sh metrics-ledger.sh metrics-trend.sh close-task.sh conformance-check.sh coverage-preflight.sh graph-sweep.sh review-module.sh coherence-cadence.sh build-plan-status.sh progress-report.py done-drift-check.sh page-scope.sh render-improvement-register.sh check-design-portability.sh check-design-reaches-app.sh check-page-shell.sh page-fidelity.js model-stamp.sh verify-model.sh install-project-hooks.sh mx-metadata-note.py session-check.sh constants-audit.sh context-pack.sh check-staged-secrets.sh" +MXTK_PROJECT_BIN="reload.sh module-close.sh process-map.js process-map.template.html app-restart.sh test-scratch.sh script-owners.sh describe-baseline.sh assemble-prototype.js prototype-route.js check-prototype-links.js wf-add-path-terminators.py wf-set-call-captions.py model-scope.py mf-canvas-facts.py mxunit_bson.py mxpatch.py _common.sh app-facts.sh _claims.sh snapshot-mpr.sh restore-mpr.sh exec.sh affected-tests.sh look-ledger.sh save-sp.sh restart-sp.sh check-sp-health.sh verify-module.sh harness-scan.sh test-stack-up.sh fixture-manifest.sh check-root-clean.sh lint-gate.sh lint-trend.sh metrics-ledger.sh metrics-trend.sh close-task.sh conformance-check.sh coverage-preflight.sh graph-sweep.sh review-module.sh coherence-cadence.sh build-plan-status.sh progress-report.py done-drift-check.sh page-scope.sh render-improvement-register.sh check-design-portability.sh check-design-reaches-app.sh check-page-shell.sh page-fidelity.js model-stamp.sh verify-model.sh install-project-hooks.sh mx-metadata-note.py session-check.sh constants-audit.sh context-pack.sh check-staged-secrets.sh" # Files in project-bin/ that are deliberately NOT installed into projects. The reverse check # below flags anything named by NEITHER list, so a new file in project-bin/ has to be either diff --git a/project-bin/module-close.sh b/project-bin/module-close.sh new file mode 100755 index 00000000..1fde4c32 --- /dev/null +++ b/project-bin/module-close.sh @@ -0,0 +1,53 @@ +#!/usr/bin/env bash +# module-close.sh — the checks that run once per module, not once per script. +# +# bin/module-close.sh Procurement +# +# Runs, in order, and writes one report: docs/progress/module-close/.md +# 1. mxcli lint, whole model; the ⚠ lines that name this module are the module's findings +# 2. model score (bin/metrics-ledger.sh score → mxcli report), once +# 3. mxcli docker check (full mxbuild check) +# 4. bin/verify-module.sh --quick (conformance, graph sweep, coverage; diagnostic only) +# 5. bin/look-ledger.sh status: pages of this module still owed a LOOK +# Then the lead does what no script can: open each owed page as each role that can reach it, +# run the module's unhappy paths, and fix every ⚠ that is real. Record those in the same report. +# +# WHY (run 2, 2026-10-26, arm C). Lint and score ran after every exec (18.3 min) and the ratchet read +# ROSE on 123 of 147 rows, because the baseline cannot be refreshed unattended. Nobody read it. Yet +# the model at row 124 held 6 SEC010 ⚠ (agent-evidence lists empty for the roles that open the page). +# Meanwhile verify-module, gate-check and the LOOK ledger ran 0 times: nothing called them. +# One command at the module boundary makes the per-module pass cheap enough to actually run. +# +# Exit: 0 report written, no ⚠ for this module · 1 report written, findings to act on · 2 usage. +set -u +mod=${1:?usage: bin/module-close.sh } +cd "$(dirname "$0")/.." || exit 2 +MPR=$(ls ./*.mpr 2>/dev/null | head -1); [ -n "$MPR" ] || { echo "no .mpr here" >&2; exit 2; } +out=docs/progress/module-close; mkdir -p "$out"; rep="$out/$mod.md" +MXB=${MXTK_MXBUILD_PATH:-} +t0=$(date +%s); now() { date '+%F %H:%M'; } + +lint=$(./mxcli lint -p "$MPR" 2>&1 | grep -v WARNING) +mine=$(printf '%s\n' "$lint" | grep -A1 '⚠' | grep -B1 -E "\b$mod\." | grep '⚠' || true) +nmine=$(printf '%s' "$mine" | grep -c '⚠' || true) +score=$(bash bin/metrics-ledger.sh score 2>&1 | tail -1) +chk=$(./mxcli docker check -p "$MPR" ${MXB:+--mxbuild-path "$MXB"} 2>&1 | grep -v WARNING | tail -3) +vm=$(bash bin/verify-module.sh "$mod" --quick 2>&1); vm_rc=$?; vm=$(printf '%s\n' "$vm" | tail -15) +look=$(bash bin/look-ledger.sh status 2>/dev/null | grep -E "\b$mod\." || true) + +{ + echo "# Module close: $mod ($(now), $(( $(date +%s) - t0 )) s of instruments)" + echo + echo "## 1. Lint ⚠ in $mod: $nmine" + echo; [ -n "$mine" ] && printf '%s\n' "$mine" | sed 's/^ */- /' || echo "none" + echo; echo "Whole model: $(printf '%s\n' "$lint" | grep -c '⚠') ⚠, $(printf '%s\n' "$lint" | grep -c 'ℹ') ℹ." + echo; echo "## 2. Score"; echo; echo "$score" + echo; echo "## 3. mx check"; echo; echo '```'; echo "$chk"; echo '```' + echo; echo "## 4. verify-module --quick (exit $vm_rc)"; echo; echo '```'; echo "$vm"; echo '```' + echo; echo "## 5. LOOK owed in $mod"; echo; [ -n "$look" ] && printf '%s\n' "$look" | sed 's/^/- /' || echo "none recorded" + echo; echo "## 6. Lead: roles × pages, unhappy paths, fixes"; echo + echo "| Page | Role | Seen (screenshot path) | Defect | Fixed in row/script |" + echo "|---|---|---|---|---|" +} > "$rep" +echo "module-close $mod: $nmine lint ⚠, verify-module exit $vm_rc, report $rep" +[ "$nmine" -eq 0 ] || exit 1 diff --git a/project-bin/reload.sh b/project-bin/reload.sh new file mode 100755 index 00000000..b4b37d53 --- /dev/null +++ b/project-bin/reload.sh @@ -0,0 +1,46 @@ +#!/usr/bin/env bash +# reload.sh — put the current model into THIS project's running Docker app, keeping its data. +# +# bin/reload.sh build the package, then hot-reload the model (no container restart) +# bin/reload.sh --css styling only (theme/SCSS already compiled into the deployment) +# bin/reload.sh --restart full build + container restart (after a Java/GenAI class change, or when +# a reload left the app odd — run 1 cloud A-43: ClassNotFoundException) +# +# WHY (run 2, 2026-10-26, arm C). `mxcli docker reload` reaches the M2EE admin API by running curl +# INSIDE the container (mxcli cmd/mxcli/docker/m2ee.go, v0.24 and v0.28 alike), and the +# eclipse-temurin JRE image has no curl. Every reload failed with "sh: 1: curl: not found", so the +# lead restarted the container after each change instead. `--direct` calls the same API over HTTP +# from the host, through the mapped admin port, with no curl. +# +# What it saves is the restart and the wait for the port, not mxbuild: the package build stays. +# So refresh the app when you need to LOOK or test, not after every apply. +# +# Falls back to --restart when the reload fails, and says so. Prints the path taken and seconds. +# Exit: 0 app serving the new model · 1 not proven up · 2 usage. +set -u +cd "$(dirname "$0")/.." || exit 2 +MPR=$(ls ./*.mpr 2>/dev/null | head -1); [ -n "$MPR" ] || { echo "reload.sh: no .mpr here" >&2; exit 2; } +MXB=${MXTK_MXBUILD_PATH:-} +mxb() { [ -n "$MXB" ] && printf -- '--mxbuild-path\n%s\n' "$MXB"; } +mode=reload +case "${1:-}" in "") ;; --css) mode=css ;; --restart) mode=restart ;; *) sed -n 2,8p "$0"; exit 2 ;; esac + +# The admin port as mapped on the host (compose maps container 8090 to it). +admin=$(cd .docker 2>/dev/null && docker compose port mendix 8090 2>/dev/null | sed 's/.*://') +app_url=$(grep -m1 -o 'http://localhost:[0-9]*' CLAUDE.md 2>/dev/null); app_url=${app_url:-http://localhost:8080} +up() { for i in $(seq 1 90); do [ "$(curl -s -o /dev/null -w '%{http_code}' "$app_url/login.html")" = 200 ] && return 0; sleep 2; done; return 1; } + +t0=$(date +%s) +if [ "$mode" != restart ] && [ -n "$admin" ]; then + args=(-p "$MPR" --direct --port "$admin"); [ "$mode" = css ] && args+=(--css) || { args+=(--skip-check); while IFS= read -r a; do args+=("$a"); done < <(mxb); } + if out=$(./mxcli docker reload "${args[@]}" 2>&1); then + up && { echo "reload.sh: $mode via --direct OK in $(( $(date +%s) - t0 )) s"; exit 0; } + fi + echo "reload.sh: $mode failed, falling back to restart: $(printf '%s' "$out" | grep -v WARNING | tail -1)" +fi +[ -z "$admin" ] && echo "reload.sh: app not running (no admin port mapped); starting it" +args=(-p "$MPR"); while IFS= read -r a; do args+=("$a"); done < <(mxb) +./mxcli docker build "${args[@]}" 2>&1 | grep -E 'rror|BUILD|CE[0-9]' | tail -3 +./mxcli docker up -p "$MPR" --detach >/dev/null 2>&1 || (cd .docker && docker compose up -d >/dev/null 2>&1) +up && { echo "reload.sh: restart OK in $(( $(date +%s) - t0 )) s"; exit 0; } +echo "reload.sh: app not answering on $app_url after $(( $(date +%s) - t0 )) s"; exit 1 diff --git a/skills/iterative-build-loop.md b/skills/iterative-build-loop.md index 229f677a..f82033e9 100644 --- a/skills/iterative-build-loop.md +++ b/skills/iterative-build-loop.md @@ -391,7 +391,14 @@ blank `mxcli new` app plus one entity and microflow, 4 cores, 3 runs each): `mx passed a Java compile error. So a check-only gate needs a full deploy whenever `javasource/` or `theme/` changed and at module close — a gate redesign, parked on #444. -**Draft the ready set in parallel, apply serially.** The unit is the build-plan row. The ready +**Draft the ready set in parallel, apply serially.** The unit is the build-plan row, and a script may +carry several rows: low-risk rows of one module that are ready together (enumerations, entities and +associations, CRUD pages over them, simple sub-microflows) go in **one script**, so the per-apply loop +(check, apply, mxbuild gate, commit) is paid once for the group. Rows that touch security (roles, access +rules), workflows, AI agents, REST, Java actions or a STOP rule 21 target stay **one row per script**: +those are where a failed apply must roll back the least. A group script names all its rows in its +header and BUILD-LOG row (run 2, 2026-10-26: arm C wrote 0.93 scripts per row against the cloud's 0.34, +and paid a 22-128 s loop per script). The ready set is every `BUILD` row whose "Depends on" rows are all applied, minus overlapping writers: two rows whose scripts `create or replace`, `alter` or drop+create the same document (the row's written documents are listed in Produces, `brd-to-build-plan.md`) are never in one ready set, @@ -402,8 +409,12 @@ refuses a second); after each apply, recompute the ready set. The failure this p that drafted one script at a time and sat idle while each draft finished. The risk it adds: a queued draft can restate a document as it was before a sibling apply — STOP rule 21's incident, where the second of two drop+creates of a shared seed aggregator silently dropped a seed call -with `mx check` at 0 errors. So before applying a draft, if any apply has landed since it was -drafted, run `mxcli diff -p