Add a DeepWork review rule for scope discipline

.deepreview

@@ -0,0 +1,84 @@
+# DeepWork review: scope discipline.
+#
+# Catches the failure mode where a PR does what was asked AND a pile of things
+# that weren't — unrequested abstractions, defensive branches for impossible
+# states, drive-by refactors, narration comments. Each one is individually
+# defensible, which is why they accumulate; the cost lands on the reviewer, who
+# pays attention per line regardless of whether the line needed to exist.
+#
+# `all_changed_files` (tripwire): the match is only the trigger. Any source
+# change hands the reviewer the entire changeset, which is the only way to judge
+# proportionality — "is this diff bigger than the task required" is a question
+# about the whole, not about any one file.
+#
+# Deliberately ONE rule: each rule spawns its own sub-agent with real overhead,
+# and scope creep is a single judgement, not several.
+
+scope-discipline:
+  description: |
+    Flag additions the task did not require: unrequested abstractions and
+    helpers, error handling for states that cannot occur, unrelated cleanup
+    bundled into an unrelated change, comments that restate the code, and
+    redundant tests. Proportionality of the diff to the stated task.
+  match:
+    include:
+      - "**/*.py"
+      - "**/*.md"
+    exclude:
+      - ".venv/**"
+      - "**/migrations/versions/**"
+  review:
+    strategy: all_changed_files
+    instructions: |
+      Judge this changeset against one question: **is everything here required
+      by the task, and is the diff proportionate to it?**
+
+      Infer the task from the PR title, the branch name, and the commit
+      messages. If you genuinely cannot tell what was asked, say so and stop —
+      do not guess a narrower task and then flag everything outside it.
+
+      Flag each of these, with `file:line` and a one-line reason:
+
+      1. **Unrequested abstraction** — a helper, wrapper, base class, or
+         indirection introduced for a single call site. A one-shot operation
+         usually does not need a helper.
+      2. **Speculative generality** — parameters, config flags, or extension
+         points with exactly one caller and no stated requirement. Designing for
+         a hypothetical future requirement.
+      3. **Impossible-state handling** — try/except, null guards, or fallbacks
+         for conditions the surrounding code or a framework guarantee already
+         rules out. Validate at system boundaries (user input, external APIs),
+         not between two functions in the same module.
+      4. **Drive-by changes** — renames, reformatting, import reordering, or
+         refactors unrelated to the task. A bug fix does not need surrounding
+         cleanup. These are the most expensive kind of noise: they inflate the
+         diff while hiding the real change inside it.
+      5. **Narration comments** — comments that restate what the next line does,
+         explain why the change is correct, or address the reviewer rather than
+         the next reader. A comment should state a constraint the code cannot
+         show. Anything that stops being true once the PR merges is noise.
+      6. **Redundant tests** — a new test whose failure conditions are already
+         covered by an existing one. More tests is not automatically better;
+         each one is code to maintain.
+      7. **Disproportionate documentation** — a multi-section document, summary
+         file, or expanded README where the change warranted a sentence.
+
+      For each finding, state the concrete recommendation: **delete**, **split
+      into a separate PR**, or **keep** with the justification that makes it
+      required. Prefer "delete" — the default answer for something the task did
+      not ask for is that it should not be in this PR.
+
+      Then give a single overall verdict:
+
+      - **PROPORTIONATE** — the diff matches the task.
+      - **BLOATED** — name the two or three changes that would shrink it most,
+        and estimate the lines they would remove.
+
+      Two things to get right, because getting them wrong makes this rule worse
+      than useless:
+
+      - **Do not flag work the task actually required**, even where it is large.
+        A big diff is not itself a finding; a big diff for a small task is.
+      - **Do not flag missing work.** Other rules and human review cover
+        correctness, coverage, and completeness. This rule only ever argues for
+        removing things, never for adding them.

deploy/unsupervised-main-prod-testing/README.md

@@ -0,0 +1,63 @@
+# Enable panopticon agents to do finder prod-testing
+
+Give unsupervised-main task agents the ability to **provision test pods, generate the finder
+executable, and profile results** against production data — the workflow currently done by hand.
+
+Two phases:
+- **Interim (now, broad role):** test pods run in **`default`** as `unsupervised-unsupervised` (the
+  existing `prod-unsupervised-main` IRSA role) to read prod exports. **No cluster/credential/IAM
+  changes** — everything needed already exists in `default`. Trade-off: agents get the broad
+  (read+write) role and there is **no ResourceQuota** — the operator turn-handoff gate is the only
+  guardrail. Files marked "phase 2" below are NOT used here.
+- **Phase 2 (later, needs IAM-admin):** scoped `finder-repro` namespace + read-only IRSA SA +
+  ResourceQuota. Blocked until an IAM-admin creates the scoped role (the `Unsupervised-Engineer` SSO
+  role can't `iam:CreateRole`). `finder-repro-rbac.yaml` + `iam-finder-repro-readonly.json` + `apply.sh
+  scope-sa` are the phase-2 path, kept ready.
+
+Common to both: **in-pod finder build (no Docker-in-Docker)** from a pre-baked Harbor builder image,
+and every prod run gated by an **operator turn-handoff**.
+
+## Two identities (keep them straight)
+- **Control** — `panopticon-repro` SA: creates/execs pods (RBAC). No data access.
+- **Run** — `finder-test` SA: the SA the test *pods* run as; IRSA -> read-only prod S3. Data
+  access rides here, not on kubectl.
+
+## Files
+
+| File | Goes to | Purpose |
+|---|---|---|
+| `finder-repro-rbac.yaml` | prod cluster (`kubectl apply`) | `finder-repro` ns + `panopticon-repro` (control) + `finder-test` (run) SAs + Role + ResourceQuota/LimitRange |
+| `iam-finder-repro-readonly.json` | **AWS (operator creates)** | IAM role templates: trust (EKS OIDC + `finder-repro:finder-test`) + read-only s3 on `unsupervised-prod-internal/internal/*` |
+| `unsupervised-main/Dockerfile.finder-builder` | unsupervised-main (PR) | Pre-baked builder image (deps installed, source overlaid per build) |
+| `unsupervised-main/publish.finder-builder.yml` | unsupervised-main (PR) | CI publish job (reuses existing `HARBOR_USERNAME`/`HARBOR_PASSWORD`) |
+| `image-layer.head.dockerfile` | `$CONFIG/layers/` (via apply.sh) | kubectl + auto-wiring wrapper (materializes kubeconfig from the injected env var) |
+| `build-finder-in-pod.sh` | image layer `/usr/local/bin` | Agent-run in-pod build |
+| `repro-pod.template.yaml` | reference | A finder test pod running as `finder-test` (IRSA S3) |
+| `prod-testing-gate.md` | unsupervised-main AGENTS.md | The turn-handoff approval rule agents must follow |
+| `apply.sh` | operator runs | `config` (wire layer+repo) / `scope-sa` (prod RBAC + rewrite kubeconfig secret) |
+
+## Guardrail = policy + backstop
+- **Policy (turn-handoff, `prod-testing-gate.md`):** before any pod in `finder-repro`, the agent
+  ends its turn with a proposal (change, pod size, **which exports it reads**, what it measures);
+  you approve in the dashboard. Trust-based, per-run, full context.
+- **Backstop (enforced by the API server):** `ResourceQuota` caps the namespace (8 pods / 128 CPU
+  / 600Gi / 1000Gi scratch), `LimitRange` caps any one pod (64 CPU / 300Gi / 500Gi). IRSA is
+  read-only, one bucket. So worst case, even if the policy is ignored, the blast radius is bounded.
+
+## Apply sequence
+1. **PR** `Dockerfile.finder-builder` + `publish.finder-builder.yml` into unsupervised-main; run the
+   CI job once -> `harbor…/images/finder-builder:latest` exists.
+2. **AWS (you):** create IAM role `prod-finder-repro-readonly` from `iam-finder-repro-readonly.json`;
+   uncomment the `finder-test` SA's `role-arn` annotation in `finder-repro-rbac.yaml`.
+3. `apply.sh config` — assemble+install the image layer, PATCH repo config, force image rebuild.
+4. `kubectl --context <prod> apply -f finder-repro-rbac.yaml`; copy the pull secret into the ns:
+   `kubectl -n default get secret unsupervised-regcred -o yaml | sed 's/namespace: default/namespace: finder-repro/' | kubectl -n finder-repro apply -f -`
+   (or just run `apply.sh scope-sa`, which does the apply + pull-secret copy + kubeconfig rewrite).
+5. `apply.sh scope-sa` — mint a `finder-repro`-scoped kubeconfig, **test it (probe pod) before
+   overwriting**, back up the old value, rewrite `PROD_REPRO_KUBECONFIG_B64`.
+6. Add `prod-testing-gate.md` to unsupervised-main's AGENTS.md.
+7. Smoke test: `build-finder-in-pod.sh` -> binary; provision a `repro-pod.template.yaml` pod; profile.
+
+## Rebuild cadence
+The pre-baked image only rebuilds when the **heavy deps** change (requirements.txt / Rust ext /
+python-utils). Ordinary `fc.py` edits are overlaid at build-in-pod time — day-to-day this is free.

deploy/unsupervised-main-prod-testing/apply.sh

@@ -0,0 +1,133 @@
+#!/usr/bin/env bash
+# One-shot operator setup to enable unsupervised-main agents' finder prod-testing.
+#
+#   apply.sh config     # INTERIM (default): wire the image layer + repo config only.
+#                        # No cluster/credential changes — agents run test pods in `default`
+#                        # as unsupervised-unsupervised (broad prod-unsupervised-main IRSA),
+#                        # and panopticon-repro's existing kubeconfig already grants pod-create
+#                        # in `default`. This is all that's needed for the broad-role interim.
+#   apply.sh scope-sa   # PHASE 2 (needs an IAM-admin first): create the finder-repro RBAC,
+#                        # mint a finder-repro-scoped kubeconfig, and REWRITE
+#                        # PROD_REPRO_KUBECONFIG_B64 (backed up first). Requires the scoped IAM
+#                        # role + finder-test SA annotation to exist — do NOT run until then.
+set -euo pipefail
+
+HERE="$(cd "$(dirname "$0")" && pwd)"
+PANOPTICON_CONFIG="${PANOPTICON_CONFIG:-$HOME/.config/panopticon}"
+LAYERS_DIR="$PANOPTICON_CONFIG/layers"
+SECRETS_DIR="$PANOPTICON_CONFIG/secrets"
+ENV_FILE="$SECRETS_DIR/unsupervised_main.env"
+SVC="${PANOPTICON_SERVICE_URL:-http://localhost:8000}"
+PROD_CTX="${PROD_CTX:-arn:aws:eks:us-east-1:037004398141:cluster/prod}"
+NS="finder-repro"
+SA="panopticon-repro"
+LAYER_NAME="unsupervised-main.dockerfile"
+
+log() { printf '\n== %s ==\n' "$*"; }
+
+config() {
+  log "assembling image layer -> $LAYERS_DIR/$LAYER_NAME"
+  mkdir -p "$LAYERS_DIR"
+  {
+    cat "$HERE/image-layer.head.dockerfile"
+    printf '\n# --- inlined build-finder-in-pod.sh (build context is Dockerfile-only, so no COPY) ---\n'
+    printf "RUN cat > /usr/local/bin/build-finder-in-pod.sh <<'FINDERBUILD' && chmod 0755 /usr/local/bin/build-finder-in-pod.sh\n"
+    cat "$HERE/build-finder-in-pod.sh"
+    printf '\nFINDERBUILD\n'
+  } > "$LAYERS_DIR/$LAYER_NAME"
+
+  log "PATCH repo config: image_layer_file=$LAYER_NAME"
+  curl -fsS -X PATCH "$SVC/repos/unsupervised-main" \
+    -H 'Content-Type: application/json' \
+    -d "{\"image_layer_file\": \"$LAYER_NAME\"}" >/dev/null
+  echo "  ok"
+
+  log "forcing image rebuild (drop cached composed tags; runner rebuilds on next spawn)"
+  for wf in github-peer-reviewed github-self-reviewed; do
+    docker rmi "panopticon-$wf-unsupervised-main" 2>/dev/null || true
+  done
+  echo "  dropped; a fresh task spawn will compose base -> workflow -> repo layer."
+}
+
+scope_sa() {
+  log "applying scoped RBAC to prod ($NS)"
+  kubectl --context "$PROD_CTX" apply -f "$HERE/finder-repro-rbac.yaml"
+
+  log "copying pull secret unsupervised-regcred into $NS"
+  kubectl --context "$PROD_CTX" -n default get secret unsupervised-regcred -o yaml \
+    | sed -e 's/^  namespace: default/  namespace: '"$NS"'/' \
+          -e '/resourceVersion:/d' -e '/uid:/d' -e '/creationTimestamp:/d' \
+    | kubectl --context "$PROD_CTX" -n "$NS" apply -f -
+
+  log "minting a long-lived token secret for $NS/$SA"
+  kubectl --context "$PROD_CTX" -n "$NS" apply -f - <<YAML
+apiVersion: v1
+kind: Secret
+metadata:
+  name: ${SA}-token
+  namespace: ${NS}
+  annotations:
+    kubernetes.io/service-account.name: ${SA}
+type: kubernetes.io/service-account-token
+YAML
+  # wait for the controller to populate the token
+  for _ in $(seq 1 30); do
+    tok="$(kubectl --context "$PROD_CTX" -n "$NS" get secret "${SA}-token" -o jsonpath='{.data.token}' 2>/dev/null || true)"
+    [ -n "$tok" ] && break; sleep 1
+  done
+  [ -n "$tok" ] || { echo "token secret never populated" >&2; exit 3; }
+
+  log "building the scoped kubeconfig"
+  server="$(kubectl --context "$PROD_CTX" config view --minify -o jsonpath='{.clusters[0].cluster.server}')"
+  ca="$(kubectl --context "$PROD_CTX" -n "$NS" get secret "${SA}-token" -o jsonpath='{.data.ca\.crt}')"
+  token="$(printf '%s' "$tok" | base64 -d)"
+  newkc="$(mktemp)"; trap 'rm -f "$newkc"' RETURN
+  cat > "$newkc" <<KC
+apiVersion: v1
+kind: Config
+clusters:
+- name: prod
+  cluster: { server: ${server}, certificate-authority-data: ${ca} }
+contexts:
+- name: prod-repro
+  context: { cluster: prod, namespace: ${NS}, user: ${SA} }
+current-context: prod-repro
+users:
+- name: ${SA}
+  user: { token: ${token} }
+KC
+
+  log "TESTING the new kubeconfig against $NS (must pass before we touch the secret)"
+  kubectl --kubeconfig "$newkc" -n "$NS" auth can-i create pods >/dev/null
+  kubectl --kubeconfig "$newkc" -n "$NS" run rbac-probe --image=public.ecr.aws/docker/library/busybox:latest \
+    --restart=Never --command -- sh -c 'exit 0' >/dev/null
+  kubectl --kubeconfig "$newkc" -n "$NS" delete pod rbac-probe --wait=false >/dev/null 2>&1 || true
+  echo "  new kubeconfig works in $NS."
+
+  log "backing up + rewriting PROD_REPRO_KUBECONFIG_B64 in $ENV_FILE"
+  cp -p "$ENV_FILE" "$ENV_FILE.bak-$(date -u +%Y%m%dT%H%M%SZ)"
+  newval="$(base64 < "$newkc" | tr -d '\n')"
+  # replace just that one line (values may contain '/', so use a python rewrite, not sed)
+  ENVFILE="$ENV_FILE" NEWVAL="$newval" python3 - <<'PY'
+import os
+p=os.environ["ENVFILE"]; nv=os.environ["NEWVAL"]
+lines=open(p).read().splitlines(); out=[]; done=False
+for ln in lines:
+    if ln.startswith("PROD_REPRO_KUBECONFIG_B64="):
+        out.append(f"PROD_REPRO_KUBECONFIG_B64={nv}"); done=True
+    else: out.append(ln)
+assert done, "PROD_REPRO_KUBECONFIG_B64 line not found"
+open(p,"w").write("\n".join(out)+"\n")
+print("  rewrote PROD_REPRO_KUBECONFIG_B64 (backup kept)")
+PY
+  echo "  done. New task spawns will use the finder-repro-scoped credential."
+}
+
+case "${1:-config}" in
+  config)   config ;;
+  scope-sa)
+    echo "PHASE 2: only run scope-sa after an IAM-admin has created prod-finder-repro-readonly"
+    echo "and you've uncommented the finder-test SA role-arn. Ctrl-C now if that's not done." >&2
+    sleep 5; scope_sa ;;
+  *) echo "usage: apply.sh [config|scope-sa]   (config = broad-role interim; scope-sa = phase 2)" >&2; exit 2 ;;
+esac

deploy/unsupervised-main-prod-testing/build-finder-in-pod.sh

@@ -0,0 +1,98 @@
+#!/usr/bin/env bash
+# Build the finder PyInstaller binary in a k8s pod (NO Docker-in-Docker).
+#
+# Baked into the panopticon repo image layer at /usr/local/bin. Uses the pre-baked Harbor builder
+# image, which carries ONLY toolchain + deps + the compiled bfinder wheel (no finder/Rust source).
+# This script overlays the agent's CURRENT finder source, recompiles the Rust wheel *iff* the Rust
+# source changed (auto-detected via RUST_SRC_HASH — a Python-only change never recompiles; a Rust
+# change is never tested stale), installs finder editable, and runs pyinstaller. kubectl is
+# auto-wired to `default` (interim) by the image-layer wrapper; the builder pod pulls via
+# unsupervised-regcred.
+#
+# Usage: build-finder-in-pod.sh [--src DIR] [--out FILE] [--ns NS] [--image IMG]
+#                               [--name POD] [--force-rust] [--keep]
+set -euo pipefail
+
+SRC="subrepos/finder"
+OUT="./finder"
+# Interim: default ns (broad prod-unsupervised-main IRSA). Phase 2: NS=finder-repro.
+NS="default"
+IMAGE="${FINDER_BUILDER_IMAGE:-harbor.unsupervised.com/images/finder-builder:latest}"
+POD=""
+FORCE_RUST=0
+KEEP=0
+
+while [ $# -gt 0 ]; do
+  case "$1" in
+    --src) SRC="$2"; shift 2;;
+    --out) OUT="$2"; shift 2;;
+    --ns) NS="$2"; shift 2;;
+    --image) IMAGE="$2"; shift 2;;
+    --name) POD="$2"; shift 2;;
+    --force-rust) FORCE_RUST=1; shift;;   # recompile the wheel even if the hash matches
+    --keep) KEEP=1; shift;;
+    *) echo "unknown arg: $1" >&2; exit 2;;
+  esac
+done
+
+[ -d "$SRC/src" ] || { echo "no finder src at $SRC/src (run from the repo root, or pass --src)" >&2; exit 2; }
+if [ -z "$POD" ]; then
+  h="$(find "$SRC/src" -type f -printf '%P %s\n' 2>/dev/null | cksum | cut -d' ' -f1)"
+  POD="finder-build-${h}"
+fi
+
+cleanup() { [ "$KEEP" = 1 ] || kubectl -n "$NS" delete pod "$POD" --ignore-not-found --wait=false >/dev/null 2>&1 || true; }
+trap cleanup EXIT
+
+echo "== launching builder pod $POD (image=$IMAGE, ns=$NS) =="
+kubectl -n "$NS" delete pod "$POD" --ignore-not-found --wait=true >/dev/null 2>&1 || true
+kubectl -n "$NS" apply -f - <<YAML
+apiVersion: v1
+kind: Pod
+metadata:
+  name: ${POD}
+  labels: { app.kubernetes.io/managed-by: panopticon, purpose: finder-build }
+spec:
+  restartPolicy: Never
+  imagePullSecrets: [{ name: unsupervised-regcred }]
+  containers:
+    - name: builder
+      image: ${IMAGE}
+      command: ["sleep", "3600"]
+      resources:
+        requests: { cpu: "4", memory: 6Gi, ephemeral-storage: 12Gi }
+        limits:   { cpu: "8", memory: 12Gi, ephemeral-storage: 20Gi }
+YAML
+
+echo "== waiting for Ready =="
+kubectl -n "$NS" wait --for=condition=Ready "pod/$POD" --timeout=300s
+kubectl -n "$NS" exec "$POD" -- test -f /build/FINDER_BUILDER_READY \
+  || { echo "pod is not a finder-builder image (missing /build/FINDER_BUILDER_READY)" >&2; exit 3; }
+
+echo "== overlaying current finder source into the pod (transient — deleted with the pod) =="
+tar -C "$SRC" --exclude=venv --exclude=.venv --exclude=target --exclude=dist \
+    --exclude=__pycache__ --exclude=.git -czf - . \
+  | kubectl -n "$NS" exec -i "$POD" -- bash -c 'mkdir -p /build/subrepos/finder && tar -C /build/subrepos/finder -xzf -'
+
+echo "== recompiling the bfinder wheel only if the Rust source changed =="
+kubectl -n "$NS" exec "$POD" -- bash -lc "
+  set -euo pipefail
+  cd /build/subrepos/finder
+  rust_hash() { find Cargo.toml Cargo.lock bfinder pybfinder -type f | sort | xargs sha256sum | sha256sum | cut -d' ' -f1; }
+  baked=\$(cat /build/RUST_SRC_HASH)
+  now=\$(rust_hash)
+  if [ '${FORCE_RUST}' = '1' ] || [ \"\$now\" != \"\$baked\" ]; then
+    echo '   Rust source changed (or --force-rust) -> recompiling wheel'
+    (cd pybfinder && maturin build -r -o /tmp/wheels && pip install --force-reinstall --no-deps /tmp/wheels/*.whl)
+  else
+    echo '   Rust unchanged -> using baked wheel (fast path)'
+  fi
+  echo '== installing finder (editable) from overlaid src + pyinstaller =='
+  pip install -e . --no-deps
+  sh pyinstaller.sh
+"
+
+echo "== copying binary out -> $OUT =="
+kubectl -n "$NS" cp "$POD:/build/subrepos/finder/dist/finder" "$OUT"
+chmod +x "$OUT"
+echo "== done: $OUT ($(wc -c < "$OUT") bytes). linux/amd64 binary — run it in a pod, not on the agent host. =="