Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
214 changes: 192 additions & 22 deletions .dev-loop/INGEST_REPORT.md

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions log.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,3 +34,4 @@ Append-only. Format: `## [YYYY-MM-DD] <ingest|revise|lint|gap|contradiction|drif
## [2026-07-12] revise | security/secrets-in-code +1 edge case: third-party HTTP client (httpx/requests) logs the full request URL — including a query-param API key — at INFO, so root/DEBUG logging leaks it; keep the client logger above INFO. Found when a standalone sync process set logging.basicConfig(INFO) and httpx wrote the data.go.kr serviceKey to the log file. last_verified bumped to 2026-07-12.
## [2026-07-13] ingest | databases +1 (query-optimization): streaming-large-result-sets — memory-bounded export of a huge single-query result. Client-side cursor pulls the whole set to libpq on execute (fetchmany caps only the Python-list explosion); only a server-side/named cursor truly streams but needs a transaction, so it fails under autocommit or a proxy that blocks BEGIN → fall back to client-side fetchmany + disk spool + openpyxl write_only (measured 300k rows 838MB→38MB). Derived from RNR-3440 (potential-listing weekly extract memory peak); QueryPie BEGIN-block generalized to "read-only access proxy", field-tested. Sources: psycopg2 usage/cursor docs (named-cursor WITHOUT HOLD + autocommit exception), openpyxl optimized-modes (write-only near-constant memory, lxml=speed-not-memory).
## [2026-07-23] ingest | databases +2: schema-design/online-schema-changes (ACCESS EXCLUSIVE lock avoidance — non-volatile default fast path, ADD CONSTRAINT NOT VALID + VALIDATE at SHARE UPDATE EXCLUSIVE, CHECK-NOT-NULL trick, CREATE INDEX CONCURRENTLY, expand-and-contract to decouple DB migration from app deploy, lock_timeout for lock-queue pile-up) + operations/autovacuum-and-wraparound (NEW category operations: per-table scale_factor/cost_limit tuning for hot tables, age(datfrozenxid)/relfrozenxid + n_dead_tup monitoring, wraparound read-only cliff and superuser VACUUM recovery, VACUUM FULL vs pg_repack). Derived from the Hatchet "Postgres survival guide"; both cross-checked against PostgreSQL official docs (sql-altertable, routine-vacuuming).
## [2026-07-30] ingest | 4 queued insights → 3 pages. platforms/processes/non-interactive-cli-invocation (a non-interactive flag is not a closed stdin: `</dev/null` + tool fail-fast switch + timeout; split a zero-output hang by whether the far side logged the request — nohup/ssh -n/BatchMode/timeout docs). platforms/shells/command-text-inspected-before-execution (NEW page in shells: a gate reads `tool_input.command` pre-expansion, so quoted args defeat its extractor while unexpanded `$VAR` resolves to a nonexistent path — two distinct refusals; write literal paths, create gate-read files in a prior command, pass dangerous-looking prose by file). qa/document-verification/editing-a-gated-document (NEW category page, author-side counterpart to spec-document gates: anchor inventory before editing, describe upstream as observed shape not with definition verbs, scope a check outside the region quoting it, record scoped conditions instead of global counts, re-run the full gate set against a baseline). Regex failure modes reproduced against hooks/pre-flush-pr-gate.sh; sources live-verified (nohup, ssh(1)/ssh_config(5), timeout(1), pgrep(1) self-exclusion, Claude Code hooks, Vale scopes/existence, markdownlint MD013).
2 changes: 2 additions & 0 deletions wiki/platforms/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ Match your situation to a "load when" line; load only matching pages.
| Page | Load when |
|------|-----------|
| [portable-shell-scripts](shells/portable-shell-scripts.md) | Writing a shell script that must run on more than one machine/OS/shell or in CI; a script that works locally fails elsewhere; choosing a shebang (bash vs sh); a bash script misbehaves in zsh or vice versa (unquoted vars, `=word`, array indexing); deciding how `set -euo pipefail` protects (and doesn't); building argument lists safely |
| [command-text-inspected-before-execution](shells/command-text-inspected-before-execution.md) | A hook, policy gate, allow-list, or audit rule blocked a command that is correct as written; composing a command that must satisfy such a gate first try; deciding whether to write a path literally or as `"$VAR"` in an inspected argument; a gate reports an argument missing or a file nonexistent though both are right; a gate must read a file your command creates; prose containing a dangerous-looking command (release notes, docs, fixtures) trips a text scanner |

## tools

Expand All @@ -41,6 +42,7 @@ Match your situation to a "load when" line; load only matching pages.
| Page | Load when |
|------|-----------|
| [background-services](processes/background-services.md) | Something must run persistently or on a schedule on a dev machine or server (daemon, watcher, cron-style job); a "started" process dies when the terminal/SSH/agent session ends; choosing nohup vs LaunchAgent vs systemd unit vs cron/timer; a job works in the terminal but fails under cron/launchd (minimal environment); wiring service logs and restart policy |
| [non-interactive-cli-invocation](processes/non-interactive-cli-invocation.md) | Calling a tool that can prompt (agent CLI, ssh, git, package manager) from a script, CI step, hook, or agent session; such a call produced no output and never returned; deciding whether a hang belongs to the client, the network, or the far-side service; choosing the stdin/timeout/fail-fast switches for an unattended call; a TTY-detecting tool changes its output format under automation |

## toolchains

Expand Down
2 changes: 1 addition & 1 deletion wiki/platforms/processes/background-services.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ sources:
- https://man7.org/linux/man-pages/man1/nohup.1.html
- https://man7.org/linux/man-pages/man1/loginctl.1.html
last_verified: 2026-07-10
related: [platforms-toolchains-version-management, platforms-shells-portable-shell-scripts]
related: [platforms-toolchains-version-management, platforms-shells-portable-shell-scripts, platforms-processes-non-interactive-cli-invocation]
---

# Keeping a Process Running Beyond the Terminal Session
Expand Down
97 changes: 97 additions & 0 deletions wiki/platforms/processes/non-interactive-cli-invocation.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
---
id: platforms-processes-non-interactive-cli-invocation
domain: platforms
category: processes
applies_to: [macos, linux]
confidence: verified
sources:
- https://man7.org/linux/man-pages/man1/nohup.1.html
- https://man.openbsd.org/ssh.1
- https://man.openbsd.org/ssh_config.5
- https://man7.org/linux/man-pages/man1/timeout.1.html
last_verified: 2026-07-30
related: [platforms-processes-background-services, platforms-shells-portable-shell-scripts, debugging-methodology-reproduce-first]
---

# Invoking an Interactive-Capable CLI From a Script, Hook, or Agent

## When this applies

You are calling a tool that can prompt (a coding-agent CLI, `ssh`, `git`, a package
manager, an installer) from a script, CI step, hook, or agent session; or such a
call produced no output and never returned, and you are deciding whether the tool,
the network, or the far-side service is at fault.

## Do this

1. **Close stdin and pass the tool's own non-interactive switch — both, not either.**
A `-p`/`--print`/`--quiet` flag suppresses *rendering*; it does not promise the
tool never reads stdin. When stdin is still a terminal, any prompt the flag
didn't cover (a permission confirmation, a passphrase, a host-key question)
blocks forever with zero bytes written. This is why `ssh -n` exists — it
"Redirects stdin from /dev/null (actually, prevents reading from stdin)" and
"must be used when ssh is run in the background" — and why `nohup` does it for
you: "If standard input is a terminal, redirect it from an unreadable file."

| Case | Do |
|------|----|
| One-shot call whose output you need in this step | `cmd … </dev/null` plus the tool's non-interactive flag |
| Call that must outlive the session | `nohup cmd … </dev/null >"$LOG" 2>&1 & disown` ([platforms-processes-background-services]) |
| Tool authenticates over the network | Add the tool's fail-fast switch so a missing credential errors instead of prompting: `ssh -o BatchMode=yes`, `GIT_TERMINAL_PROMPT=0`, `DEBIAN_FRONTEND=noninteractive` |
| Any unattended call | Wrap in a timeout so a block fails loudly: `timeout <secs> cmd …` (on macOS, `gtimeout` — see [platforms-tools-bsd-vs-gnu-cli]) |

2. **Give the call a deadline rather than watching it.** `timeout` will "Start
COMMAND, and kill it if still running after DURATION", exiting 124 when that
fires — a distinguishable signal that the call blocked, instead of an agent turn
that hangs until something else kills it.

3. **When it still hangs, locate the boundary before blaming either side.** Zero
output is compatible with two very different faults: the request never left the
client, or the far side never answered. Split them with evidence from the far
side — the server/gateway access log for that source IP and time window, or a
packet/connection count:

| Observation at the far side | Conclusion | Next action |
|-----------------------------|------------|-------------|
| No request recorded in the window | The call never got that far — it is blocked locally (stdin, auth prompt, DNS, proxy) | Re-run with `</dev/null` and a timeout; inspect the client's own debug/verbose log |
| Request recorded, no response or a slow one | The far side owns it | Take the latency/error question to the service |

4. **Re-run the identical command with stdin closed as the first diagnostic step.**
If a `</dev/null` run returns immediately, the hang was a prompt, and no
investigation of the far side is warranted.

## Edge cases

| Case | Then |
|------|------|
| Tool needs input you intend to supply | Feed it from a file or heredoc (`cmd <input.txt`), never leave it on the inherited terminal — an unfed prompt and a fed one are indistinguishable from the caller |
| Tool detects a TTY and changes output format (color codes, progress bars, pagers) | Closing stdin is not enough; also disable the pager/color explicitly (`GIT_PAGER=cat`, `--no-color`) so parsers see stable text |
| The call must run under a supervisor or scheduler | Services get no terminal at all, so the prompt fails differently there than in your shell — verify by log, not by launch exit code ([platforms-processes-background-services]) |
| Wrapper CLI shells out to a second binary that prompts | Redirecting the wrapper's stdin covers the child only if the child inherits it; confirm with a timeout run before trusting the wrapper's own flags |
| Hang reproduces even with stdin closed and a timeout | Treat it as a real far-side or network fault and continue from the boundary evidence in step 3 |

## Instead of

| If you are about to | Do this instead | Why |
|---------------------|-----------------|-----|
| Trust a `--print`/`-p` flag to make a call safe for unattended use | Add `</dev/null` and a timeout alongside the flag | The flag governs output; a prompt the flag does not cover still reads the inherited terminal and blocks with no output |
| Conclude "the model/gateway/service is slow or broken" from a call that produced nothing | Check the far side's log for the request first | A locally blocked call leaves no trace there, so a hang with no logged request is evidence about the client, not the service |
| Watch a hung foreground call and cancel it by hand | Bound it with `timeout` and read exit 124 as "blocked" | Manual cancellation loses the distinction between slow and stuck, and burns the whole wait |
| Debug the hang by re-running it the same way | Re-run it with stdin closed as the first variation | Closing stdin is a one-token change that either fixes it or eliminates the largest class of cause |

## Sources

- https://man7.org/linux/man-pages/man1/nohup.1.html — "If standard input is a terminal, redirect it from an unreadable file" — detaching a command includes taking its terminal stdin away
- https://man.openbsd.org/ssh.1 — `-n` "Redirects stdin from /dev/null (actually, prevents reading from stdin). This must be used when ssh is run in the background"
- https://man.openbsd.org/ssh_config.5 — `BatchMode=yes` disables "user interaction such as password prompts and host key confirmation requests", "useful in scripts and other batch jobs where no user is present"
- https://man7.org/linux/man-pages/man1/timeout.1.html — "Start COMMAND, and kill it if still running after DURATION"; exit status 124 "if COMMAND times out, and --preserve-status is not specified"

## Field context

Distilled from a 2026-07 session that ran an agent CLI with its non-interactive
`--tools` flag in the foreground: two runs hung (300 s and 150 s) with zero bytes
of output. The gateway access log showed **no request from that host** in either
window, which ruled out the model and the gateway; the same command launched with
stdin taken from `/dev/null` completed its tool call immediately. The far-side-log
check in step 3 is the field-derived part of this page; the stdin rules in step 1
are the documented behavior the incident rediscovered.
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
---
id: platforms-shells-command-text-inspected-before-execution
domain: platforms
category: shells
applies_to: [bash, zsh, posix-sh]
confidence: verified
sources:
- https://code.claude.com/docs/en/hooks
- https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html
last_verified: 2026-07-30
related: [platforms-shells-portable-shell-scripts, platforms-environment-path-resolution]
---

# Commands Read as Text by a Gate Before the Shell Runs Them

## When this applies

A hook, policy gate, allow-list, or audit rule inspects your command line and
decides whether it may run (agent PreToolUse hook, commit gate, sudo command
pattern, CI policy check); the gate blocked or mis-parsed a command that is
correct as written; or you are composing a command that must satisfy such a gate
on the first attempt.

## Do this

1. **Write values the gate reads as literal text.** The gate is handed the command
as an unexecuted string — a Claude Code `PreToolUse` hook receives
`tool_input.command` (`{"tool_name":"Bash","tool_input":{"command":"npm test"}}`)
and runs "Before a tool call executes". Nothing has expanded yet: `"$VAR"` is
four literal characters plus a name, and the quote marks are part of the text.
Shell expansion is defined to happen when the shell processes the line, which is
after the gate has already decided.

2. **Know which of the two failure modes you are in — the error message tells you.**
Both a quoted path and an unexpanded variable break a gate that extracts a file
argument, but for different reasons and with different symptoms:

| What you wrote | What the gate extracts | How it fails |
|----------------|------------------------|--------------|
| `--body-file /abs/path/REPORT.md` | `/abs/path/REPORT.md` | Passes |
| `--body-file="/abs/path/REPORT.md"` or `--body-file "$REPO/REPORT.md"` | nothing — the extraction pattern excludes the quote character, so the match fails outright | Gate reports the argument as **missing** ("no `--body-file` found"), which reads as a malformed command |
| `--body-file $REPO/REPORT.md` (unquoted variable) | the literal string `$REPO/REPORT.md` | Gate reports the file as **nonexistent**, which reads as a missing deliverable |

3. **Create the file a gate will read in an earlier, separate command.** The gate
runs before this command executes, so a file produced by a heredoc inside the
same command does not exist yet at inspection time and the gate fails closed.
Write the file in one call, reference it by literal path in the next.

4. **When content must contain patterns the gate treats as dangerous, put the
content in a file with a non-shell tool and pass the path.** Release notes,
docs, or fixtures containing `curl … | sh` or `rm -rf` are data, but a
text-scanning gate cannot tell data from an invocation. A file written by an
editor/Write tool is never scanned as a command; `--notes-file` / `--body-file`
then carries it.

5. **Read the gate's own extraction pattern when a correct command is refused.**
The pattern is the specification of what the gate can see. Reproduce it against
your exact command string before rewriting anything else — one run tells you
whether you are in the missing-argument or nonexistent-file mode above.

## Edge cases

| Case | Then |
|------|------|
| Blocking feedback appears without the gate's message | Exit code 2 sends the reason to **stderr**, not stdout; read stderr for the actual cause |
| The gate matches an intended-as-prose mention of a dangerous command (in a commit message, doc, or test fixture) | Move the text into a file and pass it by path (step 4) rather than reshaping the sentence |
| Path contains a space, so quoting is unavoidable | Relocate or symlink the target to a space-free path for gated commands; a gate that excludes quote characters cannot receive a quoted path at all |
| The gate needs `~` expanded | Write the absolute path; a gate that resolves `~` itself is doing so on the literal tilde, which only works if it implements the expansion |
| The same command must also be portable/robust as a script | Keep the gate-read argument literal and leave the rest of the script quoted normally ([platforms-shells-portable-shell-scripts]) — this page narrows one argument, it does not license unquoted expansions elsewhere |

## Instead of

| If you are about to | Do this instead | Why |
|---------------------|-----------------|-----|
| Interpolate `"$VAR/file"` into an argument a gate inspects | Write the resolved absolute path literally | The gate sees pre-expansion text; the quote character can defeat its extractor entirely and the variable name never resolves for it |
| Assume a blocked command means the deliverable is wrong | Reproduce the gate's extraction pattern against your literal command string first | A quoting-level extraction failure and a genuinely incomplete deliverable produce the same refusal, so fixing content wastes the round |
| Build the file the gate checks with a heredoc in the same command | Write it in a prior command and reference the path | The gate is evaluated before execution, so the file is absent at decision time |
| Reword prose to get a dangerous-looking string past a scanner | Put the prose in a file and pass `--notes-file`/`--body-file` | Editing meaning to satisfy a text scanner degrades the artifact; a file is not scanned as a command |

## Sources

- https://code.claude.com/docs/en/hooks — `PreToolUse` runs "Before a tool call executes. Can block it"; the hook's stdin JSON carries `tool_input.command` — the unexecuted Bash command string. Exit 2 blocks and "stderr text is fed back to Claude as an error message"
- https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html — the shell's order of word expansion (tilde, parameter, command substitution, field splitting, quote removal) is performed by the shell as it processes the command, so an external reader of the command text sees none of it applied

## Field context

Reproduced against the extraction pattern of this repo's own flush gate
(``--body-file[= ]+[^ '"`]+``, `hooks/pre-flush-pr-gate.sh`) on 2026-07-30: five
variants run through that pattern gave `--body-file "$REPO/…"` → empty (blocked as
missing), `--body-file $REPO/…` → literal `$REPO/…` (blocked as nonexistent),
`--body-file "/abs/…"` → empty even with a literal path, while
`--body-file /abs/…` and `--body-file=/abs/…` extracted correctly. A same-command
heredoc body-file was separately blocked as not-yet-existing until moved to a
preceding call.
2 changes: 1 addition & 1 deletion wiki/platforms/shells/portable-shell-scripts.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ sources:
- https://google.github.io/styleguide/shellguide.html
- https://www.shellcheck.net/
last_verified: 2026-07-10
related: [platforms-tools-bsd-vs-gnu-cli, platforms-toolchains-version-management]
related: [platforms-tools-bsd-vs-gnu-cli, platforms-toolchains-version-management, platforms-shells-command-text-inspected-before-execution]
---

# Shell Scripts That Must Run on More Than One Machine or Shell
Expand Down
Loading
Loading