Skip to content

docs: add inventory file for multi-server-devnet skill - #584

Merged
pablodeymo merged 6 commits into
mainfrom
docs/server-inventory
Aug 19, 2026
Merged

docs: add inventory file for multi-server-devnet skill#584
pablodeymo merged 6 commits into
mainfrom
docs/server-inventory

Conversation

@MegaRedHand

Copy link
Copy Markdown
Collaborator

🗒️ Description / Motivation

The devnet.env example included a list of servers inside a comment. This PR moves that information to a new file, with scripts to query it.

What Changed

  • List the files or areas touched
  • Brief summary of each change

Correctness / Behavior Guarantees

  • What invariants are preserved or updated?
  • Are there any behavior changes reviewers should know about?

Tests Added / Run

  • What tests were added or updated?
  • What commands did you run to verify this change?

Related Issues / PRs

  • Closes #
  • Related to #

✅ Verification Checklist

  • Ran make fmt — clean
  • Ran make lint (clippy with -D warnings) — clean
  • Ran make test (cargo test --workspace --profile release-fast) — all passing

@github-actions

Copy link
Copy Markdown

🤖 Codex Code Review

  1. inventory.sh silently accepts malformed tags fields and shifts columns instead of failing. The parser takes $3/$4/$5 as tags/nodes/subnets without validating column count or that tags contains no spaces, so a common typo like devnet-ab, aggregator is accepted as tags=devnet-ab,, nodes=aggregator, subnets=32. I reproduced this and the script exited 0 while emitting a corrupted row. For deployment tooling, that is the wrong failure mode: it should reject rows with extra columns or whitespace inside the tags column.

  2. [inventory.sh allows a host to belong to multiple devnet-* groups at once](</home/runner/work/ethlambda/ethlambda/.claude/skills/multi-server-devnet/scripts/inventory.sh:78>). validatoris derived from “contains anydevnet-*tag and notaggregator”, but the script never enforces that there is at most one such tag per host. A row like devnet-ab,devnet-ccurrently matches and exits0, even though the inventory schema only has one nodes/subnets` pair per host and the docs describe a single chain membership per server. That can make one machine appear in two independent devnets and drive the wrong SSH/prometheus targeting. This should be rejected during parsing.

No Rust consensus paths are touched in this PR, so I did not find any fork-choice, attestation, STF, XMSS, or SSZ-specific concerns in the diff itself.


Automated review by OpenAI Codex · gpt-5.4 · custom prompt

@github-actions

Copy link
Copy Markdown

🤖 Kimi Code Review

This PR introduces operational tooling (bash/AWK scripts) for multi-server devnet inventory management. It does not modify consensus-critical Rust code.

Review Summary: The implementation is robust with excellent error handling (typo-resistant tag validation, strict exit codes) and clear documentation. No security vulnerabilities or correctness issues identified.

Minor Suggestions:

  1. Fragile line numbers in help text (inventory.sh:27)

    usage() { sed -n '2,26p' "$0" | sed 's/^# \{0,1\}//'; exit "${1:-0}"; }

    Hardcoded line numbers break when the header comment is edited. Consider using a here-document or dynamic extraction:

    usage() { sed -n '/^#/,/^[^#]/{/^#!/d;p;}' "$0" | sed 's/^# \{0,1\}//; /^$/d'; exit "${1:-0}"; }
  2. AWK stderr portability (inventory.sh:78, 82, 116, etc.)
    The script uses > "/dev/stderr" within AWK, which is a GNU extension (works in gawk and mawk but not POSIX awk). Given this is for Linux devnet operations, this is acceptable, but documenting the dependency on gawk or mawk in the script header would be helpful.

  3. Grammar in empty match message (inventory.sh:143)

    printf "no host in %s matches%s%s\n", src, \
      (nw ? " tags: " want : ""), (nn ? " not: " nowant : "") > "/dev/stderr"

    When only --not-tag is specified, this prints "matches not: tag" instead of "matches (not: tag)". Consider:

    printf "no host in %s matches%s%s\n", src, \
      (nw ? " tags: " want : ""), (nn ? " (not: " nowant ")" : "") > "/dev/stderr"

Correctness Verification:

  • The derived validator tag logic (line 80-82) correctly implements "has devnet-* and not aggregator"
  • Tag substring protection via comma-wrapping (,tag,) prevents partial matches (e.g., "agg" matching "aggregator")
  • Exit code strategy is sound: 2 for operator errors (unknown tags), 1 for empty legitimate queries, 0 for success

Documentation: The SKILL.md and example file comments accurately describe the inventory format and query semantics.

Security: No injection vectors detected; user input is passed to AWK via -v variables (not interpolated), and file paths are quoted.

Approve with minor non-blocking suggestions.


Automated review by Kimi (Moonshot AI) · kimi-k2.5 · custom prompt

@github-actions

Copy link
Copy Markdown

🤖 Claude Code Review

Review: PR 584 — docs: add inventory file for multi-server-devnet skill

Scope check: this PR only touches the multi-server-devnet skill's docs and operator tooling scripts (SKILL.md, devnet.env.example, a new devnet.inventory.example, a new inventory.sh, .gitignore). No Rust/consensus code is touched, so none of the fork-choice/attestation/STF/XMSS/SSZ concerns apply here. I traced through the new inventory.sh awk logic by hand (field width handling, tag-matching, the validator derivation, the unknown-tag hard-fail path, count/exit-code semantics) and it holds up correctly — no logic bugs found there.

Findings

1. SERVERS (in devnet.env) and the new inventory's name column are now two independently-maintained lists of the same hosts.

  • devnet.env.example:8 still declares SERVERS="host-a host-b", and SKILL.md (for h in $SERVERS, line ~137) and references/operations.md keep using it as the canonical host list — while the new devnet.inventory also enumerates every host by name. SKILL.md even shows SERVERS=$(inventory.sh --field name | tr '\n' ' ') as an example, suggesting the two should be derived from one source, but nothing enforces that; an operator can still edit one file and forget the other.
  • The stated motivation for this PR is "so 'which hosts are in devnet-c' has one answer instead of living in someone's head" (devnet.inventory.example) — that goal is only partially met, since the base host list itself is now duplicated between devnet.env and devnet.inventory.
  • Not a functional bug (both files are gitignored, operator-filled, and nothing in this PR silently misuses either), but worth deciding: should devnet-env.sh/SKILL.md examples move to always deriving SERVERS from inventory.sh, or is keeping both intentional?

2. Minor: usage()'s error path writes to stdout, inconsistent with the script's own error-message convention.

  • inventory.sh:29: usage() { sed -n '2,26p' "$0" | ...; exit "${1:-0}"; }, called as usage 2 from the unknown-argument branch (line ~ *) echo "unknown argument: $1" >&2; usage 2 ;;). Every other error message in the script is explicitly sent to >&2, but the help text itself always goes to stdout regardless of whether it's an error exit (2) or a deliberate --help (0). If a caller does x=$(inventory.sh --typo-flag) without checking $?, $x silently fills with the help text rather than staying empty — a milder version of the exact "typo looks like success" failure mode this script otherwise guards against. Low impact (no current caller in this PR captures output blindly), but easy to fix by teeing the error-path help to stderr.

3. Nit: for t in $2; do tags+=("$t"); done (and the --not-tag equivalent) is unquoted, so it's subject to pathname expansion, not just word-splitting.

  • If a --tag/--not-tag value ever contained a glob metacharacter (e.g. someone typing --tag 'devnet-*' expecting glob-style matching, which isn't supported), bash would expand it against files in the current directory before the value ever reaches the tag list. In practice this fails safe — the resulting bogus "tag" (a filename) almost certainly isn't in the inventory's seen set, so the script's unknown-tag guard (line ~ "no such tag in ...") catches it and exits 2 rather than silently misbehaving. Worth a one-line mention in the script header if it's expected behavior, otherwise just a robustness nit (e.g. read -ra instead of an unquoted for).

What looks good

  • inventory.sh's unknown-tag detection (seen populated from literal tags across all rows before filtering, validator explicitly excluded from that check) is well thought out and correctly implemented.
  • The validator derivation (has a devnet-* tag and is not tagged aggregator) is correctly computed as a function rather than stored, so it can't drift out of sync with the aggregator tag — matches the stated intent.
  • File-lookup precedence (--file > $DEVNET_INVENTORY > ./devnet.inventory > script-dir) mirrors the existing devnet-env.sh convention, and an explicit-but-missing --file/$DEVNET_INVENTORY is treated as an error rather than silently falling back — good, deliberate consistency with the "don't silently act on the wrong deployment" principle stated in the script's own comments.
  • usage()'s sed -n '2,26p' line range was verified against the actual file content — it exactly captures the header comment block, so the extracted help text won't silently drift if unrelated header lines are added/removed without updating the range (though that coupling is itself a little fragile — worth a comment noting the range must track the header if either changes).
  • .gitignore addition and example-file structure follow the existing devnet.env/devnet.env.example pattern closely.

Overall: a low-risk, well-executed docs/tooling change. The one substantive point worth a second look before merging is Item 1 (the SERVERS vs. inventory name duplication) — everything else is polish-level.


Automated review by Claude (Anthropic) · sonnet · custom prompt

Six ways the selector could hand a caller the wrong host list, all of them
silent, which is the failure mode the script exists to prevent:

- `--tag ""` matched nothing, so the filter was dropped and the ENTIRE fleet
  came back at exit 0. An unset `$DEVNET` in `--tag "$DEVNET"` would act on
  hosts on other chains, the mirror image of the typo case already guarded.
- The unknown-argument path printed help to stdout, and the documented idiom
  captures stdout: a mistyped flag fed 1.4 KB of usage text into a host loop.
- `--not-tag T` hard-failed when no host carried T, contradicting the script's
  own `--tag validator` == `--tag <devnet> --not-tag aggregator` equivalence.
  An exclusion matching nothing is well-defined; only `--tag` is checked now.
- Nothing validated nodes/subnets, so one space in the tags column shifted every
  later field: a role landed in `nodes`, `--tag validator` returned the
  aggregator, and a word reached a caller expecting a count. The file is now
  refused with the row named, rather than answered from.
- Tag arguments were glob-expanded against the cwd, so `--tag 'devnet-*'` could
  silently resolve to a different group than typed.
- `usage()` sliced help from a hardcoded line range; it now ends at the first
  non-comment line, so editing the header cannot truncate `--help`.

The documented recipes were also unrunnable as written: `inventory.sh` is not on
PATH, and the `| tr` pipeline discarded the exit code the script sets so
carefully, turning any error into an empty-but-successful host list.
Follow-up to the input-validation pass, closing the rest of the review:

- `validator` was special-cased in three places (a branch in hastag, plus two
  exemptions in the unknown-tag check) with `devnet-` hardcoded inside the
  branch, so a second derived tag meant touching all three and a fleet that
  names chains differently got a silent empty answer. It is now one table entry
  (prefix + disqualifying tag), and DEVNET_TAG_PREFIX names the prefix.
- A literal `validator` in the file was silently ignored, which is exactly the
  disagreement the derivation exists to rule out: the row said one thing and the
  computation another. It now names the line and refuses.
- `--tag validator` against a file carrying no chain tag at all said only "no
  host matches", when the actionable fact is that the tag can never match there.
- The file-lookup ladder was a copy of devnet-env.sh's, comment included, and had
  already drifted (return-0 vs exit-2 on nothing found). Both now call one
  `devnet_find_file`, so the two config files of a deployment cannot end up with
  two ideas of where they live.
- The known-tag list, whose whole job is to be read next to the operator's typo,
  printed in awk's hash order.

Docs: devnet.inventory is the record of which hosts exist and `SERVERS` is a
working set filled from a query against it, rather than two hand-kept host lists
with no stated precedence. Also notes that the `for h in $SERVERS` workflows rely
on bash word-splitting, which zsh does not do, and drops the em-dashes this
branch added.
Nothing read `SERVERS`: no script did, and a shell variable can't survive from
one command to the next anyway, so as a stored config value it was only ever a
second host list to keep in step with devnet.inventory. The workflows now query
the inventory in the same command that loops over it, guarded with `|| exit` so a
typo'd tag aborts instead of leaving a loop that does nothing and reports
success, and `$(echo $hosts)` because zsh doesn't split parameter expansions.

`SSH_USER` stays: it isn't a host, the inventory has no column for it, and every
documented ssh needs it. Its section is named for what it holds now.
…evnet.env

Narrows the previous commit: replacing the `for h in $SERVERS` loops with an
inline inventory query per command went further than intended. Which hosts a
command runs against is the caller's to pass, and `SERVERS` is how they pass it,
so the workflows are back to taking it from the caller and the query that fills
it stays documented where inventory.sh is introduced.

What devnet.env loses stands: no script reads SERVERS or SSH_USER, so storing
them there only created a host list to keep in step with devnet.inventory.
@pablodeymo
pablodeymo added this pull request to the merge queue Aug 19, 2026
Merged via the queue into main with commit 13bf5e2 Aug 19, 2026
2 checks passed
@pablodeymo
pablodeymo deleted the docs/server-inventory branch August 19, 2026 20:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants