Skip to content

feat(stack): report site recipes that shadow an EESSI module - #13

Merged
HaoZeke merged 9 commits into
mainfrom
feat/docs-shadow-check
Nov 3, 2026
Merged

HaoZeke merged 9 commits into
mainfrom
feat/docs-shadow-check

Conversation

@HaoZeke

@HaoZeke HaoZeke commented Nov 3, 2026 •

Copy link
Copy Markdown
Collaborator

Reproducer

Before this change, the subcommand does not exist.
Clap rejects it before any path is read:

$ eb-stack stack shadow --easyconfigs tests/shadow_fixtures/easyconfigs --base tests/shadow_fixtures/base --base tests/shadow_fixtures/bare.txt
error: unrecognized subcommand 'shadow'

  tip: a similar subcommand exists: 'shard'

Usage: eb-stack stack <COMMAND>

For more information, try '--help'.

Exit status 2.

After, from the repository root:

$ eb-stack stack shadow \
    --easyconfigs tests/shadow_fixtures/easyconfigs \
    --base tests/shadow_fixtures/base \
    --base tests/shadow_fixtures/bare.txt
same zlib-1.3.1-GCCcore-15.2.0.eb name=zlib version=1.3.1 base=1.3.1-GCCcore-15.2.0
different FFTW-3.3.11-gompi-2023a.eb name=FFTW version=3.3.11 base=3.3.10-gompi-2023a
unversioned OnlyName-1.2-GCCcore-15.2.0.eb name=OnlyName version=1.2 base=-
same=1 different=1 unversioned=1 recipes=4 base=3
error: 3 site-layer recipes shadow a module the EESSI base stack provides

Exit status 1.
The fourth fixture recipe is site-only: the base inventory does not name it, so the report leaves it out.
The default file under the zlib module directory is not read as a version.

What and why

A site module path searched before the European Environment for Scientific Software Installations (EESSI) path hides any base module with the same name.
The site layer should hold only software the base stack does not install.
stack shadow reads the site easyconfigs and loads --base with gap::inventory_from_module_source, the reader stack gap uses for --installed.
A --base directory is an Lmod tree.
A --base file is module -t avail, module spider -t, or Name/version.

Every recipe whose name the base also lists gets one of these kinds:

Kind Meaning
same the base installs this software version
different the base installs another version of this name
unversioned the base lists this name with no version

Names match without regard to case.
EasyBuild spells an installed module version as the software version, then -, then the toolchain, then the versionsuffix.
The comparison reuses that spelling through build_order::installed_module_version and MODULE_VERSION_TOOLCHAIN_SEPARATOR.
A recipe versionsuffix has to end the base version, so a CUDA build does not match the plain build.
The command exits 1 when any recipe shadows a base module, and when a recipe does not parse.
--json prints the same report as one document.
The Model Context Protocol (MCP) tool eb_stack_shadow runs the same check.

The branch also adds how-tos for commands that had a reference entry and no worked example:

  • stack shard: split a Jenkins build list into shards with a shared prelude.
  • stack batch: plan software-layer pull requests whose builds do not overlap.
  • stack sections: regroup a Jenkins build list under the section headers of a previous list.
  • stack easystack: write a stack lock as the YAML eb --easystack reads.
  • recipe easyblock, recipe templates, recipe hierarchy: read what EasyBuild derives from a recipe.
  • package latest: report the newest stable upstream release of a recipe.
  • repro summary: read the reproduction scoreboard and hold it to a ratchet file.
  • stack shadow: report a site recipe that shadows an EESSI module.

Each how-to runs a fixture under tests/ and quotes its output.
CHANGELOG.md, AGENTS.md, the CLI reference, the formats page, and both llms files list the new command.

Not covered yet:

  • The check reads each recipe's own name, so it misses an exts_list extension that EasyBuild installs as its own module.
  • No --allow flag exists for a second version the site builds beside the base on purpose.

Tests

tests/stack_shadow.rs runs the binary and the MCP tool on tests/shadow_fixtures:

  • the full fixture prints the report above and exits 1
  • a site-only recipe prints zero counts and exits 0
  • --json gives the same counts and exits 1
  • an absent --base path appears in the error, and the run exits 1
  • a recipe that does not parse names the file and exits 1
  • an omitted required flag exits 2
  • eb_stack_shadow returns the same report as a tool error

Unit tests in src/shadow.rs cover the version spelling, a shorter version against a longer one, the versionsuffix rule, case folding across repeated --base paths, and the text report.
The CI test job now runs --test stack_shadow.

On every commit, cargo fmt --all --check, cargo clippy --locked --all-targets -- -D warnings, cargo test --locked --all-targets, and python3 scripts/gen-llms-txt.py --check exited 0.
The Sphinx build with -W --keep-going exited 0.

Reproduce

git checkout feat/docs-shadow-check
cargo test --locked --test stack_shadow
cargo test --locked --all-targets
python3 scripts/gen-llms-txt.py --check
cargo run --locked -- stack shadow \
  --easyconfigs tests/shadow_fixtures/easyconfigs \
  --base tests/shadow_fixtures/base \
  --base tests/shadow_fixtures/bare.txt
echo "exit $?"

Docs, with Org 9.7 or newer on load-path ahead of the bundled Org:

(cd docs && emacs -Q --batch --load export.el)
bash scripts/gen-rust-api.sh
EB_STACK_DOCS_PYTHON=/path/to/venv/bin/python bash scripts/sphinx-build-docs.sh

Note

Low Risk
Read-only validation and documentation; no build, auth, or data-mutation paths—behavior is localized to inventory comparison and reporting.

Overview
Adds stack shadow (and MCP eb_stack_shadow) to flag site-layer easyconfigs whose module names overlap the EESSI base inventory loaded via the same --base reader as stack gap. Hits are classified as same, different, or unversioned; the command exits 1 when any shadow is found or a recipe fails to parse. Version matching reuses new installed_module_version / MODULE_VERSION_TOOLCHAIN_SEPARATOR helpers in build_order.

Also ships eight new how-to guides (shard, batch, sections, easystack, recipe identity, package latest, repro summary, site-layer shadow), wires them into the doc index and llms*.txt, updates the CLI/formats reference and changelog, and registers --test stack_shadow in CI.

Reviewed by Cursor Bugbot for commit 57e5be8. Bugbot is set up for automated code reviews on this repo. Configure here.

A site-layer recipe whose name is already in the base module inventory hides that module when the site path is searched first.
The `stack shadow` command loads the inventory with the same reader `stack gap` uses and exits 1 when a recipe matches the software version, another version, or a bare name.
The `stack shard` command has a CLI entry and no how-to.
The page runs the fixture wave and quotes the prelude and the three shard files.
It also says which modules are shared.
The `stack batch` command was described from the other stack pages and had no how-to of its own.
The page runs the GROMACS fixture and quotes the second-version warning the batch records.
The `stack sections` command had a reference section and no how-to.
The page runs the recorded previous and next lists and quotes the regrouped file.
The `stack easystack` command was in the command reference and the format page, and had no how-to.
The page writes the baseline lock and shows the option spelling.
Those three commands have been in the CLI reference since they were added and had no how-to.
The page quotes the derived class names and the template values for zlib 1.3.1.
The `package latest` command had a reference section and no how-to.
The page runs the directory-backend fixture, whose listing is empty, and describes the kept and dropped lines a real backend prints.
The `repro summary` command had a reference section and a mention in the fidelity page, and no how-to.
The page writes two case scores and quotes the table and the ratchet line.
The shadow check had a CLI section and no how-to.
The page runs the fixture layer and quotes the same, different, and unversioned lines.
It also shows a site-only recipe that exits 0.
@HaoZeke
HaoZeke merged commit 9af2be2 into main Nov 3, 2026
19 checks passed
@HaoZeke
HaoZeke deleted the feat/docs-shadow-check branch November 4, 2026 09:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant