diff --git a/CHANGELOG.md b/CHANGELOG.md index c3894758..e37aad15 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -29,6 +29,19 @@ All notable changes to this project are documented here. ### Added +- `stack order --installed` stops the build order at modules already on + the machine. A file is a module listing and a directory is an Lmod + tree, read the same way as `stack gap --installed`. The version is the + full module version. A listed module is left out of the order, and so + are the dependencies that were installed with it. `--filter-deps` + still drops dependencies first. The modules that remain are chosen + with the synthetic GCC-family rule `recipe hierarchy --export` + prints. Stdout reports `installed=` as how many modules the walk + stopped at. `eb_stack_order` takes the same `installed` list. +- The Model Context Protocol manual lists one tool for every CLI + command. `docs/orgmode/reference/mcp.org` names each tool, including + the recipe and easyblock tools that have no command. A test fails + when a subcommand's tool is absent from that page. - `stack shadow` reports a site-layer recipe that shadows a module the EESSI base stack already provides. `--base` is an Lmod tree or a module listing, loaded with the same inventory reader as `stack gap`. The same @@ -270,6 +283,9 @@ include `stack gap`, `stack oom`, `stack from-commit`, `rebuild-set`, `source probe:`. With no `--arch` all three are still probed. With one family there is nothing to compare with, so a source it cannot fetch is `unreachable` rather than `arch-missing`, and still exits 1. + Two families compare only with each other: a miss is `arch-missing` + when the other named family fetched that source slot, and + `unreachable` when neither did. - `classify_build_failure` classes `No space left on device` and `Disk quota exceeded` as `resource`. A full disk in an EasyBuild log was `compile` before. diff --git a/README.md b/README.md index ff1ac8d9..838f2e75 100644 --- a/README.md +++ b/README.md @@ -402,7 +402,8 @@ for `pr-overlap`, `rebuild-set`, `component-reach`, and `easyblock-reach`. `eb-stack mcp --workspace ROOT` (or `EB_STACK_WORKSPACE`) is the root for the recipe file tools. -- `eb_package_inspect`, `eb_package_plan`, `eb_package_bump`; +- `eb_package_inspect`, `eb_package_plan`, `eb_package_bump`, + `eb_package_latest`; - `eb_recipe_read`, `eb_recipe_list`, `eb_recipe_edit`, `eb_recipe_write`, `eb_recipe_grep`, `eb_recipe_apply_suggestion`, `eb_recipe_diff`, `eb_dep_versions`; @@ -411,7 +412,11 @@ recipe file tools. - `eb_module_lint`, `eb_easyblock_dry_run`, `eb_build_log`, `eb_build_test` (easyblocks; `eb_build_test` only with `EB_STACK_ALLOW_BUILD=1`); - `eb_stack_solve`, `eb_stack_sbom`, `eb_stack_order`, - `eb_stack_easystack`, `eb_stack_sections`, `eb_stack_from_commit`; + `eb_stack_easystack`, `eb_stack_gap`, `eb_stack_provided`, + `eb_stack_shadow`, `eb_stack_batch`, `eb_stack_shard`, + `eb_stack_arch`, `eb_stack_files`, `eb_stack_generation`, + `eb_stack_cuda_cc`, `eb_stack_known_failures`, `eb_stack_oom`, + `eb_stack_bot_classify`, `eb_stack_sections`, `eb_stack_from_commit`; - `eb_component_reach` (same query as `component-reach`, including a purl or a CVE id); - `eb_target_list`, `eb_target_doctor`; - `eb_campaign_run`, `eb_campaign_status`; diff --git a/docs/orgmode/howto/arch-unsupported.org b/docs/orgmode/howto/arch-unsupported.org index 01374e1d..b44ed802 100644 --- a/docs/orgmode/howto/arch-unsupported.org +++ b/docs/orgmode/howto/arch-unsupported.org @@ -199,4 +199,40 @@ same source slot resolves to a file that exists on another architecture. ~--arch x86_64~ on this fixture probes only the x86_64 wheel, which the server has, and exits 0. +** Two architectures + +What it does: ~--arch x86_64 --arch aarch64 --probe-sources~ probes +those two families and no other. A target such as ~x86_64/amd/zen4~ +is still the family before the slash. Naming a family and one of its +targets, ~--arch aarch64 --arch aarch64/neoverse_v1~, probes that +family once. The order of the flags does not change the report. + +Why you'd use it: two bot targets failed the fetch and a third was +not in the run. The comparison is between the families that were +named. A family left off the command is not fetched, and it does not +turn ~arch-missing~ into ~unreachable~ or the other way around. + +#+begin_src bash +eb-stack stack arch \ + --arch x86_64 \ + --arch aarch64 \ + --easystack tests/source_probe_fixtures/easystacks/kaleido.yml \ + --easyconfigs tests/source_probe_fixtures/easyconfigs \ + --probe-sources +#+end_src + +On the same copy and server it prints: + +#+begin_example +Kaleido-0.2.1-GCCcore-15.2.0.eb source kaleido-0.2.1-manylinux2014-aarch64.whl is arch-missing on aarch64: __BASE_URL__/kaleido-0.2.1-manylinux2014-aarch64.whl (attempt 1/3 HEAD=status:404 RANGE=status:404) +parse_failed=0 unsupported=0 flagged=0 source_unreachable=0 source_arch_missing=1 +#+end_example + +How to read it: the x86_64 wheel is on the server, so the aarch64 +wheel is ~arch-missing~. riscv64 is not named and is not probed. The +same command with ~--arch aarch64 --arch riscv64~ prints both wheels +as ~unreachable~, with ~source_unreachable=2~ and +~source_arch_missing=0~, because neither probed family fetched the +source slot. The x86_64 wheel is not consulted. + See [[file:../reference/cli.org][the command-line reference]]. diff --git a/docs/orgmode/howto/solve-lock.org b/docs/orgmode/howto/solve-lock.org index 3296e290..d26edf3d 100644 --- a/docs/orgmode/howto/solve-lock.org +++ b/docs/orgmode/howto/solve-lock.org @@ -118,6 +118,25 @@ eb-stack stack order \ requirement. The hash file is one ~ ~ line per build. See [[file:../reference/cli.org][the CLI reference]]. +~--installed~ stops the walk at modules the site already has, the way +the robot does. ~--filter-deps~ still drops dependencies before that +walk. The two compose: a filtered dependency is gone, and a listed +module is a leaf whose own dependencies are not ordered. + +#+begin_src bash +eb-stack stack order \ + --easyconfigs fixtures/gromacs_2025_to_next/easyconfigs \ + --root GROMACS \ + --installed fixtures/stack_order_installed/installed.txt \ + --filter-deps FFTW \ + --out /tmp/build-order.txt +#+end_src + +The order file has no ~OpenBLAS~ path and no ~FFTW~ path. ~GROMACS~ is +still in it. Standard output ends with ~installed=1~ and then +~order=/tmp/build-order.txt builds=~. ~installed=1~ is OpenBLAS. FFTW +was filtered, so it is not a leaf the walk stopped at. + * One lock per toolchain family A foss solve only sees the foss hierarchy. diff --git a/docs/orgmode/reference/cli.org b/docs/orgmode/reference/cli.org index f0c858c2..860a75da 100644 --- a/docs/orgmode/reference/cli.org +++ b/docs/orgmode/reference/cli.org @@ -432,6 +432,7 @@ eb-stack stack order \ --easyconfigs TREE [--easyconfigs OVERLAY]... \ --root NAME [--root NAME==VERSION]... \ [--filter-deps NAME,NAME...] \ + [--installed LIST]... \ [--out /tmp/build-order.txt] \ [--oldest] \ [--easystack-out /tmp/order.yml] \ @@ -644,7 +645,20 @@ not repin a recipe. See [[file:../howto/solve-lock.org][Order every pinned build ~name==version~ pins one build. ~--filter-deps~ drops each named dependency before that order is built, together with the subtree under it, using the same matching as ~stack solve~; when the flag is absent -the command reads ~EASYBUILD_FILTER_DEPS~. ~--oldest~ takes the oldest +the command reads ~EASYBUILD_FILTER_DEPS~. ~--installed~ repeats. A file +is ~module -t avail~, ~module spider -t~, or ~Name/version~, and a +directory is an Lmod tree, both read the same way as +~stack gap --installed~. The version is the full module version, +~1.3.1-GCCcore-15.2.0~, the spelling under ~modules/all~. A system +toolchain contributes no toolchain segment. A listed module is a leaf: +it stays out of the order, and so do the dependencies that were +installed with it. A dependency the tree cannot resolve is still an +error. The walk that remains uses the synthetic GCC-family rule +~recipe hierarchy --export~ prints, so an unpinned dependency of a +~GCC~ or ~GCCcore~ recipe stays in that family. ~--filter-deps~ is +applied first, so a filtered dependency is not counted as a leaf. The +line ~installed=~ just before ~order=~ is how many distinct modules +the walk stopped at. ~--oldest~ takes the oldest admissible version of an unpinned requirement, and the default is the newest. ~--out~ is one easyconfig path per line (default ~build-order.txt~). ~--easystack-out~ writes that same sequence as an @@ -863,6 +877,9 @@ fallback, on each family ~--arch~ names, or on ~x86_64~, ~aarch64~, and ~riscv64~ with no ~--arch~. A source no probed family can fetch is ~unreachable~. A source another probed family can fetch is ~arch-missing~, so with one ~--arch~ a missing source is ~unreachable~. +Two ~--arch~ values compare only that pair: a miss is ~arch-missing~ +when the other named family fetched the same source slot, and +~unreachable~ when neither did. ~--probe-retries~ and ~--probe-timeout-s~ tune that network check. [[file:../howto/arch-unsupported.org][Refuse a plan an architecture cannot build]] @@ -1751,7 +1768,8 @@ plus sandboxed recipe and easyblock tools: (easyblocks; ~eb_build_test~ only with ~EB_STACK_ALLOW_BUILD=1~); - ~eb_stack_solve~, ~eb_stack_sbom~, ~eb_stack_order~, ~eb_stack_easystack~, ~eb_stack_gap~, ~eb_stack_provided~, - ~eb_stack_batch~, ~eb_stack_shard~, ~eb_stack_arch~, ~eb_stack_files~, + ~eb_stack_shadow~, ~eb_stack_batch~, ~eb_stack_shard~, + ~eb_stack_arch~, ~eb_stack_files~, ~eb_stack_generation~, ~eb_stack_cuda_cc~, ~eb_stack_known_failures~, ~eb_stack_oom~, ~eb_stack_bot_classify~, ~eb_stack_sections~, ~eb_stack_from_commit~; @@ -1792,8 +1810,12 @@ table and the derived-key rules. ~eb_recipe_hierarchy~ takes ~export~. The plain call prints the fixture generations. ~export~ true adds the synthetic GCC-family rule, the same split as ~recipe hierarchy --export~. ~eb_stack_order~ requires ~easyconfigs~ and ~root~, and takes ~out~, -~oldest~, ~easystack_out~, ~hashes_out~, and ~filter_deps~. +~oldest~, ~easystack_out~, ~hashes_out~, ~filter_deps~, and ~installed~. ~filter_deps~ drops the same dependencies ~--filter-deps~ drops. +~installed~ is the module list or Lmod tree ~--installed~ reads. A +module it names is left out of the order. The result field ~installed~ +is how many modules the walk stopped at, and is null when the argument +is omitted. ~eb_stack_easystack~ requires ~lock~ and takes ~out~ and ~option~, each option a ~: