From c77ed27fd2d4d72211e00891e031e1dacfeee14d Mon Sep 17 00:00:00 2001 From: Rohit Goswami Date: Tue, 3 Nov 2026 17:07:46 +0000 Subject: [PATCH 1/6] feat(stack): order builds against an installed module list `stack order` walked every dependency the roots pin, including modules already on the machine. `--installed` now stops that walk at each listed module and leaves that module and its own dependencies out of the order. `--filter-deps` still drops dependencies first, and the modules that remain follow the synthetic GCC-family hierarchy. --- CHANGELOG.md | 9 + docs/orgmode/howto/solve-lock.org | 19 ++ docs/orgmode/reference/cli.org | 22 +- docs/orgmode/reference/formats.org | 9 +- fixtures/stack_order_installed/installed.txt | 2 + llms-full.txt | 50 ++++- src/build_order.rs | 207 ++++++++++++++++++- src/main.rs | 9 + src/mcp.rs | 5 +- src/workflows.rs | 56 ++++- tests/mcp_v1.rs | 48 +++++ tests/stack_order_filter.rs | 120 +++++++++++ 12 files changed, 529 insertions(+), 27 deletions(-) create mode 100644 fixtures/stack_order_installed/installed.txt diff --git a/CHANGELOG.md b/CHANGELOG.md index c3894758..76bfec95 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -29,6 +29,15 @@ All notable changes to this project are documented here. ### Added +- `stack order --installed` stops the build order at modules already on + the machine. A file is a module listing and a directory is an Lmod + tree, read the same way as `stack gap --installed`. The version is the + full module version. A listed module is left out of the order, and so + are the dependencies that were installed with it. `--filter-deps` + still drops dependencies first. The modules that remain are chosen + with the synthetic GCC-family rule `recipe hierarchy --export` + prints. Stdout reports `installed=` as how many modules the walk + stopped at. `eb_stack_order` takes the same `installed` list. - `stack shadow` reports a site-layer recipe that shadows a module the EESSI base stack already provides. `--base` is an Lmod tree or a module listing, loaded with the same inventory reader as `stack gap`. The same diff --git a/docs/orgmode/howto/solve-lock.org b/docs/orgmode/howto/solve-lock.org index 3296e290..d26edf3d 100644 --- a/docs/orgmode/howto/solve-lock.org +++ b/docs/orgmode/howto/solve-lock.org @@ -118,6 +118,25 @@ eb-stack stack order \ requirement. The hash file is one ~ ~ line per build. See [[file:../reference/cli.org][the CLI reference]]. +~--installed~ stops the walk at modules the site already has, the way +the robot does. ~--filter-deps~ still drops dependencies before that +walk. The two compose: a filtered dependency is gone, and a listed +module is a leaf whose own dependencies are not ordered. + +#+begin_src bash +eb-stack stack order \ + --easyconfigs fixtures/gromacs_2025_to_next/easyconfigs \ + --root GROMACS \ + --installed fixtures/stack_order_installed/installed.txt \ + --filter-deps FFTW \ + --out /tmp/build-order.txt +#+end_src + +The order file has no ~OpenBLAS~ path and no ~FFTW~ path. ~GROMACS~ is +still in it. Standard output ends with ~installed=1~ and then +~order=/tmp/build-order.txt builds=~. ~installed=1~ is OpenBLAS. FFTW +was filtered, so it is not a leaf the walk stopped at. + * One lock per toolchain family A foss solve only sees the foss hierarchy. diff --git a/docs/orgmode/reference/cli.org b/docs/orgmode/reference/cli.org index f0c858c2..a0bfa245 100644 --- a/docs/orgmode/reference/cli.org +++ b/docs/orgmode/reference/cli.org @@ -432,6 +432,7 @@ eb-stack stack order \ --easyconfigs TREE [--easyconfigs OVERLAY]... \ --root NAME [--root NAME==VERSION]... \ [--filter-deps NAME,NAME...] \ + [--installed LIST]... \ [--out /tmp/build-order.txt] \ [--oldest] \ [--easystack-out /tmp/order.yml] \ @@ -644,7 +645,20 @@ not repin a recipe. See [[file:../howto/solve-lock.org][Order every pinned build ~name==version~ pins one build. ~--filter-deps~ drops each named dependency before that order is built, together with the subtree under it, using the same matching as ~stack solve~; when the flag is absent -the command reads ~EASYBUILD_FILTER_DEPS~. ~--oldest~ takes the oldest +the command reads ~EASYBUILD_FILTER_DEPS~. ~--installed~ repeats. A file +is ~module -t avail~, ~module spider -t~, or ~Name/version~, and a +directory is an Lmod tree, both read the same way as +~stack gap --installed~. The version is the full module version, +~1.3.1-GCCcore-15.2.0~, the spelling under ~modules/all~. A system +toolchain contributes no toolchain segment. A listed module is a leaf: +it stays out of the order, and so do the dependencies that were +installed with it. A dependency the tree cannot resolve is still an +error. The walk that remains uses the synthetic GCC-family rule +~recipe hierarchy --export~ prints, so an unpinned dependency of a +~GCC~ or ~GCCcore~ recipe stays in that family. ~--filter-deps~ is +applied first, so a filtered dependency is not counted as a leaf. The +line ~installed=~ just before ~order=~ is how many distinct modules +the walk stopped at. ~--oldest~ takes the oldest admissible version of an unpinned requirement, and the default is the newest. ~--out~ is one easyconfig path per line (default ~build-order.txt~). ~--easystack-out~ writes that same sequence as an @@ -1792,8 +1806,12 @@ table and the derived-key rules. ~eb_recipe_hierarchy~ takes ~export~. The plain call prints the fixture generations. ~export~ true adds the synthetic GCC-family rule, the same split as ~recipe hierarchy --export~. ~eb_stack_order~ requires ~easyconfigs~ and ~root~, and takes ~out~, -~oldest~, ~easystack_out~, ~hashes_out~, and ~filter_deps~. +~oldest~, ~easystack_out~, ~hashes_out~, ~filter_deps~, and ~installed~. ~filter_deps~ drops the same dependencies ~--filter-deps~ drops. +~installed~ is the module list or Lmod tree ~--installed~ reads. A +module it names is left out of the order. The result field ~installed~ +is how many modules the walk stopped at, and is null when the argument +is omitted. ~eb_stack_easystack~ requires ~lock~ and takes ~out~ and ~option~, each option a ~: