Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
57e339f
chore(package): drop the excludes .gitignore already keeps out of the…
HaoZeke Nov 4, 2026
7d04acd
feat(ledger): per-stack ledger as an RFC 9162 log with signed heads
HaoZeke Nov 4, 2026
2d54bee
feat(mcp): eb_ledger_append, eb_ledger_head, eb_ledger_show, eb_ledge…
HaoZeke Nov 4, 2026
0b5604f
test(ledger): round trip, removed record, rewritten log, key order
HaoZeke Nov 4, 2026
185c7c9
docs(ledger): canonical form, record fields, heads, and MCP tools
HaoZeke Nov 4, 2026
34e36e4
feat(shard): resolve from-commit and from-pr lines against their trees
HaoZeke Nov 4, 2026
4df3766
docs(shard): document commit trees for fetched build-list lines
HaoZeke Nov 4, 2026
e833b33
fix(shard): derive Debug on the shard run so its tests compile
HaoZeke Nov 4, 2026
9a08821
style: simplify two boolean expressions clippy 1.94 flags
HaoZeke Nov 4, 2026
f8f6c78
fix(shard): let a robot-path line need a module a fetched line builds
HaoZeke Nov 4, 2026
e986656
test(shard): plan fetched PETSc with the petsc4py line that needs it
HaoZeke Nov 4, 2026
b3575ed
fix(workspace): resolve absolute paths through a symlinked root
HaoZeke Nov 4, 2026
fc9683f
style(workspace): format the root alias regression
HaoZeke Nov 4, 2026
bab87cc
test(shard): cover explicit recipe aliases
HaoZeke Nov 4, 2026
7626183
fix(shard): preserve explicitly selected recipe aliases
HaoZeke Nov 4, 2026
c3181a0
style(shard): format explicit alias selection
HaoZeke Nov 4, 2026
7d14adc
fix(lint): recognize the max_parallel build limit
HaoZeke Nov 4, 2026
e0a5d4d
style(lint): format the parallelism regression
HaoZeke Nov 4, 2026
704093a
test(lint): provide easyblock context for parallelism checks
HaoZeke Nov 5, 2026
e793bb3
fix(shard): reject per-line dependency filters
HaoZeke Nov 5, 2026
72ba025
test(shard): supply the required shard count
HaoZeke Nov 5, 2026
3538e68
feat(ledger): combine installation records with shard planning
HaoZeke Nov 5, 2026
e977c96
fix(ledger): serialize writers with a POSIX record lock
HaoZeke Nov 5, 2026
a38b7f4
style(ledger): apply Rust formatting
HaoZeke Nov 5, 2026
2a44ae8
chore(ledger): lock the Merkle revision and Unix dependency
HaoZeke Nov 5, 2026
e8465ca
docs(ledger): regenerate public command references
HaoZeke Nov 5, 2026
77184b5
docs: point contribution guidance at public workflows
HaoZeke Nov 5, 2026
fbadfef
test(ledger): refuse aliases that split writer exclusion
HaoZeke Nov 5, 2026
0b167b2
fix(ledger): refuse lock aliases and validate writer inodes
HaoZeke Nov 5, 2026
556bed4
style(ledger): apply Rust formatter to lock validation
HaoZeke Nov 5, 2026
3839bf7
fix(package): declare a registry version for the ledger tree dependency
HaoZeke Nov 5, 2026
c978c7e
ci(ledger): exercise writer and proof regressions
HaoZeke Nov 5, 2026
11fb72b
test(ledger): preserve one event across writer retries
HaoZeke Nov 5, 2026
2aeb9c9
fix(ledger): acknowledge durable events across writer retries
HaoZeke Nov 5, 2026
93673d6
test(ledger): align catalog and exported discovery contracts
HaoZeke Nov 5, 2026
0755f7e
style(ledger): format event retry implementation
HaoZeke Nov 5, 2026
18130f1
fix(metadata): verify private working-directory exclusions
HaoZeke Nov 5, 2026
4b0467a
style(metadata): format ignore-policy verification
HaoZeke Nov 5, 2026
b1da826
refactor(ledger): simplify optional string validation
HaoZeke Nov 5, 2026
c05b639
test(packaging): cover checksums in Bundle components
HaoZeke Nov 5, 2026
0910230
fix(packaging): validate checksums in Bundle components
HaoZeke Nov 5, 2026
04f7184
style(packaging): format component checksum validation
HaoZeke Nov 5, 2026
38f70f5
fix(docs): close the shard tree archive span Sphinx cannot parse
HaoZeke Nov 5, 2026
aaef02e
fix(packaging): count postinstall patches in Bundle checksum coverage
HaoZeke Nov 5, 2026
97550d4
fix(ledger): recover interrupted writes and serialize signed heads
HaoZeke Nov 5, 2026
4aeb09a
Merge remote-tracking branch 'origin/eb/shard-commit-trees-v2' into e…
HaoZeke Nov 5, 2026
07787c1
fix(docs): stop link check fetching shard archive placeholders
HaoZeke Nov 5, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions .github/workflows/ci_test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,16 @@ jobs:
- name: Cargo source package
run: cargo package --locked

ledger:
name: ledger proofs and writer exclusion
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- uses: dtolnay/rust-toolchain@stable
- uses: Swatinem/rust-cache@v2
- name: Records, proofs, and POSIX writer locks
run: cargo test --locked --test ledger --test ledger_posix_lock --test ledger_lock_aliases --test ledger_retry --test ledger_durability

msrv:
name: Rust 1.88 minimum
runs-on: ubuntu-latest
Expand Down Expand Up @@ -88,6 +98,7 @@ jobs:
--test eon_foss_2026_1 \
--test qmcpack_foss_2026_1 \
--test eon_packaging \
--test bundle_component_checksums \
--test goal_scenario \
-- --nocapture

Expand Down
6 changes: 3 additions & 3 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,9 @@ target
# rustc ICE dumps
rustc-ice-*.txt

# Machine-local agent configuration
.claude/
.codex/
# Hidden working directories are private; repository CI remains visible.
.*/
!/.github/

# Sphinx build (docs/orgmode/*.org is the source of truth; docs/source/*.rst
# and graphviz SVGs under docs/orgmode/images are regenerated by
Expand Down
3 changes: 2 additions & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,7 @@ stack solve | sbom | order | easystack | sections | from-commit | gap | provi
target list | doctor
campaign run | status | finding claim | finding resolve
repro summary
ledger append | head | show | verify
easyblock-reach
pr-overlap
rebuild-set
Expand All @@ -86,7 +87,7 @@ mcp

The Model Context Protocol (MCP) catalog mirrors these commands,
including pr-overlap, rebuild-set, component-reach, easyblock-reach,
and repro summary, with `eb_`-prefixed tools. Recipe file tools
repro summary, and the ledger commands, with `eb_`-prefixed tools. Recipe file tools
(`eb_recipe_read`, `eb_recipe_list`, `eb_recipe_edit`,
`eb_recipe_apply_suggestion`, `eb_recipe_diff`, `eb_recipe_write`,
`eb_recipe_grep`, and `eb_dep_versions`), easyblock dry-run, module
Expand Down
37 changes: 37 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,22 @@ All notable changes to this project are documented here.

### Fixed

- `stack shard` resolves a build-list line with `--from-commit` or
`--from-pr` against that commit's or pull request's easyconfigs first,
then the robot path, the order EasyBuild uses. A dependency such a line
shares with another shard goes to the prelude, so two shards do not
build it at once. `petsc4py-3.25.0-foss-2026.1.eb --from-commit=...`
and the `PETSc-3.25.0-foss-2026.1.eb` line it needs plan into one
shard, PETSc first. Split over two shards, the second build stops on
PETSc's install lock. A prelude line for a module only a
fetched tree provides carries that line's `--from-commit`,
`--from-pr` and `--include-easyblocks-from-*` options. A fetched line
that still cannot be resolved stays opaque; `plan.json` gives the cause
under `opaque_reason` and the top-level `opaque`, and the command
prints a `warning:` line for it and `opaque=<n>` in its summary.
A `--commit-cache` download that fails before an HTTP status exits 1
and writes nothing; the error names the cache, the commit, the URL,
and the transport error. An HTTP status, including 404, stays opaque.
- `stack shard` plans a build-list line it cannot resolve as an opaque
line when EasyBuild can still build it: the line carries `--from-pr`
or `--from-commit`, or its easyconfig is on the robot path but does
Expand Down Expand Up @@ -44,6 +60,27 @@ All notable changes to this project are documented here.

### Added

- `stack shard --commit-cache DIR` reads the trees of `--from-commit` and
`--from-pr` lines from DIR and downloads the missing ones: a commit from
`codeload.github.com/easybuilders/easybuild-easyconfigs` (its `.eb`
files only), a pull request's changed easyconfigs at its head from
`api.github.com`. `--commit-tree COMMIT=DIR` and `--pr-tree N=DIR` name
trees already on disk. `eb_stack_shard` takes `commit_cache`,
`commit_tree` and `pr_tree`.
- `ledger append`, `ledger head`, `ledger show`, and `ledger verify` keep
a per-stack software ledger: one JSON Lines file of install, rebuild,
retire, move, access, removal, finding, and correction records. Each
line is canonical JSON (keys sorted, no whitespace, integers only) with
its `index` and `id` filled in, and its leaf is the RFC 9162 leaf hash
of the line, from the `rfc9162` crate. Heads are signed with Ed25519
under the domain `eb-stack-ledger-v1`; the key is a PEM path passed on
the command line and is refused inside the log's directory.
`ledger verify --record N --head HEAD` checks an inclusion proof and
`ledger verify --from HEAD1 --to HEAD2` a consistency proof, so a
removed record or a rewritten log is status 1. `eb_ledger_append`,
`eb_ledger_head`, `eb_ledger_show`, and `eb_ledger_verify` are the same
commands over MCP.

- `stack order --installed` stops the build order at modules already on
the machine. A file is a module listing and a directory is an Lmod
tree, read the same way as `stack gap --installed`. The version is the
Expand Down
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ Do not hand-edit generated `docs/source/**/*.rst` or `docs/source/crates/`.

## Packaging campaigns

Follow `AGENTS.md`, `skills/annual-bump/SKILL.md` (generation rebuild), and
Follow `skills/annual-bump/SKILL.md` (generation rebuild) and
`skills/new-package/SKILL.md` (conda-forge/Spack: new EasyBuild recipe). Run
`skills/verify-recipe/SKILL.md` over what they emit before a build or a push,
and `skills/tool-repair/SKILL.md` when the emitted file needed a hand
Expand Down
Loading
Loading