Skip to content

package-bundles.org

Latest commit

 

History

History
381 lines (319 loc) · 17.6 KB

package-bundles.org

File metadata and controls

381 lines (319 loc) · 17.6 KB
Raw

Package bundle reference

package inspect, package plan, and package bump write a version-one package bundle. The bundle is the hand-off between parsing, Resolvo, EasyBuild authoring, and empirical build evaluation.

Bundle layout

BUNDLE/
  package.plan.json
  package.sbom.cdx.json
  closure.plan.json
  closure.sbom.cdx.json
  build-order.json
  locks/<profile>.lock.json
  easyconfigs/<letter>/<name>/<recipe>.eb
  easyconfigs/<letter>/<name>/<patch files>
  easyblocks/<letter>/<module>.py
  packages/<name>-<version>-<toolchain>/

package inspect writes only the plan and planned Software Bill of Materials (SBOM). package plan and package bump also write one lock and one EasyBuild recipe for every requested profile. A campaign requires at least one lock and recipe.

package plan writes closure.plan.json, closure.sbom.cdx.json, and build-order.json whenever --easyconfigs is passed, because that tree is a discovery root. --package-catalog and --package-sources add further providers. Companions land under packages/<name>-<version>-<toolchain>/, beside one shared easyconfigs/ overlay. Companion providers may be foreign or easybuild-bump; bump companions keep origin = easy-build and use the annual-bump emitter. --emit-easyblock does not combine with this writer.

Canonical package plan

package.plan.json is the build manifest. Its schema_version is 1 and unknown fields are rejected by the Rust model.

FieldMeaning
originconda-forge, spack, easy-build, pypi, cran, cargo, luarocks, or raku
packageNormalized name, version, homepage, description, and license
sourcesURLs, filenames, SHA-256 values, version control system (VCS) identities, staging directories, and provenance
dependenciesNames, constraints, optional explicit EasyBuild toolchains, roles, structured conditions, virtual capabilities, and provenance
rulesPreserved conflicts and requirements
buildToolchain, easyblock, build systems, configuration options, checksummed patches, typed easyconfig parameters, and module class
profilesIndependently installable product configurations
outputsRequested profile and stack-policy pairs
residualsExplicit parser, normalization, resolution, emission, build, or verification work

Source and dependency provenance records the input span, extractor, original text, and one of exact, derived, or ambiguous. The static foreign adapters preserve facts they can prove and put dynamic or ambiguous logic in residuals instead of inventing EasyBuild metadata.

Written provenance paths are source identities rather than checkout locations: conda-forge/recipe.yaml, spack/package.py, or the corresponding EasyBuild recipe name. Line and column spans still address the named input. Absolute checkout paths are used while reading local inputs but are not serialized into the bundle, so identical inputs produce identical manifests on different hosts.

An EasyBuild-origin dependency carries toolchain when its tuple names an explicit dependency toolchain. Annual bump planning maps that toolchain family onto the target generation and requires Resolvo to select a matching artifact. An implicit tuple omits toolchain and may select the minimum non-SYSTEM member of the target hierarchy. SYSTEM is eligible only when the source tuple or parent recipe selects it explicitly.

Parser notes are diagnostics. Canonical planning never searches note text for words such as dynamic or residual. A successfully materialized url_for_version, for example, may remain visible as a note but does not create residual work. Anything that needs mechanical repair or authoring judgment is represented by a typed residual with category, severity, summary, optional evidence, and optional source provenance. Conda build.number and compiler, standard-library, or cross-python template requirements are also notes because the EasyBuild toolchain represents those concerns. An unresolved template in an ordinary package identity remains a template-evaluation residual.

Residual severity has three values:

  • mechanical: deterministic formatting or metadata work;
  • judgment: package-authoring or target evidence is required;
  • blocking: the next pipeline stage cannot make a valid claim.
{
  "id": "foreign:imperative-patch:0",
  "stage": "parse",
  "category": "imperative-patch",
  "severity": "judgment",
  "summary": "imperative patch method requires EasyBuild translation",
  "provenance": {
    "span": {"path": "package.py", "start_line": 42}
  }
}

The build manifest stores patches as artifacts:

{
  "filename": "Orbit-2.0-portability.patch",
  "sha256": "4f43b42fdcf84d0cf634d993dd944f252c8243dc612a919fe2825d56f937c8eb",
  "source": "patches/Orbit-2.0-portability.patch"
}

Relative patch sources resolve against the package-policy TOML. Planning verifies the declared SHA-256 before copying each patch beside every emitted recipe. The manifest retains the portable source path; machine-local resolved paths do not enter the bundle.

Spack patch() directives retain their checksum, exact remote URL, source provenance, and structured when condition. Package-version predicates are specialized against the selected version; profile predicates remain attached until variant materialization. A remote artifact has a conventional filename, a url, a sha256, and a condition:

{
  "filename": "26aac9a2adb9a688ab8fe2a9664450de97557cf9.patch",
  "sha256": "54484b722df264dae3fd0c1094883b17431617e278eeba2cffbd720b36c9e21a",
  "url": "https://github.com/example/project/commit/26aac9a2adb9a688ab8fe2a9664450de97557cf9.patch?full_index=1",
  "condition": {"op": "always"}
}

EasyBuild accepts HTTP(S) URLs in patches and checks the downloaded bytes against the corresponding positional checksum. Local artifacts are verified and copied into the recipe directory; remote artifacts remain URLs in the emitted recipe. Imperative Spack def patch(self) methods cannot be represented as patch files and become residual build-authoring work when their version condition applies.

easyconfig_parameters accepts strings, integers, booleans, lists, and tables. It cannot contain executable expressions. Build-level parameters are defaults for every emitted recipe; profile-level parameters replace matching keys for one variant.

Planned CycloneDX SBOM

package.sbom.cdx.json is CycloneDX 1.5 derived from the package plan. It is a pre-build inventory of requested components and dependency intent. It is not a scan of installed files and does not establish the builds or binary-verified claim.

The root component carries the first source digest that is a SHA-256; a value that is not 64 hexadecimal characters is no hash. Its external references retain VCS tag/commit identity and one distribution URL plus SHA-256 per source artifact, including derived GitHub tag archives. The archive is derived only from a github.com remote, in any spelling (scp, ssh, a port, ssh.github.com); a gist or another host gets no invented archive. Overlay extensions appear as library components with their digest. These values come from the canonical plan after any explicit positional checksum overrides.

Consumers should preserve the lifecycle and eb_stack:* properties when copying the document into a wider stack report. Build evidence belongs in the campaign state, not in the planned SBOM.

The planned SBOM omits a generation timestamp because its identity is the canonical input and dependency intent, not the wall-clock time of a CLI run. Campaign evidence records build and verification event times. This keeps the SBOM reproducible across checkout roots and repeated planning runs.

Product profiles and recipe names

A profile represents one loadable product. Each requested profile is solved and emitted independently:

  • the default profile has an empty versionsuffix and an unsuffixed recipe;
  • a non-default profile declares its complete versionsuffix, such as -complex;
  • Message Passing Interface (MPI) and OpenMP toolchain options alone do not create a separate recipe name;
  • a feature or ABI variant that users load independently gets its own profile, lock, and .eb file.

For example, the QMCPACK profile set emits:

locks/default.lock.json
locks/complex.lock.json
easyconfigs/q/QMCPACK/QMCPACK-4.3.0-foss-2026.1.eb
easyconfigs/q/QMCPACK/QMCPACK-4.3.0-foss-2026.1-complex.eb

Resolvo profile lock

locks/<profile>.lock.json records the actual Boolean satisfiability (SAT) selection used to render that recipe:

{
  "schema_version": 1,
  "package": "QMCPACK",
  "version": "4.3.0",
  "profile": "complex",
  "toolchain": {"name": "foss", "version": "2026.1"},
  "versionsuffix": "-complex",
  "dependencies": [
    {
      "name": "HDF5",
      "version": "2.1.1",
      "toolchain": {"name": "foss", "version": "2026.1"},
      "easyconfig_path": "easybuild/easyconfigs/h/HDF5/HDF5-2.1.1-foss-2026.1.eb",
      "build": false
    }
  ],
  "pin_outcomes": [],
  "exclusions": [],
  "solver": "resolvo"
}

Every locked dependency carries its selected EasyBuild identity and source recipe path. Standard robot trees retain the path beginning at easybuild/easyconfigs/; other robot layouts retain the recipe filename. Machine-local robot prefixes are not serialized. build distinguishes direct build dependencies from runtime or host dependencies.

pin_outcomes keeps stack preferences reviewable. A preferred pin may record fallback: true when Resolvo selects another jointly compatible candidate. A locked pin cannot fall back. exclusions records candidates removed from consideration with the evidence-backed reason and optional scope.

SAT compatibility is not build evidence. A selected candidate may still fail configure, compile, link, tests, install, or sanity; the campaign records that outcome as a typed finding.

EasyBuild recipe set

Generated recipes follow the conventional EasyBuild hierarchy and syntax. Sources and checksums remain positional: all source checksums precede patch checksums. Verified local patch files are copied into the recipe directory; checksum-addressed remote patches remain exact URLs. Robot overlays are searched in command-line order, with later paths overriding earlier identities.

A foreign VCS commit is immutable source identity but is not the SHA-256 of the downloaded archive. Supply one --source-checksum per source artifact when the foreign recipe lacks archive hashes. package inspect keeps the missing checksum as a blocking residual; package plan refuses to emit recipes until all source and patch checksums are complete and valid.

Run the three recipe gates before a campaign:

eb-stack recipe format BUNDLE/easyconfigs/*/*/*.eb
eb-stack recipe lint BUNDLE/easyconfigs/*/*/*.eb
eb-stack recipe check \
  --recipe BUNDLE/easyconfigs/q/QMCPACK/QMCPACK-4.3.0-foss-2026.1.eb \
  --easyconfigs tests/sbom_recorded/cbc \
  --easyconfigs BUNDLE/easyconfigs

Easyblocks included with the recipe

Some software cannot be described by an easyconfig alone. SeisSol’s HOST_ARCH sets the vector width of the kernels its code generators emit, so it has to be derived from the build host at configure time, and the binary name, and with it the sanity check, follows from that value. That logic belongs in a software-specific easyblock, EB_SeisSol, not in a hardcoded configure flag or a site hook.

A package layer keeps such an easyblock beside its easyconfigs:

[build]
easyblock = "EB_SeisSol"

[[build.easyblocks]]
filename = "seissol.py"
source = "easyblocks/seissol.py"
sha256 = "8c80992232a496096e6687679565509f2637d6cd2b03a0bbf49a3bc3a7488713"

source is relative to the layer file. A layer that sets build.easyblocks owns the complete list. build.drop_easyconfig_parameters names the parameters the easyblock derives itself, such as sanity_check_paths, test_cmd or modextrapaths; they leave the build level and every profile, so the emitted recipe carries none of them. Planning and bumping verify each module against its SHA-256, parse it (without importing it), and record the classes it defines, with their bases and extra_options keys, under build.easyblocks in the plan. Planning fails when no included module defines the class the recipe resolves to: the class named by build.easyblock, or the class EasyBuild derives from the software name when none is named.

build.config_options replaces the recipe’s configopts; when the options do not fit one line they are written one per configopts += line. build.extra_config_options keeps the recipe’s own configopts, whatever expression builds it, and adds one configopts += line per option after its last statement, every layer’s list in layer order. A bump also writes a layer’s dependencies.requirements with the build role into builddependencies. A requirement whose roles are only build or test replaces a runtime role the source recipe carried, and the tuple moves out of dependencies. source_checksums on the layer, one digest per source, is the bump checksum when the command line does not pass one. A layer parameter replaces the recipe’s whole value, += continuations included.

EasyBuild finds a software-specific easyblock by name, and its own easyconfigs never spell it out, so when build.easyblock equals the derived class (EB_SeisSol for SeisSol) the emitted recipe has no easyblock line. The writer copies each module to easyblocks/<letter>/<module>.py, the layout of the easyblocks repository. A campaign passes every module in that directory to EasyBuild as --include-easyblocks=FILE,FILE, so the class exists for the build without being installed first. examples/packages/seissol-easyblock.toml layers this over examples/packages/seissol.toml.

The bundle establishes resolves only when every requested profile has a lock and recipe. See campaign state and findings for the build and binary claims.

When a package owes an easyblock

A package that carries variant logic as data repeats that logic in every recipe. A software-specific easyblock keeps it once, and the easyconfigs lean on it. package plan and package bump decide whether a package owes one when its recipe resolves to a generic easyblock: build.easyblock names none or a class other than the one EasyBuild derives from the software name, and no included module defines the derived class. Any one of three triggers applies:

  1. Two or more profiles whose config_options differ from each other, after the build-level options. A variant switch such as QMCPACK’s -DQMC_COMPLEX=ON against -DQMC_COMPLEX=OFF is this case.
  2. The build-level or any profile easyconfig_parameters carry sanity_check_paths, sanity_check_commands, test_cmd, testopts, pretestopts, or runtest as a string command.
  3. A profile’s sanity_check_paths.files or sanity_check_commands names a binary that differs between profiles, as qmcpack and qmcpack_complex do.

The plan then records one residual. Its stage is emit, its category missing-easyblock, its severity judgment, and its id missing-easyblock:<class>, for example missing-easyblock:EB_QMCPACK. The summary names the derived class. The evidence lists the profile option differences, the parameters, and the binary names that would move into it.

--emit-easyblock on package plan and package bump (the emit_easyblock argument of the Model Context Protocol (MCP) tools) renders a skeleton instead of leaving the residual:

eb-stack package plan --emit-easyblock \
  --source fixtures/foreign_ingest/spack_qmcpack/package.py \
  --package-config examples/packages/qmcpack.toml \
  --toolchain foss/2026.1 \
  --easyconfigs tests/sbom_recorded/cbc \
  --stack-policy policy.toml \
  --out-dir BUNDLE

The flag writes easyblocks/<letter>/<module>.py in the bundle, for QMCPACK easyblocks/q/qmcpack.py, records it under build.easyblocks with its SHA-256, and sets build.easyblock to the derived class, so no residual is raised. The module derives from the plan’s easyblock (CMakeMake when the plan names none) and carries:

  • one CUSTOM parameter per feature, with the default profile’s value;
  • a configure_step that passes the options present in the profiles where a feature is true and absent where it is false, and never overrides a -D<option>= already in configopts;
  • a test_step when test_cmd, testopts, or pretestopts exist, which runs only when runtest is true;
  • a sanity_check_step built from sanity_check_paths and sanity_check_commands, with a binary name that differs between profiles chosen by the feature that flips it;
  • make_module_extra from modextrapaths.

The options the class sets leave the emitted recipes, which then select a variant with a boolean parameter such as complex = True. Options shared by every profile stay in the easyconfig. The recipes keep their test and sanity parameters, so the skeleton and the recipes agree until a human moves them.

The module is a starting point. Its docstring says so, it has not been built, and a human completes and reviews it before release. The rendered text parses with the same reader that checks included modules, keeps lines within 120 characters, and carries the EasyBuild copyright header. examples/packages/easyblocks/qmcpack.py shows a finished easyblock of this shape. package bump --emit-easyblock writes the module and updates the plan manifest; the retargeted recipe keeps the text of its source recipe. The flag does not combine with package-closure planning.