feat(container): lean pixi-based two-stage container with multi-arch support

[jcgraciosa]

Summary

• Rewrites container/Containerfile to a pixi two-stage build, fixing PETSc
  version drift between the container and Binder (root cause of the v3.1.0
  container build failure)
• Adds linux/arm64 support for native performance on ARM64 machines
  (Apple Silicon Macs and ARM64 Linux servers/cloud instances)
• Moves gmsh from PyPI to conda-forge — restores the original
  environment.yaml approach and was required to unblock arm64 support
  (PyPI gmsh has no linux-aarch64 wheels)

Details

Containerfile: Two-stage build (builder: pixi install + compile UW3;
runtime: copy env + Mesa apt packages). Drops vtk-osmesa; uses conda-forge
vtk + Mesa instead — same as Binder, works on amd64 and arm64.

pixi.toml/pixi.lock: Added linux-aarch64 platform and target sections.
gmsh moved to [dependencies] (conda-forge).

environment.yaml: Marked as human-readable reference only.

Testing

• pixi install -e runtime: linux/arm64 ✓, linux/amd64 ✓, osx-arm64 ✓
• Full container build: CI

Underworld development team with AI support from Claude Code

[lmoresi]

Adversarial review

The two-stage structure itself held up under attack (see the cleared list), but three findings block merge:

1. CRITICAL — import gmsh is broken on linux-aarch64, the PR's headline platform. Conda-forge gmsh-4.15.2 ships the app/library only — no Python module; that lives in the separate python-gmsh package, which is absent from the lock. linux-64/osx-arm64 stayed green only because pygmsh drags the pypi gmsh wheel back in; no such wheel exists for linux-aarch64, so the solver silently downgraded pygmsh to 7.1.12 and locked no gmsh Python module at all. The arm64 container cannot build a single mesh. Fix: add conda python-gmsh (the old environment.yaml listed both) and re-solve the lock; verify import gmsh in the linux-aarch64 env.
2. MAJOR — no green container build exists for the PR head. The only build attempt ran on f814251 and died at pixi install -e runtime (aarch64 unsupported); the fix commit 47ffb64 was never build-tested. The workflow has no pull_request trigger, so merging is the first real test — and it pushes to the ghcr :development tag. One demonstrated green build of the actual head (fork workflow_dispatch is enough) before merge.
3. MAJOR — no .dockerignore, and COPY . . runs after pixi install -e runtime: the documented local podman build . uploads the multi-GB host .pixi/ and build/ as context and merges the host's .pixi/envs/* over the freshly installed runtime env — a wrong-arch clobber for anyone who has built natively (as the PR author has). CI is unaffected (clean checkout); local builds are the trap. Exclude .pixi/build or add a .dockerignore.

Minor: environment.yaml's new header calls itself the runtime reference but still says petsc=3.24 while the lock ships 3.25.3 — wrong on exactly the pin that caused #355; lock churn on development now triggers full dual-arch rebuilds with arm64 under QEMU and no build cache configured; curl pixi.sh/install.sh | sh is unpinned; the runtime image still carries the full compiler toolchain, so "lean" is relative.

Attacks that failed: no pip install -e anywhere (the only -e is pixi's env selector); two-stage copy is path-identical so rpaths and the UW3 .so survive; kernelspec copied; workflow secrets/permissions sane (GITHUB_TOKEN, packages:write, no per-PR trigger); pixi.lock internally consistent (petsc==petsc4py==3.25.3, numpy 2.4.6 < 2.5, python 3.12, identical across envs/platforms); dev-env pixi tasks untouched.

Not run: an actual container build (runtime boot, OSMesa render, QEMU arm64 duration unverified).

Also for the record: #355 is already closed by #356; this PR supersedes #356's mechanism (environment.yaml stops being a build input) but doesn't close anything.

[jcgraciosa]

Thanks for the review. All five issues addressed, plus two additional fixes:

• python-gmsh: Added to [dependencies] in pixi.toml + regenerated lock. Root cause: conda-forge gmsh ships binary only; the Python module is a separate package python-gmsh. No PyPI gmsh wheel exists for linux-aarch64, so import gmsh silently failed. Side effect: the "PyPI overwrites conda gmsh" warning is also gone (pip no longer installs PyPI gmsh as pygmsh's transitive dep).
• .dockerignore: Created, excluding .pixi/, build/, .git/, docs_legacy/, **/__pycache__/.
• CI build: Both linux/amd64 and linux/arm64 built successfully. Job failed only at the push step — fork's GITHUB_TOKEN can't push to ghcr.io/underworldcode. Build logs confirm both platforms assembled cleanly.
• environment.yaml: petsc=3.24 → petsc=3.25.
• Pixi version: Pinned to PIXI_VERSION=0.63.2.
• GHA build cache: Added cache-from/cache-to: type=gha,mode=max to the workflow so repeated arm64 QEMU builds reuse cached layers instead of re-solving from scratch.

Compiler toolchain in the runtime image is acknowledged, deferred to a follow-up PR.

Noted on #355/#356 - this PR improves on #356's approach: environment.yaml is no longer a build input (but still useful as human readable record); pixi.lock drives the container instead, eliminating version drift between the container and the Binder environment.

Underworld development team with AI support from Claude Code

[lmoresi]

Re-verification at head 5392b0b — all five findings CLEARED

#	Finding	Status
1	CRITICAL: no gmsh Python module on linux-aarch64	VERIFIED — gmsh + python-gmsh both in [dependencies]; lock re-solved with python-gmsh-4.15.2 in every env × platform incl. runtime/linux-aarch64; zero pypi gmsh wheels remain in the lock, so the papering-over mechanism is gone rather than masked.
2	MAJOR: no green container build	VERIFIED — fork run 30968087225 at exactly this head: linux/amd64 and linux/arm64 both complete builder 11/11 + runtime 9/9 with manifests exported; the only error is the expected permission_denied pushing upstream ghcr from a fork token.
3	MAJOR: no .dockerignore	VERIFIED — excludes .pixi/, build/, .git/, docs_legacy/, pycache; Containerfile COPY --exclude as belt-and-braces.
4	environment.yaml petsc mismatch	VERIFIED — petsc/petsc4py=3.25 + "not a build input" header; lock ships 3.25.3 across platforms.
5	unpinned pixi install	VERIFIED — PIXI_VERSION=0.63.2.

Extras sane: the GHA build cache (type=gha,mode=max + actions: write) also answers our lock-churn/QEMU-rebuild complaint; workflow triggers correctly re-keyed to pixi.toml/pixi.lock. The merge of development is clean — the diff vs development touches exactly six packaging files, no src/ or tests/. test_uw3 red at head was the known #461 MANA-NIC flake (it fired twice, logged on the issue); the re-run at this head is green.

For the record, non-blocking residuals: pygmsh resolves 7.1.12 on linux-aarch64 vs 7.1.17 elsewhere (harmless — nothing in src/ imports pygmsh); no import gmsh smoke ran inside the built arm64 image (mechanism matches the old working micromamba container); linux-64 default/dev envs grow ~110 packages from conda gmsh's dep tree (inherent to the aarch64-correct fix).

Underworld development team with AI support from Claude Code