Skip to content

[PRODENG-3593] Add Ubuntu 26.04 (Resolute) host configurer - #661

Merged
james-nesbitt merged 3 commits into
mainfrom
PRODENG-3593
Sep 29, 2026
Merged

james-nesbitt merged 3 commits into
mainfrom
PRODENG-3593

Conversation

@james-nesbitt

@james-nesbitt james-nesbitt commented Sep 2, 2026 •

Copy link
Copy Markdown
Collaborator

What

Register a host configurer for Ubuntu 26.04 (Resolute Raccoon) so launchpad recognizes the OS.

Why

PRODENG-3593 asks for Ubuntu 26.04 support. The configurer piece is cheap groundwork; it does not by itself make installs succeed, see caveat below.

How

  • Add pkg/configurer/ubuntu/resolute.go: ResoluteConfigurer registered for os.ID == "ubuntu" && os.Version == "26.04", same pattern as bionic/focal/jammy/noble/xenial
  • No version-specific logic needed - ubuntu.Configurer.InstallMCR already derives the APT repo URL generically from VERSION_CODENAME
  • Add the compile-time HostConfigurer assertion in pkg/product/mke/config/configurer_test.go

Caveat

MCR has not published Docker EE packages for Ubuntu 26.04 on repos.mirantis.com or repos-internal.mirantis.com as of 2026-09-02 (both list only trusty/xenial/bionic/focal/jammy/noble). InstallMCR will fail on this platform until MCR ships support - same gap already tracked by TestCuttingEdgeCluster in PR #642. This PR only unblocks OS recognition.

Testing

  • go build ./..., go vet, go test ./pkg/product/mke/config/... ./pkg/configurer/...
  • golangci-lint run: no new issues

Links

Checklist

  • Tests added or updated
  • Docs updated if user-visible behaviour changed
  • No debug output or dead code left in

Written by AI: claude-sonnet-5

@james-nesbitt
james-nesbitt marked this pull request as draft September 21, 2026 07:26
@james-nesbitt
james-nesbitt marked this pull request as ready for review September 29, 2026 16:58
Registers ResoluteConfigurer for os.ID == "ubuntu" && os.Version ==
"26.04", matching the existing per-codename pattern (bionic, focal,
jammy, noble, xenial). No version-specific behavior needed: the shared
ubuntu.Configurer.InstallMCR already builds the APT repo URL generically
from VERSION_CODENAME.

MCR has not published Docker EE packages for Ubuntu 26.04 on either
repos.mirantis.com or repos-internal.mirantis.com as of 2026-09-02 (both
list only trusty/xenial/bionic/focal/jammy/noble). InstallMCR will fail
on this platform until MCR ships support - this configurer only makes
launchpad recognize the OS, it does not unblock installs. Same gap
already tracked by TestCuttingEdgeCluster in PR #642.

Verified: go build ./..., go vet, go test ./pkg/product/mke/config/...
./pkg/configurer/..., golangci-lint run (no new issues).
PRODENG-3594 (#651) migrated OS module registration from
k0sproject/rig + rig/os/registry (func(rig.OSVersion) bool) to
pkg/configurer.RegisterOSModule + rigos.Release (func(*rigos.Release)
bool), landed on main after this branch was created. Update
ResoluteConfigurer to match the current pattern used by
NobleConfigurer; without this the branch fails to compile against
main.

Written by AI: claude-sonnet-5

Signed-off-by: James Nesbitt <jnesbitt@mirantis.com>
- test/platforms.go: add Ubuntu26 platform entry (ubuntu_26.04)
- test/smoke/smoke_test.go: add MCRRepoURL to smokeConfig, wired into
  the launchpad tfvars map as mcr_repo_url when non-empty (defaults
  to the public repo otherwise, via PR #669's optional tfvar); add
  TestCuttingEdgeCluster exercising a single ubuntu_26.04 manager
  against repos-internal.mirantis.com channel test-29, the only
  place Ubuntu 26.04 MCR packages exist as of 2026-09-29

Verified: TestCuttingEdgeCluster passed against real AWS infra
(docker-testing-533267045383, us-east-1), MCR engine 29.8.0m1-rc1
installed and detected by MKE 3.9.2, cluster reported healthy,
Reset succeeded, all 28 AWS resources destroyed cleanly.

Written by AI: claude-sonnet-5

Signed-off-by: James Nesbitt <jnesbitt@mirantis.com>
@james-nesbitt
james-nesbitt merged commit 7b8a4ba into main Sep 29, 2026
15 checks passed
@james-nesbitt
james-nesbitt deleted the PRODENG-3593 branch September 29, 2026 17:32
james-nesbitt added a commit that referenced this pull request Nov 3, 2026
…gle-node test

MCR package availability on repos.mirantis.com, checked 2026-11-03:

  rhel/10 (also serves rocky10)  stable-29.2 ... stable-29.8.0
  ubuntu resolute                stable-29.8.0 and test-29.8.0 only
  sles/16                        no directory; sles/ has 12, 12.3, 15

Changes following from that:

  - WrkSles16 removed from TestCuttingEdgeCluster and the Sles16 entry
    removed from test/platforms.go, its only consumer. A sles16 worker
    cannot install MCR at all, so it can only ever fail the run.
  - MCRChannel stable-29.6.1 -> stable-29.8.0. 29.6.1 does not exist for
    resolute, so the previous value could not install on ubuntu26.
  - MKEVersion 3.9.6 -> 3.9.7, the pairing with MCR 29.8.0 exercised live
    in PRODENG-4062.

main's TestCuttingEdgeCluster (added by #661) is removed and replaced by
this matrix. That test provisioned one ubuntu_26.04 manager against
repos-internal.mirantis.com test-29 because resolute packages existed
nowhere else on 2026-09-29. repos.mirantis.com now carries them, so the
public-repo matrix covers ubuntu26 in both manager and worker roles. The
cost is that no smoke test exercises the repos-internal path any more;
MCRRepoURL in smokeConfig is retained for the next test that needs it.

The provision-aws module constraint is the exact pin 0.1.8 from main
rather than this branch's >= 0.1.7: 0.2.0-0.2.2 fail at plan time.

Not run: the smoke job itself. The RHEL10/Rocky10 MCR-daemon-start gap
(missing xt_* kernel modules, nftables vs swarm) is unresolved and may
still fail this matrix.

Written by AI: claude-opus-5

Signed-off-by: James Nesbitt <jnesbitt@mirantis.com>
james-nesbitt added a commit that referenced this pull request Nov 11, 2026
* smoke: add TestCuttingEdgeCluster (rhel10/rocky10/ubuntu24/sles15)

Adds a new smoke test targeting the latest gen-10 distributions:
- RHEL 10, Rocky 10, Ubuntu 24.04 as managers and workers
- SLES 15 as an additional worker

Also adds:
- Rhel10 and Rocky10 platform definitions in test/platforms.go
- smoke-cutting-edge Makefile target (50m timeout)
- smoke-cutting-edge CI job triggered by the smoke-cutting-edge label

NOTE: This test depends on terraform-mirantis-modules/terraform-mirantis-provision-aws
PR #22 being merged (adds rhel_10 and rocky_10 platform keys to the
provision-aws module). The test will fail until the module is updated and
the version bumped in examples/terraform/aws-simple/.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: James Nesbitt <jnesbitt@mirantis.com>

* smoke: unblock TestCuttingEdgeCluster on released provision-aws v0.1.7

- shorten Name "cutting-edge" -> "cuttingedge" (11 chars): the 12-char
  value overflowed the 32-char AWS LB/target-group name limit and was
  the actual CI failure ("name" cannot be longer than 32 characters)
- pin examples/terraform/aws-simple to provision-aws >= 0.1.7, which
  ships the rhel_10/rocky_10 platform keys (provision-aws#22, released)
- refresh test doc comment now the upstream dependency is merged/released

gofmt also normalized two pre-existing violations in smoke_test.go.

* smoke: document cutting-edge test as merged ahead of MCR support

rhel10/rocky10 AMIs and provision-aws platform keys (v0.1.7) are already
available, but launchpad's own smoke coverage previously failed with
'package manager could not install docker-ee' because repos.mirantis.com
had not yet published docker-ee for rhel/10 or rocky/10.

Checked repos.mirantis.com directly: docker-ee is now present in both
rhel/10/x86_64/stable-29.4 and rocky/10/x86_64/stable-29.4 -- that gap
looks closed, matching the AMI-availability timing pattern seen
elsewhere this cycle (Oracle Linux, Rocky 8).

Strategy going forward: keep TestCuttingEdgeCluster merged and
label-gated rather than blocking on it going green immediately, so it
picks up MCR support automatically as soon as it lands, with no further
code changes. Documented that intent on the Makefile target and the
GitHub Actions job.

Written by AI: claude-sonnet-5

* smoke(cutting-edge): switch worker from sles15 to sles16

Adds a Sles16 platform definition (sles_16) and switches
TestCuttingEdgeCluster's additional worker from sles15 to sles16.

NOTE: MCR has not published Docker EE packages for SLES 16 as of
2026-07-22 (repos.mirantis.com/sles/ only has 12/12.3/15). This node
is expected to fail at Install MCR until MCR ships SLES16 support -
same category of gap as RHEL10/Rocky10 (docker-ee installs but the
daemon won't start), tracked separately for that pair. Flagged in a
doc comment on the test so CI failures here are understood, not
mysterious.

gofmt also cleaned two pre-existing, unrelated violations in these
files (trailing blank line in platforms.go; WrkWin2025 alignment in
TestFIPSCluster).

* smoke(cutting-edge): switch to ubuntu 26.04 (resolute)

Adds a Ubuntu26 platform definition (ubuntu_26.04, codename resolute)
and switches TestCuttingEdgeCluster's Ubuntu manager+worker from 24.04
to 26.04 - the current LTS as of 2026-07 (24.04 remains fully
supported, just no longer the newest release).

The ubuntu_26.04 key is already present in the pinned provision-aws
module (>= v0.1.7), AMI pattern confirmed against a live published AMI
(ubuntu-resolute-26.04-amd64-server-*, published as recently as
2026-07-22).

NOTE: MCR has not published Docker EE packages for Ubuntu 26.04 yet
(repos.mirantis.com/ubuntu/dists/ only has trusty/xenial/bionic/focal/
jammy/noble - no resolute). This node is expected to fail at Install
MCR alongside the existing SLES16 and RHEL10/Rocky10 gaps, until MCR
ships support. Flagged in the test's doc comment.

* smoke(cutting-edge): bump to latest MCR (stable-29.6.1) and MKE (3.9.6)

Re-checking whether the RHEL10/Rocky10 MCR-daemon-start gap and the
Ubuntu 26.04/SLES 16 missing-package gaps (documented in this PR's
description) have closed with the newest available releases. Was
stable-29.4/MKE 3.9.2 (docker-ee-29.4.1m1); repos.mirantis.com now also
publishes stable-29.6.1 (docker-ee-29.6.1m2) for rhel/10.
MSRVersion unchanged (3.1.18), already current elsewhere.

Signed-off-by: James Nesbitt <jnesbitt@mirantis.com>

* fix: surface docker service start failure reason via journalctl

EnableMCR's error on a failed "systemctl start docker" was always a
bare "exit status 1" with no way to tell why: rig's StartService
issues "systemctl start docker 2> /dev/null", discarding systemd's own
diagnostic output.

Reproduced live on RHEL10/Rocky10 in TestCuttingEdgeCluster (PR #642)
with the latest available MCR (stable-29.6.1) and MKE (3.9.6): package
install succeeds, but docker.service still fails to start, with no
way from launchpad's own logs to see why -- exactly the gap this PR's
description flags as "root cause not yet isolated (stderr suppressed
by the install command)".

Best-effort fetch journalctl output for the docker unit on failure and
include it in the returned error, so the actual systemd failure reason
is visible without needing rig changes or manual SSH access. Ignore a
failure to fetch the diagnostic itself, since that must not mask the
original error.

Written by AI: claude-sonnet-5

Signed-off-by: James Nesbitt <jnesbitt@mirantis.com>

* smoke(cutting-edge): drop sles16, bump to MCR 29.8.0, replace the single-node test

MCR package availability on repos.mirantis.com, checked 2026-11-03:

  rhel/10 (also serves rocky10)  stable-29.2 ... stable-29.8.0
  ubuntu resolute                stable-29.8.0 and test-29.8.0 only
  sles/16                        no directory; sles/ has 12, 12.3, 15

Changes following from that:

  - WrkSles16 removed from TestCuttingEdgeCluster and the Sles16 entry
    removed from test/platforms.go, its only consumer. A sles16 worker
    cannot install MCR at all, so it can only ever fail the run.
  - MCRChannel stable-29.6.1 -> stable-29.8.0. 29.6.1 does not exist for
    resolute, so the previous value could not install on ubuntu26.
  - MKEVersion 3.9.6 -> 3.9.7, the pairing with MCR 29.8.0 exercised live
    in PRODENG-4062.

main's TestCuttingEdgeCluster (added by #661) is removed and replaced by
this matrix. That test provisioned one ubuntu_26.04 manager against
repos-internal.mirantis.com test-29 because resolute packages existed
nowhere else on 2026-09-29. repos.mirantis.com now carries them, so the
public-repo matrix covers ubuntu26 in both manager and worker roles. The
cost is that no smoke test exercises the repos-internal path any more;
MCRRepoURL in smokeConfig is retained for the next test that needs it.

The provision-aws module constraint is the exact pin 0.1.8 from main
rather than this branch's >= 0.1.7: 0.2.0-0.2.2 fail at plan time.

Not run: the smoke job itself. The RHEL10/Rocky10 MCR-daemon-start gap
(missing xt_* kernel modules, nftables vs swarm) is unresolved and may
still fail this matrix.

Written by AI: claude-opus-5

Signed-off-by: James Nesbitt <jnesbitt@mirantis.com>

* Install kernel-modules-extra on the rhel10/rocky10 smoke nodes

TestCuttingEdgeCluster failed on all four EL10 nodes at Install MCR:
dockerd refused to start because its bridge driver could not append
"-m addrtype --dst-type LOCAL" to nat-PREROUTING. xt_addrtype lives in
kernel-modules-extra, which EL10 minimal/cloud images do not carry;
iptables 1.8.11-15.el10 stopped recommending it unconditionally and
docker-ee does not Require it (docker/docker-ce-packaging#1340).

Measured on EC2 in us-east-1, against the AMIs provision-aws v0.3.0
resolves for rhel_10 and rocky_10:

  control, no fix   start rc=1, is-active "activating",
                    "Cannot connect to the Docker daemon"
  rhel_10 + fix     active, docker info -> 29.8.0m1 overlayfs
  rocky_10 + fix    active, docker info -> 29.8.0m1 overlayfs

The control is the more interesting half: installing docker-ee there
pulled kernel-core, kernel-modules and kernel-modules-extra for
6.12.0-211.74.1 onto a box running 6.12.0-211.16.1, so the module was
still not loadable. That is why the install pins
kernel-modules-extra-$(uname -r) instead of leaving it to the resolver.

The nftables firewall backend is not an alternative: Docker 29 refuses
to enable it in Swarm mode, which MKE requires.

The payload needs a shebang. provision-aws passes user_data to the
launch template verbatim -- base64encode, no wrapping -- so cloud-init
runs only what it recognises. Measured on the control instance, which
carries the same bare "sudo firewall-cmd ..." string every other Linux
platform in this file uses: cloud-init logged "Unhandled non-multipart"
and /var/lib/cloud/instance/scripts/ was empty. Those firewall rules
have never executed, on any platform, in any smoke run.

No firewall rules in the new payload either: firewalld is not installed
on either EL10 AMI, firewall-cmd is not on PATH, and node-to-node
traffic is already carried by the security group's self rule.

On the race between cloud-init's dnf and launchpad's own: a second dnf
waits ("Waiting for process with pid ... to finish") rather than
failing, and sshd becomes reachable 27s (rhel) to 59s (rocky) before
cloud-init finishes, against roughly five minutes between instance
creation and Install MCR in CI.

Also aligns the smoke-cutting-edge job with the six other suites in
smoke-tests.yaml: arc-runner-set-mirantis-public, SHA-pinned checkout
and setup-terraform, persist-credentials off, terraform_wrapper off,
and the Set up Go step it lacked. Its Makefile and workflow comments
said the job was merged red pending MCR packages for rhel/10 and
rocky/10; those packages exist and install cleanly, so the comments are
corrected rather than kept.

Written by AI: claude-opus-5

Signed-off-by: James Nesbitt <jnesbitt@mirantis.com>

* Pass the EL10 user_data as a single HCL-safe line

The previous commit's user_data was a multi-line Go raw string. terratest
renders Options.Vars into `-var 'nodegroups={...}'` without escaping, so
terraform received a quoted HCL string split over several lines and
TestCuttingEdgeCluster died in 17s with nineteen parse errors --
"Invalid multi-line string" and "Invalid character" against
<value for var.nodegroups> -- before provisioning anything.

Written on one line with literal \n escapes instead. HCL unescapes them,
so the instance still gets a normal multi-line script. Verified by
rendering Platforms["Rhel10"].GetWorker() through TerraformOptions into
an offline fixture and reading the value back out of a terraform output:
shebang on its own line, one command per line.

The double quotes around kernel-modules-extra-$(uname -r) are gone for
the same reason: they would terminate the HCL string. The argument is a
single shell word, so the quoting was never load-bearing.

Written by AI: claude-opus-5

Signed-off-by: James Nesbitt <jnesbitt@mirantis.com>

---------

Signed-off-by: James Nesbitt <jnesbitt@mirantis.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant