Skip to content

✨ Adding Support for CAPI >= 1.11 (v1beta2) - #346

Open
schmidax wants to merge 15 commits into
mainfrom
migration/capi-v1.11
Open

schmidax wants to merge 15 commits into
mainfrom
migration/capi-v1.11

Conversation

@schmidax

Copy link
Copy Markdown

What this PR does / why we need it:
Migrate cluster-stack-operator from CAPI v1.8.1 to CAPI v1.11.0 (Stage 1).
Key changes:

  • Upgrade sigs.k8s.io/cluster-api dependency from v1.8.1 to v1.11.0
  • Update import paths: api/v1beta1 → api/core/v1beta1, exp/runtime/hooks/api/v1alpha1 → api/runtime/hooks/v1alpha1
  • Use deprecated v1beta1 conditions packages (util/deprecated/v1beta1/conditions + util/deprecated/v1beta1/patch) per Stage 1 migration guidance
  • Fix breaking API changes: external.Get (3 args), predicates.ResourceNotPausedAndHasFilterLabel (added *runtime.Scheme), removed external.GetObjectReference
  • Fix ClusterAddon webhook to implement webhook.CustomValidator (old Validator interface removed in controller-runtime)
  • Fix Helm chart templates: use dynamic namespaces ({{ .Release.Namespace }}) and release-name-based secret references
  • Fix cross-namespace ClusterClass lookup: use cluster.Spec.Topology.ClassNamespace (CAPI v1.11+ supports ClusterClasses in different namespaces than the Cluster resource)
    This is a Stage 1 migration only — conditions still use the clusterv1.Conditions type via the deprecated v1beta1 packages. Migration to metav1.Conditions (Stage 2/3) is intentionally deferred.

Special notes for your reviewer:

  • No image version changes — this PR only changes operator code, Helm chart templates, and vendored dependencies
  • Verified against kind cluster with CAPI v1.11 providers: operator reconciles ClusterStack, ClusterStackRelease, and ClusterAddon correctly

@schmidax
schmidax force-pushed the migration/capi-v1.11 branch from baa2fc0 to 331d9f2 Compare August 21, 2026 09:10
@Nils98Ar

Nils98Ar commented Sep 9, 2026

Copy link
Copy Markdown
Member

@schmidax What about the failing checks? Is there an image available somewhere, or do I need to build one myself for testing?

@schmidax

schmidax commented Sep 14, 2026 •

Copy link
Copy Markdown
Author

@Nils98Ar I'll take a look at the failing checks. And I haven't uploaded the image anywhere yet.

- Upgrade CAPI dependency from v1.8.1 to v1.11.0
- Update import paths to v1beta1/v1beta2 locations
- Use deprecated v1beta1 conditions packages (Stage 1)
- Fix API signature changes (external.Get, predicates)
- Fix ClusterAddon webhook (CustomValidator interface)
- Fix Helm chart templates (dynamic namespaces)
- Add MIGRATION.md documentation

Per CAPI v1.11 migration guide:
https://release-1-11.cluster-api.sigs.k8s.io/developer/providers/migrations/v1.10-to-v1.11

Signed-off-by: Axel Schmidt <axel.schmidt@rz.uni-augsburg.de>
- Upgrade CAPI dependency from v1.8.1 to v1.11.0
- Update import paths to v1beta1/v1beta2 locations
- Use deprecated v1beta1 conditions packages (Stage 1)
- Fix API signature changes (external.Get, predicates)
- Fix ClusterAddon webhook (CustomValidator interface)
- Fix Helm chart templates (dynamic namespaces)

Per CAPI v1.11 migration guide:
https://release-1-11.cluster-api.sigs.k8s.io/developer/providers/migrations/v1.10-to-v1.11

Signed-off-by: Axel Schmidt <axel.schmidt@rz.uni-augsburg.de>
- Use cluster.Spec.Topology.ClassNamespace when looking up ClusterStackRelease
- Fallback to cluster.Namespace if classNamespace is not set
- Supports CAPI v1.11+ cross-namespace ClusterClass references

Signed-off-by: Axel Schmidt <axel.schmidt@rz.uni-augsburg.de>
Signed-off-by: Axel Schmidt <axel.schmidt@rz.uni-augsburg.de>
Signed-off-by: Axel Schmidt <axel.schmidt@rz.uni-augsburg.de>
@schmidax
schmidax force-pushed the migration/capi-v1.11 branch from bb549c6 to e19e719 Compare September 17, 2026 07:22
Signed-off-by: Axel Schmidt <axel.schmidt@rz.uni-augsburg.de>
Signed-off-by: Axel Schmidt <axel.schmidt@rz.uni-augsburg.de>
@jschoone

Copy link
Copy Markdown
Contributor

Hi @schmidax and @Nils98Ar, checks are green again.
To get the image you can run the Build cso Image workflow on any branch you like. I just did that, you can use ghcr.io/sovereigncloudstack/cso-staging:sha-ef9fb41
The hash is also the commit id, which is the last commit in this PR.

@jschoone
jschoone requested review from Nils98Ar and garloff October 26, 2026 22:46

@Nils98Ar Nils98Ar left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good and works for us

@jschoone

jschoone commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Hi @garloff and @schmidax,
Nils and I were successful with testing the CSO based on this PR. It supports CAPI up to 1.14 now. With that we don't have to pin it to 1.10 longer for the existing Cluster Stacks.
The support for 1.15 must be added with a separate PR, as it will likely break with the scs2 Cluster Stack. The development has already been started, I'll create a PR against main as soon as this is merged.
Do you have any objections before merging?

@schmidax

Copy link
Copy Markdown
Author

I can also confirm that in my environment the cso based on this PR works. Therefore, I have no objections.

@jschoone
jschoone force-pushed the migration/capi-v1.11 branch from bcd61c7 to e3f8a0d Compare November 13, 2026 07:54
…tion

- external.Get in CAPI >= 1.11 uses the namespace of the reference instead of an explicit
  namespace argument. A ClusterStack with a providerRef without namespace (as documented)
  could therefore not find its template anymore. Always resolve the template in the
  namespace of the ClusterStack, like before, and default the namespace of the Cluster
  references used for the addon builtins. Add a test for a providerRef without namespace.
- cleanUpResources talks to the workload cluster, where resources without a namespace
  belong to the addon namespace (kube-system). The namespace of the ClusterAddon object in
  the management cluster is not meaningful there, so restore kube-system as the default.

Signed-off-by: Jan Schoone <6106846+jschoone@users.noreply.github.com>
Clusters of any namespace can use the ClusterClasses of a namespace since CAPI 1.11. The
in-use checks of the ClusterStack and ClusterStackRelease webhooks and controllers only
looked at the clusters of the own namespace, so a ClusterStackRelease could be deleted while
a cluster in another namespace still used its ClusterClass, and clusters of the own
namespace using a ClusterClass of another namespace were wrongly counted.

Add pkg/clusterclass with a helper listing the clusters that use the ClusterClasses of a
namespace and a test for getUsedClusterClasses.

Signed-off-by: Jan Schoone <6106846+jschoone@users.noreply.github.com>
When a cluster switches to a release of the same Kubernetes version, the objects of the old
release that are not part of the new one are deleted. The objects of the old release were
collected with the stages of the clusteraddon.yaml of the new release, but from the charts
of the old release. An addon that only exists in the old release (removed from the cluster
stack) was therefore never found and its objects stayed in the workload cluster.

Read the stages from the clusteraddon.yaml of the old release.

Signed-off-by: Jan Schoone <6106846+jschoone@users.noreply.github.com>
Cluster API 1.13 and newer send the runtime hook request with a v1beta2 Cluster. The hook
server decodes it as v1beta1, where the class is spec.topology.class; in v1beta2 it is
spec.topology.classRef.name. The class was therefore empty and the hook record key
(name/namespace/clusterclass) was the same for every ClusterClass of a cluster.

After the first upgrade of a cluster, every further upgrade of the same cluster in the same
CSO process then skipped the patch of the ClusterAddon: BeforeClusterUpgrade returned success
without applying the addons of the new release and the ClusterAddon stayed ready. Seen with
CAPI 1.13.7: after 1.35 -> 1.36 the nodes ran v1.36.6 while Cilium, the cloud-controller-manager
and the Cinder CSI plugin were still on the 1.35 versions.

If the class is missing in the request, read the Cluster from the API server (served as
v1beta1 by Cluster API 1.11 to 1.14). This uses the API reader of the manager and not the
cached client: Cluster API calls the hook right after it has seen the new class, and the cache
of the CSO can still have the previous one (a first version that read from the cache did not
fix the skipped hook). Requests of Cluster API 1.11 and 1.12 carry the class and are handled
as before. A missing topology no longer causes a nil pointer dereference.

No change of the Cluster API version of the CSO.

Signed-off-by: Jan Schoone <6106846+jschoone@users.noreply.github.com>
…stack

The integration tests used releases in the old cluster stack format (docker-ferrol 1.27),
with a single Helm chart per release and a version label on the objects. Replace them with
two releases built from the docker/scs cluster stack of cluster-stacks (Kubernetes 1.35),
which use the addon stages:
- the hook applies the gateway-api, cni and metrics-server addons, including a value that
  is templated from the Cluster object,
- a release with a changed addon updates the objects,
- a release without the gateway-api addon removes its objects.
test/releases/README.md describes how the releases are built. The old releases stay for the
unit tests of the controllers.

Run the tests on a current kind node image. The kindest/node v1.26.6 image used so far
crashes kube-proxy and kindnet on current runner kernels (the iptables of its Debian 11 base
no longer works), so the node never becomes ready. Use kind v0.31.0 with its node image
v1.35.0, detect the container runtime instead of forcing docker (KIND_EXPERIMENTAL_PROVIDER
selects another one) and move hack/kind-dev.sh to the same Kubernetes version.

Raise the timeout of the specs that wait for the cluster addon to apply Helm charts to the
kind cluster from 3 to 30 seconds. The specs still return as soon as the condition is met.

Signed-off-by: Jan Schoone <6106846+jschoone@users.noreply.github.com>
… code

The builder image 1.1.35 contains golangci-lint v2.10.1. It reports two findings in
code that exists only on migration/capi-v1.11:

- prealloc: allocate the object list in utils_test.go with its final capacity.
- revive enforce-switch-style: add a default clause to the switch in
  storageVariantFromCAPI. It does nothing, the behaviour is unchanged.

Signed-off-by: Jan Schoone <6106846+jschoone@users.noreply.github.com>
The Eventually timeout of 2 seconds is too short when the machine running envtest is
busy. Different specs of the controller suite timed out, for example
"ClusterAddonReconciler creates the clusterAddon object" and "ClusterStackReconciler
checks ProviderClusterstackrelease is deleted when version is removed from spec".

Raise the timeout to 10 seconds. Eventually returns as soon as the condition holds, so
passing runs are not slower, and no spec uses Consistently.

On a temporary VM with Kubernetes 1.35.0 in envtest, 2 of 10 runs failed with 2 seconds
and 0 of 16 runs failed with 10 seconds.

Signed-off-by: Jan Schoone <6106846+jschoone@users.noreply.github.com>
@jschoone
jschoone force-pushed the migration/capi-v1.11 branch from e3f8a0d to fd2eaa8 Compare November 13, 2026 07:57
Signed-off-by: Jan Schoone <6106846+jschoone@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants