Skip to content

blocked-edges: Unblock 4.22 z-stream for MissingOpenVSwitch - #10903

Merged
openshift-merge-bot[bot] merged 1 commit into
openshift:masterfrom
hongkailiu:MCO-2603-z-is-fine
Oct 2, 2026
Merged

openshift-merge-bot[bot] merged 1 commit into
openshift:masterfrom
hongkailiu:MCO-2603-z-is-fine

Conversation

@hongkailiu

Copy link
Copy Markdown
Member

Testing has shown that the problematic logic will not be triggered on a z-stream upgrade within 4.22.

Drop the 4.22.0-4.22.14 alternation from the from regex, leaving only 4.21 sources flagged.

/cc @sdodson

Testing has shown that the problematic logic will not be triggered on a
z-stream upgrade within 4.22.

Drop the 4.22.0-4.22.14 alternation from the `from` regex, leaving only
4.21 sources flagged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@openshift-ci
openshift-ci Bot requested a review from sdodson October 2, 2026 17:19
@coderabbitai

coderabbitai Bot commented Oct 2, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 88c34739-aea9-4ff4-bf5a-fa29445d478b

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Comment @coderabbitai help to get the list of available commands.

@openshift-ci openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Oct 2, 2026
@sdodson

sdodson commented Oct 2, 2026

Copy link
Copy Markdown
Member

/lgtm

@openshift-ci openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label Oct 2, 2026
@openshift-ci

openshift-ci Bot commented Oct 2, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: hongkailiu, sdodson

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci

openshift-ci Bot commented Oct 2, 2026

Copy link
Copy Markdown
Contributor

@hongkailiu: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@openshift-merge-bot
openshift-merge-bot Bot merged commit 7b67aff into openshift:master Oct 2, 2026
4 checks passed
@hongkailiu

Copy link
Copy Markdown
Member Author

@wking

wking commented Oct 2, 2026 •

Copy link
Copy Markdown
Member

Filling in some details on the testing, our initial expectation that 4.22.older were exposed was based on an incorrect understanding of their skopeo version vs. this check. But it turns out that 4.22.0's skopeo is new enough (>= 1.22.2) to not trip that conditional, with the release controller diff between 4.22.0 and 4.22.15 showing skopeo 2:1.22.2-1.el9_8 → 2:1.22.2-7.el9_8 and rpm-ostree-2026.1-6.el9_8. Those are both in the RHCOS image, and not out of the boot image. But for updates out of a level 4.22.0 cluster, it's that RHCOS image (and not the boot image) that Nodes are actually running.

Testing to confirm this read of the code, with ClusterBot launch 4.22.14 azure,multi source (logs):

$ oc get -o json clusterversion version | jq -c '.status.desired | {version, architecture}'
{"version":"4.22.14","architecture":"Multi"}
$ oc debug -q "$(oc get -o name nodes | head -n1)" -- chroot /host sh -c 'skopeo --version && rpm-ostree --version'
skopeo version 1.22.2 commit: cd48f3da7baca8d4e7bbf6af27f7810754ceb504
rpm-ostree:
 Version: '2026.1'
 Git: d956146df9cddea2e51ae6498764eb5ce5eabf06
 Features:
  - rust
  - compose
  - container
  - rhsm
$ oc patch clusterversion version --type json -p '[{"op": "remove", "path": "/spec/overrides"}]'
$ oc adm upgrade --to 4.22.15
Requested update to 4.22.15
$ oc adm upgrade | head -n1
info: An upgrade is in progress. Working towards 4.22.15: 134 of 1023 done (13% complete), waiting on etcd, kube-apiserver

And then an ~hour later, a successful update completion:

$ oc adm upgrade
Cluster version is 4.22.15
...

Similarly with ClusterBot launch 4.22.0 azure,multi (logs):

$ oc get -o json clusterversion version | jq -c '.status.desired | {version, architecture}'
{"version":"4.22.0","architecture":"Multi"}
$ oc debug -q "$(oc get -o name nodes | head -n1)" -- chroot /host sh -c 'skopeo --version && rpm-ostree --version'
skopeo version 1.22.2 commit: 4db1c56e8003ad07759a4d899254643d445dbe04
rpm-ostree:
 Version: '2026.1'
 Git: d956146df9cddea2e51ae6498764eb5ce5eabf06
 Features:
  - rust
  - compose
  - container
  - rhsm
$ oc patch clusterversion version --type json -p '[{"op": "remove", "path": "/spec/overrides"}]'
clusterversion.config.openshift.io/version patched
$ oc adm upgrade --to 4.22.15
Requested update to 4.22.15
$ oc adm upgrade | head -n1
info: An upgrade is in progress. Working towards 4.22.15: 28 of 1023 done (2% complete)

And then an ~hour later, another successful update completion:

$ oc adm upgrade
Cluster version is 4.22.15
...

And then, because 4.22.15 predates openshift/installer#10884, we still have an old 4.20 boot image to use to confirm post-update scale-up works:

$ oc -n openshift-machine-api get -o json machinesets.machine.openshift.io | jq -c '.items[].spec.template.spec.providerSpec.value.image' | sort | uniq -c
      3 {"offer":"aro4","publisher":"azureopenshift","resourceID":"","sku":"420-v2","type":"MarketplaceNoPlan","version":"9.6.20251015"}
$ oc -n openshift-machine-api get machinesets.machine.openshift.io
NAME                                          DESIRED   CURRENT   READY   AVAILABLE   AGE
ci-ln-3y1j0tt-1d09d-j52n8-worker-centralus1   1         1         1       1           142m
ci-ln-3y1j0tt-1d09d-j52n8-worker-centralus2   1         1         1       1           142m
ci-ln-3y1j0tt-1d09d-j52n8-worker-centralus3   1         1         1       1           142m
$ oc -n openshift-machine-api scale --replicas 2 machinesets.machine.openshift.io ci-ln-3y1j0tt-1d09d-j52n8-worker-centralus1

And a few minutes later, there's a healthy new Node:

$ oc -n openshift-machine-api get -o wide machines.machine.openshift.io | grep centralus1
ci-ln-3y1j0tt-1d09d-j52n8-worker-centralus1-x7h6l   Running   Standard_D4as_v5   centralus   1      4m5s   ci-ln-3y1j0tt-1d09d-j52n8-worker-centralus1-x7h6l   azure:///subscriptions/...   Running
ci-ln-3y1j0tt-1d09d-j52n8-worker-centralus1-zqcjx   Running   Standard_D4as_v5   centralus   1      142m   ci-ln-3y1j0tt-1d09d-j52n8-worker-centralus1-zqcjx   azure:///subscriptions/...   Running
$ oc get nodes | grep centralus1
ci-ln-3y1j0tt-1d09d-j52n8-worker-centralus1-x7h6l   Ready    worker                 5m31s   v1.35.6
ci-ln-3y1j0tt-1d09d-j52n8-worker-centralus1-zqcjx   Ready    worker                 141m    v1.35.6

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants