Fix XCom sidecar helper mutating the caller's pod volumes - #72522
Open
henry3260 wants to merge 1 commit into
Open
Fix XCom sidecar helper mutating the caller's pod volumes#72522henry3260 wants to merge 1 commit into
henry3260 wants to merge 1 commit into
Conversation
henry3260
requested review from
hussein-awala,
jedcunningham and
jscheffl
as code owners
September 4, 2026 15:52
boring-cyborg
Bot
added
area:providers
provider:cncf-kubernetes
add_xcom_sidecar deep-copies the input pod so callers keep an untouched object, but it then pointed the copy's volumes back at the original pod's list before inserting the xcom volume. Any caller reusing the same pod object accumulated one extra "xcom" volume per call, which the Kubernetes API rejects as a duplicate. KubernetesPodOperator was only shielded because reconcile_pods happens to deep-copy first. The default V1Volume and V1VolumeMount were also inserted as the shared module-level instances, so editing them on one pod silently changed every other pod built by the helper and the defaults themselves.
henry3260
force-pushed
the
fix-xcom-sidecar-input-pod-mutation
branch
from
September 4, 2026 15:57
95135a8 to
4f432aa
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
add_xcom_sidecardeep-copies the input pod so the caller's object stays untouched, but it then re-pointed the copy'sspec.volumesat the original pod's volume list before inserting thexcomvolume. Any caller that reuses the same pod object (custom operators or subclasses callingbuild_pod_request_objmore than once, or direct users of the helper) accumulated one extraxcomvolume per call. Kubernetes requires volume names to be unique, so the API rejects the second pod as a duplicate.KubernetesPodOperatoritself was only shielded becausereconcile_podshappens to deep-copy first.The helper also inserted the module-level
PodDefaults.VOLUMEandPodDefaults.VOLUME_MOUNTinstances directly, so mutating them on one pod silently changed every other pod built by the helper, as well as the defaults themselves.What
providers/cncf/kubernetes/src/airflow/providers/cncf/kubernetes/utils/xcom_sidecar.py: assignpod_cp.spec.volumesfrom the copy instead of the original pod, and insert deep copies ofPodDefaults.VOLUMEandPodDefaults.VOLUME_MOUNT.providers/cncf/kubernetes/tests/unit/cncf/kubernetes/utils/test_xcom_sidecar.py: add two regression tests. One asserts the caller's pod keeps its original volume list while the returned copy gains thexcomvolume. The other asserts the inserted volume and mount are not the shared default instances. Both fail onmainand pass with this change.Was generative AI tooling used to co-author this PR?