Task SDK: Fix executor_config aliasing across tasks sharing default_args - #72556
Open
bujjibabukatta wants to merge 1 commit into
Open
Task SDK: Fix executor_config aliasing across tasks sharing default_args#72556bujjibabukatta wants to merge 1 commit into
bujjibabukatta wants to merge 1 commit into
Conversation
bujjibabukatta
requested review from
XD-DENG,
amoghrajesh,
ashb and
kaxil
as code owners
September 5, 2026 09:18
bujjibabukatta
force-pushed
the
fix/#72544
branch
from
September 5, 2026 16:04
f69e9cb to
8d70eba
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
executor_configset in a shareddefault_argsdict was being shared by reference across tasks and DAGs, not copied. So if one task mutated itsexecutor_configat runtime (a normal thing to do with the Kubernetes executor), it could silently change the config for other tasks using the samedefault_args.Closes #72544.
Root Cause
Every place
executor_configgets passed around does a shallow copy of the outer dict, but nobody ever copied theexecutor_configdict itself:BaseOperator.__init__just didexecutor_config or {}— no copy.TaskInstancethen copied it straight from the task object again, so even after a fix in one place, it still ends up as the same shared dict at runtime.Callback fields like
on_failure_callbackalready avoid this by rebuilding a fresh list each time.executor_confignever got the same treatment.Fix
In
BaseOperator.__init__,executor_configis now wrapped incopy.copy()before being assigned, so it no longer just falls back toexecutor_config or {}directly. The same change was made inTaskInstance, wheretask.executor_configis now copied instead of assigned as-is. Together, these ensure every task and task instance gets its own independentexecutor_configdict instead of sharing one.This is a shallow copy, consistent with how callback fields are already handled. If
executor_configever holds nested mutable objects (like aV1Pod), those inner parts would still be shared by reference — open to switching to a deep copy if reviewers think that's needed.Was generative AI tooling used ?
Generated-by: Claude following the guidelines