Repository navigation
[RELEASE ONLY CHANGES] Prepare ExecuTorch 1.6 release dependencies - #24199
Merged
JacobSzwejbka merged 13 commits intoOct 30, 2026
Merged
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/24199
Note: Links to docs will display an error until the docs builds have been completed. ❌ 62 New Failures, 112 Cancelled Jobs, 3 Unrelated Failures, 7 Unclassified FailuresAs of commit 6543438 with merge base 4db778e ( NEW FAILURES - The following jobs have failed:
UNCLASSIFIED FAILURES - DrCI could not classify the following jobs because the workflow did not run on the merge base. The failures may be pre-existing on trunk or introduced by this PR:
CANCELLED JOBS - The following jobs were cancelled. Please retry:
FLAKY - The following jobs failed but were likely due to flakiness present on trunk:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
github-actions
Bot
added
the
ciflow/docker
JacobSzwejbka
had a problem deploying
to
cadence
October 27, 2026 16:09 — with 
GitHub Actions
Failure
GitHub Actions
Failure
JacobSzwejbka
had a problem deploying
to
cadence
October 27, 2026 16:09 — with
GitHub Actions
Failure
GitHub Actions
Failure
JacobSzwejbka
had a problem deploying
to
cadence
October 27, 2026 16:09 — with
GitHub Actions
Failure
GitHub Actions
Failure
JacobSzwejbka
had a problem deploying
to
cadence
October 27, 2026 16:09 — with
GitHub Actions
Failure
GitHub Actions
Failure
meta-cla
Bot
added
the
CLA Signed
JacobSzwejbka
had a problem deploying
to
upload-benchmark-results
October 27, 2026 18:30 — with
GitHub Actions
Failure
GitHub Actions
Failure
JacobSzwejbka
marked this pull request as ready for review
October 28, 2026 20:46
shoumikhin
approved these changes
Oct 28, 2026
JacobSzwejbka
added a commit
that referenced
this pull request
Oct 29, 2026
Cherry-pick of #24238 onto `release/1.6`, preserving the original author and recording the merged source commit `0db7a88ff808ecd39fc61e794952cbe033bb4619`. This brings all five Windows CUDA fixes to the release: the distutils logging call, CUDA 13.2 selection for the installed runner toolkits, copying the required ExecuTorch DLLs beside the example runners, driver installation and reliable model-test failure checks, and the test-infra pin that supports final Python 3.15 wheel builds. Linux model exports retain CUDA 13.0. Validation: the cherry-pick applied without conflicts and its stable patch ID matches the merged source commit exactly. Both changed workflows parse, `setup.py` parses, and `git diff --check` passes. A merge-tree check with the open release dependency PR #24199 is clean. Windows GPU tests and wheel builds require CI. Cherry-pick prepared with Codex assistance; original implementation by Anthony Shoumikhin. Co-authored-by: Anthony Shoumikhin <anthony@shoumikh.in>
JacobSzwejbka
had a problem deploying
to
upload-benchmark-results
October 29, 2026 03:14 — with
GitHub Actions
Failure
GitHub Actions
Failure
Use the conda-forge GCC compiler and matching runtime libraries for ROCm builds so the Red Hat nonshared libstdc++ objects are not mixed with conda shared libraries. Assisted by Codex.
Require glibc 2.28 development headers so conda cannot select an older sysroot missing timespec_get while using the new C++ compiler. Assisted by Codex.
JacobSzwejbka
removed
the
ciflow/docker
Select MSVC through CUDAHOSTCXX after PyTorch overrides the CMake variable during CUDA initialization. Authored with Codex assistance.
github-actions
Bot
added
the
ciflow/docker
PyTorch 2.15 wrap_triton export requires the cumulative-sum USE_GRAPH argument explicitly instead of relying on the eager Triton default. Authored with Codex assistance.
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Release-only preparation for ExecuTorch 1.6.0 and PyTorch 2.15, following #21563 and #22724. Relates to #23608.
Review the installer and
torch_pin.pyfirst: they select PyTorch 2.15.0, torchvision 0.29.2, torchaudio 2.11.1, and production PyTorch indexes. The full wheel requirestorch==2.15.0,torchao>=0.18.0, and unpinnedpytorch-tokenizers. The minimal wheel excludes torch and TorchAO.pyproject.tomldocuments these dynamically generated requirements. Wheninstall_executorch.py --use-pt-pinned-commitbuilds a CI wheel, its metadata instead pins the already installed source build so pip cannot replace the torch used to link its extensions. Ordinary release wheel metadata retains the exact stable torch pin.The PyTorch source pin is the final
v2.15.0tag,31a77fc9c0a7c6fff15a1ff58c743f53b1a9d457, including the Inductor prefetch revert. The other dependency edits align QNN, TorchCodec 0.17.0, and the Windows smoke check with the release packages.CI follow-up: the branch now includes the current
release/1.6cherry-picks, including the device placement tests that the release workflow already invokes. CUDA CI uses the published 13.2 and 13.4 channels and their verified available container images. Every CUDA install step constrains torch and its domain libraries to the selected local wheel variant; requesting an unpublished 12.6/13.0 variant fails before pip and directs source-build users to--use-pt-pinned-commit. The existing separate nightly TorchAO indexes and aarch64 CPU fallback are retained with the shared minimum requirement.ROCm CI uses the published torch 2.15.0+rocm7.14 wheel and the available ROCm 7.14 builder image. That channel has only Python-only TorchAO wheels, so both ROCm jobs build the existing pinned TorchAO source with
USE_CPP=1 USE_CPU_KERNELS=1against the installed torch/HIP libraries. The CPU kernel option supplies the extension sources on ROCm; without it the pinned setup produces a Python-only wheel. The installer addsrocm[devel]at the already installed ROCm SDK version and usespython -m rocm_sdk path --rootfor the build and runtime paths, including the HIP linker library. The ROCm jobs use conda-forge GCC 16.2 with matching libstdc++/libgcc and a glibc 2.28 sysroot matching the builder image, avoiding the observed Red Hat GCC-toolset nonshared-library link failure when mixed with conda runtime libraries. The shared installer explicitly loads the resulting extension and fails on a missing or unloadable library instead of accepting TorchAO's logged-and-suppressed import errors.FLA is updated from 0.4.2 to 0.6.0 in CUDA CI and the Qwen 3.5 requirements. The wrapper is updated for the new kernel arguments, flattened launch grids and base-2 gate scaling. Numerical test tolerances are unchanged; an additional GPU regression covers multiple batches, value tiles and partial chunks. The cumulative-sum launch now passes
USE_GRAPH=Falseexplicitly: PyTorch 2.15 export does not supply the Triton default, causing the observed missing-argument failures in the export tests and Qwen export. Windows CUDA wheel builds select MSVC throughCUDAHOSTCXX, which CMake reads after PyTorch overridesCMAKE_CUDA_HOST_COMPILERwith clang-cl. This avoids the observed CUDA/MSVC math-header collision without changing the C++ compiler. The earlier CMake argument was ineffective because PyTorch overwrote it. The Linux Windows-export image and workflow now also use CUDA 13.2, with the verified available 13.2.2 Windows cross-compilation toolkit. Vulkan GPU jobs explicitly disable the CUDA delegate, and the installer honors that CMake option by choosing CPU torch even when nvcc is present. The Vulkan tests still execute against the real NVIDIA Vulkan driver. Upload and aggregate jobs retain their existing build dependencies.Before this can merge
Platform CI must validate installation and runtime compatibility with the final production packages. The local checks below do not execute CUDA or ROCm kernels or the Windows compiler.
pytorch-tokenizers 1.6.0 is published on PyPI: https://pypi.org/project/pytorch-tokenizers/1.6.0/. All 20 published wheel hashes match the release manifest from
v1.6.0-rc2, commitf54baeab00d5b664bfc4a977b884ff679fd3f15a; a downloaded TestPyPI wheel passed all 17 Python tests. The existing tokenizer gitlink is retained; final source-tag verification and tokenizer source finalization still need a release-only follow-up.Validation
All 16 installer dependency tests, 6 CUDA workflow tests, and the ROCm dependency sequencing/failure-propagation test pass locally. Checked all 222 literal FLA 0.6.0 autotune keys against their kernel argument names. Checked the wrapper's six launch signatures, grid dimensions and gate scaling against the published FLA source for ordinary, batched, and partial-chunk inputs. Changed Python files pass Black; shell scripts and workflow shell blocks pass syntax checks; YAML parses and
git diff --checkpasses. Production torch CUDA 13.2/13.4 and ROCm 7.14 wheel availability and the corresponding CI container manifests are verified. The CUDA 13.2 Ubuntu development packages and Windows 13.2.2 installer are verified available. The pinned TorchAO source selection is checked with and without the CPU kernel option. The CUDA 13.2 and 13.4 build/runtime probe jobs passed on commit8846440476. TorchAO compiled and loaded on ROCm after the SDK path fix at commitd7c54f49d4; that run then exposed the mixed C++ toolchain link failure addressed by the latest compiler change. The compiler migration also explicitly pinssysroot_linux-64=2.28: conda otherwise selected 2.12 headers, which lack thetimespec_getdeclaration used by the newer C++ headers. Verified the host-compiler selection block from CI's CMake 3.31.2 with an MSVC-style path containing spaces:CUDAHOSTCXXreplaces PyTorch's clang-cl selection. This local check does not run the Windows compiler. The ROCm Voxtral runtime job passed onb376004507. CUDA system tests passed on8846440476; kernel tests passed 225 cases and failed two export cases on the omittedUSE_GRAPHargument now addressed. The stricter local launch-contract check fails without that argument and passes with it. Full kernel, ROCm AOTI and Windows runtime results remain pending in CI.Authored with Codex assistance.