CUDA backend: per-program weights file names and antialiased resize by pytorchbot · Pull Request #24324 · pytorch/executorch

pytorchbot · 2026-10-30T04:09:59Z

This change fixes several problems in the CUDA backend. Each fix is its own commit.

Name the weights file of each program

The backend saves every program's weights to aoti_cuda_blob.ptd. Exporting several programs into one folder overwrites the weights, and the earlier programs then fail to load. Renaming each file after export works, but is easy to miss. A new option names the file:

partitioner = CudaPartitioner([
    CudaBackend.generate_method_name_compile_spec("forward"),
    CudaBackend.generate_data_name_compile_spec("encoder"),
])  # writes encoder.ptd

Without the option nothing changes. The name is checked: up to 200 ASCII letters, digits, _, - and ., not starting with . or -, not ending with .ptd, and not a Windows device name such as NUL. All methods of a program must share it. A backend that keeps its data inside the .pte, such as Metal, refuses it.

Lower antialiased resizes

F.interpolate(x, mode="bilinear" or "bicubic", antialias=True) failed to lower with "missing fallback kernels". The backend now rewrites it into plain tensor math, with PyTorch's window and weight formulas, for dynamic sizes and float32, float16 and bfloat16. The results are not bit-identical: for inputs between 0 and 1, float32 results stay within about 2e-5 of PyTorch's kernel for downscales, such as to 224x224, and within about 2e-4 for large upscales, such as 2048x2048 to 3000x3000.

Tests

Ran on an A100 GPU (Linux x86_64): test_cuda_export.py, test_antialiased_resize.py, test_missing_weights_blob.py, test_merge_ptes.py and test_cuda_weight_metadata.py. Each test added for a fix fails when that fix is removed.

The last commits come from #24326:

The antialiased resize torch.compile test is deleted. It could not fail, and with torch 2.14 in CI it failed for an unrelated reason, which an earlier commit from CUDA backend: CUDA graph fixes for mask indexing and changed input shapes, plus data name and resize follow-ups #24326 had patched. test_bicubic_antialias_resize already runs an exported program at an output height of 1.
The comment on the resize's min and max now says that export, not torch.compile, is why it avoids a conditional.
A backend that keeps its data in the .pte now refuses a hand-built data_name compile spec too, not only one from generate_data_name_compile_spec, and it does so before compiling anything.
The docs warn that Windows and macOS usually treat data names that differ only in case as the same file.

cc @digantdesai @freddan80 @per @zingo @oscarandersson8218 @mansnils @Sebastian-Larsson @robell @rascani

…and in exir (#24222) Fixes found while exporting models to ExecuTorch backends. ## Fixes - **exir:** `constant_prop_pass` keeps tensors that in-place ops write, also through views or in `no_grad`/`autocast` blocks. `to_backend` keeps the output node last. `to_edge` lowers `torch._check_tensor_all` and `torch.sym_ite`. `GroupBasedPartitioner` no longer makes partitions that depend on each other, unless caller node groups already do. Tensor constants get static shapes; deduplicated constants keep their data. - **AOTI (AOTInductor) and CUDA:** the missing-kernel check catches scatter and index_put fallbacks. `detach` lowers, also inside `torch.cond`. Strided outputs are copied. Transposed convolutions (1-D, 2-D, no dilation) and `torch._check_tensor_all` lower. - **XNNPACK:** the quantizer skips bfloat16 tensors, and `cat` respects scoped configs. ELU is delegated when XNNPACK can run it. - **Qualcomm:** autocast regions are inlined correctly. Unquantized bfloat16 nodes, data-dependent shapes and HTP partitions without runtime inputs stay on the CPU. Fixes for stacks, 0-d selects, RMSNorm and int64 constants. - **OpenVINO:** model data stays alive while OpenVINO 2026.4 reads it. Partitions are converted with their input types. - **Core ML:** bfloat16 ops, and ops above rank 5 outside static layout chains, stay on the CPU. - **MLX, Arm, Cortex-M, Vulkan, NXP, portable:** dtype checks for MLX tensor lists, one constant slot per weight, slice updates on a constant base; TOSA keeps unused delegate inputs for VGF; `MatmulToBmmPass` takes a constant operand; Vulkan constants above 4 dims; non-strict programs in the NXP quantizer; `cummin.out`. ## Behavior changes - A CUDA `.pte` loads only the weights it was exported with. A `.ptd` with other weights, for example after fine-tuning, needs a newly exported `.pte`. - CUDA graphs turn off when outputs do not match the ETensor layout. If only the capture call sees such a layout, that call returns an error. - `GroupBasedPartitioner` is about 2x slower on large graphs, and some graphs get a different number of partitions. `to_backend` keeps the original order of top-level nodes. - XNNPACK: weight-only quantized linears, and 8da4w or 8da8w linears (8-bit dynamic activations, 4-bit or 8-bit weights) lowered with `to_backend`, fail at export instead of giving wrong results. Delegated ELU flushes subnormal values to zero. Cats outside a scoped quantizer config are no longer quantized. - Ethos-U raises an error for a delegate with unused inputs instead of returning wrong values. We would like to cherry-pick this PR to release/1.6 after it merges. It needs #23915 there first. ## Testing Each fix has a test that fails without it. Run on an A100 (CUDA, Vulkan), an x86 CPU (OpenVINO, QNN SDK), macOS (Core ML, MLX), the TOSA reference model and VGF emulation (Arm), and the eIQ Neutron SDK (NXP). Not run: Metal, ROCm, Windows CUDA. (cherry picked from commit 054de1f)

This branch was picked before #24263 and #24295 reached release/1.6, so three files carried their content only in part. Take release/1.6 and use the versions a clean pick of 054de1f gives for backends/qualcomm/partition/qnn_partitioner.py, backends/qualcomm/tests/test_passes.py and exir/tests/test_pass_infra.py.

…fer check (#24299) ## What is wrong today Robot policies and follow-up checks on #24222 found these bugs in the CUDA backend and the runtime: 1. **CUDA mask indexing:** `x[mask]` and `x[mask] = values` have no C shims, so lowering fails with "missing fallback kernels". This blocks SmolVLA. 2. **CUDA graphs:** a channels-last output turns CUDA graphs off, although a raw copy is correct. 3. **CUDA graph shapes:** when the captured call has a new batch size, it and every replay return the previous output shape. 4. **Output buffers:** `Method::set_output_data_ptr` checks a buffer against the output's current size. After a dynamic output shrinks, a too-small buffer is accepted and a later, larger call writes past it. ## What this change does One commit per fix: 1. New shims for index and index_put follow ATen's indexing rules. The Windows import library gains the new names. Inside a CUDA graph capture, a mask index fails with a clear error. 2. The check compares output strides with the strides the ETensor's dim order gives. 3. The capture step resizes each output to the captured shape. 4. The buffer is checked against the output's upper bound. Code that binds a smaller buffer after a call now gets `InvalidArgument`. ## Tests Each new test fails without its fix. - **CUDA (A100):** mask reads and writes, also inside a capture; the raw-copy check; a capture call with a new batch size. - **Runtime:** a too-small output buffer is refused after a dynamic output shrinks; the runtime tests pass. On LeRobot policies, SmolVLA now lowers to CUDA and runs in 43 ms per call on an A100, against 504 ms in eager PyTorch (cosine similarity 0.99996). (cherry picked from commit ff1bab3)

…fer check (#24299) ## What is wrong today Robot policies and follow-up checks on #24222 found these bugs in the CUDA backend and the runtime: 1. **CUDA mask indexing:** `x[mask]` and `x[mask] = values` have no C shims, so lowering fails with "missing fallback kernels". This blocks SmolVLA. 2. **CUDA graphs:** a channels-last output turns CUDA graphs off, although a raw copy is correct. 3. **CUDA graph shapes:** when the captured call has a new batch size, it and every replay return the previous output shape. 4. **Output buffers:** `Method::set_output_data_ptr` checks a buffer against the output's current size. After a dynamic output shrinks, a too-small buffer is accepted and a later, larger call writes past it. ## What this change does One commit per fix: 1. New shims for index and index_put follow ATen's indexing rules. The Windows import library gains the new names. Inside a CUDA graph capture, a mask index fails with a clear error. 2. The check compares output strides with the strides the ETensor's dim order gives. 3. The capture step resizes each output to the captured shape. 4. The buffer is checked against the output's upper bound. Code that binds a smaller buffer after a call now gets `InvalidArgument`. ## Tests Each new test fails without its fix. - **CUDA (A100):** mask reads and writes, also inside a capture; the raw-copy check; a capture call with a new batch size. - **Runtime:** a too-small output buffer is refused after a dynamic output shrinks; the runtime tests pass. On LeRobot policies, SmolVLA now lowers to CUDA and runs in 43 ms per call on an A100, against 504 ms in eager PyTorch (cosine similarity 0.99996).

The pick was made before #24298 (the #24222 pick) was on release/1.6, so three files carried partial copies of #24222. This merges #24298's branch and restores the files so that, on top of #24298, the change is exactly #24299 (ff1bab3).

…24314) This change fixes several problems in the CUDA backend. Each fix is its own commit. ## Name the weights file of each program The backend saves every program's weights to `aoti_cuda_blob.ptd`. Exporting several programs into one folder overwrites the weights, and the earlier programs then fail to load. Renaming each file after export works, but is easy to miss. A new option names the file: ```python partitioner = CudaPartitioner([ CudaBackend.generate_method_name_compile_spec("forward"), CudaBackend.generate_data_name_compile_spec("encoder"), ]) # writes encoder.ptd ``` Without the option nothing changes. The name is checked: up to 200 ASCII letters, digits, `_`, `-` and `.`, not starting with `.` or `-`, not ending with `.ptd`, and not a Windows device name such as `NUL`. All methods of a program must share it. A backend that keeps its data inside the `.pte`, such as Metal, refuses it. ## Lower antialiased resizes `F.interpolate(x, mode="bilinear" or "bicubic", antialias=True)` failed to lower with "missing fallback kernels". The backend now rewrites it into plain tensor math, with PyTorch's window and weight formulas, for dynamic sizes and float32, float16 and bfloat16. The results are not bit-identical: in float32 they stay within about 2e-5 of PyTorch's kernel for inputs between 0 and 1. ## Tests Ran on an A100 GPU (Linux x86_64): `test_cuda_export.py`, `test_antialiased_resize.py`, `test_missing_weights_blob.py`, `test_merge_ptes.py` and `test_cuda_weight_metadata.py`. Each test added for a fix fails when that fix is removed. (cherry picked from commit 7586c16)

pytorch-bot · 2026-10-30T04:10:03Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/24324

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 86 Pending, 1 Unclassified Failure

As of commit b535a66 with merge base 4db778e ():

UNCLASSIFIED FAILURE - DrCI could not classify the following job because the workflow did not run on the merge base. The failure may be pre-existing on trunk or introduced by this PR:

Test CUDA Windows Export and E2E / export-model-cuda-windows-artifact (mistralai, Voxtral-Mini-4B-Realtime-2602, quantized-int4-tile... / linux-job (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
undefined reference to aoti_torch_dtype_element_size'`

This comment was automatically generated by Dr. CI and updates every 15 minutes.

…24314) This change fixes several problems in the CUDA backend. Each fix is its own commit. ## Name the weights file of each program The backend saves every program's weights to `aoti_cuda_blob.ptd`. Exporting several programs into one folder overwrites the weights, and the earlier programs then fail to load. Renaming each file after export works, but is easy to miss. A new option names the file: ```python partitioner = CudaPartitioner([ CudaBackend.generate_method_name_compile_spec("forward"), CudaBackend.generate_data_name_compile_spec("encoder"), ]) # writes encoder.ptd ``` Without the option nothing changes. The name is checked: up to 200 ASCII letters, digits, `_`, `-` and `.`, not starting with `.` or `-`, not ending with `.ptd`, and not a Windows device name such as `NUL`. All methods of a program must share it. A backend that keeps its data inside the `.pte`, such as Metal, refuses it. ## Lower antialiased resizes `F.interpolate(x, mode="bilinear" or "bicubic", antialias=True)` failed to lower with "missing fallback kernels". The backend now rewrites it into plain tensor math, with PyTorch's window and weight formulas, for dynamic sizes and float32, float16 and bfloat16. The results are not bit-identical: in float32 they stay within about 2e-5 of PyTorch's kernel for inputs between 0 and 1. ## Tests Ran on an A100 GPU (Linux x86_64): `test_cuda_export.py`, `test_antialiased_resize.py`, `test_missing_weights_blob.py`, `test_merge_ptes.py` and `test_cuda_weight_metadata.py`. Each test added for a fix fails when that fix is removed.

The pick was made before #24298 and #24320 were on release/1.6, so it carried partial copies of earlier changes. This stacks it on #24320's branch so that, on top of #24320, the change is exactly #24314 (7586c16).

Some comments added with the CUDA mask indexing change say the wrong thing: - index.h described accumulation as out[indices] += values. With a repeated index, += keeps only one of the values, while index_put_ with accumulate adds every value, and so does the shim. - index.cu said CUB counts items in an int for the index_put sort. That is true for the mask path, but the radix sort takes a wider count. The limit there comes from the int the code passes to the sort. - index.cu said the accumulation types are ATen's. That holds for half and bfloat16, which add in float. Narrower integers add in int64 here, which gives the same wrapped result. - method.h named the output index i in the set_output_data_ptr docs, but the parameter is output_idx. Only comments change.

A backend that keeps its data in the .pte, such as Metal, refuses a data_name compile spec. preprocess only checked it after moving the program to the device and compiling it with AOTInductor, which took about 18 seconds and left the compiled library on disk. It now checks the name first. The data name test calls the shared preprocess with such a spec, so moving the check back behind the compile, or dropping it, fails the test.

…ick reverted The bot pick took the older main versions of these two files, which undid the release/1.6 fix from #24289.

shoumikhin added 6 commits October 29, 2026 16:09

pytorchbot requested a review from mergennachin as a code owner October 30, 2026 04:10

pytorchbot mentioned this pull request Oct 30, 2026

CUDA backend: per-program weights file names and antialiased resize #24314

Merged

meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Oct 30, 2026

pytorchbot temporarily deployed to cadence October 30, 2026 04:10 — with GitHub Actions Inactive

shoumikhin added 2 commits October 29, 2026 21:10

Rebase the #24314 cherry-pick onto the #24299 cherry-pick

44980c5

The pick was made before #24298 and #24320 were on release/1.6, so it carried partial copies of earlier changes. This stacks it on #24320's branch so that, on top of #24320, the change is exactly #24314 (7586c16).

shoumikhin requested review from JacobSzwejbka, SS-JIA, digantdesai, larryliu0820, metascroy, psiddh, rascani, robert-kalmar and shoumikhin as code owners October 30, 2026 04:11

shoumikhin mentioned this pull request Oct 30, 2026

[v1.6.0] Release Schedule and Tracker #23608

Open

github-actions Bot added ciflow/trunk module: arm Issues related to arm backend labels Oct 30, 2026

shoumikhin temporarily deployed to cadence October 30, 2026 04:11 — with GitHub Actions Inactive

shoumikhin temporarily deployed to cadence October 30, 2026 11:07 — with GitHub Actions Inactive

shoumikhin temporarily deployed to cadence October 30, 2026 11:10 — with GitHub Actions Inactive

shoumikhin temporarily deployed to cadence October 30, 2026 11:11 — with GitHub Actions Inactive

shoumikhin deployed to upload-benchmark-results October 30, 2026 12:16 — with GitHub Actions Active

shoumikhin added 2 commits October 30, 2026 06:34

Merge the updated #24299 cherry-pick into the #24314 cherry-pick

2ee2baa

shoumikhin temporarily deployed to cadence October 30, 2026 13:35 — with GitHub Actions Inactive

shoumikhin temporarily deployed to cadence October 30, 2026 13:49 — with GitHub Actions Inactive

shoumikhin temporarily deployed to cadence October 30, 2026 13:50 — with GitHub Actions Inactive

shoumikhin temporarily deployed to cadence October 30, 2026 13:53 — with GitHub Actions Inactive

shoumikhin temporarily deployed to cadence October 30, 2026 14:27 — with GitHub Actions Inactive

shoumikhin temporarily deployed to cadence October 30, 2026 14:39 — with GitHub Actions Inactive

shoumikhin temporarily deployed to cadence October 30, 2026 14:43 — with GitHub Actions Inactive

shoumikhin temporarily deployed to cadence October 30, 2026 14:44 — with GitHub Actions Inactive

shoumikhin deployed to upload-benchmark-results October 30, 2026 15:51 — with GitHub Actions Active

shoumikhin added 2 commits October 30, 2026 09:23

Restore release/1.6 Windows Voxtral CI files that the #24299 cherry-p…

cdb488e

…ick reverted The bot pick took the older main versions of these two files, which undid the release/1.6 fix from #24289.

Merge the updated #24299 cherry-pick into the #24314 cherry-pick

b535a66

shoumikhin temporarily deployed to cadence October 30, 2026 16:24 — with GitHub Actions Inactive

shoumikhin temporarily deployed to cadence October 30, 2026 16:37 — with GitHub Actions Inactive

shoumikhin deployed to cadence October 30, 2026 16:42 — with GitHub Actions Active

Navigation Menu

Repository navigation

CUDA backend: per-program weights file names and antialiased resize - #24324

CUDA backend: per-program weights file names and antialiased resize#24324
pytorchbot wants to merge 23 commits into
release/1.6from
cherry-pick-24314-by-pytorch_bot_bot_

pytorchbot commented Oct 30, 2026 •

edited by shoumikhin

Loading

Uh oh!

pytorch-bot Bot commented Oct 30, 2026 •

edited

Loading

Uh oh!

This branch was successfully deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

pytorchbot commented Oct 30, 2026 • edited by shoumikhin Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Name the weights file of each program

Lower antialiased resizes

Tests

Uh oh!

pytorch-bot Bot commented Oct 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/24324

❌ 86 Pending, 1 Unclassified Failure

Uh oh!

This branch was successfully deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

pytorchbot commented Oct 30, 2026 •

edited by shoumikhin

Loading

pytorch-bot Bot commented Oct 30, 2026 •

edited

Loading