[executorch][native] Land the bundled op sample harness stack by digantdesai · Pull Request #24698 · pytorch/executorch

digantdesai · 2026-11-05T22:26:18Z

Lands the bundled op sample harness stack on main as its 20 original commits, in order. Merge with "Rebase and merge" to keep them as 20 commits.

ghstack PRs: #24355, #24356, #24357, #24358, #24359, #24360, #24361, #24362, #24363, #24364, #24365, #24366, #24367, #24368, #24369, #24370, #24371, #24372, #24373, #24405

This replaces the merge-bot PRs #24530 to #24537 and #24549. #24538 to #24548 were squashed into intermediate bot branches only, and #24549 now conflicts with its base.

@diff-train-skip-merge

pytorch-bot · 2026-11-05T22:26:26Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/24698

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 14 New Failures, 17 Cancelled Jobs, 2 Unrelated Failures, 12 Unclassified Failures

As of commit 3016b89 with merge base 0f8e096 ():

NEW FAILURES - The following jobs have failed:

pull / test-arm-backend-public-api-backward-compatibility / linux-job (gh)
RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
pull / test-arm-cortex-m-size-test (bare_metal) / linux-job (gh)
RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
pull / test-arm-cortex-m-size-test (zephyr-preset) / linux-job (gh)
RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
pull / test-mcu-cortex-m-backend / linux-job (gh)
RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
pull / test-vulkan-operators-linux / linux-job (gh)
RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 127
Test Vulkan Backend (specialized runners) / test-vulkan-nvidia / linux-job (gh)
RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 127
trunk / test-cortex-m-e2e / run (mv2, cortex-m0plus) / mv2-cortex-m0plus (gh)
RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
trunk / test-cortex-m-e2e / run (mv2, cortex-m55) / mv2-cortex-m55 (gh)
RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
trunk / test-cortex-m-e2e / run (mv2, cortex-m7) / mv2-cortex-m7 (gh)
RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
trunk / test-cortex-m-e2e / run (mv3, cortex-m0plus) / mv3-cortex-m0plus (gh)
RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
trunk / test-cortex-m-e2e / run (mv3, cortex-m55) / mv3-cortex-m55 (gh)
RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
trunk / test-cortex-m-e2e / run (mv3, cortex-m7) / mv3-cortex-m7 (gh)
RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
trunk / test-cortex-m-ops / run (cortex-m0plus) / cortex-m-ops-cortex-m0plus (gh)
RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
trunk / test-cortex-m-ops / run (cortex-m7) / cortex-m-ops-cortex-m7 (gh)
RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1

UNCLASSIFIED FAILURES - DrCI could not classify the following jobs because the workflow did not run on the merge base. The failures may be pre-existing on trunk or introduced by this PR:

Build Aarch64 Linux CUDA Wheels / pytorch/executorch / build-wheel-py3_11-cuda-aarch6413_2-aarch64 (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
Build Aarch64 Linux CUDA Wheels / pytorch/executorch / upload / upload-wheel-py3_11-cuda-aarch6413_2-aarch64 (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
Unable to download artifact(s): Artifact not found
Build Linux CUDA Wheels / pytorch/executorch / build-manywheel-py3_11-cuda13_2 (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
Build Linux CUDA Wheels / pytorch/executorch / upload / upload-manywheel-py3_11-cuda13_2 (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
Unable to download artifact(s): Artifact not found
Build macOS Wheels / pytorch/executorch / build-wheel-py3_11-cpu (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
Build Windows CUDA Wheels / bundle-cuda-windows-wheel-e2e-13.2 (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
Unable to download artifact(s): Artifact not found
Build Windows CUDA Wheels / pytorch/executorch / build-wheel-py3_11-cuda13_2 (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
##[error]The operation was canceled.
Build Windows CUDA Wheels / pytorch/executorch / upload / upload-wheel-py3_11-cuda13_2 (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
Unable to download artifact(s): Artifact not found
Build Windows CUDA Wheels / test-cuda-windows-wheel-e2e-13.2 / windows-job (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
Unable to download artifact(s): Artifact not found
Build Windows Wheels / pytorch/executorch / build-wheel-py3_11-cpu (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
Build Windows Wheels / pytorch/executorch / upload / upload-wheel-py3_11-cpu (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
Unable to download artifact(s): Artifact not found
pull / nxp-mcuxpresso-test / run / linux-job (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)

CANCELLED JOBS - The following jobs were cancelled. Please retry:

pull / android / run-emulator (gh)
##[error]The operation was canceled.
pull / test-arm-backend-no-driver (test_pytest_ops_tosa) / linux-job (gh)
pull / test-arm-backend-vkml (test_pytest_ops_vkml) / linux-job (gh)
pull / test-lora-linux / linux-job (gh)
pull / test-models-linux (phi_4_mini, portable, mt-l-x86iavx512-16-128) / linux-job (gh)
pull / test-qnn-delegate-linux / linux-job (gh)
pull / test-qnn-testsuite-linux / package-golden-artifacts (gh)
pull / test-qnn-testsuite-linux / test-backend-linux (qnn, models) / linux-job (gh)
[runner-container-hooks] FATAL: received SIGINT
pull / test-sqnr-static-llm-qnn-linux (smollm2_135m) / linux-job (gh)
[ERROR] [Qnn ExecuTorch]: <E> unexpected GraphHtpSettings option 66
pull / test-voxtral-realtime-xnnpack-linux / linux-job (gh)
pull / unittest / linux / linux-job (gh)
##[error]received SIGINT
pull / unittest / macos / macos-job (gh)
pull / unittest / windows / windows-job (gh)
pull / unittest-buck / macos / macos-job (gh)
pull / unittest-editable / linux / linux-job (gh)
##[error]received SIGINT
pull / unittest-editable / macos / macos-job (gh)
pull / unittest-nxp-neutron / linux-job (gh)
##[error]received SIGINT

FLAKY - The following job failed but was likely due to flakiness present on trunk:

pull / unittest-editable / windows / windows-job (gh) (matched win rule in flaky-rules.json)
##[error]The operation was canceled.

BROKEN TRUNK - The following job failed but was present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

Test CoreML Backend / test-coreml / test-backend-macos (coreml, models) / macos-job (gh) (trunk failure)
##[error]The operation was canceled.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

github-actions · 2026-11-05T22:26:59Z

This PR needs a `release notes:` label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

Copilot

🟡 Changes recommended

A hard-coded PYTHONPATH separator blocks Windows sample-test builds, while state conversion breaks integer buffers and one-shot dtype iterators silently omit later cases.

3 open issues

What changed in this PR

Adds a bundled operator-sample harness for ExecuTorch’s native engines, with shared numerical checks and expanded linear coverage.

Changes:

Generates floating-point and quantized .ptn samples with eager and high-precision references, including chained dynamic-shape calls.
Adds shared ggml/Vulkan test drivers, explicit activation-quantization opt-in, and ggml CPU sample testing in CI.
Fixes Vulkan rank-1 quantization, weight-sum strides, and half-precision image-store rounding.

File	Description
`backends/vulkan/test/vulkan_compute_api_test.cpp`	Tests half-store rounding and rank-1 quantization.
`backends/vulkan/test/spv_codegen/test_gen_vulkan_spv.py`	Tests generated image-store wrappers.
`backends/vulkan/test/glsl/fill_texture__test.glsl`	Uses generated image declarations.
`backends/vulkan/test/glsl/binary_op_nobroadcast__test.glsl`	Uses generated image declarations.
`backends/vulkan/runtime/graph/ops/impl/QuantizeDequantize.cpp`	Treats rank-1 inputs as single rows.
`backends/vulkan/runtime/graph/ops/glsl/repeat_channel.glsl`	Uses generated image declarations.
`backends/vulkan/runtime/graph/ops/glsl/gemm/linear/qw/dq8ca_q4gsw/linear_dq8ca_q4gsw_tiled.glsl`	Corrects padded weight-sum row strides.
`backends/vulkan/runtime/graph/ops/glsl/convolution/conv2d/transposed/pack_weights/conv_transpose2d_prepack_weights.glsl`	Uses generated texture declarations.
`backends/vulkan/runtime/graph/ops/glsl/convolution/conv2d/pack_weights/conv2d_prepack_weights.glsl`	Uses generated texture declarations.
`backends/vulkan/runtime/graph/ops/glsl/convolution/conv2d/pack_weights/conv2d_dw_prepack_weights.glsl`	Uses generated texture declarations.
`backends/vulkan/runtime/graph/ComputeGraph.h`	Documents rank-1 matrix acceptance.
`backends/vulkan/runtime/graph/ComputeGraph.cpp`	Accepts rank-1 tensors as matrices.
`backends/vulkan/runtime/gen_vulkan_spv.py`	Generates nearest-even half-store rounding.
`backends/test/suite/operators/test_linear.py`	Aligns inputs with model feature counts.
`backends/native/test/samples/test_record.py`	Tests recording, references, and dynamic chains.
`backends/native/test/samples/test_output_check.cpp`	Tests numerical comparison rules.
`backends/native/test/samples/schemes.py`	Defines quantization schemes and reference decoding.
`backends/native/test/samples/sample_test_main.cpp`	Runs bundled samples through shared engine tests.
`backends/native/test/samples/record.py`	Records methods and reference sample sets.
`backends/native/test/samples/OutputCheck.h`	Defines numerical statistics and policies.
`backends/native/test/samples/OutputCheck.cpp`	Implements error and bias checks.
`backends/native/test/samples/ops.py`	Registers sample-supported operators.
`backends/native/test/samples/linear.py`	Adds suite, FACTO, quantization, and dynamic cases.
`backends/native/test/samples/generate.py`	Generates packages across operators and schemes.
`backends/native/test/samples/cases.py`	Provides case collection and input generation.
`backends/native/test/samples/BUCK`	Defines sample-generation and harness targets.
`backends/native/test/runtime/vulkan/vulkan_ptn_runner.cpp`	Rejects unsupported activation-quantization opt-in.
`backends/native/test/runtime/vulkan/targets.bzl`	Adds reusable host and sample-test targets.
`backends/native/test/runtime/PtnRunner.h`	Defines shared engine-host options.
`backends/native/test/runtime/ptn_runner_main.cpp`	Forwards activation-quantization opt-in.
`backends/native/test/runtime/ggml/test_ggml_ops.py`	Updates quantized tests and runner invocation.
`backends/native/test/runtime/ggml/test_ggml_engine.cpp`	Tests default rejection and explicit opt-in.
`backends/native/test/runtime/ggml/ggml_ptn_runner.cpp`	Applies shared options to ggml hosts.
`backends/native/test/runtime/ggml/BUCK`	Adds sample-test variants and execution settings.
`backends/native/test/runtime/BUCK`	Separates the shared engine-host interface.
`backends/native/test/e2e/test_cross_engine.py`	Passes engine-specific quantization flags.
`backends/native/test/CMakeLists.txt`	Integrates generation, checks, and engine sample tests.
`backends/native/runtime/vulkan/TODO.md`	Records remaining linear support gaps.
`backends/native/runtime/engine/ggml/GgmlExecutor.cpp`	Gates weight-only quantized linears on opt-in.
`backends/native/runtime/engine/ggml/GgmlEngine.h`	Exposes the activation-quantization option.
`backends/native/runtime/deserialize/Samples.h`	Adds sample-tensor equality comparison.
`backends/native/README.md`	Documents rounding, sample tests, and quantization opt-in.
`backends/native/examples/hf_llm/runtime/run_hf_llm_ggml.cpp`	Exposes opt-in in the LLM runner.
`.github/workflows/test-native.yml`	Adds generator dependencies and workflow triggers.
`.ci/scripts/test-native.sh`	Runs numerical checks and ggml CPU samples.

_{Review effort: Balanced}

+add_custom_command(
+  OUTPUT ${_native_op_samples}.stamp
+  COMMAND
+    ${CMAKE_COMMAND} -E env MKL_CBWR=COMPATIBLE ONEDNN_MAX_CPU_ISA=AVX2
+    ATEN_CPU_CAPABILITY=avx2 OMP_NUM_THREADS=1 MKL_NUM_THREADS=1
+    "PYTHONPATH=${EXECUTORCH_ROOT}/src:${EXECUTORCH_ROOT}/backends/cadence/utils/FACTO"


+    dtypes: Iterable[str] = tuple(DTYPES),
+) -> None:
+    """Writes no package, removing any at `path`, when the scheme fits no case."""
+    programs, golden, precise, golden_act = {}, [], [], []


+            state = {
+                k: scheme.decode(v).double() for k, v in method.state_dict().items()
+            }


…s 67 input features Pull Request resolved: #24355 Eight of the fourteen `test_linear.py` models were fed 64-feature inputs, but `Model` defaults to `in_features=67`, so eager rejected them and every backend skipped them. The dtype, no-bias, batch-size and leading-batch tests now pass 67-feature inputs. Authored with Claude Code. @bypass-github-export-checks @bypass-github-pytorch-ci-checks @bypass-github-executorch-ci-checks ghstack-source-id: 463657740 @exported-using-ghexport Differential Revision: [D128254891](https://our.internmc.facebook.com/intern/diff/D128254891/)

Pull Request resolved: #24356 Drivers other than `ptn_runner_main` need to link an engine without its runner binary. Each engine's `make_engine_host()` moves into a `*_engine_host` library and `PtnRunner.h` into `engine_host`; the runners are unchanged. Authored with Claude Code. Differential Revision: [D128254879](https://our.internmc.facebook.com/intern/diff/D128254879/) **NOTE FOR REVIEWERS**: This PR has internal Meta-specific changes or comments, please review them on [Phabricator](https://our.internmc.facebook.com/intern/diff/D128254879/)! ghstack-source-id: 463657752

…se samples Pull Request resolved: #24357 Engines need op-level reference data that doesn't depend on any engine or device. `op_samples` writes one .ptn per op and quantization scheme, bundling `golden` (eager) and `precise` (float64, weights dequantized, activations unquantized) samples over the same inputs. Linear is the first op. Schemes are float, int4wo g32/g128 and 8da4w g32/g128, each with f32, f16 and bf16 methods; int8 weight-only is left out because no native engine runs it. Linear cases are the backend test suite's models, reused through a stand-in runner, plus decode, prefill, large-K, bias-free leading-batch, outlier-channel and zero-row cases. With CMake, the `native_op_samples` target writes them; it is not in ALL, so a plain build never needs torch. Ops and schemes are independent axes. An op is an `Op` in `ops()`, a function returning `Case`s plus the dtypes to record them in; a scheme is a `Scheme` in `schemes()` that decides which cases it fits, quantizes a model and decodes its weights. `generate` records every op under every scheme, one spawned process per package, and writes no package when a scheme fits none of an op's cases. `cases.py` holds the case type and the backend test suite adapter, `schemes.py` the quantization, and `record.py` the packaging, which knows neither torchao nor any op. Authored with Claude Code. @bypass-github-export-checks @bypass-github-pytorch-ci-checks @bypass-github-executorch-ci-checks ghstack-source-id: 463657757 @exported-using-ghexport Differential Revision: [D128254877](https://our.internmc.facebook.com/intern/diff/D128254877/)

Pull Request resolved: #24358 Eager golden outputs differed by an ulp or two between generating hosts, because MKL and oneDNN pick kernels by ISA and MKL splits work by thread count. The `op_samples` genrule and the `native_op_samples` CMake target now pin MKL to its reproducible path, oneDNN and ATen to AVX2, and OpenMP and MKL to one thread. `MKL_NUM_THREADS` is set as well as `OMP_NUM_THREADS` so a host's own value cannot override it. Authored with Claude Code. @bypass-github-export-checks @bypass-github-pytorch-ci-checks @bypass-github-executorch-ci-checks ghstack-source-id: 463657753 @exported-using-ghexport Differential Revision: [D128254894](https://our.internmc.facebook.com/intern/diff/D128254894/)

…samples Pull Request resolved: #24359 A fixed tolerance either hides engine bugs or fails legitimate numerics, so an engine is judged by its distance from `precise` relative to eager's. `OutputCheck` bounds RMSE and max error by multiples of eager's plus `rtol`, and the fitted absolute and relative bias likewise, plus `sigmas` standard errors. Non-finite values must equal golden. CMake builds it and its test too. Authored with Claude Code. @bypass-github-export-checks @bypass-github-pytorch-ci-checks @bypass-github-executorch-ci-checks ghstack-source-id: 463657743 @exported-using-ghexport Differential Revision: [D128254878](https://our.internmc.facebook.com/intern/diff/D128254878/)

Pull Request resolved: #24360 The bundled samples need a harness that runs on any engine. `sample_test_main` registers one gtest per package and method and judges each output with `OutputCheck`; unsupported ops, dtypes and shapes fail. `ggml_sample_test{,_cuda,_hip}` and `vulkan_sample_test` run it on `op_samples`. With CMake, `native_ggml_sample_test` (CPU) and `native_vulkan_sample_test` depend on `native_op_samples` and run it on its output. The sample tests are manual for now, since gaps fail: `ci.skip_test` in Buck and `DISABLED` in CTest. The README says how to run them and what the generator needs. `vulkan_sample_test` runs on SwiftShader like `vulkan_engine_test`; preloading a driver's `libvulkan.so.1` runs it on that driver. The ggml CUDA targets preload the CUDA driver stub through their env, so `test_ggml_ops.py` no longer puts the runner's directory on `LD_LIBRARY_PATH`. Authored with Claude Code. Differential Revision: [D128254880](https://our.internmc.facebook.com/intern/diff/D128254880/) **NOTE FOR REVIEWERS**: This PR has internal Meta-specific changes or comments, please review them on [Phabricator](https://our.internmc.facebook.com/intern/diff/D128254880/)! ghstack-source-id: 463657748

… activation quant is allowed Pull Request resolved: #24361 ggml quantizes the activations of its quantized matmuls, so weight-only int4 and GGUF linears silently ran with numerics the PTN doesn't describe. Compile now rejects them unless `GgmlEngineOptions::allow_activation_quant_for_weight_only` is set. `ggml_ptn_runner` and `run_hf_llm_ggml` expose it as `--allow_activation_quant_for_weight_only`, which GGUF LLMs on ggml now need. Authored with Claude Code. @bypass-github-export-checks @bypass-github-pytorch-ci-checks @bypass-github-executorch-ci-checks ghstack-source-id: 463657742 @exported-using-ghexport Differential Revision: [D128254887](https://our.internmc.facebook.com/intern/diff/D128254887/)

…t8-activation golden Pull Request resolved: #24362 With activation quantization allowed, ggml's weight-only linears no longer compute what `golden` describes. Weight-only packages now carry `golden-int8_act`, eager with int8 per-token dynamic activations, which `sample_test_main` judges against under `--allow_activation_quant_for_weight_only`. The flag is a `HostOptions` field that Vulkan rejects; new `ggml_sample_test_act_quant` targets and the `native_ggml_sample_test_act_quant` ctest set it. Authored with Claude Code. @bypass-github-export-checks @bypass-github-pytorch-ci-checks @bypass-github-executorch-ci-checks ghstack-source-id: 463657755 @exported-using-ghexport Differential Revision: [D128254888](https://our.internmc.facebook.com/intern/diff/D128254888/)

…ple tests Pull Request resolved: #24363 By default ggml-CUDA's cuBLAS GEMMs compute in f16 for f16 and dequantized weights, and in TF32 for f32 ones, so the linear samples measured those defaults rather than ggml's kernels. The CUDA sample-test targets set `GGML_CUDA_CUBLAS_COMPUTE_TYPE=f32` and `NVIDIA_TF32_OVERRIDE=0`; llama.cpp's defaults, `test_ggml_ops` and the bundle are unchanged. Authored with Claude Code. Differential Revision: [D128254889](https://our.internmc.facebook.com/intern/diff/D128254889/) **NOTE FOR REVIEWERS**: This PR has internal Meta-specific changes or comments, please review them on [Phabricator](https://our.internmc.facebook.com/intern/diff/D128254889/)! ghstack-source-id: 463657744

…values Pull Request resolved: #24364 Rounding toward zero when narrowing f32 to f16, which Vulkan and WebGPU allow, biases every stored value toward zero, and the bias compounds across layers. The README suggests rounding to nearest, ties to even, as eager does. Authored with Claude Code. @bypass-github-export-checks @bypass-github-pytorch-ci-checks @bypass-github-executorch-ci-checks ghstack-source-id: 463657754 @exported-using-ghexport Differential Revision: [D128254885](https://our.internmc.facebook.com/intern/diff/D128254885/)

Pull Request resolved: #24365 The hand-written cases cover few shapes. `facto_cases` turns the argument tuples FACTO draws for an op spec from a fixed seed into one-sample cases, `facto_<i>`, and `linear.py` keeps every fifth `linear.default` tuple. The spec lives here, so ET backend tests are unchanged. The `native_op_samples` CMake target puts the vendored FACTO submodule on `PYTHONPATH` and reruns when its sources change. Authored with Claude Code. @bypass-github-export-checks @bypass-github-pytorch-ci-checks @bypass-github-executorch-ci-checks ghstack-source-id: 463657759 @exported-using-ghexport Differential Revision: [D128254881](https://our.internmc.facebook.com/intern/diff/D128254881/)

Pull Request resolved: #24366 PTN carries GGUF k-quant constants, but no op package exercised them. A `Gguf` scheme, as `gguf_q4_k` and `gguf_q6_k`, writes random valid blocks to a .gguf with gguf-py and loads them back as a checkpoint would. `golden-int8_act` now comes from per-token int8 pre-hooks on the weight-only model, which also work for GGUF, so `Scheme.weight_only` replaces `Scheme.act_quant`. FACTO also keeps the tuples whose input features fit 256-element blocks. Authored with Claude Code. Differential Revision: [D128254890](https://our.internmc.facebook.com/intern/diff/D128254890/) **NOTE FOR REVIEWERS**: This PR has internal Meta-specific changes or comments, please review them on [Phabricator](https://our.internmc.facebook.com/intern/diff/D128254890/)! ghstack-source-id: 463657745

…samples Pull Request resolved: #24367 `precise` used weights dequantized in the method's dtype, so f16 and bf16 references held already-rounded weights while engines decode in float32. The `Gguf` and `Torchao` schemes now decode their weights in float32 before the float64 math and raise on any other torchao weight. Authored with Claude Code. @bypass-github-export-checks @bypass-github-pytorch-ci-checks @bypass-github-executorch-ci-checks ghstack-source-id: 463657760 @exported-using-ghexport Differential Revision: [D128254893](https://our.internmc.facebook.com/intern/diff/D128254893/)

…-block scales in the op samples Pull Request resolved: #24368 GGUF samples used fixed positive power-of-two super-block scales, so engines' decoders never saw negative, rounded or subnormal ones. A model's i-th `nn.Linear` now takes the i-th scale family, and a new `scale_families` case gives each family its own output. A zero-d family waits behind a TODO. With q4_k's d at 0 every weight is -dmin·m, so a token's 32 outputs share one activation-quantization error (correlation about 0.74) and `OutputCheck`'s bias standard error, which assumes independent outputs, comes out about 4.9 times too small; with q6_k's d at 0 every weight is zero and the outputs are just the bias. Authored with Claude Code. @bypass-github-export-checks @bypass-github-pytorch-ci-checks @bypass-github-executorch-ci-checks ghstack-source-id: 463657756 @exported-using-ghexport Differential Revision: [D128254896](https://our.internmc.facebook.com/intern/diff/D128254896/)

Pull Request resolved: #24369 `linear_dq8ca_q4gsw` failed on rank-1 activations: `is_2d_matrix` rejected them, and `get_quantized_input_num_blocks` read the size at `ndim - 2`. Both now treat a 1-D tensor as a single row. Authored with Claude Code. @bypass-github-export-checks @bypass-github-pytorch-ci-checks @bypass-github-executorch-ci-checks ghstack-source-id: 463657751 @exported-using-ghexport Differential Revision: [D128254882](https://our.internmc.facebook.com/intern/diff/D128254882/)

… 1 to 4 Pull Request resolved: #24370 `linear_dq8ca_q4gsw_tiled` strode each group's weight sums by `div_up(N, 4)` texels, but the sums are laid out `{groups, align_up(N, 8)}`, as both the ET-VK AOT pass (`compute_per_group_sums`) and Native's `FuseQuantizedLinear` write them. When `div_up(N, 4)` is odd, every group after the first read another row's sums; the shader now strides by `2 * div_up(N, 8)`, which equals the old stride for every other N. Authored with Claude Code. @bypass-github-export-checks @bypass-github-pytorch-ci-checks @bypass-github-executorch-ci-checks ghstack-source-id: 463657747 @exported-using-ghexport Differential Revision: [D128254883](https://our.internmc.facebook.com/intern/diff/D128254883/)

…near gaps Pull Request resolved: #24371 The Vulkan engine leaves int4 weight-only linears unlowered when N isn't a multiple of 4, so op-sample cases with N of 1 and 43 fail. It also computes a linear on a rank-4 input wrong, while rank-3 inputs pass. `TODO.md` records both, with the fix for the first and where to look for the second. Authored with Claude Code. @bypass-github-export-checks @bypass-github-pytorch-ci-checks @bypass-github-executorch-ci-checks ghstack-source-id: 463657741 @exported-using-ghexport Differential Revision: [D128254884](https://our.internmc.facebook.com/intern/diff/D128254884/)

Pull Request resolved: #24372 Vulkan lets the device choose how stores to rgba16f images round, and devices that round toward zero bias f16 outputs; shader float controls don't govern that conversion. `gen_vulkan_spv.py` now wraps `imageStore` on writable rgba16f images in a round-to-nearest-even helper, and six shaders that declared such images by hand, two of them test shaders, now use the codegen. On devices that round toward zero, stores of 65520 and above now give inf, as the CPU conversion does, instead of 65504. The four texture shaders that already call `round_to_half_rte` on their outputs (`binary_scalar_texture`, `full_texture`, `reduce`, `scalar_tensor`) now round twice, with identical results; removing those calls is a follow-up. `aten._to_copy` from f32 to f16 textures is a `vkCmdBlitImage`, which the codegen doesn't reach, so `test_to_copy` keeps its one-ulp tolerance. Authored with Claude Code. @bypass-github-export-checks @bypass-github-pytorch-ci-checks @bypass-github-executorch-ci-checks ghstack-source-id: 463657749 @exported-using-ghexport Differential Revision: [D128254886](https://our.internmc.facebook.com/intern/diff/D128254886/)

Pull Request resolved: #24373 `ggml_sample_test_act_quant` (CPU) is the one sample test that passes every package, so it leaves the manual set and runs in internal CI. The OSS ggml job now runs it as `native_ggml_sample_test_act_quant`, along with `native_output_check_test`. The rest stay manual with a TODO: plain `ggml_sample_test` fails the weight-only packages by design, and the Vulkan, CUDA and HIP ones have gaps. The OSS ggml job installs `gguf` for the op sample generator, and the workflow also runs on changes to FACTO and `test_linear.py`, which the samples draw from. Authored with Claude Code. @bypass-github-export-checks @bypass-github-pytorch-ci-checks @bypass-github-executorch-ci-checks ghstack-source-id: 463657737 @exported-using-ghexport Differential Revision: [D128668496](https://our.internmc.facebook.com/intern/diff/D128668496/)

…samples Pull Request resolved: #24405 Engines resize dynamic inputs between inferences, so the op samples now cover a shape changing mid-run. A case built with `chain` is exported with its `Dim`s and recorded as one sample of four calls. Each `Dim` starts mid-range, where export traces it, visits both bounds and returns, every other `Dim` in the opposite order. The harness already runs a sample's calls in order on one executable from one context, resizing inputs between them, so only its header comment changes. The new linear cases are the XNNPACK delegate's dynamic linear tests, each also followed by a ReLU: one and two batch dims, one `Dim` on two dims or two inputs, and sequential and parallel linears, with batch dims down to 1, which tracing mid-range keeps export from specializing. The cases every quantization scheme fits take 256 input features over up to 64 tokens: `dynamic_tokens` in 2-D and, with a static batch of one as in an LLM, in 3-D, two batch dims sharing one `Dim` of up to 16, and the sequential and parallel linears. Authored with Claude Code. Differential Revision: [D128770754](https://our.internmc.facebook.com/intern/diff/D128770754/) **NOTE FOR REVIEWERS**: This PR has internal Meta-specific changes or comments, please review them on [Phabricator](https://our.internmc.facebook.com/intern/diff/D128770754/)! ghstack-source-id: 463657750

digantdesai · 2026-11-05T22:59:15Z

/easycla

digantdesai requested review from SS-JIA and larryliu0820 as code owners November 5, 2026 22:26

Copilot AI balanced review requested due to automatic review settings November 5, 2026 22:26

meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Nov 5, 2026

digantdesai temporarily deployed to cadence November 5, 2026 22:26 — with GitHub Actions Inactive

Copilot started reviewing on behalf of digantdesai November 5, 2026 22:26

View session

Copilot AI reviewed Nov 5, 2026

View reviewed changes

JakeStevens approved these changes Nov 5, 2026

View reviewed changes

digantdesai temporarily deployed to cadence November 5, 2026 22:44 — with GitHub Actions Inactive

digantdesai temporarily deployed to cadence November 5, 2026 22:46 — with GitHub Actions Inactive

JakeStevens approved these changes Nov 5, 2026

View reviewed changes

digantdesai temporarily deployed to cadence November 5, 2026 22:48 — with GitHub Actions Inactive

digantdesai added 13 commits November 5, 2026 16:52

digantdesai added 7 commits November 5, 2026 16:52

digantdesai force-pushed the gh/digantdesai/201/orig branch from 3016b89 to f67eb1a Compare November 5, 2026 22:52

digantdesai temporarily deployed to cadence November 5, 2026 22:52 — with GitHub Actions Inactive

digantdesai mentioned this pull request Nov 5, 2026

[executorch][native] Record dynamic-shape linears as chained bundled samples #24549

Closed

digantdesai merged commit 7f7ddc4 into main Nov 5, 2026
670 of 672 checks passed

digantdesai deleted the gh/digantdesai/201/orig branch November 5, 2026 23:03

digantdesai temporarily deployed to cadence November 5, 2026 23:12 — with GitHub Actions Inactive

digantdesai temporarily deployed to cadence November 5, 2026 23:13 — with GitHub Actions Inactive

digantdesai temporarily deployed to cadence November 5, 2026 23:15 — with GitHub Actions Inactive

Navigation Menu

Repository navigation

[executorch][native] Land the bundled op sample harness stack - #24698

[executorch][native] Land the bundled op sample harness stack#24698
digantdesai merged 20 commits into
mainfrom
gh/digantdesai/201/orig

digantdesai commented Nov 5, 2026

Uh oh!

pytorch-bot Bot commented Nov 5, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Nov 5, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

digantdesai commented Nov 5, 2026

Uh oh!

Uh oh!

This branch was previously deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

digantdesai commented Nov 5, 2026

Uh oh!

pytorch-bot Bot commented Nov 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/24698

❌ 14 New Failures, 17 Cancelled Jobs, 2 Unrelated Failures, 12 Unclassified Failures

Uh oh!

github-actions Bot commented Nov 5, 2026

This PR needs a release notes: label

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

digantdesai commented Nov 5, 2026

Uh oh!

Uh oh!

This branch was previously deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

pytorch-bot Bot commented Nov 5, 2026 •

edited

Loading

This PR needs a `release notes:` label