OpenVINO: LeRobot policies that do not run yet, and what blocks them #24325

We exported the LeRobot policies to ExecuTorch and ran them with the OpenVINO and TensorRT backends. This issue lists what still keeps policies from working on the CPU backends (OpenVINO, XNNPACK, Qualcomm, Core ML, MLX), so it can be picked up. The GPU side (CUDA and TensorRT) is tracked in #24353.

Asks by party

• ExecuTorch (OpenVINO, XNNPACK, portable kernels, Core ML, Qualcomm): review the open fixes below and pick the merged ones into release/1.6. Several gaps have no fix or owner yet, listed per backend.
• OpenVINO: items 1 to 5, 7, 8 and 11 below are in OpenVINO's PyTorch frontend or CPU plugin. Items 1 and 8 are worked around in ExecuTorch for now, but the real fix is in OpenVINO. Items 6 and 10 are in ExecuTorch's OpenVINO backend. Item 9 is not narrowed yet.
• coremltools: 3 conversion bugs, worked around in ExecuTorch for now.
• LeRobot: 2 policy options and 3 checkpoint or tolerance gaps, listed at the end.

Results

These numbers are from Oct 31, on an older stack. A refresh on the current stack is in progress. Since then, groot, lawam and vla_jepa (item 1) and eo1 (item 8) pass on OpenVINO; their new numbers are not in the table yet.

Milliseconds per action chunk, batch size 1. Every number matches eager PyTorch on the same weights, except the 2 marked * and eo1 on OpenVINO (item 8).

• OpenVINO: Intel Xeon Platinum 8339HC server CPU, 16 threads, OpenVINO 2026.4.1, with the fixes in Fix 0-d tensors, dynamic output sizes and integer reductions, and default to float32 on CPU, in the OpenVINO backend #24294.
• TensorRT: NVIDIA H100, Jetson AGX Thor, Jetson Orin Nano 8 GB.
• On the Orin Nano, pi0 and xvla use a low-memory export, and pi05 and groot are split into several programs.
• - does not export or run yet (fastwam on Jetson Thor is not measured yet). x does not fit in 8 GB.

* tdmpc is sensitive to bf16 (eager PyTorch in bf16 also differs). pi0_fast in fp32 matches a float32 eager reference.

Already fixed or in review

• OpenVINO
  • Fix crashes, wrong outputs and lowering failures in several backends and in exir #24222 (merged; release/1.6 pick Fix crashes, wrong outputs and lowering failures in several backends and in exir #24298 open): crashes, bool-mask and bf16 inputs, stray assert nodes
  • Fix 0-d tensors, dynamic output sizes and integer reductions, and default to float32 on CPU, in the OpenVINO backend #24294 (merged; release/1.6 pick Fix 0-d tensors, dynamic output sizes and integer reductions, and default to float32 on CPU, in the OpenVINO backend #24377 open): 0-d tensors, float32 by default, data-dependent sizes
  • OpenVINO: fix bool min/max and masked read lowering, and follow-ups to #24294 #24396 (merged; release/1.6 pick OpenVINO: fix bool min/max and masked read lowering, and follow-ups to #24294 #24446 open): min, max, clamp and sign on bool inputs (gaussian_actor), masked reads (works around item 1), follow-ups to Fix 0-d tensors, dynamic output sizes and integer reductions, and default to float32 on CPU, in the OpenVINO backend #24294. It also makes the OpenVINO test runner fail when a test fails. Before, it printed the failures but exited with success, so CI stayed green.
  • OpenVINO: fix wrong outputs when a partition returns a copy of its input #24466 (merged; release/1.6 pick requested): a copy of a partition input that the partition returns now runs on the CPU (works around item 8, eo1)
• Portable kernels and the export flow
  • Keep Half and BFloat16 sums in float in portable matmul, var, cumsum and pooling kernels, and fix portable cat #24401 (merged; pick Keep Half and BFloat16 sums in float in portable matmul, var, cumsum and pooling kernels, and fix portable cat #24443 merged): Half and BFloat16 sums kept in float (smolvla, groot)
  • Add a portable kernel for antialiased bicubic resize #24400 (merged; release/1.6 pick Add a portable kernel for antialiased bicubic resize #24447 open): antialiased bicubic resize (vla_jepa, wall_x)
  • Keep ops above 6 dims out of XNNPACK and fix quantized ops it leaves out #24380 (merged; pick Keep ops above 6 dims out of XNNPACK and fix quantized ops it leaves out #24445 open): ops above 6 dimensions kept out of XNNPACK
  • Fix mixed-dtype cat and stack in to_edge and the edge check #24381 (merged; pick Fix mixed-dtype cat and stack in to_edge and the edge check #24450 merged): mixed-dtype cat and stack
  • Support 3-D convolution in the portable convolution kernel #24385 (merged; release/1.6 pick Support 3-D convolution in the portable convolution kernel #24460 open): 3-D convolution in the portable kernels (lawam, lingbot_va)
  • Fix export of zero-stride constants and inputs, and of constants that delegates share #24404: zero-stride constants and constants that delegates share (wall_oss_05)
  • Limit the default thread pool to the allowed CPUs on Linux (not Android) #24395 (merged; pick Limit the default thread pool to the allowed CPUs on Linux (not Android) #24449 merged): default thread pool sized to the allowed CPUs on Linux
• XNNPACK
  • XNNPACK: fix enable_bf16 load failures and BatchNorm folding into a 3-D linear output #24464 (merged; release/1.6 pick XNNPACK: fix enable_bf16 load failures and BatchNorm folding into a 3-D linear output #24477 open): with enable_bf16=True, bf16 programs load: ops XNNPACK cannot run in bf16 stay on the portable kernels, and a bf16 bmm computes in fp32 and converts the result to bf16 (smolvla, groot, pi05, vla_jepa, evo1)
• Core ML
  • Core ML: run ops that read uint8 and int8 inputs on the CPU, refuse ops coremltools gets wrong, and fix fills, tanh GELU, RMSNorm and repeated outputs #24317: uint8 and int8 readers on the CPU, ops coremltools gets wrong, fills, tanh GELU, RMSNorm, repeated outputs (fixes the xvla compile)
  • Fix wrong dtypes and values in Core ML conversion #24424: wrong dtypes and values in conversion
  • Refuse ops coremltools cannot convert in the Core ML partitioner, and say why #24425: refuse ops coremltools cannot convert, and say why
• Qualcomm
  • Qualcomm AI Engine Direct - Keep node names unique across the QNN and Samsung passes #24316 (merged): node names unique across the QNN and Samsung passes
  • Qualcomm AI Engine Direct - Decompose kept-whole ops that HTP leaves on CPU #24429: decompose kept-whole ops that HTP leaves on the CPU

What blocks OpenVINO

1. Boolean-mask indexing does not convert (vla_jepa, lawam)

Worked around in ExecuTorch. Conversion stops with Dynamic rank for aten::index input is not supported. Both policies use the Qwen3-VL DeepStack code in transformers, which writes and then reads the same masked rows on several layers in a row (_deepstack_process):

local_this = hidden_states[visual_pos_masks, :] + visual_embeds
hidden_states[visual_pos_masks, :] = local_this

On the second layer, the boolean mask reaches aten.index with a dynamic rank. The cause is in the FX decoder: TorchFXPythonDecoder.get_shape_for_value reads only node.meta["tensor_meta"], which is None after ExecuTorch's to_edge, and does not fall back to node.meta["val"] as get_found_shape already does. A fallback to meta["val"] fixes it; a small Qwen3-VL text model with DeepStack inputs then converts and matches PyTorch.

ExecuTorch now fills tensor_meta before conversion (#24396), so both convert. With it, lawam and vla_jepa pass LeRobot's export check on OpenVINO. The fallback in the FX decoder is still the right fix, so that other callers do not hit the error.

2. cumsum over a bool tensor wraps after 255 (pi0, pi0.5, pi0_fast)

Partly fixed. The pi0 family builds position ids with a cumulative sum over a bool padding mask (pi0, pi0.5):

position_ids = torch.cumsum(pad_masks, dim=1) - 1

PyTorch returns int64 here. The OpenVINO frontend keeps the narrow input type for the accumulation, so the count wraps after 255 and prompts longer than 255 tokens get wrong position ids. ExecuTorch now casts the input of cumsum and prod to int64 (#24294), but aten::cumsum over bool or integer inputs, and aten::prod over integer inputs, should follow PyTorch's type promotion in the frontend itself.

3. int64 reductions are computed in 32 bits on the CPU plugin

Even with the int64 cast from item 2, the CPU plugin of OpenVINO 2026.4.1 computes prod, cumsum and sum in 32 bits, so a result past the int32 range comes back wrong with no error. For example, prod of the int32 tensor [50000, 50000], cast to int64 first, returns -1794967296 instead of 2500000000.

4. Integer cumsum over constants is wrong on the CPU plugin

Partly fixed. Masks built from an int64 or int32 cumsum over a constant can come back wrong on CPU from the second inference on: they keep only what earlier calls allowed. We saw this on 2025.4.1, 2026.0.0 and 2026.4.1, but not on the nightly. The pi0 family builds its masks this way. ExecuTorch now computes these cumsums during lowering (#24294). Without that step, pi0.5 stopped matching eager PyTorch after a few action chunks. With it, pi0.5 matches on 16 action chunks in a row. The plugin bug remains in released versions.

5. HETERO devices

Lowering for HETERO:CPU crashes in export_model(), and for AUTO:CPU the same call fails with Not Implemented. Also, OpenVINO rejects set_property on a HETERO: device string (2025.1.0 to 2026.4.1), which is how the ExecuTorch runtime applies inference_precision. Setting the hint on each listed device works.

6. A failed run gives no reason

Partly fixed. Without #24294, molmoact2 lowered to 2 OpenVINO partitions plus CPU ops, then the first delegate call failed with CALL_DELEGATE execute failed at instruction 0: 0x1. With #24294 it runs and matches eager PyTorch. Failures in OpenvinoBackend::execute still do not log OpenVINO's own message (ov_get_last_err_msg()), and some, such as a failure to set an input or output tensor, return with no log at all.

7. Lowering is slow for large models (wall_x, flux3, lingbot_vla_v2)

wall_x now finishes, but lowering alone takes about 8.3 hours on this CPU, with other jobs sharing it (a 61,000-node graph, 12 OpenVINO partitions). lingbot_vla_v2 exports, but OpenVINO lowering does not finish in 12 hours in either precision. flux3 lowers for about 9 hours and then fails in conversion. AtenIndexPutReplacer fails on an aten.index_put whose indices are a list of tensors: while building the index it adds an int64 tensor to an int32 one, and OpenVINO rejects the mixed types:

While validating node 'opset1::Add Add_18585 (opset1::Add Add_18584[0]:i64[1,1,1,1],
  opset1::Unsqueeze aten_unsqueeze_copy_default_23/...[0]:i32[128,1]) -> (dynamic[...])'

Part of the time is ExecuTorch partitioning, which slows down sharply for very large graphs. A profile of the OpenVINO conversion step would show how much is on the OpenVINO side.

8. An output that is just a model input is not written (eo1)

Worked around in ExecuTorch. eo1 missed its accuracy check on OpenVINO: 2.06 against a tolerance of 0.2. The cause is in the CPU plugin. Some model outputs are just a model input, passed through (a Result fed directly by a Parameter). If the caller sets its own output tensors with set_output_tensor() after setting the inputs, the plugin does not write such an output. If the same input also feeds other nodes, the plugin overwrites the caller's input buffer with the old contents of that output buffer, so those results are wrong too.

The PyTorch frontend creates this shape by itself: it converts aten.clone and aten.alias_copy into nothing, so a partition that returns x.clone() returns its input directly. eo1 has one such copy. Plain OpenVINO, no ExecuTorch:

import numpy as np
import openvino as ov
import openvino.opset13 as ops

x = ops.parameter([2, 3], ov.Type.f32, name="x")
y = ops.parameter([2, 3], ov.Type.f32, name="y")
# Output 0 is input x itself. Output 1 is computed from x, output 2 from y.
model = ov.Model([ops.result(x), ops.result(ops.sin(x)), ops.result(ops.sin(y))], [x, y])
request = ov.Core().compile_model(model, "CPU").create_infer_request()

xv, yv = np.ones((2, 3), np.float32), np.ones((2, 3), np.float32)
outs = [np.full((2, 3), -7.0, np.float32) for _ in range(3)]
request.set_input_tensor(0, ov.Tensor(xv, shared_memory=True))
request.set_input_tensor(1, ov.Tensor(yv, shared_memory=True))
for i, out in enumerate(outs):
    request.set_output_tensor(i, ov.Tensor(out, shared_memory=True))
request.infer()
print(xv[0], outs[0][0], outs[1][0], outs[2][0])

On 2026.4.1, x and output 0 both hold -7, and output 1 is sin(-7) = -0.657 instead of sin(1) = 0.841. Only output 2 is right. We see the same in 2025.2.0, 2025.4.1, 2026.2.1 and the 2026.5 nightly. Setting the output tensors before the input tensors avoids it, and so does a same-type Convert between the Parameter and the Result.

ExecuTorch now leaves such copies out of the partition, so its own CPU kernel makes the copy (#24466). With it, eo1 passes on OpenVINO in LeRobot (largest difference 0.063, tolerance 0.2; 1.44 times faster than eager PyTorch). The real fix belongs in the CPU plugin: an output fed by a Parameter should be written whatever order the tensors are set in. An OpenVINO maintainer reproduced it and is taking it to the core OpenVINO team.

9. bf16 constants are rounded differently

A bf16 constant can come back rounded differently from PyTorch. Small effect, low priority.

10. Partition inputs and outputs ignore the dim order (ExecuTorch side)

ExecuTorch's OpenVINO runtime gives OpenVINO only the sizes of each partition input and output, not the dim order. So OpenVINO reads and writes them in the default (contiguous) order, and a channels-last tensor at a partition edge gets wrong values, with no error. Two examples, lowered with OpenvinoPartitioner for CPU:

x = torch.randn(2, 3, 4, 5)


class Output(torch.nn.Module):  # run with x: max abs error 8.87
    def forward(self, x):
        return (x * 2).to(memory_format=torch.channels_last)


class Input(torch.nn.Module):  # run with x.to(memory_format=torch.channels_last): max abs error 0.29
    def forward(self, x):
        return x.mean(dim=(2, 3))

Some ops hide it: x * 2 on a channels-last input gives a channels-last output, so the wrong order going in and the wrong order coming out cancel out. The fix is in ExecuTorch's OpenVINO backend: tell OpenVINO the dim order of each partition input and output (for example with a Transpose at the partition edge), or reorder the data in the runtime. An OpenVINO maintainer offered to fix it in a separate pull request. No LeRobot policy hits it today: none of the 99 OpenVINO programs we checked (21 policies) has a partition input or output in a non-default dim order.

11. A partition that returns a value and a copy of it does not convert

Conversion fails with:

Check 'is_number(name)' failed at src/frontends/pytorch/src/translate_session.cpp:434

The PyTorch frontend converts aten.clone and aten.alias_copy into nothing (as in item 8), so both outputs become the same OpenVINO tensor. With s = torch.sin(x) inside the partition:

The fix belongs in the PyTorch frontend: accept 2 different outputs that end up as the same OpenVINO tensor, as it already does for s, s. ExecuTorch could also leave such copies out of the partition, as #24466 does for copies of inputs. No LeRobot policy hits it today.

What blocks XNNPACK

• bf16 policies: by default a bf16 policy's matmuls run on ExecuTorch's single-threaded portable kernels, and one action chunk takes minutes. With XNNPACK: fix enable_bf16 load failures and BatchNorm folding into a 3-D linear output #24464 and enable_bf16=True, smolvla, groot, pi05, vla_jepa and evo1 pass their checks and take 8.8 s, 6.3 s, 13.3 s, 6.6 s and 33.1 s per chunk on the same Xeon (before, one chunk did not finish in 15 minutes). That is still 2 to 5 times slower than eager PyTorch: no matmul is left on the portable kernels, but convolutions and other bf16 ops still are. LeRobot's export also needs to pass enable_bf16=True to the XNNPACK partitioner.
• Some float32 bundles are slower than eager on the same 16 cores: xvla 0.8x, fineart_vla 0.7x, pi0_fast 0.4x. No owner yet.
• lawam: the bundle fails at run time inside the delegate: Propagating input shapes failed.
• A long Method.execute keeps the Python GIL, so other Python threads in the same process stall while it runs (seen with 2 XNNPACK bundles in one process). No owner yet.

What blocks Qualcomm (export only, checked on the x86 HTP emulator)

• smolvla, eo1, vla_jepa: the recomposed RMSNorm drops its bf16 cast, so lowering fails (eo1: masked_scatter gets bfloat16 and float32; vla_jepa: Missing out variants: {'aten::rms_norm'}). Qualcomm AI Engine Direct - Keep the dtype of a recomposed RMSNorm #24428 fixed it but was closed without merging. No owner yet.
• smolvla: with the Qualcomm AI Engine Direct - Keep the dtype of a recomposed RMSNorm #24428 change every program compiled, but the bundle missed its check on the emulator (2.55 against 0.2). Being narrowed.
• groot, pi05: loading fails with Can't create context from binary. Error 1002. This is the HTP memory limit, not an ExecuTorch bug: one session has about 3.75 GB of usable address space, and these policies have about 6 GB of fp16 weights. They need the large parts quantized (16a8w or 16a4w), or fewer programs loaded at once.
• wall_oss_05: after a 49 minute compile, the int64-to-int32 pass leaves an index with an invalid type. No small repro yet.
• fineart_vla: with the Qualcomm AI Engine Direct - Keep the dtype of a recomposed RMSNorm #24428 change and Qualcomm AI Engine Direct - Decompose kept-whole ops that HTP leaves on CPU #24429, the int64 ops were left on the CPU. The HTP compile then needs about 5.2 GB, over the 3.7 GB limit, so like groot and pi05 it needs the large parts quantized.

What blocks Core ML

• gaussian_actor: when a partition returns a tensor next to a same-shape expand of it, Core ML hands the tensor back as zeros on the CPU and GPU compute units. The fix is in the next Core ML: run ops that read uint8 and int8 inputs on the CPU, refuse ops coremltools gets wrong, and fix fills, tanh GELU, RMSNorm and repeated outputs #24317 update.
• pi0 (float32): a Core ML delegate over 1 GiB is copied and hashed inside to_executorch, which costs about 3 times its size in memory, so the export needs about 61 GiB.
• wall_oss_05: the Core ML partitioner's pre-pass repeats the partition search and stalls (85 minutes with no conversion started).
• vla_jepa (float32): misses its check by 5e-3 on both the CPU and the CPU-and-GPU compute units, and crashes on the "all" compute unit. Not narrowed yet.
• coremltools: an index_put with a whole slice and a multi-entry index, rms_norm exported without its eps argument, and an integer full with no dtype are converted wrongly. Core ML: run ops that read uint8 and int8 inputs on the CPU, refuse ops coremltools gets wrong, and fix fills, tanh GELU, RMSNorm and repeated outputs #24317 keeps these on the CPU for now.

What blocks MLX

• eo1, wall_oss_05: .numpy() not supported for tensor subclasses. Seen on an older stack; needs a run on the current one.

Outside the backends

• dm05, wall_x, molmoact2: stock torch 2.15 cannot save or load these programs; the PyTorch fixes are tracked in TensorRT: LeRobot policies that do not run yet, and what blocks them #24353.
• fastwam: the published lerobot/fastwam_base checkpoint has no action head, so it does not load for export. A fix for its CPU float32 layers is ready but waits behind LeRobot's open pull request limit.
• lawam: fix(lawam): let the VLM-to-LAM head take bfloat16 hidden states on the CPU huggingface/lerobot#5214 lets its head take bf16 on the CPU. The OpenVINO blocker was then item 1, which OpenVINO: fix bool min/max and masked read lowering, and follow-ups to #24294 #24396 works around.
• lingbot_va: it exports now, but the bf16 check fails on the CPU for the same reason as on the GPU (the check is stricter than rounding noise, see TensorRT: LeRobot policies that do not run yet, and what blocks them #24353). In float32 the check passes, and the export then stops in ExecuTorch's edge verifier.
• flux3: the published so101 checkpoint uses history conditioning, which the export does not serve yet. fix(flux3): hand F3_NATTEN_BACKEND to NATTEN's kernels huggingface/lerobot#5213 and a queued pull request handle the NATTEN kernels.
• smolvla: its VLM always loads in bf16 (no dtype option), so on Core ML it runs on portable kernels, and on XNNPACK it needs enable_bf16=True (XNNPACK: fix enable_bf16 load failures and BatchNorm folding into a 3-D linear output #24464).
• pi0 on XNNPACK (float32): the check gives 2.10e-05 against a tolerance of 2e-05.
• tdmpc: exports on OpenVINO and XNNPACK, but lerobot-rollout cannot feed it on the test robot yet.

cc @suryasidd @daniil-lyakhov @anzr299