MiniMax-H3: float64 rotary position grid cannot be moved to MPS (`Cannot convert a MPS Tensor to float64`)

[4suino]

Describe the bug

Both MiniMax-H3 layout blocks build their rotary position grid in float64 on CPU (deliberately — the grid has to reproduce the released model's fp64 coordinates exactly) and then move it to the execution device with a plain .to(device):

• MiniMaxH3PrepareLayoutStep, before_denoise.py line 444 (v0.40.0): block_state.position_ids = position_ids.to(device)
• MiniMaxH3Ref2VAPrepareLayoutStep, line 768: same statement

Metal has no float64, so on MPS every workflow (t2va, fl2va, ref2va) dies at that line before the first denoising step. No MPS-specific handling exists anywhere on that path.

Reproduction

Reproduction

The failing operation in isolation (this is exactly what the layout step does):

import torch
position_ids = torch.zeros(8, 3, dtype=torch.float64)   # what MiniMaxH3PrepareLayoutStep builds
position_ids.to("mps")

With the real pipeline: MiniMaxH3ModularPipeline on device="mps", any workflow, fails inside prepare_layout with the same error.

Logs

TypeError: Cannot convert a MPS Tensor to float64 dtype as the MPS framework doesn't support float64. Please use float32 instead.

Proposed fix

Keep the fp64 computation untouched and drop the dtype only at the device transfer, only for MPS (two sites):

# before_denoise.py, MiniMaxH3PrepareLayoutStep and MiniMaxH3Ref2VAPrepareLayoutStep
if device is not None and torch.device(device).type == "mps":
    block_state.position_ids = position_ids.to(device, dtype=torch.float32)
else:
    block_state.position_ids = position_ids.to(device)

Precision impact, measured on the grids the code actually produces (_spatial_position_grid, _temporal_position_grid, audio rows) for a large layout — 64×64 latent frame with patch 2 (1024×1024 output), 102 latent frames (~15 s), 4000 audio latents:

grid	max |coord|	max abs error after fp32 cast
spatial h / w	31.0	0 (exact)
temporal	568.3	2.0e-5
audio time	4567.3	1.6e-4

Worst case is 1.6e-4 rad of rotary phase at the highest frequency (inv_freq = 1), about 24× below the bf16 precision (3.9e-3) the transformer itself runs in. With this patch applied locally, all three workflows run end to end on an M5 Max (t2va, fl2va with keyframes, ref2va with up to 12 references), outputs reproducible across runs. Happy to open a PR if this shape of fix is acceptable.

Logs

System Info

• 🤗 Diffusers version: 0.40.0 (sites verified against the v0.40.0 tag; also present on main)
• Platform: macOS-26.6.2-arm64-arm-64bit-Mach-O
• Python version: 3.14.5
• PyTorch version (GPU?): 2.14.0.dev20260808 (MPS)
• Transformers version: 5.14.1
• Accelerate version: 1.14.0
• Accelerator: Apple M5 Max, 128 GB unified memory
• Using GPU in script?: yes (MPS)
• Using distributed or parallel set-up in script?: no

Who can help?

@apolinario (MiniMax-H3 integration, #14355)