MiniMax H3 LoRA training hangs indefinitely at “loading model (nf4)” on RTX 5090 (sm_120)

[refrsher0]

H3 LoRA training consistently hangs after the weight files finish loading. The process never reaches the first training step.
Environment

GPU: NVIDIA GeForce RTX 5090 (compute capability sm_120 / Blackwell)
OS: Windows 11
Inline Studio / Core: current (v1.2.64-era)
PyTorch: 2.13.0+cu130
System RAM: 64 GB (plenty free during the hang)
Model: minimax_h3_fl2va_bf16.safetensors (local)

Steps to reproduce

Start a MiniMax H3 LoRA training run (stills or clips).
Latent caching and caption caching complete normally.
Log reaches:textloading model (nf4)
Loading weights: 100%|██████████| 1058/1058 [...]
Process then sits at “training - 0/1500” / “loading model (nf4)” indefinitely.

Observed behaviour

After “Loading weights: 100%” the process becomes nearly idle:
~0–5 % CPU
0 disk activity
~14 GB RAM used
GPU utilisation near zero

Left running for 30–90+ minutes with no further progress and no error raised.
Reproduced multiple times after clean restarts, numpy reinstall, and bitsandbytes upgrade.

Expected behaviour
Model load + nf4 quantisation/factorisation should complete in roughly 5–15 minutes (as documented), then training steps should begin.
Notes

The hang is specific to the nf4 conversion step of the large H3 transformer.
Generation with H3 works; only the Trainer’s model-load path is affected.
Same symptoms on repeated clean runs, so it does not appear to be a one-off lock or corrupted cache.

Happy to provide full logs or run any diagnostic commands if useful.

[imprsnst]

Hi @refrsher0
Thanks for the report, Can you help me debug this, please share these details:

1. Find the trainer process. It is a separate python from the server, so grab the one running
inline_core.training. In PowerShell while it is hung:

Get-CimInstance Win32_Process -Filter "Name='python.exe'" |
  Select-Object ProcessId, CommandLine | Format-List

2. Dump its stack. This is the one that actually answers it: it shows the exact line it is
sitting on. Run the terminal as Administrator, py-spy needs it to attach on Windows.

pip install py-spy
py-spy dump --pid <the ProcessId from step 1>

If it looks idle in a CUDA call rather than Python, add --native:

py-spy dump --native --pid <ProcessId>

3. bitsandbytes self-test. Training quantises with bitsandbytes while generation uses torchao,
which is why generation works and training does not. This prints which CUDA binary bnb resolved:

cd Inline-Studio\core
.venv\Scripts\python -m bitsandbytes

Plus versions, one line:

.venv\Scripts\python -c "import torch, bitsandbytes; print(torch.__version__, torch.version.cuda, bitsandbytes.__version__)"

The stack dump from step 2 is the important one. Paste all of it, even the parts that look like noise.

[refrsher0]

Hi firstly wanted to say thank you for a great tool you provide...

Here are the requested diagnostics.

Versions

.venv\Scripts\python -c "import torch, bitsandbytes; print(...)"
→ 2.13.0+cu130 13.0 0.50.0

.venv\Scripts\python -m bitsandbytes
→ bitsandbytes v0.50.0
→ PyTorch 2.13.0+cu130, CUDA 13.0, Highest CC (12, 0)
→ Checking that the library is importable and callable... SUCCESS!

Important correction

This does not appear to be stuck in nf4 model load.

The process holding the GPU is:
PID 17224
"C:\Python311\python.exe" -m inline_core.training
C:\Users\web\InlineStudioProjects\mmh3-lora-train.inlinestudio\training_runs\fa3bf681-bdc0-43fd-bd30-64602e72ac55\manifest.json

While “hung”, nvidia-smi showed:

• GPU-Util 100%
• Memory ~31238MiB / 32607MiB
• Compute process: PID 17224 C:\Python311\python.exe

Note: trainer is running via system Python (C:\Python311), not core.venv.
py-spy could not attach to the venv python.exe (“Failed to find python version from target process” / BSS version string failure).

py-spy dump --pid 17224

Thread 25628 (idle): "MainThread"
engine_run_backward (torch\autograd\graph.py:979)
backward (torch\autograd_init.py:395)
backward (torch_tensor.py:623)
backward (accelerate\accelerator.py:2850)
train (inline_core\training\trainer.py:262)
main (inline_core\training_main_.py:58)
(inline_core\training_main_.py:74)
_run_code (:88)
_run_module_as_main (:198)
Thread 15240 (idle): "tqdm_monitor"
wait (threading.py:331)
wait (threading.py:629)
run (tqdm_monitor.py:69)
_bootstrap_inner (threading.py:1045)
_bootstrap (threading.py:1002)
Thread 6580 (idle)

py-spy dump --native --pid 17224

Thread 25628 (idle): "MainThread"
ZwWaitForAlertByThreadId (ntdll.dll)
RtlSleepConditionVariableSRW (ntdll.dll)
SleepConditionVariableSRW (KERNELBASE.dll)
Cnd_wait (msvcp140.dll)
torch::autograd::Engine::is_checkpoint_valid (torch_cpu.dll)
torch::autograd::Engine::thread_main (torch_cpu.dll)
torch::autograd::Engine::execute_with_graph_task (torch_cpu.dll)
...
engine_run_backward (torch\autograd\graph.py:979)
backward (torch\autograd_init.py:395)
backward (torch_tensor.py:623)
backward (accelerate\accelerator.py:2850)
train (inline_core\training\trainer.py:262)
main (inline_core\training_main_.py:58)
...

Thread 6580 (idle)
ZwWaitForAlertByThreadId (ntdll.dll)
...
0x7ffd70839cf0 (nvcuda64.dll)
cuStreamBatchMemOp_v2_ptsz (nvcuda64.dll)
cuFuncGetAttribute (nvcuda64.dll)
cublasLtMatrixTransformDescSetAttribute (cublasLt64_13.dll)
...
cublasLtTSTMatmul (cublasLt64_13.dll)
...
cublasGemmEx (cublas64_13.dll)
at::cuda::blas::gelsBatched<c10::complex > (torch_cuda.dll)
...
at::native::_mm_dtype_out_cuda (torch_cuda.dll)
at::native::structured_mm_out_cuda::impl (torch_cuda.dll)
...
at::_ops::mm::call (torch_cpu.dll)
...
torch::autograd::Engine::evaluate_function (torch_cpu.dll)
torch::autograd::Engine::thread_main (torch_cpu.dll)
torch::autograd::Engine::thread_init (torch_cpu.dll)
...

Summary

UI looked stuck on “loading model (nf4)” / 0/1500, but stack shows it has entered trainer.train() and is blocked in the first backward, specifically in a CUDA matmul (cublasGemmEx / mm_out_cuda), with GPU at 100% and nearly full VRAM.

Happy to collect anything else you need.

[refrsher0]

Just thinking; Possible directions / workarounds from my side:

1. Progress UI mismatch
   Stack shows we’re already in trainer.train(), backward, while UI still looked like “loading model (nf4)” / 0/1500. Might be worth checking status reporting around model load vs first step.

2. Wrong interpreter
   Trainer/server were running as C:\Python311\python.exe, not core.venv\Scripts\python.exe.
   Could the launcher prefer the venv explicitly? Happy to retest if you want a clean venv-only repro.

3. Local workarounds I can try if you think they’re useful

• Cut batch size to 1, lower resolution / frames / clip length
• Lower LoRA rank
• Enable gradient checkpointing / any activation offload if exposed
• Confirm whether a non-nf4 / bf16 / torchao training path exists for H3 (generation already uses torchao; training uses bitsandbytes)

4. Suspected failure point
   Native stack is in first backward CUDA matmul:
   cublasGemmEx / at::native::structured_mm_out_cuda, GPU ~100% and ~31GB/32GB VRAM.
   So this may be an sm_120 cuBLAS / autograd issue on the first step rather than bitsandbytes import/load failure (bnb self-test SUCCESS).

If you have a preferred minimal config for 5090 32GB, I’ll run that next.

[imprsnst]

Great thinking & suggestion, that might solve this.

CPU offload: the dropdown exists but is silently ignored for H3. Our code disables it whenever the
base is quantised, and H3 is always 4-bit, so setting it On does nothing today. That check made
sense for the image models where the base is the big thing, but for H3 clips the base is only
11.7GB and it is the activations that overflow. So the one setting that would have helped you was
switched off. I am fixing it so an explicit on wins.

Minimal config for your 5090, please try this next:

Resolution 512
Clip length 1 (or leave empty to train on stills)
Rank 16
Steps 100 just to see it move

If that steps normally, it is memory pressure. If it still crawls at 512 with stills, then you are right and it is something sm_120 specific, and I will chase cuBLAS next.

[imprsnst]

Published a new release, please update & run training again.
Check details here: https://github.com/inlineresearch/Inline-Studio/releases/tag/v1.2.66

To update:

cd core && uv pip install --python .venv/bin/python -e . --no-deps

Let me know if this works.