feat: add native Intel XPU (torch.xpu) device support

[LexiconCode]

Summary

Native Intel XPU (torch.xpu) support for Arc / Battlemage GPUs.

Disclaimer: AI was used. However you can expect mutual respect for InvokeAI and person to person communication. I would appreciate feedback.

I've tried to keep this pr minimal. Most of the changes are for the new xpu back-end or memory probe — CUDA (incl. ROCm), MPS, and CPU paths are unchanged.

• xpu / xpu:N device selection, fp16 default
• VRAM detection via torch.xpu.mem_get_info(), with a fallback for setups missing the SYCL free-memory aspect (e.g. GPU passthrough VMs)
• FP8 layerwise casting enabled via a cached runtime probe (fp8 storage + upcast; no fp8 matmul needed)
• VAE auto-tiling, partial loading, OOM handling, and VRAM stats on XPU
• [xpu] torch 2.13.0+
• idle-GPU offload fix
• Mock-based XPU tests mirroring the CUDA/MPS suites

Known limitations:

• On the tested stack, VRAM exhaustion overcommits into host RAM and hangs rather than raising torch.OutOfMemoryError; the preflight mem_get_info budgeting is the operative defense.
• BitsAndBytes NF4/INT8 both work on XPU (bitsandbytes supports Arc officially; verified on the B70 — NF4 needs the Level-Zero dev headers for its SYCL JIT). InvokeAI's own bnb wrappers still hardcode .cuda(), so enabling them is a small follow-up; until then GGUF and fp8 are the in-app quantized paths. However I'm unsure of the downstream effects of upgrading bitsandbytes on other back-ends.
• Intel Arc Alchemist (A-series) has no native FP64. Battle Mage has FP64 hardware acceleration
• Would benefit from testing on Windows and other Intel graphics cards besides B70 pro.

Related branches: fix/fp8-dequant-bf16 (fp8 checkpoint load-RAM fix), fix/flux-diffusers-vae (FLUX VAE classification fix).

Related Issues / Discussions

• [enhancement]: Support for OpenVINO Backend for Intel oneAPI/oneDNN #2751 (OpenVINO backend request — this PR takes the native torch.xpu route instead, which needs no separate runtime; ONNX/OpenVINO could complement it later)

QA Instructions

Verified end to end on an Intel Arc Pro B70 (torch 2.7.1+xpu, Linux):

• txt2img across SD1.5, SDXL, FLUX.1 (fp8 + GGUF), FLUX.2 (fp8 + GGUF), SD3.5, Z-Image, and CogView4
• Wan 2.2 video generation (TI2V-5B), including the XPU VAE-tiling path on an oversized decode
• fp8 layerwise casting verified on FLUX.1-dev (transformer resident at 11.3GB vs 22.7GB bf16)
• Benchmarks land 1.3–1.7× behind an RTX 4090 on the same models/settings

To reproduce on Arc hardware: pip install invokeai[xpu], leave device: auto, generate. On any machine: pytest tests/backend/util/test_devices.py (44 tests, no GPU required).

Merge Plan

No special care needed. uv.lock is re-resolved for the [xpu] extra (additive); openapi.json/schema.ts regenerated for the device field values.

WHL file for Windows or Linux

invokeai-6.14.0a0-py3-none-any.zip Updated 8/2/2026

uv venv --python 3.12 invoke
invoke\Scripts\activate
uv pip install "invokeai-6.14.0a0-py3-none-any.whl[xpu]" --extra-index-url https://download.pytorch.org/whl/xpu --index-strategy unsafe-best-match
invokeai-web

Checklist

• The PR has a short but descriptive title, suitable for a changelog
• Tests added / updated (if applicable)
• ❗Changes to a redux slice have a corresponding migration (n/a — no slice changes)
• Documentation added / updated (if applicable) (happy to add an Intel install docs section if desired)
• Updated What's New copy (if doing a release after this PR)

[lstein]

Thanks for this — it's a genuinely well-scoped contribution. I ran an adversarial review (the goal being to break it rather than to evaluate it), and I want to lead with what held up, because a lot did:

• No CUDA / MPS / CPU regression found. Every refactor I traced is behaviour-preserving on the existing backends: the wan_latents_to_video restructure into total_vram: int | None is identical for CUDA, CPU, MPS and cpu_only VAEs; torch.cuda.OutOfMemoryError → torch.OutOfMemoryError is a no-op (they're the same object); the model_cache.py device-label change still yields "cuda device" for an index-less CUDA device.
• Packaging is clean. uv export --frozen --extra {cpu,cuda,rocm} is byte-identical before and after, except one added comment line. The 958-line uv.lock churn is entirely conflict-marker rewriting from adding xpu to the conflicts group — zero removals, zero version changes on shared packages. uv lock --check passes under both current uv and the CI-pinned 0.6.10.
• The partial-loading machinery really is device-agnostic. I read cached_model_with_partial_load.py and all of torch_module_autocast/ looking for CUDA assumptions now reachable via running_with_dedicated_vram — no pinned memory, no streams, no .cuda(), no dtype assumptions. Your call to enable it on XPU is sound.
• Your thread-locality comment is correct. I checked c10/xpu/XPUFunctions.cpp in v2.7.1: thread_local DeviceIndex curDeviceIndex. The multi-worker pinning is sound, and I couldn't construct an interleaving where normalize("xpu") resolves to another worker's GPU.
• The test suite is thorough and passes locally (3859 passed, 119 skipped, 8 xfailed).

Requesting changes for the CI failures and a handful of real defects below.

---

1. CI is red on four checks (all mechanical)

invokeai/app/services/invocation_stats/invocation_stats_default.py:98 — the new ternary is ~158 chars; uv tool run ruff@0.11.2 format --check . rewrites it. This is the python-checks failure.

invokeai/app/services/events/events_common.py:143 — the source still reads "...(set only when running on a CUDA GPU)", but the committed openapi.json:45725 and schema.ts:18420 read "...on a GPU". It looks like the generated artifacts picked up an edit that didn't land in the Python. That single line is the entire openapi-checks and typegen-checks failure — fix the description in events_common.py and regenerate.

docs/src/generated/settings.json — not regenerated after the device / generation_devices description changes in config_default.py. This is the check-and-build failure (pnpm run generate-docs-data).

---

2. The XPU mem_get_info fallback feeds a formula that assumes different semantics

TorchDevice.xpu_mem_get_info falls back to total_memory - torch.xpu.memory_reserved(device). But memory_reserved() counts only this process's caching-allocator blocks, whereas torch.cuda.mem_get_info's free is a driver-global query. ModelCache._get_vram_available then applies the CUDA formula to both:

vram_available_to_process = vram_free + vram_allocated

which is only sound when vram_free is driver-global. The fallback is therefore high by (other processes + the Level-Zero context + compiled kernel binaries) — on Intel that's routinely 0.5–1.5 GB, and the Arc is usually also the display GPU.

Concrete trigger, using the exact scenario the docstring cites as the fallback's reason to exist — Arc 16 GiB in a passthrough VM, ~2 GiB held by the compositor + L0 context, fresh process, default device_working_mem_gb = 3:

1. Fallback reports free = 16 GiB → vram_available_to_process = 16 GiB
2. _load_locked_model sees 13 GiB available and fully loads a 12 GiB transformer
3. Real free is now ~2 GiB against a 3 GiB working-memory contract → OOM on the first denoise step
4. The cache still believes it has room, so _offload_unlocked_models frees nothing on retry

The docstring's "budget hint rather than a guarantee" is fair, but the caller treats it as a hard budget. The same root cause hits attention.py:28 (auto_detect_slice_size picks "balanced" where it should pick "max").

Sub-case that's worse: if get_device_properties also fails, the function returns (0, 0). Work the arithmetic through _get_vram_available:

vram_available_to_process = 0 + allocated
vram_total_available_to_cache = allocated - working_mem
vram_cur_available_to_cache  = allocated - working_mem - _get_vram_in_use()   # == allocated
                             = -working_mem      # -3 GiB, forever, regardless of device state

Every subsequent lock() then evicts all unlocked models, unloads the model being locked, and calls partial_load_to_vram(-3221225472) — so everything runs permanently with per-layer CPU→GPU autocast and no error is surfaced. Meanwhile _calc_ram_available_to_model_cache takes the total or None branch and sizes a healthy-looking 32 GB RAM cache, so the logs look fine. Note this doesn't need a probe failure: max(total - reserved, 0) produces the same lock-in whenever reserved > total.

Minor, same function: the except (RuntimeError, AttributeError) doesn't catch what _lazy_init actually raises —

>>> TorchDevice.xpu_mem_get_info(torch.device('xpu', 0))
AssertionError: Torch not compiled with XPU enabled

Same for get_device_name at devices.py:158. Reaching it needs an XPU execution device on a non-XPU torch, which get_generation_devices rejects at startup — but the clause doesn't match its own documented failure mode. (Also: torch.xpu.mem_get_info does exist in 2.7.1, so the AttributeError arm is dead weight rather than a bug.)

---

3. The FP8 probe runs on the wrong device during idle-GPU offload

Three things compound here:

1. _device_supports_fp8_storage(...) is the first statement in _should_use_fp8, before every exclusion — so it fires on the very first model load of any kind (tokenizer, VAE, anything).
2. torch.zeros(2, device="xpu") is index-less, so it resolves through the thread's current XPU device.
3. _maybe_offload_to_idle_gpu re-pins only InvokeAI's thread-local via TorchDevice.set_session_device(borrowed_device) — it never calls torch.xpu.set_device, unlike worker startup at session_processor_default.py:631-636, which does both.

So with generation_devices: [xpu:0, xpu:1] and offload_text_encoders_to_idle_gpus on (the default):

1. Worker A is pinned to xpu:0 (torch.xpu.set_device(xpu:0) on that thread)
2. A hits a CompelInvocation (idle_gpu_offloadable=True) and borrows xpu:1
3. The ModelLoader's _torch_device is xpu:1
4. The probe allocates on xpu:0 — the busy denoise GPU the offload existed to protect — and xpu:1's answer is inferred from xpu:0

load_model is also reachable from API/install threads, where the probe forces XPU lazy SYCL init on the FastAPI thread.

Two related issues in the same helper:

• @lru_cache is keyed on the device type, not the device. Your own docstring says float8 support is build/driver dependent and "emerging" on Xe2 — that's a per-device property. On a discrete Arc + Iris Xe iGPU, whichever device probes first decides for both; on the wrong one, _apply_fp8_to_nn_module casts every Linear/Conv weight and the pre-hook upcast raises at forward time.
• A transient failure is memoized forever, silently. except Exception: return False with maxsize=None, no logging, and cache_clear() is never called outside tests. The probe fires during a model load — i.e. exactly when the device may be full — so torch.OutOfMemoryError (a RuntimeError subclass), a UR/L0 transient, or "Cannot re-initialize XPU in forked subprocess" all permanently disable FP8 for the process. The only symptom is 2× VRAM and thrashing; the only cure is a restart. At minimum: log the exception, and don't cache failures.

Lastly, the probe doesn't exercise the runtime path. It tests float32 → float8_e4m3fn → float16 on XPU, but at runtime the storage cast happens on CPU (params are still CPU-resident at load time), and the XPU-side operations are the fp8 host→device copy plus the pre-hook's fp8 → bf16 upcast (compute_dtype is typically bf16 for Krea-2/FLUX). A build where fp8→fp16 works but fp8→bf16 doesn't would probe True and then fail at forward.

---

4. try_borrow lost its device-type filter

device_pool.py:99 checks only exclude.type not in _OFFLOAD_DEVICE_TYPES, then takes any registered key != exclude_key. Before this PR only CUDA devices could ever be registered, so cross-type borrowing was structurally impossible; now both types share one pool.

generation_devices: ["cuda:0", "xpu:0"] is accepted by both validators (config_default.py:285, app_info.py:123) and by TorchDevice.get_generation_devices, so a cuda:0 session would be handed xpu:0 for its text encoder. Reachability is low (the pinned +cu128 / +xpu wheels are mutually exclusive, so this needs a custom build), but the fix is one line — filter candidates by key.startswith(exclude.type).

---

5. Design question: Intel integrated GPUs

Unlike CUDA, where every enumerated device is a discrete accelerator, Level Zero enumerates the CPU's iGPU alongside any discrete card (and on Data Center GPU Max, each tile separately unless ZE_FLAT_DEVICE_HIERARCHY=COMPOSITE). Two consequences:

• devices.py:220-221 — _all_available_devices takes range(torch.xpu.device_count()) with no filter, so on the mainstream Arc configuration (iGPU + discrete Arc) generation_devices: auto resolves to [xpu:0, xpu:1] and dispatches half the queue to the iGPU. _OFFLOAD_DEVICE_TYPES also makes it a borrow target, so try_borrow may hand it an 8.9 GB text encoder.
• model_cache.py:599 — running_with_dedicated_vram = type in ("cuda", "xpu") treats an iGPU as having dedicated VRAM. The comment three lines above states the policy MPS is excluded for — "memory is shared with the CPU" — which is exactly an iGPU's topology, yet it takes the elif "xpu" branch at :1103 rather than the MPS branch at :1107 that deliberately uses psutil.virtual_memory().available.

On, say, a 16 GB Lunar Lake laptop: get_device_properties(igpu).total_memory reports a large share of system RAM, so heuristic 2 sizes the RAM cache from it and _get_vram_available budgets the same DRAM again — doubled once more by keep_ram_copy_of_weights=True. Heuristic 2's stated intent ("cap the RAM cache at 1× VRAM") is meaningless when "VRAM" is the RAM being capped.

I don't think torch exposes a clean is_integrated flag (I checked XPUDeviceProp.h — there's architecture and gpu_eu_count, but nothing direct), so this may be a name/arch heuristic or simply a documented ZE_AFFINITY_MASK note. Happy to hear what you think is right; I'd rather not guess for you.

---

6. Smaller items

• pins.json has no xpu entry, so the launcher will have no XPU install option — users would be stuck with the manual pip install invokeai[xpu] from your description. Worth adding alongside the extra.
• vae_working_memory.py:207 (untouched by this PR) is the only estimator that branches on backend, and it branches is_rocm = torch.version.hip is not None — so XPU silently gets the CUDA constants (2900/1600), which assume flash/efficient attention. If XPU's SDPA falls back to math attention the way ROCm's does, the right constants are ~2–4× larger. Given your note that XPU exhaustion hangs rather than raising, this is an unpleasant one to guess wrong on, and Qwen isn't in your QA list. Worth a spot-check on Qwen Image / Qwen Image Edit.
• invocation_stats_default.py:31, memory_snapshot.py:50, model_cache.py:1333 all dispatch on torch.cuda.is_available() first. On a mixed NVIDIA + Arc box running on xpu:0, node stats and the summary report 0.0 GB while the Arc is full, and the cache log prints "CUDA Memory Allocated: 0.0 MB". Diagnostic-only, but it'd make XPU bug reports hard to act on. Unlike _get_vram_in_use, none of these consult self._execution_device.
• anima_latents_to_image.py:59 — "out_of_host_memory" is Intel's catch-all for driver-side resource failures (kernel compilation, handle exhaustion), not just host OOM, so a genuinely broken decode gets a pointless tiled retry. Bounded and re-raises, so cosmetic. (The lowercasing itself is correct — I checked every needle.)
• Metadata only: buildSD1Graph.ts:144 / buildSDXLGraph.ts:159 write rand_device: 'cuda' into image metadata on XPU machines.
• Docs: nothing under docs/ mentions xpu or the [xpu] extra. You offered an Intel install section — yes please.

---

Nothing here is architectural; the shape of the change is right. §1 is mechanical, §2 and §3 are the ones I'd want fixed before merge, and §5 is a genuine question rather than a demand. Thanks again for the care that went into this, and for the detailed QA notes — they made the review much easier.

[fishd72]

Tried this on Fedora 44 on my Framework 13 Pro which has the new PantherLake X7 385H with Intel Arc B390 integrated graphics, 32GB system RAM and it ran just fine. Performance seems better than my Apple MacBook Pro 14 with M2 Max chip (30GPU cores).

Full install requires:

1. Download the zip file above, extract to a directory
2. Install Intel compute runtimes with sudo dnf install intel-compute-runtime
3. Create a directory for invoke (I use ~/invokeai), change into this directory
4. Create a venv folder using uv venv --python 3.12 .venv
5. Activate venv using source .venv/bin/activate
6. Install Invoke using uv pip <path to extracted file>/invokeai-6.14.0a0-py3-none-any.whl[xpu] --extra-index-url https://download.pytorch.org/whl/xpu --index-strategy unsafe-best-match
7. On my device, to increase performance I used the latest Torch versions by removing the prior versions: uv pip uninstall torch torchvision pytorch-triton-xpu, then installed the versions from the nightly repo: uv pip install torch torchvision --index-url https://download.pytorch.org/whl/nightly/xpu/
8. Launch invoke from the command line using .venv/bin/invokeai-web

Only tested SDXL so far but will try other models shortly.

[LexiconCode]

@lstein I've tried my best to address your concerns including igpu.

§1 All four CI failures fixed — the 151-char line, events_common.py wording, and regenerated settings.json.

§2 Fixed, and measured. On torch 2.7.1 mem_get_info() genuinely raises on my Arc Pro B70 — so the blind estimate was the only source, exactly as you described. Added a Level Zero Sysman tier between native and the estimate; it returned 29.04/31.72 GiB where native failed. With 16 GiB held by another process the estimate over-reports by 16.17 GiB while Sysman is correct. (0, 0) now raises rather than collapsing the budget, and the except is broad since the type moves between torch releases. attention.py:28 fixes with it.

§3 Probe moved below the exclusions, targets the given device, caches per-device, no longer memoises failures, and now mirrors the runtime path (CPU cast → host→device copy → bf16/fp16 upcast).

§4 Borrows restricted to the excluded device's own type.

§5 Level Zero does expose it — ZE_DEVICE_PROPERTY_FLAG_INTEGRATED, read via ctypes against the loader that already ships with torch+xpu. iGPUs are dropped from auto when a discrete GPU exists, and no longer counted as dedicated VRAM. Unknown answers and iGPU-only machines keep current behaviour. I don't have the hardware to test this!

§6 All done. On the VAE constants — I measured XPU SDPA scaling instead of guessing: peak memory doubles as sequence length doubles (2.00× across 2048→16384), so XPU is in CUDA's O(area) regime and those constants are right.

Also bumped the extra to torch 2.13.0+xpu. Measured on the same hardware, identical code, only torch differing: SD1.5 −9%, SDXL −10%, Z-Image −16%, Wan video −33%. cpu/cuda/rocm exports are unchanged package-for-package.

One open question. @fishd72 reports an Arc B390 iGPU below. My §5 change switches partial loading off there, and makes heuristic 2's cap inert. I've asked for numbers before assuming that's an improvement. Related: _calc_ram_available_to_model_cache sizes from virtual_memory().total while _get_vram_available uses .available — on shared memory those are the same pool. Worth a separate look.

@fishd72 Thanks integrated-GPU report! Four things if you have a moment:

1. With a model loaded, what does free -g show in the available column?
2. What number follows Calculated model RAM cache size in the startup log?
3. Tried anything bigger than SDXL — FLUX.1 or Qwen Image?
4. Could you run the snippet below? It reports whether Level Zero flags the B390 as integrated, which is what the iGPU handling keys off.

import ctypes, ctypes.util, struct, psutil, torch
lib = ctypes.CDLL(ctypes.util.find_library("ze_loader") or "libze_loader.so.1")
u32p, vpp = ctypes.POINTER(ctypes.c_uint32), ctypes.POINTER(ctypes.c_void_p)
for fn, args in (("zeInit",[ctypes.c_uint32]), ("zeDriverGet",[u32p,vpp]),
                 ("zeDeviceGet",[ctypes.c_void_p,u32p,vpp]),
                 ("zeDeviceGetProperties",[ctypes.c_void_p,ctypes.c_void_p])):
    f = getattr(lib, fn); f.argtypes, f.restype = args, ctypes.c_int
assert lib.zeInit(0) == 0, "zeInit failed"
def enum(fn, parent=None):
    n = ctypes.c_uint32(0); head = [parent] if parent is not None else []
    getattr(lib, fn)(*head, ctypes.byref(n), None)
    arr = (ctypes.c_void_p * n.value)()
    getattr(lib, fn)(*head, ctypes.byref(n), arr)
    return list(arr[:n.value])
for i, d in enumerate(dv for drv in enum("zeDriverGet") for dv in enum("zeDeviceGet", drv)):
    buf = ctypes.create_string_buffer(368); struct.pack_into("I", buf, 0, 3)
    lib.zeDeviceGetProperties(d, buf)
    name = buf.raw[112:368].split(b"\0")[0].decode(errors="replace")
    print(f"device {i}: integrated={bool(struct.unpack_from('I', buf, 28)[0] & 1)}  name={name!r}")
vm = psutil.virtual_memory()
print(f"torch {torch.__version__}  xpu devices {torch.xpu.device_count()}")
for i in range(torch.xpu.device_count()):
    p = torch.xpu.get_device_properties(torch.device("xpu", i))
    print(f"  xpu:{i} total_memory {p.total_memory/2**30:.1f} GB")
print(f"RAM total {vm.total/2**30:.1f} GB  available {vm.available/2**30:.1f} GB")

Read-only — it just queries the driver.

[fishd72]

1. Result of free -g mid-generation

               total        used        free      shared  buff/cache   available
Mem:              30          17           3           0          11          13
Swap:              7           0           7

2. Startup log

[2026-08-02 15:03:05,294]::[InvokeAI]::INFO --> Using torch device: Intel(R) Arc(TM) B390 GPU
>> patchmatch.patch_match: INFO - Compiling and loading c extensions from "/home/fishd/invokeai/.venv/lib/python3.12/site-packages/patchmatch".
>> patchmatch.patch_match: ERROR - patchmatch failed to load or compile (Command 'make clean && make' returned non-zero exit status 2.).
>> patchmatch.patch_match: INFO - Refer to https://invoke-ai.github.io/InvokeAI/installation/060_INSTALL_PATCHMATCH/ for installation instructions.
[2026-08-02 15:03:07,107]::[InvokeAI]::INFO --> Patchmatch not loaded (nonfatal)
[2026-08-02 15:03:07,817]::[InvokeAI]::INFO --> InvokeAI version 6.14.0-alpha
[2026-08-02 15:03:07,818]::[InvokeAI]::INFO --> Root directory = /home/fishd/invokeai
[2026-08-02 15:03:07,818]::[InvokeAI]::INFO --> Initializing database at /home/fishd/invokeai/databases/invokeai.db
[2026-08-02 15:03:07,828]::[InvokeAI]::INFO --> JWT secret loaded from database
[2026-08-02 15:03:07,830]::[ModelManagerService]::INFO --> [MODEL CACHE] Calculated model RAM cache size: 13761.99 MB. Heuristics applied: [1].
[2026-08-02 15:03:07,830]::[ModelManagerService]::INFO --> Model cache global RAM budget: 13.44 GB across 1 device cache(s).
[2026-08-02 15:03:07,832]::[ModelInstallService]::INFO --> Restoring incomplete installs
[2026-08-02 15:03:07,832]::[ModelInstallService]::INFO --> Finished restoring incomplete installs
[2026-08-02 15:03:07,886]::[InvokeAI]::INFO --> Invoke running on http://127.0.0.1:9090 (Press CTRL+C to quit)

3. Not yet, I'll give that a go tonight. My internet is terrible (rural UK based) so takes me a while to download the larger models
4. Result of above code

device 0: integrated=True  name='Intel(R) Arc(TM) B390 GPU'
torch 2.14.0.dev20260731+xpu  xpu devices 1
  xpu:0 total_memory 28.6 GB
RAM total 30.9 GB  available 16.6 GB

[LexiconCode]

Heuristic 2 previously only applied on CUDA (total_cuda_vram_bytes); this PR extended it to XPU. On an integrated GPU that caps the RAM cache against the same RAM, which is your point exactly. Fixed by treating integrated devices as having no dedicated VRAM, so total_vram_bytes stays None — restoring the pre-PR behaviour for that device class rather than inventing a new one.

Separately, and pre-existing: heuristic 1 sizes from psutil.virtual_memory().total. On @fishd72's B390 that yields a 13.44 GB cache against 13 GB actually available mid-generation — 103% of what exists. Sizing from available would give ~10 GB, but it's unstable to sample once at startup and it changes MPS too, so I've left it alone. Any other thoughts on memory management?

[fishd72]

Got round to testing Anima, Flux and Z-Image... I don't know a lot about these models but Anima Base and NovaCartoonAM work fine, I tried Anima Turbo V1-Q8 and it quickly ran out of RAM and crashed, Flux.1 schnell seems ok, and Z-image Turbo Q4 seemed ok, but Q8 would run out of RAM and crash.

In case it matters, I have the following settings in my invokeai.yaml:

device: xpu
generation_devices:
- xpu:0

enable_partial_loading: true