Skip to content

Pick the right CUDA wheel per GPU, surface integer versions H3 + clip window select #34

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
imprsnst merged 3 commits into from
Aug 8, 2026
Merged

Pick the right CUDA wheel per GPU, surface integer versions H3 + clip window select #34

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
e1f462b
pick the right CUDA wheel per GPU, and stop warning on cards that work
imprsnst Aug 8, 2026
405d337
action test fix
imprsnst Aug 8, 2026
0808b63
bumped version to 1.2.65
imprsnst Aug 8, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
111 changes: 111 additions & 0 deletions .github/workflows/launcher.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
# The launchers pick which PyTorch wheel index a GPU needs, and webui.bat cannot be exercised from
# a Linux dev box, so two field reports shipped before anyone noticed it chose wrong. cmd runs
# natively here and the GPU is stubbable, so the decision is assertable without hardware.
name: launcher

on:
push:
paths:
- "core/webui.bat"
- "core/webui.sh"
- "core/tests/test_webui_install.py"
- "core/src/inline_core/device/**"
- ".github/workflows/launcher.yml"
pull_request:
paths:
- "core/webui.bat"
- "core/webui.sh"
- "core/tests/test_webui_install.py"
- "core/src/inline_core/device/**"
- ".github/workflows/launcher.yml"

jobs:
windows-torch-index:
name: "webui.bat picks ${{ matrix.expect }} (${{ matrix.name }})"
runs-on: windows-latest
strategy:
fail-fast: false
matrix:
include:
- name: blackwell on a current driver
probe: "12.0, 610.88"
expect: cu130
reason: autodetect
# cu128 still serves and was the first index with sm_120, so this machine has exactly one
# workable choice. A hard error here would be worse than what v1.2.64 did.
- name: blackwell on a pre-R580 driver
probe: "12.0, 575.57"
expect: cu128
reason: driver-floor-cu128
- name: ada
probe: "8.9, 580.82"
expect: cu126
reason: autodetect
- name: hopper
probe: "9.0, 580.82"
expect: cu126
reason: autodetect
# An old driver answers an unknown query with a word. A string comparison would rank it
# above 10 and hand an unknown card cu130, so this asserts the set /a coercion holds.
- name: a driver that cannot answer the query
probe: "Unknown Error"
expect: cu126
reason: autodetect
steps:
- uses: actions/checkout@v4

- name: Put a fake nvidia-smi first on PATH
shell: pwsh
run: |
$dir = Join-Path $env:RUNNER_TEMP 'stub'
New-Item -ItemType Directory -Force -Path $dir | Out-Null
# Answers -L with success (so the GPU reads as present) and the query with the matrix line.
@"
@echo off
echo %* | findstr /c:"compute_cap" >nul && echo ${{ matrix.probe }}
exit /b 0
"@ | Set-Content -Path (Join-Path $dir 'nvidia-smi.bat') -Encoding ascii
Add-Content -Path $env:GITHUB_PATH -Value $dir

- name: Check the stub is reachable and returns
shell: cmd
run: |
where nvidia-smi || (echo stub not on PATH & exit /b 1)
call nvidia-smi --query-gpu=compute_cap,driver_version --format=csv,noheader
echo stub returned control, errorlevel %errorlevel%

- name: Report the decision
id: decide
shell: cmd
working-directory: core
run: |
call .\webui.bat --print-torch-index > decision.txt 2>&1
echo ---------- decision.txt ----------
type decision.txt
echo ----------------------------------

- name: Assert the index and the reason
shell: cmd
working-directory: core
run: |
findstr /c:"torch-index: ${{ matrix.expect }}" decision.txt >nul || (echo EXPECTED "torch-index: ${{ matrix.expect }}" but got: & type decision.txt & exit /b 1)
findstr /c:"reason: ${{ matrix.reason }}" decision.txt >nul || (echo EXPECTED "reason: ${{ matrix.reason }}" but got: & type decision.txt & exit /b 1)

- name: Assert it installed nothing
shell: cmd
working-directory: core
run: |
if exist .venv (echo --print-torch-index must not create a venv & exit /b 1)

posix-launcher-tests:
name: webui.sh launcher tests
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: astral-sh/setup-uv@v5
- name: Install the engine and test tooling
working-directory: core
run: uv venv && uv pip install --python .venv/bin/python -e ".[server,dev]"
- name: Launcher and device tests
working-directory: core
run: .venv/bin/python -m pytest tests/test_webui_install.py tests/test_device_detect.py -q
46 changes: 27 additions & 19 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,9 @@ git clone https://github.com/inlineresearch/Inline-Studio.git
cd Inline-Studio\core
.\webui.bat --install --extra all
.\webui.bat

rem If the CUDA build turns out wrong for your card, name the index yourself:
.\webui.bat --install --extra all --torch-index cu130
```

That's it: `--install` sets up the environment and installs everything once, then `webui.sh` / `webui.bat` runs the app on one port. See **[Command-line options](#command-line-options)** for every flag (`--listen`, `--port`, `--lowvram`, `--multi-gpu`, …).
Expand Down Expand Up @@ -82,17 +85,20 @@ Honest status - what's actually been run, versus what has a code path but no one

RTX 50-series cards (5060/5070/5080/5090 and the RTX PRO Blackwell line) are compute capability **sm_120**, and no PyTorch wheel built for CUDA 12.4 or 12.6 has kernels for them. `--install` handles this: it reads the compute capability off the driver and picks `cu130`, so a plain `.\webui.bat --install --extra all` is all you need.

Two cases where you may want to say it yourself:
**Old driver?** CUDA 13 needs driver R580 or newer. If yours predates it, `--install` picks `cu128` for you and says so: cu128 still has `sm_120` but is **frozen at torch 2.11** and will never update, so updating the driver and re-running `--install` is worth doing when you can.

To name an index yourself:

```powershell
rem Blackwell card, but a driver older than CUDA 13 (R580) - cu128 has sm_120 and a lower floor
.\webui.bat --install --extra all --torch-index cu128
.\webui.bat --install --extra all --torch-index cu130

rem Or set it once for the shell, same effect
set INLINE_TORCH_INDEX=cu128
set INLINE_TORCH_INDEX=cu130
```

`--torch-index` takes a short name (`cu130`, `cu128`, `cu126`), a full index URL, or `cpu` to force the CPU-only build. `webui.sh` takes the same flag. If the installed build turns out to have no kernels for your card, Core says so by name at startup rather than leaving you with PyTorch's own `sm_120 is not compatible` warning.
`--torch-index` takes a short name (`cu130`, `cu128`, `cu126`), a full index URL, or `cpu` to force the CPU-only build. `webui.sh` takes the same flag. Naming it explicitly also **replaces** an already-installed torch, which a plain re-run will not do, so you rarely need `--recreate`.

Not sure what you have? `.\webui.bat --print-torch-index` prints what the driver reported and which index would be used, and installs nothing. Paste that into a bug report. If the installed build turns out to have no kernels for your card, Core also says so by name at startup rather than leaving you with PyTorch's own `sm_120 is not compatible` warning.

#### AMD (ROCm) setup

Expand Down Expand Up @@ -163,20 +169,22 @@ The friendly launcher (in `core/`) maps flags onto the engine's `INLINE_*` envir
<details>
<summary><strong>Show all command-line flags</strong></summary>

| `webui.sh` / `main.py` flag | Env var | What it does |
| ---------------------------------- | ------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `--listen` | `INLINE_HOST=0.0.0.0` | Bind all interfaces so other machines can reach it |
| `--host ADDR` | `INLINE_HOST` | Bind a specific address (default `127.0.0.1`) |
| `--port N` | `INLINE_PORT` | Port to serve on (default `8848`) |
| `--models-dir PATH` | `INLINE_MODELS_DIR` | Where model weights are scanned from (default `./models`) |
| `--data-dir PATH` | `INLINE_DATA_DIR` | Where runs + takes are written (default `./.inline`) |
| `--lowvram` | `INLINE_PROFILE=lowvram` | Tight-VRAM profile (VAE tiling/slicing, attention slicing) |
| `--cpu` | `INLINE_PROFILE=cpu` | Force CPU generation |
| `--profile NAME` | `INLINE_PROFILE` | Set the profile explicitly: `gpu-max` \| `lowvram` \| `cpu` |
| `--vram-budget GB` | `INLINE_VRAM_BUDGET_GB` | Treat the GPU as having GB of usable VRAM |
| `--multi-gpu [SPEC]` | `INLINE_PARALLEL` | Split one image's denoise across GPUs (e.g. `pipefusion=2`); auto with 2+ GPUs |
| `--front-end-root DIR` _(main.py)_ | `INLINE_FRONTEND_ROOT` | Serve a local SPA build instead of the installed UI package (dev) |
| `--rebuild` _(webui.sh)_ | n/a | Force a fresh SPA build (`npm run build:spa`) from source and serve it on the one port; use after UI changes when not running `--dev`. Needs the repo checkout + Node/npm |
| `webui.sh` / `main.py` flag | Env var | What it does |
| ---------------------------------- | ------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `--listen` | `INLINE_HOST=0.0.0.0` | Bind all interfaces so other machines can reach it |
| `--host ADDR` | `INLINE_HOST` | Bind a specific address (default `127.0.0.1`) |
| `--port N` | `INLINE_PORT` | Port to serve on (default `8848`) |
| `--models-dir PATH` | `INLINE_MODELS_DIR` | Where model weights are scanned from (default `./models`) |
| `--data-dir PATH` | `INLINE_DATA_DIR` | Where runs + takes are written (default `./.inline`) |
| `--lowvram` | `INLINE_PROFILE=lowvram` | Tight-VRAM profile (VAE tiling/slicing, attention slicing) |
| `--cpu` | `INLINE_PROFILE=cpu` | Force CPU generation |
| `--profile NAME` | `INLINE_PROFILE` | Set the profile explicitly: `gpu-max` \| `lowvram` \| `cpu` |
| `--vram-budget GB` | `INLINE_VRAM_BUDGET_GB` | Treat the GPU as having GB of usable VRAM |
| `--multi-gpu [SPEC]` | `INLINE_PARALLEL` | Split one image's denoise across GPUs (e.g. `pipefusion=2`); auto with 2+ GPUs |
| `--front-end-root DIR` _(main.py)_ | `INLINE_FRONTEND_ROOT` | Serve a local SPA build instead of the installed UI package (dev) |
| `--rebuild` _(webui.sh)_ | n/a | Force a fresh SPA build (`npm run build:spa`) from source and serve it on the one port; use after UI changes when not running `--dev`. Needs the repo checkout + Node/npm |
| `--torch-index WHICH` | `INLINE_TORCH_INDEX` | With `--install`, override the PyTorch wheel index picked from your GPU's compute capability. A short name (`cu130`, `cu128`, `cu126`), a full index URL, or `cpu`. Naming it also replaces an already-installed torch |
| `--print-torch-index` | n/a | Print what the GPU probe read and which index would be used, then exit without installing. The one line to paste into a bug report |

</details>

Expand Down
2 changes: 1 addition & 1 deletion core/pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
[project]
# PyPI name; the import package is `inline_core` (src/inline_core).
name = "inline-core"
version = "1.2.64"
version = "1.2.65"
description = "The generation engine behind Inline Studio."
readme = "README.md"
license = "GPL-3.0-or-later"
Expand Down
41 changes: 37 additions & 4 deletions core/src/inline_core/device/detect.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,9 @@

from __future__ import annotations

from collections.abc import Iterable
from string import ascii_letters

from .types import Device, DeviceKind


Expand Down Expand Up @@ -80,6 +83,37 @@ def cpu_only_torch_warning() -> str | None:
)


def _parse_arch(arch: str) -> tuple[int, int] | None:
"""``sm_120`` to ``(12, 0)``.

The minor is the **last digit** and the major is everything before it, so ``sm_100`` is (10, 0).
Reading left to right instead gives (1, 20) and puts the bug precisely on the Blackwell parts.
Tuned variants (``sm_90a``) drop their trailing letters.
"""
digits = arch[3:].rstrip(ascii_letters)
if len(digits) < 2 or not digits.isdigit():
return None
return int(digits[:-1]), int(digits[-1])


def arch_list_covers(arches: Iterable[str], major: int, minor: int) -> bool:
"""Whether a wheel's arch list has kernels that will run on a ``(major, minor)`` device.

CUDA binary compatibility runs **upward within one major only**: an ``sm_8x`` cubin executes on
any ``sm_8y`` where ``y >= x``. So ``sm_86`` covers an sm_89 Ada card, which is why exact
matching told every RTX 40-series owner their install was broken. Crossing a major never works,
so ``sm_90`` does not cover sm_120 and the Blackwell warning stays correct.

PTX entries (``compute_90``) JIT forward but are not matched here: they do not start with
``sm_``, so a PTX-carrying wheel still warns. Rare, and warning is the safe way to be wrong.
"""
for arch in arches:
parsed = _parse_arch(arch)
if parsed is not None and parsed[0] == major and parsed[1] <= minor:
return True
return False


def unsupported_arch_warning() -> str | None:
"""A warning when the installed torch has no kernels for the GPU it is about to run on.

Expand All @@ -105,16 +139,15 @@ def unsupported_arch_warning() -> str | None:
return None # a CPU-only build; cpu_only_torch_warning owns that case
major, minor = get_capability(0)
target = f"sm_{major}{minor}"
# startswith, because a wheel lists tuned variants like sm_90a for the same architecture.
if any(arch.startswith(target) for arch in arches):
if arch_list_covers(arches, major, minor):
return None
name = _device_name(torch) or "The detected NVIDIA GPU"
return (
f"{name} is compute capability {target}, but this install's PyTorch only has kernels "
f"for {' '.join(arches)}. Generation will fail or fall back to the CPU. Re-run "
"`webui.sh --install` (Windows: `.\\webui.bat --install`) to pick the wheel index that "
"matches the card, or force one with `--torch-index cu130` - `cu128` if the driver "
"predates CUDA 13."
"matches the card, or force one with `--torch-index cu130`. On a Blackwell card whose "
"driver predates R580, use `cu128`, which still has sm_120 but is frozen at torch 2.11."
)
except Exception: # noqa: BLE001 - a diagnostic must never break startup
return None
Expand Down
Loading
Loading