Skip to content

Honour CPU offload on H3, fix the training status, unblock large HF downloads - #37

Merged
imprsnst merged 4 commits into
mainfrom
offload-control-h3
Aug 8, 2026
Merged

Honour CPU offload on H3, fix the training status, unblock large HF downloads#37
imprsnst merged 4 commits into
mainfrom
offload-control-h3

Conversation

@imprsnst

@imprsnst imprsnst commented Aug 8, 2026

Copy link
Copy Markdown
Collaborator

Three fixes from field reports, plus the version bump.

CPU offload was silently dead on H3. resolve_offload tested the quantisation before the user's explicit choice, so On returned False for any 4-bit base. H3 is always 4-bit, so the one control that helps its users did nothing. Reordered: on and off win outright, auto keeps the old conservative behaviour. H3 breaks the "a quantized base already fits" assumption, because its base is 11.7GB and it is the clip activations that overflow the card.

Progress was only emitted after a step finished, so the UI showed "loading model (nf4)" for the whole of step one. A slow first step read as a hung loader and sent a reporter and me chasing bitsandbytes. Now announced before the loop, with a test pinning the order.
Possible fix to #32

Large HF downloads failed. hf_xet became a hard dependency of huggingface_hub at 0.32, and our floor was 0.23, so an older resolve could not fetch the 66GB H3 transformer at all. Floor raised.

Also carries the --no-sources launcher fix: --no-sources-package only exists in uv 0.10+, so the installer hard-errored for anyone older.

@imprsnst
imprsnst merged commit 034f7a4 into main Aug 8, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant