feat(minimax-h3): model support — vendored classes, taxonomy, probe, loader - #139
Open
lstein wants to merge 1 commit into
Open
feat(minimax-h3): model support — vendored classes, taxonomy, probe, loader#139lstein wants to merge 1 commit into
lstein wants to merge 1 commit into
Conversation
…loader Adds installable model support for MiniMax H3 (Hailuo 3.0) FL2VA, the 33B omni-modal transformer that jointly denoises video and stereo audio. - Vendors MiniMaxH3Transformer3DModel, AutoencoderKLMiniMaxH3, AutoencoderKLMiniMaxH3Audio and MiniMaxH3Scheduler from diffusers PR #14355 at abc5e9bf71 (unreleased branch) into invokeai/backend/minimax_h3. Only local changes: absolute imports + ruff import sorting; all referenced symbols exist in the pinned diffusers 0.39.0. B905 is ignored for the vendored files to keep them upstream-identical. - Taxonomy: BaseModelType.MiniMaxH3, MiniMaxH3VariantType.FL2VA, SubModelType.AudioVAE + SubModelType.Processor (the H3 layout needs both; the Qwen3VLProcessor is used even for text-only encoding). - Probe: Main_Diffusers_MiniMaxH3_Config keys on the Modular Diffusers root config (modular_model_index.json, class MiniMaxH3ModularPipeline) plus the audio_vae config as a partial-download guard. The repo's FL2VA/-Ref2VA/ remote-code subtrees declare MiniMaxH3Pipeline and are deliberately not matched. Ref2VA-only installs fail identification rather than mislabeling. modular_model_index.json is now a recognized root config marker in the factory scan (H3 installs have hundreds of shard files and would otherwise be rejected as non-models). - Loader: explicit submodel dispatch (vendored classes bf16; Qwen3-VL-32B via Qwen3VLForConditionalGeneration with the shared rope-config fix, lifted from krea2.py into model_manager/util/qwen3_vl.py; audio VAE kept fp32 - half-precision audio artifacts are audible). - Default UI settings: steps=50, cfg 1.0 (checkpoints are guidance-distilled - no CFG), native 1344x768 canvas. - Identification fixture built from the real HF repo config files (stripped stub shards); tests/model_identification passes 88/88. Part of the MiniMax H3 stack (PR 2 of 5); depends on the audio-mux PR. Post-review additions: - ModelRecordChanges.variant accepts fl2va (editing an installed H3 model otherwise 422s on the PATCH route). - Regenerated openapi.json + schema.ts, with the matching type-level frontend registration (zBaseModelType/zMainModelBase/zSubModelType/ zAnyModelVariant entries and the exhaustive MODEL_BASE_TO_* records) so typegen-checks and tsc stay green. UI behavior lands in the linear-UI PR. - _is_known_model_marker guards non-str _class_name (a list-valued _class_name in modular_model_index.json raised TypeError instead of reading as not-a-marker). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
github-actions
Bot
added
root
python-deps
python
python-tests
services
backend
frontend
labels
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Stacked PR 2 of 5 for MiniMax H3 (Hailuo 3.0) support — the 33B omni-modal transformer that jointly denoises video and 32 kHz stereo audio, open-sourced 2026-08-03. This PR makes the FL2VA checkpoint installable and loadable; the invocation nodes come in the next PR.
Depends on #138 (branch is stacked on
minimax-h3/01-audio-mux).invokeai/backend/minimax_h3/):MiniMaxH3Transformer3DModel,AutoencoderKLMiniMaxH3(video VAE),AutoencoderKLMiniMaxH3Audio,MiniMaxH3Scheduler— from diffusers PR #14355 atabc5e9bf71(unreleased branch; verified byte-identical except absolute-import rewrite + import sorting; every referenced symbol exists in the pinned diffusers 0.39.0). When a tagged diffusers ships these, the vendoring deletes cleanly.BaseModelType.MiniMaxH3,MiniMaxH3VariantType.FL2VA,SubModelType.AudioVAE+SubModelType.Processor(H3 needs the Qwen3VLProcessor even for text-only encoding).modular_model_index.json, classMiniMaxH3ModularPipeline) plus theaudio_vaeconfig as a partial-download guard, and the FL2VA transformer folder for the variant. The HF repo'sFL2VA//Ref2VA/remote-code subtrees (classMiniMaxH3Pipeline) are deliberately rejected; Ref2VA-only downloads fail identification rather than being mislabeled.modular_model_index.jsonis now a recognized root-config marker in the directory scan — H3 installs have hundreds of shard files and would otherwise be rejected by the 50-file guard.Qwen3VLForConditionalGenerationwith the rope-config fix lifted from krea2 intomodel_manager/util/qwen3_vl.py; audio VAE kept fp32 — it's 0.6 GB and half-precision audio artifacts are audible).openapi.json+schema.ts, with the minimal type-level frontend registration (zod enums + exhaustiveMODEL_BASE_TO_*records) sotscand typegen-checks stay green. All UI behavior lands in the linear-UI PR.An adversarial review of this diff was run; confirmed findings fixed:
ModelRecordChanges.variantunion (editing an installed H3 model 422'd), typegen drift, and aTypeErroron non-string_class_namein the marker check.Testing
tests/model_identification: 88/88, including a new LFS fixture built from the real HF repo config files (stripped stub shards). Reviewer verified exactly-one-config-matches, and that Ref2VA-only / missing-audio-VAE folders fail identification as designed.tests/app/services/model_install|model_records|model_load: 82 passed.tscclean, eslint/prettier clean, 1718 unit tests pass.Manual test gate for this PR: install the H3 diffusers layout via Model Manager (HF repo
MiniMaxAI/MiniMax-H3, root layout minustransformer_ref— ~106 GB, or wait for PR 5's starter entry) and confirm it probes asminimax-h3 / fl2vawith working submodel loads.🤖 Generated with Claude Code