Repository navigation
add modular pipeline for qwen-image 2.1 - #14976
Conversation
|
Hi @akshan-main, thanks for the PR! It does not appear to link an issue it fixes. If this PR addresses an existing issue, please add a closing keyword (e.g. Please note that PRs without a linked issue are likely to be automatically closed 10 days after this notice. Once the PR links an issue (or gets the |
yiyixuxu
left a comment
There was a problem hiding this comment.
thanks!
sorry i'm a bit late on review, really nice PR!
left some comments
| Components: | ||
| text_encoder (`Qwen3VLForConditionalGeneration`) processor (`Qwen3VLProcessor`) guider | ||
| (`ClassifierFreeGuidance`) |
There was a problem hiding this comment.
can we remove guider as a component for tex encoder bock?
(a lot of our current text_encoder lists guider as a components, but i write a bit on the preferred pattern here #14469)
There was a problem hiding this comment.
removed from both text encoder blocks. On their own they encode a negative_prompt when given; next to the denoiser they follow the guider and warn when one is dropped.
| scheduler (`FlowMatchEulerDiscreteScheduler`) | ||
| Configs: | ||
| sample_sigmas (default: None): Default sampling grid of the checkpoint, used when `sigmas` is not passed. |
There was a problem hiding this comment.
ohh I think this is only for distilled checkpoint, actually can we seperate the blockset for distilled checkpiont
code is in this PR #14950
repo https://huggingface.co/Qwen/Qwen-Image-2.1-Turbo
flux2 is good reference https://github.com/huggingface/diffusers/tree/main/src/diffusers/modular_pipelines/flux2
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
What does this PR do?
Adds a modular pipeline for Qwen-Image 2.1:
QwenImage21AutoBlocks+QwenImage21ModularPipeline, covering text-to-image and image-conditioned generation with one or more condition images. Same shape as flux2 (text_encoder/vae_encoder/denoise/decode, with the image-conditioned variants selected onimageandimage_latents). The denoiser is shared by both workflows and keeps one prefix KV cache per guidance branch, so the cache also works when the guider starts or stops mid-run. The Turbo checkpoint maps to its own blockset (QwenImage21TurboAutoBlocks/QwenImage21TurboModularPipeline) in the flux2 klein shape: no guider, nonegative_prompt, and thesample_sigmasgrid it ships with, which is also whatModularPipeline.from_pretrainedroutes on.Outputs match
QwenImage21Pipelineexactly with shared components and seed, on the tiny components and onQwen/Qwen-Image-2.1andQwen/Qwen-Image-2.1-Turboin bf16 (text-to-image, edit, two-image edit, with and without guidance and the KV cache). Test components live in akshan-main/tiny-qwenimage21-modular-pipe and akshan-main/tiny-qwenimage21-turbo-modular-pipe.Also registers
QwenImage21Pipelinein the auto text-to-image mapping soModularPipeline.from_pretrained("Qwen/Qwen-Image-2.1")resolves the modular class, and makes the test helper that readsmodular_model_index.jsonskipnullconfig values.Before submitting
Who can review?
@yiyixuxu @sayakpaul