Skip to content

add modular pipeline for qwen-image 2.1 - #14976

Merged
DN6 merged 4 commits into
huggingface:mainfrom
akshan-main:qwenimage21-modular
Oct 15, 2026
Merged

DN6 merged 4 commits into
huggingface:mainfrom
akshan-main:qwenimage21-modular

Conversation

@akshan-main

@akshan-main akshan-main commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

What does this PR do?

Adds a modular pipeline for Qwen-Image 2.1: QwenImage21AutoBlocks + QwenImage21ModularPipeline, covering text-to-image and image-conditioned generation with one or more condition images. Same shape as flux2 (text_encoder / vae_encoder / denoise / decode, with the image-conditioned variants selected on image and image_latents). The denoiser is shared by both workflows and keeps one prefix KV cache per guidance branch, so the cache also works when the guider starts or stops mid-run. The Turbo checkpoint maps to its own blockset (QwenImage21TurboAutoBlocks / QwenImage21TurboModularPipeline) in the flux2 klein shape: no guider, no negative_prompt, and the sample_sigmas grid it ships with, which is also what ModularPipeline.from_pretrained routes on.

Outputs match QwenImage21Pipeline exactly with shared components and seed, on the tiny components and on Qwen/Qwen-Image-2.1 and Qwen/Qwen-Image-2.1-Turbo in bf16 (text-to-image, edit, two-image edit, with and without guidance and the KV cache). Test components live in akshan-main/tiny-qwenimage21-modular-pipe and akshan-main/tiny-qwenimage21-turbo-modular-pipe.

Also registers QwenImage21Pipeline in the auto text-to-image mapping so ModularPipeline.from_pretrained("Qwen/Qwen-Image-2.1") resolves the modular class, and makes the test helper that reads modular_model_index.json skip null config values.

Before submitting

  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • Did you read the contributor guideline?
  • Did you read our philosophy doc (important for complex PRs)?
  • Was this discussed/approved via a GitHub issue or the forum? Discussed on Slack with @sayakpaul @yiyixuxu
  • Did you make sure to update the documentation with your changes?
  • Did you write any new necessary tests?

Who can review?

@yiyixuxu @sayakpaul

@github-actions github-actions Bot added size/L PR with diff > 200 LOC documentation Improvements or additions to documentation tests modular-pipelines utils pipelines and removed size/L PR with diff > 200 LOC labels Oct 7, 2026
@github-actions

github-actions Bot commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

Hi @akshan-main, thanks for the PR! It does not appear to link an issue it fixes. If this PR addresses an existing issue, please add a closing keyword (e.g. Fixes #1234) to the PR description so the issue is linked. See the contribution guide for more details. If this PR intentionally does not fix a tracked issue, a maintainer can add the no-issue-needed label to silence this reminder.

Please note that PRs without a linked issue are likely to be automatically closed 10 days after this notice.

Once the PR links an issue (or gets the no-issue-needed label), you can ignore this message — it stays here as a comment, but it no longer applies.

@yiyixuxu yiyixuxu added the no-issue-needed for PRs that do not require link to an issue label Oct 7, 2026

@yiyixuxu yiyixuxu left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks!
sorry i'm a bit late on review, really nice PR!
left some comments

Components:
text_encoder (`Qwen3VLForConditionalGeneration`) processor (`Qwen3VLProcessor`) guider
(`ClassifierFreeGuidance`)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we remove guider as a component for tex encoder bock?
(a lot of our current text_encoder lists guider as a components, but i write a bit on the preferred pattern here #14469)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed from both text encoder blocks. On their own they encode a negative_prompt when given; next to the denoiser they follow the guider and warn when one is dropped.

scheduler (`FlowMatchEulerDiscreteScheduler`)
Configs:
sample_sigmas (default: None): Default sampling grid of the checkpoint, used when `sigmas` is not passed.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ohh I think this is only for distilled checkpoint, actually can we seperate the blockset for distilled checkpiont
code is in this PR #14950
repo https://huggingface.co/Qwen/Qwen-Image-2.1-Turbo

flux2 is good reference https://github.com/huggingface/diffusers/tree/main/src/diffusers/modular_pipelines/flux2

@HuggingFaceDocBuilderDev

Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@github-actions github-actions Bot added the size/L PR with diff > 200 LOC label Oct 15, 2026
Comment thread tests/modular_pipelines/qwenimage21/test_modular_pipeline_qwenimage21.py Outdated
Comment thread tests/modular_pipelines/qwenimage21/test_modular_pipeline_qwenimage21_turbo.py Outdated
@DN6
DN6 merged commit 62cde2b into huggingface:main Oct 15, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation modular-pipelines no-issue-needed for PRs that do not require link to an issue pipelines size/L PR with diff > 200 LOC tests utils

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants