Skip to content

Fix HiDiffusion with SDXL ControlNet - #9454

Open
DustyShoe wants to merge 2 commits into
invoke-ai:mainfrom
DustyShoe:fix/hidiffusion-sdxl-controlnet
Open

Fix HiDiffusion with SDXL ControlNet#9454
DustyShoe wants to merge 2 commits into
invoke-ai:mainfrom
DustyShoe:fix/hidiffusion-sdxl-controlnet

Conversation

@DustyShoe

Copy link
Copy Markdown
Collaborator

Summary

  • make HiDiffusion detect ControlNet when InvokeAI passes a bare UNet
  • resize ControlNet residuals to the active HiDiffusion feature-map dimensions
  • preserve separate text-to-image and image-to-image RAU-Net threshold behavior
  • avoid replacing the full UNet forward for window-attention-only runs

Root cause

InvokeAI runs ControlNet as a separate extension, so the vendored HiDiffusion code could not detect it from the bare UNet. RAU-Net changed the UNet feature-map dimensions while ControlNet residuals retained their original dimensions, causing tensor-size mismatches during residual addition.

This is a follow-up to #8787.

Validation

  • uv tool run ruff@0.11.2 format --check on all changed files
  • uv tool run ruff@0.11.2 check on all changed files
  • pytest tests/backend/stable_diffusion/test_hidiffusion_utils.py -q (14 passed)

@DustyShoe
DustyShoe marked this pull request as ready for review August 3, 2026 18:47
@github-actions github-actions Bot added python PRs that change python files invocations PRs that change invocations backend PRs that change backend files python-tests PRs that change python tests labels Aug 3, 2026
@JPPhoto JPPhoto self-assigned this Aug 4, 2026
@JPPhoto JPPhoto added the 6.14.0 label Aug 4, 2026
@JPPhoto JPPhoto moved this to 6.14.x Theme: USER EXPERIENCE in Invoke - Community Roadmap Aug 4, 2026
@JPPhoto

JPPhoto commented Aug 4, 2026

Copy link
Copy Markdown
Collaborator

@DustyShoe I don't have these models or space, have you tried this feature?

@DustyShoe

Copy link
Copy Markdown
Collaborator Author

@JPPhoto
Yes, i did test it with almost all a controlnets available for SD1.5 and SDXL by generating images and it does fix the issue.
Not with SD2 though. I don't have room on my HDD for that.

@JPPhoto

JPPhoto commented Aug 4, 2026

Copy link
Copy Markdown
Collaborator

@JPPhoto Yes, i did test it with almost all a controlnets available for SD1.5 and SDXL by generating images and it does fix the issue. Not with SD2 though. I don't have room on my HDD for that.

I think somebody can make an issue if there are any problems running on that architecture. Everything looks good and this is approved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

6.14.0 backend PRs that change backend files invocations PRs that change invocations python PRs that change python files python-tests PRs that change python tests

Projects

Status: 6.14.x Theme: USER EXPERIENCE

Development

Successfully merging this pull request may close these issues.

2 participants