Repository navigation
[Looking for community contribution] support Wan 2.2 S2V: an audio-driven cinematic video generation model #12257
Description
We're super excited about the Wan 2.2 S2V (Speech-to-Video) model and want to get it integrated into Diffusers! This would be an amazing addition, and we're looking for experienced community contributors to help make this happen.
- Project Page: https://humanaigc.github.io/wan-s2v-webpage/
- Source Code: https://github.com/Wan-Video/Wan2.2#run-speech-to-video-generation
- Model Weights: https://huggingface.co/Wan-AI/Wan2.2-S2V-14B
This is a priority for us, so we will try review fast and actively collabrate with you throughout the process :)
Activity
VittoriaLanzo commentedon Apr 2, 2026
Hi everyone, I'd love to take ownership of this integration!
My approach will be to leverage our existing Wan implementation, extending the cross-attention modules to natively handle the S2V audio-conditioning based on the original Wan 2.2 source repository.
To make sure everything is solid, I'm going to spend some time running local equivalence tests against the original repo's forward pass before I open a PR. I'll drop a [WIP] draft PR here in a week or two once I have a baseline forward pass passing local assertions.
Let me know if anyone is already actively poking at this locally, otherwise I will get to work!
VittoriaLanzo commentedon Apr 2, 2026
Hi @tolgacangoz — Just a heads-up that I’m picking this up for the S2V integration.
I’ve been reviewing the technical mapping you did in #12258; thanks for the foundational work on the tensor logic, it’s a huge help for understanding the conditioning layers.
My goal is to standardize the implementation to fit the current Wan architecture as a strictly 'MVP' refactor. I’m focusing on a very clean, documented version that’s easy for the maintainers to review/merge.
I’ll be using your research as the primary technical reference and will make sure to credit your work in the final PR. If you have any 'lessons learned' on the audio conditioning you'd want me to watch out for, I'm all ears—otherwise, I'm getting to work on the local tests!
tolgacangoz commentedon Apr 3, 2026
Actually, I didn't leave that PR :/
VittoriaLanzo commentedon Apr 3, 2026
Ah, my apologies! I completely misread the thread and thought the PR was inactive.
I’ll step back so I don't disrupt your momentum. Good luck with the rest of the integration—I'll be looking forward to seeing it merged!
LeonxLJX commentedon Sep 1, 2026
I'd like to take this one (
[Looking for community contribution] support Wan 2.2 S2V: an). I'll look into the root cause and follow up with a PR. (claiming via @LeonxLJX)tolgacangoz commentedon Sep 1, 2026
Actually, I didn't leave that PR :/
LeonxLJX commentedon Sep 1, 2026
I'd like to take this one (
[Looking for community contribution] support Wan 2.2 S2V: an). I'll look into the root cause and follow up with a PR. (claiming via @LeonxLJX)sharmis-tp commentedon Oct 22, 2026
I'd like to work on this if it's still open. I'd port the S2V transformer with its audio conditioning and the pipeline from the Wan2.2 reference code into Diffusers, convert the published Wan2.2-S2V-14B weights, and check output parity against the original implementation on a short clip with fixed seeds. Is anyone from the earlier comments already on it, and would you like the transformer and pipeline in separate PRs?
tolgacangoz commentedon Oct 22, 2026
Hi. My PR was closed but people still use it from my branch. https://huggingface.co/tolgacangoz/Wan2.2-S2V-14B-Diffusers says "Total downloads 138,468 (all time)".