Skip to content

Add opt-in low-VRAM mode for Wan generation - #9462

Open
JPPhoto wants to merge 3 commits into
invoke-ai:mainfrom
JPPhoto:wan-memory-optimization
Open

Add opt-in low-VRAM mode for Wan generation#9462
JPPhoto wants to merge 3 commits into
invoke-ai:mainfrom
JPPhoto:wan-memory-optimization

Conversation

@JPPhoto

@JPPhoto JPPhoto commented Aug 5, 2026

Copy link
Copy Markdown
Collaborator

Summary

Adds an opt-in Wan memory optimization mode:

wan_memory_optimization: true

The option defaults to false.

When enabled, Invoke:

  • Limits resident Wan transformer weights to about 2 GiB and streams remaining layers from RAM.
  • Enforces the residency limit after model-cache loading, including already-resident experts and explicit VRAM cache configurations.
  • Chunks pointwise transformer operations while preserving global self-attention and cross-attention.
  • Compacts TI2V per-token timestep conditioning to its unique timestep states.
  • Streams untiled causal VAE decode chunks directly into the MP4 writer instead of retaining the complete RGB video in GPU or system memory.
  • Uses a streaming-aware VAE working-memory estimate.

This applies to Wan image and video denoise, including dual-expert A14B models.

The main tradeoff is speed: aggressive weight streaming can make generation substantially slower and requires enough system RAM for offloaded weights. Spatially tiled VAE decode retains its existing path.

Related Issues / Discussions

Related design reference: #9460.

QA Instructions

  1. Add this setting to invokeai.yaml:

    wan_memory_optimization: true
  2. Restart Invoke.

  3. Run Wan image and video generation with representative configurations:

    • TI2V-5B
    • A14B T2V or I2V
    • A14B dual-expert generation
    • CFG enabled and disabled
  4. Confirm the log contains:

    Wan memory optimization: limiting resident transformer weights to about 2 GiB
    
  5. Confirm the model-cache log reports substantially reduced resident transformer weights and peak VRAM is lower.

  6. Confirm generated videos have the expected dimensions, frame count, duration, and playback.

  7. Repeat with wan_memory_optimization: false and confirm existing behavior is unchanged.

Merge Plan

Checklist

  • The PR has a short but descriptive title, suitable for a changelog
  • Tests added / updated (if applicable)
  • ❗Changes to a redux slice have a corresponding migration
  • Documentation added / updated (if applicable)
  • Updated What's New copy (if doing a release after this PR)

@JPPhoto JPPhoto added the 6.14.0 label Aug 5, 2026
@JPPhoto
JPPhoto requested a review from blessedcoolant as a code owner August 5, 2026 00:25
@JPPhoto JPPhoto moved this to 6.14.x Theme: USER EXPERIENCE in Invoke - Community Roadmap Aug 5, 2026
@github-actions github-actions Bot added python PRs that change python files invocations PRs that change invocations backend PRs that change backend files services PRs that change app services frontend PRs that change frontend files python-tests PRs that change python tests docs PRs that change docs labels Aug 5, 2026
@JPPhoto
JPPhoto force-pushed the wan-memory-optimization branch 2 times, most recently from 2a6af17 to ce2ccc1 Compare August 5, 2026 16:21
@JPPhoto
JPPhoto force-pushed the wan-memory-optimization branch from ce2ccc1 to a25ef93 Compare August 5, 2026 17:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

6.14 Nice-to-Have 6.14.0 backend PRs that change backend files docs PRs that change docs frontend PRs that change frontend files invocations PRs that change invocations python PRs that change python files python-tests PRs that change python tests services PRs that change app services

Projects

Status: 6.14.x Theme: USER EXPERIENCE

Development

Successfully merging this pull request may close these issues.

2 participants