Skip to content

lr_scheduler step bug in examples code #15087

Open

Description

@weixiaohao

Describe the bug

for example, the code in " diffusers--examples--dreambooth--train_dreambooth_lora_qwen_image.py": lr_scheduler = get_scheduler()这里前后的代码,The official diffusers training template for Accelerate has a mistake in scheduler creation. This causes lr scheduler total steps / warmup steps to be N times larger than real global update steps, learning rate never decays normally. Also there is a ordering bug:
The script calculates training steps & creates lr_scheduler before accelerator.prepare(train_dataloader).
At this point len(train_dataloader) is the non-distributed value (not sharded). After prepare, dataloader length changes, but scheduler has already been initialized with wrong step count.

This is a potential bug that is difficult to detect in schedule constant mode.

Reproduction

  1. Use diffusers official training script + Accelerator
  2. 8 GPU, per_device_batch_size=1, gradient_accumulation_steps=4
  3. Before prepare(): len(train_dataloader)=3000
  4. Create scheduler using this len, multiplied by accelerator.num_processes
  5. After prepare(), real len(train_dataloader)=375, but scheduler already built.

Logs

System Info

Run diffusers-cli env and paste output here

  • diffusers version: 0.36
  • accelerate version: 1.10
  • OS: ubuntu

Who can help?

No response

Activity

  1. akhil9tiet commented on Oct 20, 2026

    @akhil9tiet

    I'd like to work on this — please assign it to me.

  2. weixiaohao commented on Oct 20, 2026

    @weixiaohao
    Author

    @sayakpaul

    Thanks! I’ve outlined the root cause and fix steps in the issue. Looking forward to your PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions