[DDIMInverseScheduler] inf values at first iteration when set_alpha_to_one=True and prediction_type="sample"

@yiyixuxu

Describe the bug

I got inf values in

diffusers/src/diffusers/schedulers/scheduling_ddim_inverse.py

Line 347 in 560fb5f

    
           pred_epsilon = (sample - alpha_prod_t ** (0.5) * pred_original_sample) / beta_prod_t ** (0.5)

because beta_prod_t is 0.0 at the first iteration, when timestep is < 0 and beta_prod_t is 0.0 (because alpha_prod_t is set to 1.0 when set_alpha_to_one=True).

diffusers/src/diffusers/schedulers/scheduling_ddim_inverse.py

Line 338 in 560fb5f

beta_prod_t = 1 - alpha_prod_t

Reproduction

import torch as th 
from diffusers import DDIMInverseScheduler

ddim_inverse_scheduler = DDIMInverseScheduler(
    num_train_timesteps=1000,
    prediction_type="sample",
    set_alpha_to_one=True
)
ddim_inverse_scheduler.set_timesteps(num_inference_steps=50)

with th.no_grad():
    pred = th.randn((1, 1, 2, 2, 2))
    samples = th.randn((1, 1, 2, 2, 2))
    t = 0
    samples = ddim_inverse_scheduler.step(pred, t, samples).prev_sample
    assert not th.isinf(samples).any(), "samples contain inf values"

Logs

System Info

🤗 Diffusers version: 0.32.2
Platform: macOS-15.2-arm64-arm-64bit
Running on Google Colab?: No
Python version: 3.10.8
PyTorch version (GPU?): 2.5.1 (False)
Flax version (CPU?/GPU?/TPU?): not installed (NA)
Jax version: not installed
JaxLib version: not installed
Huggingface_hub version: 0.29.1
Transformers version: not installed
Accelerate version: not installed
PEFT version: not installed
Bitsandbytes version: not installed
Safetensors version: 0.5.2
xFormers version: not installed
Accelerator: Apple M1 Pro
Using GPU in script?: No (Using MPS)
Using distributed or parallel set-up in script?: No

Who can help?

@yiyixuxu @sayakpaul

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

I encountered the same problem. Currently I am using the scheduler as follows:

def ddim_inverse(self, x_0, cond, inverse_scheduler, mask=None, **kwargs):
      if mask is None:
          mask = torch.ones(x_0.shape[:-1], device=x_0.device)
      bs = x_0.shape[0]
      timesteps = inverse_scheduler.timesteps
      x_t = x_0

      for i, t in enumerate(timesteps[1:]):
          t = t.expand(bs)
          model_output = self.denoiser(x_t, cond, t, mask=mask, **kwargs)
          res = inverse_scheduler.step(model_output, t[0], x_t, return_dict=True)
          x_t = res.prev_sample

      return x_t

But I am not sure if this will have a negative effect on the quality of the result. I think this problem still need to be addressed.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

I'd like to help with this issue. I reproduced it at a71e62e on Windows/CPU with Diffusers 0.41.0.dev0, Python 3.11.15 and PyTorch 2.14.0+cpu.

The original example returns Inf. With both sample and model_output set to the same finite tensor, it returns NaN instead (0/0), so an exact x0 prediction still hits this boundary.

At source alpha_prod_t=1, the noise coefficient is zero, so the current sample and x0 prediction do not uniquely determine pred_epsilon. Assigning zero would introduce an additional initialization convention.

Would a clear ValueError when prediction_type="sample" encounters an actual zero source noise coefficient be appropriate, with documentation and CPU regression tests? Or is there a preferred initialization convention to support this boundary? I'd be happy to implement the agreed behavior.

For reference, the unmodified baseline python -X utf8 -m pytest -q tests/schedulers/test_scheduler_ddim_inverse.py gives 34 passed, 1 skipped. Its prediction-type test covers epsilon and v_prediction, but not sample.

Still reproduces on current main (beta_prod_t is 0 at the first inverse step with set_alpha_to_one=True + prediction_type="sample", so pred_epsilon division by beta_prod_t**0.5 diverges). I'd like to work on a fix: guard the epsilon division for the sample-prediction path (mirroring how the forward schedulers avoid divide-by-zero) and add the issue's repro as a regression test in tests/schedulers/test_scheduler_ddim_inverse.py.

A brief follow-up on the first-step sample-prediction boundary. I see #14822 is now closed without merge; before preparing another implementation, could a maintainer clarify whether this zero-noise boundary should raise a clear error or use an explicit initialization convention? I would be happy to refresh the CPU reproduction and add tests for the agreed behavior, with the AI assistance disclosed. Thanks for your time!

AI-assisted follow-up, posted at the contributor's request.

Navigation Menu

Repository navigation

[DDIMInverseScheduler] `inf` values at first iteration when `set_alpha_to_one=True` and `prediction_type="sample"` #10920

Describe the bug

Reproduction

Logs

System Info

Who can help?

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

[DDIMInverseScheduler] inf values at first iteration when set_alpha_to_one=True and prediction_type="sample" #10920

Description

Describe the bug

Reproduction

Logs

System Info

Who can help?

Activity

github-actions commented on Mar 29, 2025

Cr7st commented on Oct 27, 2025

github-actions commented on Feb 3, 2026

NaCr05 commented on Sep 11, 2026

DawnofGenX commented on Sep 20, 2026

NaCr05 commented on Oct 17, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

[DDIMInverseScheduler] `inf` values at first iteration when `set_alpha_to_one=True` and `prediction_type="sample"` #10920