Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/source/en/api/pipelines/bria_fibo.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ Text-to-image models have mastered imagination - but not control. FIBO changes t

FIBO is trained on structured JSON captions up to 1,000+ words and designed to understand and control different visual parameters such as lighting, composition, color, and camera settings, enabling precise and reproducible outputs.

With only 8 billion parameters, FIBO provides a new level of image quality, prompt adherence and proffesional control.
With only 8 billion parameters, FIBO provides a new level of image quality, prompt adherence and professional control.

FIBO is trained exclusively on a structured prompt and will not work with freeform text prompts.
you can use the [FIBO-VLM-prompt-to-JSON](https://huggingface.co/briaai/FIBO-VLM-prompt-to-JSON) model or the [FIBO-gemini-prompt-to-JSON](https://huggingface.co/briaai/FIBO-gemini-prompt-to-JSON) to convert your freeform text prompt to a structured JSON prompt.
Expand Down
2 changes: 1 addition & 1 deletion docs/source/en/api/pipelines/wan.md
Original file line number Diff line number Diff line change
Expand Up @@ -531,7 +531,7 @@ export_to_video(output, "animated_advanced.mp4", fps=30)

- Try lower `shift` values (`2.0` to `5.0`) for lower resolution videos and higher `shift` values (`7.0` to `12.0`) for higher resolution images.

- Wan 2.1 and 2.2 support using [LightX2V LoRAs](https://huggingface.co/Kijai/WanVideo_comfy/tree/main/Lightx2v) to speed up inference. Using them on Wan 2.2 is slightly more involed. Refer to [this code snippet](https://github.com/huggingface/diffusers/pull/12040#issuecomment-3144185272) to learn more.
- Wan 2.1 and 2.2 support using [LightX2V LoRAs](https://huggingface.co/Kijai/WanVideo_comfy/tree/main/Lightx2v) to speed up inference. Using them on Wan 2.2 is slightly more involved. Refer to [this code snippet](https://github.com/huggingface/diffusers/pull/12040#issuecomment-3144185272) to learn more.

- Wan 2.2 has two denoisers. By default, LoRAs are only loaded into the first denoiser. One can set `load_into_transformer_2=True` to load LoRAs into the second denoiser. Refer to [this](https://github.com/huggingface/diffusers/pull/12074#issue-3292620048) and [this](https://github.com/huggingface/diffusers/pull/12074#issuecomment-3155896144) examples to learn more.

Expand Down
4 changes: 2 additions & 2 deletions docs/source/en/optimization/memory.md
Original file line number Diff line number Diff line change
Expand Up @@ -156,7 +156,7 @@ pipeline = StableDiffusionXLPipeline.from_pretrained(
)
```

Diffusers uses the maxmium memory of all devices by default, but if they don't fit on the GPUs, then you'll need to use a single GPU and offload to the CPU with the methods below.
Diffusers uses the maximum memory of all devices by default, but if they don't fit on the GPUs, then you'll need to use a single GPU and offload to the CPU with the methods below.

- [`~DiffusionPipeline.enable_model_cpu_offload`] only works on a single GPU but a very large model may not fit on it
- [`~DiffusionPipeline.enable_sequential_cpu_offload`] may work but it is extremely slow and also limited to a single GPU
Expand Down Expand Up @@ -470,7 +470,7 @@ apply_layerwise_casting(

## torch.channels_last

[torch.channels_last](https://pytorch.org/tutorials/intermediate/memory_format_tutorial.html) flips how tensors are stored from `(batch size, channels, height, width)` to `(batch size, heigh, width, channels)`. This aligns the tensors with how the hardware sequentially accesses the tensors stored in memory and avoids skipping around in memory to access the pixel values.
[torch.channels_last](https://pytorch.org/tutorials/intermediate/memory_format_tutorial.html) flips how tensors are stored from `(batch size, channels, height, width)` to `(batch size, height, width, channels)`. This aligns the tensors with how the hardware sequentially accesses the tensors stored in memory and avoids skipping around in memory to access the pixel values.

Not all operators currently support the channels-last format and may result in worst performance, but it is still worth trying.

Expand Down
2 changes: 1 addition & 1 deletion src/diffusers/guiders/magnitude_aware_guidance.py
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ class MagnitudeAwareGuidance(BaseGuidance):
prompt, while lower values allow for more freedom in generation. Higher values may lead to saturation and
deterioration of image quality.
alpha (`float`, defaults to `8.0`):
The alpha parameter for the magnitude-aware guidance. Higher values cause more aggressive supression of
The alpha parameter for the magnitude-aware guidance. Higher values cause more aggressive suppression of
guidance scale when the magnitude of the guidance update is large.
guidance_rescale (`float`, defaults to `0.0`):
The rescale factor applied to the noise predictions. This is used to improve image quality and fix
Expand Down
2 changes: 1 addition & 1 deletion src/diffusers/models/attention.py
Original file line number Diff line number Diff line change
Expand Up @@ -1651,7 +1651,7 @@ def forward(
#
# The reasoning for the change here is `torch.where` became a bottleneck at some point when golfing memory
# spikes. It is particularly noticeable when the number of frames is high. My understanding is that this comes
# from tensors being copied - which is why we resort to spliting and concatenating here. I've not particularly
# from tensors being copied - which is why we resort to splitting and concatenating here. I've not particularly
# looked into this deeply because other memory optimizations led to more pronounced reductions.
hidden_states = torch.cat(
[
Expand Down
2 changes: 1 addition & 1 deletion src/diffusers/models/attention_dispatch.py
Original file line number Diff line number Diff line change
Expand Up @@ -2070,7 +2070,7 @@ def backward(ctx, grad_outputs):
return (None, grad_input, None, None)


# Below are helper functions to handle abritrary head num and abritrary sequence length for Ulysses Anything Attention.
# Below are helper functions to handle arbitrary head num and arbitrary sequence length for Ulysses Anything Attention.
def _maybe_pad_qkv_head(x: torch.Tensor, H: int, group: dist.ProcessGroup) -> tuple[torch.Tensor, int]:
r"""Maybe pad the head dimension to be divisible by world_size.
x: torch.Tensor, shape (B, S_LOCAL, H, D) H: int, original global head num return: tuple[torch.Tensor, int], padded
Expand Down
4 changes: 2 additions & 2 deletions src/diffusers/models/transformers/transformer_ltx2.py
Original file line number Diff line number Diff line change
Expand Up @@ -975,7 +975,7 @@ def prepare_audio_coords(
start=shift, end=num_frames + shift, step=self.patch_size_t, dtype=torch.float32, device=device
)

# 2. Calculate start timstamps in seconds with respect to the original spectrogram grid
# 2. Calculate start timestamps in seconds with respect to the original spectrogram grid
audio_scale_factor = self.scale_factors[0]
# Scale back to mel spectrogram space
grid_start_mel = grid_f * audio_scale_factor
Expand All @@ -984,7 +984,7 @@ def prepare_audio_coords(
# Convert mel bins back into seconds
grid_start_s = grid_start_mel * self.hop_length / self.sampling_rate

# 3. Calculate start timstamps in seconds with respect to the original spectrogram grid
# 3. Calculate start timestamps in seconds with respect to the original spectrogram grid
grid_end_mel = (grid_f + self.patch_size_t) * audio_scale_factor
grid_end_mel = (grid_end_mel + self.causal_offset - audio_scale_factor).clip(min=0)
grid_end_s = grid_end_mel * self.hop_length / self.sampling_rate
Expand Down
4 changes: 2 additions & 2 deletions src/diffusers/pipelines/chroma/pipeline_chroma.py
Original file line number Diff line number Diff line change
Expand Up @@ -628,14 +628,14 @@ def __call__(
The height in pixels of the generated image. This is set to 1024 by default for the best results.
width (`int`, *optional*, defaults to self.unet.config.sample_size * self.vae_scale_factor):
The width in pixels of the generated image. This is set to 1024 by default for the best results.
num_inference_steps (`int`, *optional*, defaults to 50):
num_inference_steps (`int`, *optional*, defaults to 35):
The number of denoising steps. More denoising steps usually lead to a higher quality image at the
expense of slower inference.
sigmas (`list[float]`, *optional*):
Custom sigmas to use for the denoising process with schedulers which support a `sigmas` argument in
their `set_timesteps` method. If not defined, the default behavior when `num_inference_steps` is passed
will be used.
guidance_scale (`float`, *optional*, defaults to 3.5):
guidance_scale (`float`, *optional*, defaults to 5.0):
Guidance scale as defined in [Classifier-Free Diffusion
Guidance](https://huggingface.co/papers/2207.12598). `guidance_scale` is defined as `w` of equation 2.
of [Imagen Paper](https://huggingface.co/papers/2205.11487). Guidance scale is enabled by setting
Expand Down
2 changes: 1 addition & 1 deletion src/diffusers/pipelines/chroma/pipeline_chroma_img2img.py
Original file line number Diff line number Diff line change
Expand Up @@ -698,7 +698,7 @@ def __call__(
Custom sigmas to use for the denoising process with schedulers which support a `sigmas` argument in
their `set_timesteps` method. If not defined, the default behavior when `num_inference_steps` is passed
will be used.
guidance_scale (`float`, *optional*, defaults to 3.5):
guidance_scale (`float`, *optional*, defaults to 5.0):
Guidance scale as defined in [Classifier-Free Diffusion
Guidance](https://huggingface.co/papers/2207.12598). `guidance_scale` is defined as `w` of equation 2.
of [Imagen Paper](https://huggingface.co/papers/2205.11487). Guidance scale is enabled by setting
Expand Down
8 changes: 4 additions & 4 deletions src/diffusers/pipelines/chroma/pipeline_chroma_inpainting.py
Original file line number Diff line number Diff line change
Expand Up @@ -828,20 +828,20 @@ def __call__(
with the same aspect ratio of the image and contains all masked area, and then expand that area based
on `padding_mask_crop`. The image and mask_image will then be cropped based on the expanded area before
resizing to the original image size for inpainting.
num_inference_steps (`int`, *optional*, defaults to 35):
num_inference_steps (`int`, *optional*, defaults to 28):
The number of denoising steps. More denoising steps usually lead to a higher quality image at the
expense of slower inference.
sigmas (`List[float]`, *optional*):
Custom sigmas to use for the denoising process with schedulers which support a `sigmas` argument in
their `set_timesteps` method. If not defined, the default behavior when `num_inference_steps` is passed
will be used.
guidance_scale (`float`, *optional*, defaults to 3.5):
guidance_scale (`float`, *optional*, defaults to 7.0):
Guidance scale as defined in [Classifier-Free Diffusion
Guidance](https://huggingface.co/papers/2207.12598). `guidance_scale` is defined as `w` of equation 2.
of [Imagen Paper](https://huggingface.co/papers/2205.11487). Guidance scale is enabled by setting
`guidance_scale > 1`. Higher guidance scale encourages to generate images that are closely linked to
the text `prompt`, usually at the expense of lower image quality.
strength (`float, *optional*, defaults to 0.9):
strength (`float, *optional*, defaults to 0.6):
Conceptually, indicates how much to transform the reference image. Must be between 0 and 1. image will
be used as a starting point, adding more noise to it the larger the strength. The number of denoising
steps depends on the amount of noise initially added. When strength is 1, added noise will be maximum
Expand Down Expand Up @@ -900,7 +900,7 @@ def __call__(
The list of tensor inputs for the `callback_on_step_end` function. The tensors specified in the list
will be passed as `callback_kwargs` argument. You will only be able to include variables listed in the
`._callback_tensor_inputs` attribute of your pipeline class.
max_sequence_length (`int` defaults to 512): Maximum sequence length to use with the `prompt`.
max_sequence_length (`int` defaults to 256): Maximum sequence length to use with the `prompt`.

Examples:

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -1068,7 +1068,7 @@ def forward(
Returns:
`tuple`: Returns relevant parts of the model's noise prediction: the first element of the tuple is tbe VAE
`tuple`: Returns relevant parts of the model's noise prediction: the first element of the tuple is the VAE
image embedding, the second element is the CLIP image embedding, and the third element is the CLIP text
embedding.
"""
Expand Down
2 changes: 1 addition & 1 deletion src/diffusers/pipelines/ltx2/pipeline_ltx2.py
Original file line number Diff line number Diff line change
Expand Up @@ -271,7 +271,7 @@ def __init__(
self.vae_temporal_compression_ratio = (
self.vae.temporal_compression_ratio if getattr(self, "vae", None) is not None else 8
)
# TODO: check whether the MEL compression ratio logic here is corrct
# TODO: check whether the MEL compression ratio logic here is correct
self.audio_vae_mel_compression_ratio = (
self.audio_vae.mel_compression_ratio if getattr(self, "audio_vae", None) is not None else 4
)
Expand Down
2 changes: 1 addition & 1 deletion src/diffusers/pipelines/ltx2/pipeline_ltx2_image2video.py
Original file line number Diff line number Diff line change
Expand Up @@ -274,7 +274,7 @@ def __init__(
self.vae_temporal_compression_ratio = (
self.vae.temporal_compression_ratio if getattr(self, "vae", None) is not None else 8
)
# TODO: check whether the MEL compression ratio logic here is corrct
# TODO: check whether the MEL compression ratio logic here is correct
self.audio_vae_mel_compression_ratio = (
self.audio_vae.mel_compression_ratio if getattr(self, "audio_vae", None) is not None else 4
)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -223,7 +223,7 @@ def check_inputs(
f"got {type(task_prompt)} and {type(content_prompt)}"
)
if len(content_prompt) != len(task_prompt):
raise ValueError("`task_prompt` and `content_prompt` must have the same length whe they are lists.")
raise ValueError("`task_prompt` and `content_prompt` must have the same length when they are lists.")

for sample in image:
if not isinstance(sample, list) or not isinstance(sample[0], list):
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -442,7 +442,7 @@ def check_inputs(
f"got {type(task_prompt)} and {type(content_prompt)}"
)
if len(content_prompt) != len(task_prompt):
raise ValueError("`task_prompt` and `content_prompt` must have the same length whe they are lists.")
raise ValueError("`task_prompt` and `content_prompt` must have the same length when they are lists.")

for sample in image:
if not isinstance(sample, list) or not isinstance(sample[0], list):
Expand Down
Loading