Add EdenLive

docs/source/en/_toctree.yml

@@ -349,6 +349,8 @@
         title: DiTTransformer2DModel
       - local: api/models/easyanimate_transformer3d
         title: EasyAnimateTransformer3DModel
+      - local: api/models/edenlive_transformer_3d
+        title: EdenLiveTransformer3DModel
       - local: api/models/ernie_image_transformer2d
         title: ErnieImageTransformer2DModel
       - local: api/models/flux2_transformer
@@ -729,6 +731,8 @@
         title: Cosmos
       - local: api/pipelines/cosmos3
         title: Cosmos3
+      - local: api/pipelines/edenlive
+        title: EdenLive
       - local: api/pipelines/framepack
         title: Framepack
       - local: api/pipelines/helios
@@ -798,6 +802,8 @@
       title: DPMSolverSDEScheduler
     - local: api/schedulers/singlestep_dpm_solver
       title: DPMSolverSinglestepScheduler
+    - local: api/schedulers/edenlive
+      title: EdenLiveFlowMatchEulerDiscreteScheduler
     - local: api/schedulers/edm_multistep_dpm_solver
       title: EDMDPMSolverMultistepScheduler
     - local: api/schedulers/edm_euler

docs/source/en/api/models/edenlive_transformer_3d.md

@@ -0,0 +1,52 @@
+<!-- Copyright 2026 The VAST-AI Team. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
+an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
+specific language governing permissions and limitations under the License.
+-->
+
+# EdenLiveTransformer3DModel
+
+[EdenLive](https://github.com/VAST-AI-Research/EdenLive-Official) uses a video diffusion transformer conditioned on
+text, a reference image, and projected DINO proxy features. The same model class supports bidirectional and causal
+attention. See the [pipeline guide](../pipelines/edenlive) for video generation.
+
+```python
+import torch
+from diffusers import EdenLiveTransformer3DModel
+
+transformer = EdenLiveTransformer3DModel.from_pretrained(
+    "VAST-AI/EdenLive-5B-Diffusers", subfolder="transformer", torch_dtype=torch.bfloat16
+)
+```
+
+Use `subfolder="transformer_causal"` for the causal weights. Keep model timesteps in FP32 when calling the transformer
+directly, including when its weights are BF16.
+
+## EdenLiveTransformer3DModel
+
+[[autodoc]] EdenLiveTransformer3DModel
+    - forward
+    - create_causal_state
+
+## EdenLiveProxyProjection
+
+Projects DINO features into the transformer's conditioning space. Use the trained projection and DINO encoder from
+the same checkpoint; the projection returns FP32 features.
+
+[[autodoc]] EdenLiveProxyProjection
+    - forward
+
+## EdenLiveCausalState
+
+The causal pipeline manages this state automatically. For custom loops, create a separate state for each CFG branch
+with `transformer.create_causal_state()`. Denoise with `cache_update=False`, then commit each clean chunk at timestep
+zero with `cache_update=True`. Call `reset()` before an independent video.
+
+[[autodoc]] EdenLiveCausalState
+    - reset

docs/source/en/api/pipelines/edenlive.md

@@ -0,0 +1,140 @@
+<!-- Copyright 2026 The VAST-AI Team. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
+an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
+specific language governing permissions and limitations under the License.
+-->
+
+# EdenLive
+
+[EdenLive](https://github.com/VAST-AI-Research/EdenLive-Official) generates video from a proxy video, a reference
+image, and a text prompt. [`EdenLivePipeline`] supports bidirectional generation; [`EdenLiveCausalPipeline`]
+generates a complete video in causal chunks and supports changing prompts between chunks.
+
+## Basic usage
+
+Prepare an RGB reference image and a proxy video with at least 49 frames sampled at 24 fps. The image provides
+appearance, while the proxy provides motion and scene structure.
+
+```python
+import torch
+from diffusers import EdenLivePipeline
+from diffusers.utils import export_to_video, load_image, load_video
+
+pipe = EdenLivePipeline.from_pretrained(
+    "VAST-AI/EdenLive-5B-Diffusers",
+    torch_dtype={"default": torch.bfloat16, "proxy_encoder": torch.float32, "proxy_projection": torch.float32},
+).to("cuda")
+
+image = load_image("reference.png")
+video = load_video("proxy-24fps.mp4")
+
+frames = pipe(
+    image=image,
+    video=video,
+    prompt="A character explores a snowy landscape.",
+    num_frames=49,
+).frames[0]
+export_to_video(frames, "edenlive.mp4", fps=24)
+```
+
+Keep the proxy encoder and projection in FP32 when loading the other models in BF16, as shown above.
+
+## Causal generation
+
+Load the causal transformer and scheduler from the same repository. This works with either model size and uses
+four fixed denoising steps per chunk.
+
+```python
+from diffusers import EdenLiveCausalPipeline, EdenLiveTransformer3DModel, EdenLiveFlowMatchEulerDiscreteScheduler
+
+causal_pipe = EdenLiveCausalPipeline.from_pretrained(
+    "VAST-AI/EdenLive-5B-Diffusers",
+    transformer=EdenLiveTransformer3DModel.from_pretrained(
+        "VAST-AI/EdenLive-5B-Diffusers", subfolder="transformer_causal", torch_dtype=torch.bfloat16
+    ),
+    scheduler=EdenLiveFlowMatchEulerDiscreteScheduler.from_pretrained(
+        "VAST-AI/EdenLive-5B-Diffusers", subfolder="scheduler_causal"
+    ),
+    torch_dtype={"default": torch.bfloat16, "proxy_encoder": torch.float32, "proxy_projection": torch.float32},
+).to("cuda")
+```
+
+Use `prompt` for fixed text, or `chunk_prompts` to change it during a single video. At the default chunk size,
+49 frames require five prompts. Reusing the image and video above:
+
+```python
+frames = causal_pipe(
+    image=image,
+    video=video,
+    chunk_prompts=[
+        "A character walks through snow.",
+        "A character walks through snow.",
+        "The character approaches a campfire.",
+        "The character warms their hands beside the campfire.",
+        "The character rests beside the campfire.",
+    ],
+    num_frames=49,
+).frames[0]
+export_to_video(frames, "chunk-prompts.mp4", fps=24)
+```
+
+`chunk_prompts` overrides a single `prompt` and cannot be combined with `prompt_embeds`. The negative prompt remains
+fixed. A list passed as `prompt` represents a batch of independent videos. Each pipeline call starts a new video.
+
+## LightTAE decoder
+
+The full Wan VAE is the default. For faster decoding, pass the optional [`EdenLiveLightTAEDecoder`] to either pipeline:
+
+```python
+from diffusers import EdenLiveLightTAEDecoder
+
+pipe = EdenLivePipeline.from_pretrained(
+    "VAST-AI/EdenLive-5B-Diffusers",
+    decoder=EdenLiveLightTAEDecoder.from_pretrained(
+        "VAST-AI/EdenLive-5B-Diffusers", subfolder="decoder", torch_dtype=torch.bfloat16
+    ),
+    torch_dtype={"default": torch.bfloat16, "proxy_encoder": torch.float32, "proxy_projection": torch.float32},
+).to("cuda")
+```
+
+## Notes
+
+| Checkpoint | Default height × width | Example fps | Bidirectional steps |
+|---|---|---|---|
+| [5B](https://huggingface.co/VAST-AI/EdenLive-5B-Diffusers) | 704 × 1280 | 24 | 48 |
+| [14B](https://huggingface.co/VAST-AI/EdenLive-14B-Diffusers) | 480 × 832 | 16 | 30 |
+
+- Sample proxy frames at the desired output frame rate. `num_frames` must be $4k+1$ and fit within the input video;
+  omitting it uses the largest valid prefix. Custom dimensions must be multiples of 32 for 5B or 16 for 14B.
+- Guidance defaults to 3 for bidirectional generation and 1 for causal generation. Set `guidance_scale` above one
+  to use a `negative_prompt`.
+- For CPU offload, load without `.to("cuda")` and call `enable_model_cpu_offload()`.
+- See the [example README](https://github.com/huggingface/diffusers/tree/main/examples/edenlive) for installation,
+  video resampling, multi-GPU inference, and checkpoint conversion.
+
+## EdenLivePipeline
+
+[[autodoc]] EdenLivePipeline
+    - all
+    - __call__
+
+## EdenLiveCausalPipeline
+
+[[autodoc]] EdenLiveCausalPipeline
+    - all
+    - __call__
+
+## EdenLiveLightTAEDecoder
+
+[[autodoc]] EdenLiveLightTAEDecoder
+    - decode
+
+## EdenLivePipelineOutput
+
+[[autodoc]] pipelines.edenlive.pipeline_output.EdenLivePipelineOutput

docs/source/en/api/schedulers/edenlive.md

@@ -0,0 +1,27 @@
+<!-- Copyright 2026 The VAST-AI Team. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
+an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
+specific language governing permissions and limitations under the License.
+-->
+
+# EdenLiveFlowMatchEulerDiscreteScheduler
+
+[`EdenLiveFlowMatchEulerDiscreteScheduler`] implements deterministic flow Euler sampling for EdenLive 14B
+bidirectional generation and stochastic sampling for the distilled causal checkpoints. The 5B bidirectional model
+uses [`UniPCMultistepScheduler`].
+
+Use the scheduler saved with the checkpoint: `scheduler` for bidirectional generation and `scheduler_causal` for
+causal generation. The released causal schedules have four fixed steps per chunk. See the
+[pipeline guide](../pipelines/edenlive#causal-generation) for loading the causal components.
+
+## EdenLiveFlowMatchEulerDiscreteScheduler
+
+[[autodoc]] EdenLiveFlowMatchEulerDiscreteScheduler
+    - set_timesteps
+    - step

examples/edenlive/README.md

@@ -0,0 +1,75 @@
+# EdenLive inference
+
+Generate video from a reference image, proxy video, and prompt. See the [pipeline guide](../../docs/source/en/api/pipelines/edenlive.md)
+for the Python API and model defaults.
+
+## Setup
+
+Install a CUDA-compatible [PyTorch](https://pytorch.org/get-started/locally/) and torchvision, then run from the
+Diffusers repository root:
+
+```bash
+pip install -e . "transformers>=4.56.0" accelerate sentencepiece ftfy av imageio imageio-ffmpeg
+```
+
+The [requirements file](requirements.txt) also lists the example dependencies.
+
+## Single GPU
+
+```bash
+python examples/edenlive/inference_edenlive.py \
+  --model VAST-AI/EdenLive-5B-Diffusers --causal \
+  --video proxy.mp4 --image reference.png \
+  --prompt "A character explores a snowy landscape." \
+  --num-frames 49
+```
+
+Use `--causal` for the causal weights or omit it for bidirectional generation. `--model` also accepts a local
+Diffusers directory. The script resamples the proxy video to the model's default frame rate or `--fps`.
+`--num-frames` must be $4k+1$ and fit within the resampled video; omit it to use the largest valid prefix.
+
+The default output is `edenlive.mp4`, with timing and input details in `edenlive.json`. Set `--output` to change it.
+Use `--lighttae` for faster decoding or `--vae-tiling` to reduce VAE memory use. Run
+`python examples/edenlive/inference_edenlive.py --help` for all options.
+
+## Multiple GPUs
+
+Use Ulysses context parallelism and FSDP2 on two or four GPUs:
+
+```bash
+torchrun --standalone --nproc_per_node=2 examples/edenlive/inference_edenlive.py \
+  --model VAST-AI/EdenLive-14B-Diffusers --causal \
+  --video proxy.mp4 --image reference.png \
+  --prompt "A character explores a snowy landscape." \
+  --num-frames 81
+```
+
+Set `--nproc_per_node=4` for four GPUs. Add `--no-fsdp` to keep transformer weights replicated across GPUs.
+
+## Changing prompts
+
+With `--causal`, add `--chunk-prompts prompts.json`. For 49 frames, supply five prompts:
+
+```json
+[
+  "A character walks through snow.",
+  "A character walks through snow.",
+  "The character approaches a campfire.",
+  "The character warms their hands beside the campfire.",
+  "The character rests beside the campfire."
+]
+```
+
+The list replaces `--prompt`, with one prompt per chunk. The negative prompt remains fixed.
+
+## Convert original checkpoints
+
+To convert a local original release instead of loading the published Diffusers weights:
+
+```bash
+python scripts/convert_edenlive_to_diffusers.py \
+  --source /path/to/EdenLive --size 5b --output /path/to/EdenLive-5B-Diffusers
+```
+
+Use `--size 14b` and a separate output directory for 14B. Each package includes both sampling variants and the
+optional LightTAE decoder.