AnimateDiff text-to-video on SD 1.5, with FreeNoise long clips

DESCRIPTION

@@ -1,6 +1,6 @@
 Package: diffuseR
 Title: Functional Interface to Diffusion Models in R
-Version: 0.2.2.24
+Version: 0.2.2.25
 Authors@R: c(
     person("Troy", "Hernandez", email = "troy@cornball.ai", role = c("aut", "cre"),
            comment = c(ORCID = "0009-0005-4248-604X")),

NAMESPACE

@@ -1,5 +1,6 @@
 # tinyrox says don't edit this manually, but it can't stop you!
 
+export(animatediff_load_pipeline)
 export(auto_devices)
 export(bpe_tokenizer)
 export(clear_vram)
@@ -9,12 +10,14 @@ export(convert_sd21_pt_to_diffusers)
 export(ddim_scheduler_create)
 export(ddim_scheduler_step)
 export(decode_bpe)
+export(download_animatediff)
 export(download_component)
 export(download_flux1)
 export(download_flux2_klein)
 export(download_flux3_klein)
 export(download_ltx2)
 export(download_model)
+export(download_sd15)
 export(download_sd21)
 export(download_sdxl)
 export(download_zimage_turbo)
@@ -105,6 +108,7 @@ export(load_gemma3_text_encoder)
 export(load_gemma4_nf4)
 export(load_gemma4_text_encoder)
 export(load_model_component)
+export(load_motion_adapter)
 export(load_pipeline)
 export(load_qwen3_text_encoder)
 export(load_t5_text_encoder)
@@ -202,6 +206,7 @@ export(ltx23_vocoder_resblock)
 export(ltx23_vocoder_with_bwe)
 export(ltx25_open_checkpoint)
 export(models2devices)
+export(motion_adapter)
 export(offload_to_cpu)
 export(post_quant_conv)
 export(preprocess_image)
@@ -241,7 +246,9 @@ export(txt2img_flux3)
 export(txt2img_sd21)
 export(txt2img_sdxl)
 export(txt2img_zimage)
+export(txt2vid_animatediff)
 export(txt2vid_ltx2)
+export(unet_motion)
 export(unet_native)
 export(unet_native_from_safetensors)
 export(unet_native_from_torchscript)

NEWS.md

@@ -1,3 +1,41 @@
+# diffuseR 0.2.2.25
+
+* AnimateDiff text-to-video on Stable Diffusion 1.5:
+  `animatediff_load_pipeline()` and `txt2vid_animatediff()`, with the
+  v3 or v2 motion module (Apache-2.0). `download_animatediff()`
+  converts the original `.ckpt` in R; `download_sd15()` fetches SD 1.5.
+  - Prompt keyframes (`prompt = c("1" = ..., "16" = ...)`) blend the
+    prompt embeddings frame by frame, so the subject changes over the
+    clip.
+  - Clips longer than 16 frames use FreeNoise: the motion modules
+    attend within overlapping 16-frame windows (`context_length`,
+    `context_stride`) and later windows start from shuffled copies of
+    earlier noise. The modules were trained on 16-frame clips; in one
+    window of 20, 24 or 32 frames both diffusers and diffuseR give
+    noise, so `context_length` above 16 warns. The last partial window
+    is averaged with per-frame weights; diffusers counts it with one
+    scalar weight, which at 26 frames turned the last frame into
+    swirls.
+  - SD 1.5 support in the native UNet (`num_attention_heads`, 1x1 conv
+    projections) and `sd_pipeline_from_safetensors(model_name = "sd15")`.
+  - The SD UNet spatial transformers use fused SDPA and GroupNorm eps
+    1e-6, as in diffusers.
+  - Verified against diffusers on the real weights (fp32 CPU): text
+    encoder 3.7e-6, image UNet 7.7e-6, motion UNet 6.7e-6 (v3) and
+    7.9e-6 (v2) max abs, FreeNoise forward at 24 frames 6.8e-6;
+    end to end at 256x256x8 88.5 dB PSNR. GPU fp16 at 512x512x32
+    against diffusers on the GPU: 43.6 dB.
+  - Memory: the VAE decodes one frame at a time (`decode_chunk = 1`;
+    4 frames set a 13.9 GiB peak at 512x512x16). The loader sizes the
+    allocator gc gates from the GPU memory free at load and turns on
+    expandable segments, and the motion UNet runs a minor gc after each
+    layer (`diffuseR.block_gc`): R's gc had taken 69-78 s of a 95-105 s
+    render.
+  - Measured on the RTX 5060 Ti 16 GB, 25 steps, with a second process
+    holding the card to 10.3 GiB free: 512x512x16 37 s (allocated peak
+    9.3 GiB), 512x512x32 112 s (9.6 GiB), 768x768x16 106 s (9.9 GiB);
+    the same speed as on the free card. Pipeline load about 13 s.
+
 # diffuseR 0.2.2.24
 
 * Image models in a ~10.5 GiB GPU budget (a 16 GB card that keeps a