[TroyHernandez]

LTX keyframes: condition a generation on latents, an image or frames at any frame, at any strength.

txt2vid_ltx2(prompt, pipe, keyframes = list(
  ltx23_keyframe(start_img, frame = 0L),
  ltx23_keyframe(mid_still, frame = 48L, strength = 0.8),
  ltx23_keyframe(end_img, frame = 96L)))

Change

• ltx23_keyframe(x, frame, strength). x can be latents ([1, 128, k, H/32, W/32]), an image path, or an image or frames array.
• Frame 0. The keyframe replaces the first latent frames. image, condition_video and condition_latents are now frame-0 keyframes and can be combined with others.
• Any other frame. The keyframe's latent tokens are appended to the sequence, positioned at that pixel frame, then dropped before decoding:
  • no causal first-frame adjustment;
  • a single image spans one frame.
• Strength below 1. Uses a per-token conditioning mask:
  • timestep (1 - s) * sigma;
  • each step's x0 estimate is blended back toward the clean keyframe.
  • The transformer's compact per-token timestep index now carries one entry per strength value.
• Source. Ported from diffusers pipelines/ltx2/pipeline_ltx2_condition.py (Apache-2.0). Recorded in inst/REFERENCES.md.
• Not included:
  • Replacing latents at interior frames. It exists only in Lightricks' monorepo, which diffuseR uses for facts, not code.
  • Keyframes with two_stage. This is still refused.

Checks

• Tests: new test_keyframes_ltx23.R (36 results) on the tiny CPU pipeline, plus the existing test_txt2vid_ltx23.R (44). They cover:

  • the constructor;
  • the starting state (replaced, free and appended tokens, masks, coordinates);
  • frame-0 equality with condition_latents and image;
  • partial strength;
  • the coordinate math against diffusers' _prepare_keyframe_coords.

• Unchanged paths: image, condition_video, and condition_video with cond_noise_scale give bit-identical output to main.

• GPU, LTX-2.5 NF4, 512x512x97 (PSNR of the target frame against its keyframe):

  Case	With keyframe	Without
  Still at frame 48	24.7 dB	14.7 dB
  A clip's first frame repeated at frame 96 (loop)	23.6 dB	—

  Interior keyframes guide strongly but don't copy.

Found while testing, not fixed here

• Third-render OOM on main. With NF4 on the default native allocator, a third render in one process that VAE-encodes pixels runs out of memory: about 11 GiB is reserved but unallocated, and a 2 MiB request fails. It runs with expandable_segments. Separate PR.
• No H.264 recompression of conditioning stills. diffusers re-encodes conditioning stills with H.264 before the VAE (CRF 18 for 2.5, 33 for 2.3), and diffuseR doesn't yet.