You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Community pipeline: rerank Stable Diffusion candidates by CLIP score #14721
A single sample from a txt2img pipeline can miss the prompt (wrong count, dropped attribute, ignored token) even when the average output at that seed/guidance combo is fine. I put together a community pipeline that draws N candidates for one prompt, scores each one against the prompt with a CLIP model, and returns them sorted best-first.
It wraps a plain StableDiffusionPipeline built from the same components you'd pass to the regular one, so it works with whatever UNet/VAE/scheduler combination that pipeline already accepts. It doesn't touch the denoising trajectory itself, unlike CLIPGuidedStableDiffusion, which steers sampling with CLIP gradients as it goes. This one just generates independently and picks.
No GPU on the box I'm on, so I verified it against the tiny fixtures diffusers' own test suite uses (hf-internal-testing/tiny-stable-diffusion-torch + a CLIP model built from tiny-random-clip's own config) rather than a full-size checkpoint. That confirms the plumbing, not image quality:
num images returned: 4
clip_scores (best-first): [1.9110276699066162, 1.8932260274887085, 1.8369966745376587, 1.784890055656433]
default return_all=False -> images: 1 scores: [3.9261884689331055]
generator/num_candidates mismatch correctly rejected: Got 1 generators for num_candidates=2.
batched prompt correctly rejected: CLIPRerankStableDiffusionPipeline reranks candidates for one prompt at a time; call it once per prompt if you have a batch.
OK: all checks passed
Also ran ruff check and ruff format --check clean against the file.
One open question: would you rather this lived as a method/flag on the existing pipeline (return_dict-style opt-in) than a separate community pipeline class? I went with a separate class since that's the shape the rest of examples/community uses, but a rerank option is a smaller surface if that's preferred.
Happy to open a PR with this, including a couple of tests against the same tiny fixtures.
Checking back on this one: would this fit better as a new community pipeline, or as an opt-in flag on the existing pipeline? Either way I can open the PR with tests against the tiny fixtures above.
Activity
AmirF194 commentedon Sep 20, 2026
Checking back on this one: would this fit better as a new community pipeline, or as an opt-in flag on the existing pipeline? Either way I can open the PR with tests against the tiny fixtures above.