LCM Lora Gets an Update and Its Crazy Fast
Image generated with SDXL in 4 steps using an LCM LoRA.
LCM LoRA - Make Stable Diffusion images up to 10x FASTER! Sd 1.5, SDXL & SSD-1B
What are LCM LoRA models?
Latent Consistency Models (LCM) are a way to decrease the number of steps required to generate an image with Stable Diffusion and are very promising for multiple image generations.
LoRA stands for Low-Rank Adaptation. It allows you to use low-rank adaptation technology to quickly fine-tune diffusion models.
what’s the trick?
For latent consistency distillation, each model needs to be distilled separately. The core idea with LCM LoRA is to train just a small number of adapters, known as LoRA layers, instead of the full model. The resulting LoRAs can then be applied to any fine-tuned version of the model without having to distil them separately. If you are itching to see how this looks in practice, just jump to the next section to play with the inference code. If you want to train your own LoRAs, this is the process you’d use:
Select an available teacher model from the Hub. For example, you can use SDXL (base), or any fine-tuned or dreamboothed version you like.
Train a LCM LoRA on the model. LoRA is a type of performance-efficient fine-tuning, or PEFT, that is much cheaper to accomplish than full model fine-tuning. For additional details on PEFT, please check this blog post or the diffusers LoRA documentation.
Use the LoRA with any SDXL diffusion model and the LCM scheduler; bingo! You get high-quality inference in just a few steps.
LCM LoRAs and Models:
Latent Consistency Models LoRAs Collection
latent-consistency/lcm-lora-sdxl. LCM LoRA for SDXL 1.0 base, as seen in the examples above.
latent-consistency/lcm-lora-sdv1-5. LCM LoRA for Stable Diffusion 1.5.
latent-consistency/lcm-lora-ssd-1b. LCM LoRA for segmind/SSD-1B, a distilled SDXL model that's 50% smaller and 60% faster than the original SDXL.
latent-consistency/lcm-sdxl. Full fine-tuned consistency model derived from SDXL 1.0 base.
latent-consistency/lcm-ssd-1b. Full fine-tuned consistency model derived from segmind/SSD-1B.
How to install?
download LCMLoRA_SD15 here.
and copy it to “ \stable-diffusion-webui\models\Lora “
LCM collection for other models:
https://huggingface.co/collections/latent-consistency/latent-consistency-models-loras-654cdd24e111e16f0865fba6
