Awesome Efficient Video Generation

A curated list of recent efficient video generation methods.

🗂️ Table of Contents

• Scheduler
• Distillation
• Quantization
• Caching
• Pruning
• Parallelism
• Others

📄 Papers

Scheduler

• Pyramidal Flow Matching for Efficient Video Generative Modeling
  arXiv 2024 | Paper | Code |

• AdaDiff: Adaptive Step Selection for Fast Diffusion Models
  arXiv 2023 | Paper

• Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference
  ICLR 2024 | Paper | Code |

• DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models
  arXiv 2022 | Paper | Code |

• Flow Matching for Generative Modeling
  ICLR 2023 (Spotlight) | Paper

• Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow
  ICLR 2023 (Spotlight) | Paper | Code |

• DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps
  NeurIPS 2022 | Paper | Code |

• Denoising Diffusion Implicit Models
  ICLR 2021 | Paper | Code |

Distillation

• Distilling Diffusion Models into Conditional GANs
  ECCV 2024 | Paper

• Improved Distribution Matching Distillation for Fast Image Synthesis
  NeurIPS 2024 (Oral) | Paper | Code |

• Fast High-Resolution Image Synthesis with Latent Adversarial Diffusion Distillation
  arXiv 2024 | Paper

• LCM-LoRA: A Universal Stable-Diffusion Acceleration Module
  arXiv 2023 | Paper | Code |

• One-step Diffusion with Distribution Matching Distillation
  CVPR 2024 | Paper

• Adversarial Diffusion Distillation
  ECCV 2024 | Paper

Quantization

Caching

• Adaptive Caching for Faster Video Generation with Diffusion Transformers
  arXiv 2024 | Paper | Code |

• FasterCache: Training-Free Video Diffusion Model Acceleration with High Quality
  arXiv 2024 | Paper | Code |

• Accelerating Diffusion Transformers with Token-wise Feature Caching
  arXiv 2024 | Paper | Code |

• Real-Time Video Generation with Pyramid Attention Broadcast
  arXiv 2024 | Paper | Code |

• ∆-DiT: A Training-Free Acceleration Method Tailored for Diffusion Transformers
  arXiv 2024 | Paper

• DiTFastAttn: Attention Compression for Diffusion Transformer Models
  arXiv 2024 | Paper | Code |

• Faster Diffusion via Temporal Attention Decomposition
  arXiv 2024 | Paper | Code |

• Cross-Attention Makes Inference Cumbersome in Text-to-Image Diffusion Models
  arXiv 2024 | Paper | Code |

Pruning

Parallelism

• xDiT: an Inference Engine for Diffusion Transformers (DiTs) with Massive Parallelism
  arXiv 2024 | Paper | Code |

• PipeFusion: Patch-level Pipeline Parallelism for Diffusion Transformers Inference
  arXiv 2024 | Paper | Code |

• DSP: Dynamic Sequence Parallelism for Multi-Dimensional Transformers
  arXiv 2024 | Paper | Code |

• DistriFusion: Distributed Parallel Inference for High-Resolution Diffusion Models
  CVPR 2024 (Highlight) | Paper | Code |

Others