🤗 Introduction
🚀🚀🚀A curated list of papers on controllable video generation. Please join us for more comprehensive summary. If you have any additions to the list, please raise them in the issue section. 欢迎补充👏
💖 Citation
If you find this repo is useful for your research, welcome to 🌟 this repo and cite our work using the following BibTeX:
@article{ma2025controllable,
title={Controllable video generation: A survey},
author={Ma, Yue and Feng, Kunyu and Hu, Zhongyuan and Wang, Xinyu and Wang, Yucheng and Zheng, Mingzhe and He, Xuanhua and Zhu, Chenyang and Liu, Hongyu and He, Yingqing and others},
journal={arXiv preprint arXiv:2507.16869},
year={2025}
}
📋 Contents
- 🌐 Structure Control
- 🧬 ID Control
- 🖼️ Image Control
- ⏳ Temporal Control
- 🔊 Audio Control
- 🧩 Other Controls
- 🌀 Universal Control
💘 Tips
- ✅ Paper searching via catalog: directly clicking the content of the catalog to select the area of your research and browse related papers.
- ✅ Paper searching via author name: Feel free to search papers of a specific author via
ctrl + Fand then type the author name. The dropdown list of authors will automatically expand when searching. - ✅ Multicontrol indicator: The tag
indicates "multi-control" following the title of the paper.
🌐 Structure Control
🏙️ Layout-Guided
- DynamicCity: Large-Scale 4D Occupancy Generation from Dynamic Scenes
[ICLR 2025 Spotlight] Hengwei Bian, Lingdong Kong, et al.
Hengwei Bian, Lingdong Kong, Haozhe Xie, Liang Pan, Yu Qiao, Ziwei Liu
🕺 Pose-Guided
- Hand2World: Autoregressive Egocentric Interaction Generation via Free-Space Hand Gestures (10 Feb 2026)
Yuxi Wang, Wenqi Ouyang, Tianyi Wei, et al.
Yuxi Wang, Wenqi Ouyang, Tianyi Wei, Yi Dong, Zhiqi Shen, Xingang Pan
- MTVCrafter: 4D Motion Tokenization for Open-World Human Image Animation
Yanbo Ding, Xirui Hu, Zhizhi Guo, et al.
Yanbo Ding, Xirui Hu, Zhizhi Guo, Chi Zhang, Yali Wang
- HyperMotion: DiT-Based Pose-Guided Human Image Animation of Complex Motions
Shuolin Xu, Siming Zheng, Ziyi Wang, et al.
Shuolin Xu, Siming Zheng, Ziyi Wang, HC Yu, Jinwei Chen, Huaqi Zhang, Bo Li, Peng-Tao Jiang
- FlexiAct: Towards Flexible Action Control in Heterogeneous Scenarios
[SIGGRAPH 2025] Shiyi Zhang*, Junhao Zhuang*, et al.
Shiyi Zhang*, Junhao Zhuang*, Zhaoyang Zhang, Ying Shan, Yansong Tang
- DualReal: Joint Training for Lossless Identity-Motion Fusion in Video Customization
Wenchuan Wang, Mengqi Huang, Yijing Tu, et al.
Wenchuan Wang, Mengqi Huang, Yijing Tu, Zhendong Mao
- DanceTogether! Identity-Preserving Multi-Person Interactive Video Generation
Junhao Chen, Mingjin Chen, Jianjin Xu, et al.
Junhao Chen, Mingjin Chen, Jianjin Xu, Xiang Li, Junting Dong, Mingze Sun, Puhua Jiang, Hongxiang Li, Yuhang Yang, Hao Zhao, Xiaoxiao Long, Ruqi Huang
- TASTE-Rob: Advancing Video Generation of Task-Oriented Hand-Object Interaction for Generalizable Robotic Manipulation
Hongxiang Zhao, Xingchen Liu, et al.
Hongxiang Zhao, Xingchen Liu, Mutian Xu, Yiming Hao, Weikai Chen, Xiaoguang Han
- Point-to-Point Video Generation (14 Mar 2025)
Tsun-Hsuan Wang, Yen-Chi Cheng, et al.
Tsun-Hsuan Wang, Yen-Chi Cheng, Chieh Hubert Lin, Hwann-Tzong Chen, Min Sun
- Pose Guided Human Video Generation (14 Mar 2025)
Ceyuan Yang, Zhe Wang, et al.
Ceyuan Yang, Zhe Wang, Xinge Zhu, Chen Huang, Jianping Shi, Dahua Lin
- SkyReels-A1: Expressive Portrait Animation in Video Diffusion Transformers
Di Qiu, Zhengcong Fei, et al.
Di Qiu, Zhengcong Fei, Rui Wang, Jialin Bai, Changqian Yu, Mingyuan Fan, Guibin Chen, Xiang Wen
- AnyCharV: Bootstrap Controllable Character Video Generation with Fine-to-Coarse Guidance
Zhao Wang, Hao Wen, et al.
Zhao Wang, Hao Wen, Lingting Zhu, Chenming Shang, Yujiu Yang, Qi Dou
- Animate Anyone 2: High-Fidelity Character Image Animation with Environment Affordance
Li Hu, Guangyuan Wang, et al.
Li Hu, Guangyuan Wang, Zhen Shen, Xin Gao, Dechao Meng, Lian Zhuo, Peng Zhang, Bang Zhang, Liefeng Bo
- DirectorLLM for Human-Centric Video Generation (19 Dec 2024)
Kunpeng Song, Tingbo Hou, et al.
Kunpeng Song, Tingbo Hou, Zecheng He, Haoyu Ma, Jialiang Wang, Animesh Sinha, Sam Tsai, Yaqiao Luo, Xiaoliang Dai, Li Chen, Xide Xia, Peizhao Zhang, Peter Vajda, Ahmed Elgammal, Felix Juefei-Xu
- Generative Inbetweening through Frame-wise Conditions-Driven Video Generation
[CVPR 2025] Tianyi Zhu, Dongwei Ren, et al.
Tianyi Zhu, Dongwei Ren, Qilong Wang, Xiaohe Wu, Wangmeng Zuo
- DisPose: Disentangling Pose Guidance for Controllable Human Image Animation
[ICLR2025] Hongxiang Li, Yaowei Li, et al.
Hongxiang Li, Yaowei Li, Yuhang Yang, Junjie Cao, Zhihong Zhu, Xuxin Cheng, Long Chen
- StableAnimator: High-Quality Identity-Preserving Human Image Animation
[CVPR 2025] Shuyuan Tu, Zhen Xing, Xintong Han, et al.
Shuyuan Tu, Zhen Xing, Xintong Han, Zhi-Qi Cheng, Qi Dai, Chong Luo, Zuxuan Wu
- AnimateAnywhere: Context-Controllable Human Video Generation with ID-Consistent One-shot Learning
[ACMMM 2024] Hengyuan Liu, Xiaodong Chen, Xinchen Liu, et al.
Hengyuan Liu, Xiaodong Chen, Xinchen Liu, Xiaoyan Gu, Wu Liu
- ControlNeXt: Powerful and Efficient Control for Image and Video Generation
Bohao Peng, Jian Wang, et al.
Bohao Peng, Jian Wang, Yuechen Zhang, Wenbo Li, Ming-Chang Yang, Jiaya Jia
- TCAN: Animating Human Images with Temporally Consistent Pose Guidance using Diffusion Models
Jeongho Kim, Min-Jung Kim, Junsoo Lee, et al.
Jeongho Kim, Min-Jung Kim, Junsoo Lee, Jaegul Choo
- MimicMotion: High-Quality Human Motion Video Generation with Confidence-aware Pose Guidance
Yuang Zhang, Jiaxi Gu, Li-Wen Wang, et al.
Yuang Zhang, Jiaxi Gu, Li-Wen Wang, Han Wang, Junqi Cheng, Yuefeng Zhu, Fangyuan Zou
- Follow-Your-Pose v2: Multiple-Condition Guided Character Image Animation for Stable Pose Control
Jingyun Xue, Hongfa Wang, Qi Tian, et al.
Jingyun Xue, Hongfa Wang, Qi Tian, Yue Ma, Andong Wang, Zhiyuan Zhao, Shaobo Min, Wenzhe Zhao, Kaihao Zhang, Heung-Yeung Shum, Wei Liu, Mengyang Liu, Wenhan Luo
- UniAnimate: Taming Unified Video Diffusion Models for Consistent Human Image Animation
Xiang Wang, Shiwei Zhang, Changxin Gao, et al.
Xiang Wang, Shiwei Zhang, Changxin Gao, Jiayu Wang, Xiaoqiang Zhou, Yingya Zhang, Luxin Yan, Nong Sang
- VividPose: Advancing Stable Video Diffusion for Realistic Human Image Animation
Qilin Wang, Zhengkai Jiang, Chengming Xu, et al.
Qilin Wang, Zhengkai Jiang, Chengming Xu, Jiangning Zhang, Yabiao Wang, Xinyi Zhang, Yun Cao, Weijian Cao, Chengjie Wang, Yanwei Fu
- Disentangling Foreground and Background Motion for Enhanced Realism in Human Video Generation
Jinlin Liu, Kai Yu, Mengyang Feng, et al.
Jinlin Liu, Kai Yu, Mengyang Feng, Xiefan Guo, Miaomiao Cui
- PoseCrafter: One-Shot Personalized Video Synthesis Following Flexible Pose Control
[ECCV 2024] Yong Zhong, Min Zhao, Zebin You, et al.
Yong Zhong, Min Zhao, Zebin You, Xiaofeng Yu, Changwang Zhang, Chongxuan Li
- Zero-shot High-fidelity and Pose-controllable Character Animation
[IJCAI 2024] Bingwen Zhu, Fanyi Wang, Tianyi Lu, et al.
Bingwen Zhu, Fanyi Wang, Tianyi Lu, Peng Liu, Jingwen Su, Jinxiu Liu, Yanhao Zhang, Zuxuan Wu, Guo-Jun Qi, Yu-Gang Jiang
- Champ: Controllable and Consistent Human Image Animation with 3D Parametric Guidance
[ECCV 2024] Shenhao Zhu, Junming Leo Chen, Zuozhuo Dai, et al.
Shenhao Zhu, Junming Leo Chen, Zuozhuo Dai, Qingkun Su, Yinghui Xu, Xun Cao, Yao Yao, Hao Zhu, Siyu Zhu
- Do You Guys Want to Dance: Zero-Shot Compositional Human Dance Generation with Multiple Persons
Zhe Xu, Kun Wei, et al.
Zhe Xu, Kun Wei, Xu Yang, Cheng Deng
- DreaMoving: A Human Video Generation Framework based on Diffusion Models
Mengyang Feng, Jinlin Liu, Kai Yu, et al.
Mengyang Feng, Jinlin Liu, Kai Yu, Yuan Yao, Zheng Hui, Xiefan Guo, Xianhui Lin, Haolan Xue, Chen Shi, Xiaowen Li, Aojie Li, Xiaoyang Kang, Biwen Lei, Miaomiao Cui, Peiran Ren, Xuansong Xie
- Generative Rendering: Controllable 4D-Guided Video Generation with 2D Diffusion Models
[CVPR 2024] Shengqu Cai, Duygu Ceylan, et al.
Shengqu Cai, Duygu Ceylan, Matheus Gadelha, Chun-Hao Paul Huang, Tuanfeng Yang Wang, Gordon Wetzstein
- Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animation
[CVPR 2024] Li Hu, Xin Gao, Peng Zhang, et al.
Li Hu, Xin Gao, Peng Zhang, Ke Sun, Bang Zhang, Liefeng Bo
- MagicAnimate: Temporally Consistent Human Image Animation using Diffusion Model
[CVPR 2024] Zhongcong Xu, Jianfeng Zhang, et al.
Zhongcong Xu, Jianfeng Zhang, Jun Hao Liew, Hanshu Yan, Jia-Wei Liu, Chenxu Zhang, Jiashi Feng, Mike Zheng Shou
- MagicPose: Realistic Human Poses and Facial Expressions Retargeting with Identity-aware Diffusion
[ICML 2024] Di Chang, Yichun Shi, Quankai Gao, et al.
Di Chang, Yichun Shi, Quankai Gao, Jessica Fu, Hongyi Xu, Guoxian Song, Qing Yan, Yizhe Zhu, Xiao Yang, Mohammad Soleymani
- Dancing Avatar: Pose and Text-Guided Human Motion Videos Synthesis with Image Diffusion Model (15 Aug 2023)
Bosheng Qin, Wentao Ye, et al.
Bosheng Qin, Wentao Ye, Qifan Yu, Siliang Tang, Yueting Zhuang
- DISCO: Disentangled Control for Realistic Human Dance Generation
[CVPR 2024] Tan Wang, Linjie Li, Kevin Lin, et al.
Tan Wang, Linjie Li, Kevin Lin, Yuanhao Zhai, Chung-Ching Lin, Zhengyuan Yang, Hanwang Zhang, Zicheng Liu, Lijuan Wang
- DreamPose: Fashion Image-to-Video Synthesis via Stable Diffusion
[ICCV 2023] Johanna Karras, Aleksander Holynski, Ting-Chun Wang, et al.
Johanna Karras, Aleksander Holynski, Ting-Chun Wang, Ira Kemelmacher-Shlizerman
- Follow Your Pose: Pose-Guided Text-to-Video Generation using Pose-Free Videos (3 Apr 2023)
[AAAI 2024] Yue Ma, Yingqing He, et al.
Yue Ma, Yingqing He, Xiaodong Cun, Xintao Wang, Siran Chen, Ying Shan, Xiu Li, Qifeng Chen
- vid2vid:Video-to-Video Synthesis
Ting-Chun Wang, Ming-Yu Liu, Jun-Yan Zhu, et al.
Ting-Chun Wang, Ming-Yu Liu, Jun-Yan Zhu, Guilin Liu, Andrew Tao, Jan Kautz, Bryan Catanzaro
📏 Depth-Guided
- Frame Guidance: Training-Free Guidance for Frame-Level Control in Video Diffusion Models
Sangwon Jang, Taekyung Ki, Jaehyeong Jo, et al.
Sangwon Jang, Taekyung Ki, Jaehyeong Jo, Jaehong Yoon, Soo Ye Kim, Zhe Lin, Sung Ju Hwang
- OmniVDiff: Omni Controllable Video Diffusion for Generation and Understanding
Dianbing Xi, Jiepeng Wang, Yuanzhi Liang, et al.
Xi Qiu, Yuchi Huo, Rui Wang, Chi Zhang, Xuelong Li
- DreamDance: Animating Human Images by Enriching 3D Geometry Cues from 2D Poses
Yatian Pang, Bin Zhu, Bin Lin, et al.
Mingzhe Zheng, Francis E. H. Tay, Ser-Nam Lim, Harry Yang, Li Yuan
- ControlNeXt: Powerful and Efficient Control for Image and Video Generation
Bohao Peng, Jian Wang, Yuechen Zhang, et al.
Wenbo Li, Ming-Chang Yang, Jiaya Jia
- Champ: Controllable and Consistent Human Image Animation with 3D Parametric Guidance
[ECCV 2024] Shenhao Zhu, Junming Leo Chen, Zuozhuo Dai, et al.
Qingkun Su, Yinghui Xu, Xun Cao, Yao Yao, Hao Zhu, Siyu Zhu
- MoonShot: Towards Controllable Video Generation and Editing with Multimodal Conditions
David Junhao Zhang, Dongxu Li, Hung Le, et al.
Mike Zheng Shou, Caiming Xiong, Doyen Sahoo
- SparseCtrl: Adding Sparse Controls to Text-to-Video Diffusion Models (28 Nov 2023)
[ECCV 2024] Yuwei Guo, Ceyuan Yang, Anyi Rao, et al.
Maneesh Agrawala, Dahua Lin, Bo Dai
- GD-VDM: Generated Depth for better Diffusion-based Video Generation (19 Jun 2023)
Ariel Lapid, Idan Achituve, Lior Bracha, et al.
Ethan Fetaya
- Make-Your-Video: Customized Video Generation Using Textual and Structural Guidance
[IEEE TVCG 2024] Jinbo Xing, Menghan Xia, Yuxin Liu, et al.
Yuechen Zhang, Yong Zhang, Yingqing He, Hanyuan Liu, Haoxin Chen, Xiaodong Cun, Xintao Wang, Ying Shan, Tien-Tsin Wong
- Control-A-Video: Controllable Text-to-Video Diffusion Models with Motion Prior and Reward Feedback Learning
Weifeng Chen, Yatai Ji, Jie Wu, et al.
Hefeng Wu, Pan Xie, Jiashi Li, Xin Xia, Xuefeng Xiao, Liang Lin
- ControlVideo: Training-free Controllable Text-to-Video Generation (22 May 2023)
[ICLR 2024] Yabo Zhang, Yuxiang Wei, Dongsheng Jiang, et al.
Xiaopeng Zhang, Wangmeng Zuo, Qi Tian
🗺️ Landmark-Guided
- HunyuanPortrait: Implicit Condition Control for Enhanced Portrait Animation
[CVPR 2025] Zunnan Xu, Zhentao Yu, et al.
Zunnan Xu, Zhentao Yu, Zixiang Zhou, Jun Zhou, Xiaoyu Jin, Fa-Ting Hong, Xiaozhong Ji, Junwei Zhu, Chengfei Cai, Shiyu Tang, Qin Lin, Xiu Li, Qinglin Lu
- TASTE-Rob: Advancing Video Generation of Task-Oriented Hand-Object Interaction for Generalizable Robotic Manipulation
Hongxiang Zhao, Xingchen Liu, et al.
Hongxiang Zhao, Xingchen Liu, Mutian Xu, Yiming Hao, Weikai Chen, Xiaoguang Han
- Identity-Preserving Text-to-Video Generation by Frequency Decomposition
[CVPR 2025] Shenghai Yuan, Jinfa Huang, et al.
Shenghai Yuan, Jinfa Huang, Xianyi He, Yunyuan Ge, Yujun Shi, Liuhan Chen, Jiebo Luo, Li Yuan
- EchoMimicV2: Towards Striking, Simplified, and Semi-Body Human Animation
[CVPR 2025] Rang Meng, Xingyu Zhang, Yuming Li, et al.
Chenguang Ma
- Takin-ADA: Emotion Controllable Audio-Driven Animation with Canonical and Landmark Loss Optimization
Bin Lin, Yanzhen Yu, et al.
Bin Lin, Yanzhen Yu, Jianhao Ye, Ruitao Lv, Yuguang Yang, Ruoye Xie, Pan Yu, Hongbin Zhou
- EchoMimic: Lifelike Audio-Driven Portrait Animations through Editable Landmark Conditions
[AAAI 2025] Zhiyuan Chen, Jiajiong Cao, Zhiquan Chen, et al.
Yuming Li, Chenguang Ma
- LivePortrait: Efficient Portrait Animation with Stitching and Retargeting Control
Jianzhu Guo, Dingyun Zhang, et al.
Jianzhu Guo, Dingyun Zhang, Xiaoqiang Liu, Zhizhou Zhong, Yuan Zhang, Pengfei Wan, Di Zhang
- Follow-Your-Emoji: Fine-Controllable and Expressive Freestyle Portrait Animation
[Siggraph Asia 2024] Yue Ma, Hongyu Liu, et al.
Yue Ma, Hongyu Liu, Hongfa Wang, Heng Pan, Yingqing He, Junkun Yuan, Ailing Zeng, Chengfei Cai, Heung-Yeung Shum, Wei Liu, Qifeng Chen
- Follow Your Pose: Pose-Guided Text-to-Video Generation using Pose-Free Videos (3 Apr 2023)
[AAAI 2024] Yue Ma, Yingqing He, et al.
Yue Ma, Yingqing He, Xiaodong Cun, Xintao Wang, Siran Chen, Ying Shan, Xiu Li, Qifeng Chen
✏️ Sketch-Guided
- Frame Guidance: Training-Free Guidance for Frame-Level Control in Video Diffusion Models
Sangwon Jang, Taekyung Ki, Jaehyeong Jo, et al.
Sangwon Jang, Taekyung Ki, Jaehyeong Jo, Jaehong Yoon, Soo Ye Kim, Zhe Lin, Sung Ju Hwang
- OmniVDiff: Omni Controllable Video Diffusion for Generation and Understanding (15 Apr 2025)
Dianbing Xi, Jiepeng Wang, Yuanzhi Liang, et al.
Xi Qiu, Yuchi Huo, Rui Wang, Chi Zhang, Xuelong Li
- SketchVideo: Sketch-based Video Generation and Editing
Feng-Lin Liu, Hongbo Fu, Xintao Wang, et al. Feng-Lin Liu, Hongbo Fu, Xintao Wang, Weicai Ye, Pengfei Wan, Di Zhang, Lin Gao
- LayerAnimate: Layer-level Control for Animation
Yuxue Yang, Lue Fan, Zuzeng Lin, et al. Feng Wang, Zhaoxiang Zhang
- CameraCtrl II: Dynamic Scene Exploration via Camera-controlled Video Diffusion Models
Hao He, Ceyuan Yang, Shanchuan Lin, et al. Yinghao Xu, Meng Wei, Liangke Gui, Qi Zhao, Gordon Wetzstein, Lu Jiang, Hongsheng Li
- CameraCtrl: Enabling Camera Control for Text-to-Video Generation
[ICLR 2025] Hao He, Yinghao Xu, Yuwei Guo, et al. Gordon Wetzstein, Bo Dai, Hongsheng Li, Ceyuan Yang
- VidSketch: Hand-drawn Sketch-Driven Video Generation with Diffusion Control
Lifan Jiang, Shuang Chen, Boxi Wu, et al. Xiaotong Guan, Jiahui Zhang
- AniDoc: Animation Creation Made Easier
[CVPR 2025] Yihao Meng, Hao Ouyang, Hanlin Wang, et al. Qiuyu Wang, Wen Wang, Ka Leong Cheng, Zhiheng Liu, Yujun Shen, Huamin Qu
- Latent-Reframe: Enabling Camera Control for Video Diffusion Model without Training
Zhenghong Zhou, Jie An, Jiebo Luo, et al.
- CamI2V: Camera-Controlled Image-to-Video Diffusion Model
Guangcong Zheng, Teng Li, Rui Jiang, et al. Yehao Lu, Tao Wu, Xi Li
- Motion Prompting: Controlling Video Generation with Motion Trajectories(frame+track+text)
Daniel Geng, Charles Herrmann, Junhwa Hur, et al. Forrester Cole, Serena Zhang, Tobias Pfaff, Tatiana Lopez-Guevara, Carl Doersch, Yusuf Aytar, Michael Rubinstein, Chen Sun, Oliver Wang, Andrew Owens, Deqing Sun
- MOTIONFLOW: Learning Implicit Motion Flow for Complex Camera Trajectory Control in Video Generation
Author list not fully provided
- I2VControl: Disentangled and Unified Video Motion Synthesis Control
Zhiyuan Zhang, Dongdong Chen, Jing Liao
- Trajectory Attention: Enhancing Video Generation with Fine-Grained Motion Control
Zeqi Xiao, Wenqi Ouyang, Yifan Zhou, et al. Shuai Yang, Lei Yang, Jianlou Si, Xingang Pan
- Open-Sora Plan: Open-Source Large Video Generation Model
Bin Lin, Yunyang Ge, Xinhua Cheng, et al. Zongjian Li, Bin Zhu, Shaodong Wang, Xianyi He, Yang Ye, Shenghai Yuan, Liuhan Chen, Tanghui Jia, Junwu Zhang, Zhenyu Tang, Yatian Pang, Bin She, Cen Yan, Zhiheng Hu, Xiaoyi Dong, Lin Chen, Zhang Pan, Xing Zhou, Shaoling Dong, Yonghong Tian, Li Yuan
- ToonCrafter: Generative Cartoon Interpolation
[ACM TOG] Jinbo Xing, Hanyuan Liu, Menghan Xia, et al. Yong Zhang, Xintao Wang, Ying Shan, Tien-Tsin Wong
- MagicStick: Controllable Video Editing via Control Handle Transformations
[WACV 2025] Yue Ma, Xiaodong Cun, Sen Liang, et al. Jinbo Xing, Yingqing He, Chenyang Qi, Siran Chen, Qifeng Chen
- Video Diffusion Models are Training-free Motion Interpreter and Controller
Zeqi Xiao, Yifan Zhou, Shuai Yang, Xingang Pan, et al.
- DimensionX: Create Any 3D and 4D Scenes from a Single Image with Controllable Video Diffusion
Wenqiang Sun, Shuo Chen, Fangfu Liu, et al. Zilong Chen, Yueqi Duan, Jun Zhang, Yikai Wang
- MotionClone: Training-Free Motion Cloning for Controllable Video Generation
Pengyang Ling, Jiazi Bu, Pan Zhang, et al. Xiaoyi Dong, Yuhang Zang, Tong Wu, Huaian Chen, Jiaqi Wang, Yi Jin
- Boosting Camera Motion Control for Video Diffusion Transformers
Soon Yau Cheong, Duygu Ceylan, Armin Mustafa, et al. Andrew Gilbert, Chun-Hao Paul Huang
- Cavia: Camera-controllable Multi-view Video Diffusion with View-Integrated Attention
Dejia Xu, Yifan Jiang, Chen Huang, et al. Liangchen Song, Thorsten Gernoth, Liangliang Cao, Zhangyang Wang, Hao Tang
- LVCD: Reference-based Lineart Video Colorization with Diffusion Models
Zhitong Huang, Mohan Zhang, Jing Liao
- EasyControl: Transfer ControlNet to Video Diffusion for Controllable Generation and Interpolation
Cong Wang, Jiaxi Gu, Panwen Hu, et al. Haoyu Zhao, Yuanfan Guo, Jianhua Han, Hang Xu, Xiaodan Liang
- ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesis
Wangbo Yu, Jinbo Xing, Li Yuan, et al. Wenbo Hu, Xiaoyu Li, Zhipeng Huang, Xiangjun Gao, Tien-Tsin Wong, Ying Shan, Yonghong Tian
- VD3D: Taming Large Video Diffusion Transformers for 3D Camera Control
Sherwin Bahmani, Ivan Skorokhodov, Aliaksandr Siarohin, et al. Sherwin Bahmani, Ivan Skorokhodov, Aliaksandr Siarohin, Willi Menapace, Guocheng Qian, Michael Vasilkovsky, Hsin-Ying Lee, Chaoyang Wang, Jiaxu Zou, Andrea Tagliasacchi, David B. Lindell, Sergey Tulyakov
- MotionCtrl: A Unified and Flexible Motion Controller for Video Generation
Zhouxia Wang, Ziyang Yuan, Xintao Wang, et al. Zhouxia Wang, Ziyang Yuan, Xintao Wang, Tianshui Chen, Menghan Xia, Ping Luo, Ying Shan
- Ctrl-Adapter: An Efficient and Versatile Framework for Adapting Diverse Controls to Any Diffusion Model
[ICLR 2025] Han Lin, Jaemin Cho, Abhay Zala, et al. Mohit Bansal
- A Recipe for Scaling up Text-to-Video Generation with Text-free Videos (25 Dec 2023)
[CVPR 2024] Xiang Wang, Shiwei Zhang, Hangjie Yuan, et al.
Zhiwu Qing, Biao Gong, Yingya Zhang, Yujun Shen, Changxin Gao, Nong Sang
- VideoLCM: Video Latent Consistency Model
Xiang Wang, Shiwei Zhang, Han Zhang, et al. Yu Liu, Yingya Zhang, Changxin Gao, Nong Sang
- SparseCtrl: Adding Sparse Controls to Text-to-Video Diffusion Models
Yuwei Guo, Ceyuan Yang, Anyi Rao, et al. Maneesh Agrawala, Dahua Lin, Bo Dai
- Make Pixels Dance: High-Dynamic Video Generation (18 Nov 2023)
[CVPR 2024] Yan Zeng, Guoqiang Wei, Jiani Zheng, et al.
Jiaxin Zou, Yang Wei, Yuchen Zhang, Hang Li
- TaleCrafter: Interactive Story Visualization with Multiple Characters
[SIGGRAPH Asia 2023] Yuan Gong, Youxin Pang, Xiaodong Cun, et al. Menghan Xia, Yingqing He, Haoxin Chen, Longyue Wang, Yong Zhang, Xintao Wang, Ying Shan, Yujiu Yang
- Sketching the Future(STF): Applying Conditional Control Techniques to Text-to-Video Models
Rohan Dhesikan, Vignesh Rajmohan
- SketchBetween: Video-to-Video Synthesis for Sprite Animation via Sketches (1 Sep 2022)
Dagmar Lukka Loftsdóttir, Matthew Guzdial
- Sketch Me A Video (10 Oct 2021)
Haichao Zhang, Gang Yu, Tao Chen, et al. Haichao Zhang, Gang Yu, Tao Chen, Guozhong Luo
- vid2vid:Video-to-Video Synthesis
Ting-Chun Wang, Ming-Yu Liu, Jun-Yan Zhu, et al.
Ting-Chun Wang, Ming-Yu Liu, Jun-Yan Zhu, Guilin Liu, Andrew Tao, Jan Kautz, Bryan Catanzaro
📦 BBox-Guided
- HoloDrive: Holistic 2D-3D Multi-Modal Street Scene Generation for Autonomous Driving
Zehuan Wu, Jingcheng Ni, Xiaodong Wang, et al.
Zehuan Wu, Jingcheng Ni, Xiaodong Wang, Yuxin Guo, Rui Chen, Lewei Lu, Jifeng Dai, Yuwen Xiong
- Higher fidelity autonomous vehicle video generation with bounding-box controlled object motion
Ge Ya Luo, Zhi Hao Luo, et al.
Ge Ya Luo, Zhi Hao Luo, Anthony Gosselin, Alexia Jolicoeur-Martineau, Christopher Pal
- MagicDrive-V2: High-Resolution Long Video Generation for Autonomous Driving with Adaptive Control
Ruiyuan Gao, Kai Chen, Bo Xiao, et al.
Ruiyuan Gao, Kai Chen, Bo Xiao, Lanqing Hong, Zhenguo Li, Qiang Xu
- DreamForge: Motion-Aware Autoregressive Video Generation for Multi-View Driving Scenes
Jianbiao Mei, Tao Hu, Xuemeng Yang, et al.
Jianbiao Mei, Tao Hu, Xuemeng Yang, Licheng Wen, Yu Yang, Tiantian Wei, Yukai Ma, Min Dou, Botian Shi, Yong Liu
- DriveScape: Towards High-Resolution Controllable Multi-View Driving Video Generation
Wei Wu, Xi Guo, Weixuan Tang, et al.
Wei Wu, Xi Guo, Weixuan Tang, Tingxuan Huang, Chiyu Wang, Dongyue Chen, Chenjing Ding
- DriveScape: Towards High-Resolution Controllable Multi-View Driving Video Generation
Wei Wu, Xi Guo, Weixuan Tang, et al.
Wei Wu, Xi Guo, Weixuan Tang, Tingxuan Huang, Chiyu Wang, Dongyue Chen, Chenjing Ding
- DiVE: DiT-based Video Generation with Enhanced Control
Junpeng Jiang, Gangyi Hong, Lijun Zhou, et al.
Junpeng Jiang, Gangyi Hong, Lijun Zhou, Enhui Ma, Hengtong Hu, Xia Zhou, Jie Xiang, Fan Liu, Kaicheng Yu, Haiyang Sun, Kun Zhan, Peng Jia, Miao Zhang
- Unleashing Generalization of End-to-End Autonomous Driving with Controllable Long Video Generation
Enhui Ma, Lijun Zhou, Tao Tang, et al.
Enhui Ma, Lijun Zhou, Tao Tang, Zhan Zhang, Dong Han, Junpeng Jiang, Kun Zhan, Peng Jia, Xianpeng Lang, Haiyang Sun, Di Lin, Kaicheng Yu
- Boximator: Generating Rich and Controllable Motions for Video Synthesis
Jiawei Wang, Yuchen Zhang, Jiaxin Zou, et al. Yan Zeng, Guoqiang Wei, Liping Yuan, Hang Li
- Panacea: Panoramic and Controllable Video Generation for Autonomous Driving
Yuqing Wen, Yucheng Zhao, Yingfei Liu, et al.
Yuqing Wen, Yucheng Zhao, Yingfei Liu, Fan Jia, Yanhui Wang, Chong Luo, Chi Zhang, Tiancai Wang, Xiaoyan Sun, Xiangyu Zhang
- MagicDrive: Street View Generation with Diverse 3D Geometry Control
Ruiyuan Gao, Kai Chen, Enze Xie, et al.
Ruiyuan Gao, Kai Chen, Enze Xie, Lanqing Hong, Zhenguo Li, Dit-Yan Yeung, Qiang Xu
- DriveDreamer: Towards Real-world-driven World Models for Autonomous Driving
Xiaofeng Wang, Zheng Zhu, Guan Huang, et al.
Xiaofeng Wang, Zheng Zhu, Guan Huang, Xinze Chen, Jiagang Zhu, Jiwen Lu
- LLM-grounded Video Diffusion Models (29 Sep 2023)
Long Lian, Baifeng Shi, Adam Yala, et al.
Long Lian, Baifeng Shi, Adam Yala, Trevor Darrell, Boyi Li
- Multi-object Video Generation from Single Frame Layouts
Yang Wu, Zhibin Liu, Hefeng Wu, et al.
Yang Wu, Zhibin Liu, Hefeng Wu, Liang Lin
🧬 ID Control
🧑 Person-Guided
- DanceTogether! Identity-Preserving Multi-Person Interactive Video Generation
Junhao Chen, Mingjin Chen, Jianjin Xu, et al.
Junhao Chen, Mingjin Chen, Jianjin Xu, Xiang Li, Junting Dong, Mingze Sun, Puhua Jiang, Hongxiang Li, Yuhang Yang, Hao Zhao, Xiaoxiao Long, Ruqi Huang
- MTVCrafter: 4D Motion Tokenization for Open-World Human Image Animation
Yanbo Ding, Xirui Hu, Zhizhi Guo, et al.
Yanbo Ding, Xirui Hu, Zhizhi Guo, Chi Zhang, Yali Wang
- Frame In-N-Out: Unbounded Controllable Image-to-Video Generation (27 May 2025)
Boyang Wang, Xuweiyi Chen, Matheus Gadelha, et al. Boyang Wang, Xuweiyi Chen, Matheus Gadelha, Zezhou Cheng
- SkyReels-A2: Compose Anything in Video Diffusion Transformers
Zhengcong Fei, Debang Li, Di Qiu, et al.
Zhengcong Fei, Debang Li, Di Qiu, Jiahua Wang, Yikun Dou, Rui Wang, Jingtao Xu, Mingyuan Fan, Guibin Chen, Yang Li, Yahui Zhou
- AnimateAnywhere: Rouse the Background in Human Image Animation
Xiaoyu Liu, Mingshuai Yao, Yabo Zhang, et al.
Xiaoyu Liu, Mingshuai Yao, Yabo Zhang, Xianhui Lin, Peiran Ren, Xiaoming Li, Ming Liu, Wangmeng Zuo
- Concat-ID: Towards Universal Identity-Preserving Video Synthesis
Yong Zhong, Zhuoyi Yang, Jiayan Teng, et al.
Yong Zhong, Zhuoyi Yang, Jiayan Teng, Xiaotao Gu, Chongxuan Li
- PERSONALVIDEO: High ID-Fidelity Video Customization with Static Images (16 Mar 2025)
Hengjia Li, Haonan Qiu, Shiwei Zhang, Xiang Wang, et al.
Hengjia Li, Haonan Qiu, Shiwei Zhang, Xiang Wang, Yujie Wei, Zekun Li, Yingya Zhang, Boxi Wu, Deng Cai
- AnyCharV: Bootstrap Controllable Character Video Generation with Fine-to-Coarse Guidance
Zhao Wang, Hao Wen, Lingting Zhu, et al.
Zhao Wang, Hao Wen, Lingting Zhu, Chenming Shang, Yujiu Yang, Qi Dou
- Movie Weaver: Tuning-Free Multi-Concept Video Personalization with Anchored Prompts (4 Feb 2025)
Feng Liang, Haoyu Ma, Zecheng He, et al.
Feng Liang, Haoyu Ma, Zecheng He, Tingbo Hou, Ji Hou, Kunpeng Li, Xiaoliang Dai, Felix Juefei-Xu, Samaneh Azadi, Animesh Sinha, Peizhao Zhang, Peter Vajda, Diana Marculescu
- EchoVideo: Identity-Preserving Human Video Generation by Multimodal Feature Fusion (23 Jan 2025)
Jiangchuan Wei, Shiyue Yan, Wenfeng Lin, et al.
Jiangchuan Wei, Shiyue Yan, Wenfeng Lin, Boyuan Liu, Renjie Chen, Mingyu Guo
- VideoGen-of-Thought: Step-by-step generating multi-shot video with minimal manual intervention (03 Dec 2024)
Mingzhe Zheng, Yongqi Xu, Haojian Huang, et al.
Mingzhe Zheng, Yongqi Xu, Haojian Huang, Xuran Ma, Yexin Liu, Wenjie Shu, Yatian Pang, Feilong Tang, Qifeng Chen, Harry Yang, Ser-Nam Lim
[](https://arxiv.org/abs/2412.02259)
[](https://github.com/DuNGEOnmassster/VideoGen-of-Thought)
[](https://cheliosoops.github.io/VGoT/)
- Identity-Preserving Text-to-Video Generation by Frequency Decomposition (25 Nov 2024)
Shenghai Yuan, Jinfa Huang, et al.
Shenghai Yuan, Jinfa Huang, Xianyi He, Yunyuan Ge, Yujun Shi, Liuhan Chen, Jiebo Luo, Li Yuan
- MIMO: Controllable Character Video Synthesis with Spatial Decomposed Modeling
Yifang Men, Yuan Yao, Miaomiao Cui, et al.
Yifang Men, Yuan Yao, Miaomiao Cui, Liefeng Bo
[](https://arxiv.org/abs/2409.16160)
[](https://github.com/menyifang/MIMO)
[](https://menyifang.github.io/projects/MIMO/index.html)
- ID-Animator: Zero-Shot Identity-Preserving Human Video Generation (25 Jun 2024)
Xuanhua He, Quande Liu, et al.
Xuanhua He, Quande Liu, Shengju Qian, Xin Wang, Tao Hu, Ke Cao, Keyu Yan, Jie Zhang
- Magic-Me: Identity-Specific Video Customized Diffusion (14 Feb 2024)
Ze Ma, Daquan Zhou, Chun-Hsiao Yeh, et al.
Ze Ma, Daquan Zhou, Chun-Hsiao Yeh, Xue-She Wang, Xiuyu Li, Huanrui Yang, Zhen Dong, Kurt Keutzer, Jiashi Feng
[](https://arxiv.org/abs/2402.09368)
[](https://github.com/Zhen-Dong/Magic-Me)
- Vlogger: Make Your Dream A Vlog (17 Jan 2024)
Shaobin Zhuang, Kunchang Li, Xinyuan Chen, et al.
Shaobin Zhuang, Kunchang Li, Xinyuan Chen, Yaohui Wang, Ziwei Liu, Yu Qiao, Yali Wang
[](https://arxiv.org/abs/2401.09414)
[](https://github.com/zhuangshaobin/Vlogger)
🧑🤝🧑 Subject-Guided
- DualReal: Joint Training for Lossless Identity-Motion Fusion in Video Customization
Wenchuan Wang, Mengqi Huang, Yijing Tu, et al.
Wenchuan Wang, Mengqi Huang, Yijing Tu, Zhendong Mao
- Concat-ID: Towards Universal Identity-Preserving Video Synthesis
Yong Zhong, Zhuoyi Yang, Jiayan Teng, et al.
Yong Zhong, Zhuoyi Yang, Jiayan Teng, Xiaotao Gu, Chongxuan Li
- VideoMage: Multi-Subject and Motion Customization of Text-to-Video Diffusion Models
Chi-Pin Huang, Yen-Siang Wu, Hung-Kai Chung, et al.
Chi-Pin Huang, Yen-Siang Wu, Hung-Kai Chung, Kai-Po Chang, Fu-En Yang, Yu-Chiang Frank Wang
- DreamRelation: Relation-Centric Video Customization (10 Mar 2025)
Yujie Wei, Shiwei Zhang, Hangjie Yuan, et al.
Yujie Wei, Shiwei Zhang, Hangjie Yuan, Biao Gong, Longxiang Tang, Xiang Wang, Haonan Qiu, Hengjia Li, Shuai Tan, Yingya Zhang, Hongming Shan
- Get In Video: Add Anything You Want to the Video (8 Mar 2025)
Shaobin Zhuang, Zhipeng Huang, Binxin Yang, et al.
Shaobin Zhuang, Zhipeng Huang, Binxin Yang, Ying Zhang, Fangyikang Wang, Canmiao Fu, Chong Sun, Zheng-Jun Zha, Chen Li, Yali Wang
- Phantom: Subject-consistent Video Generation via Cross-modal Alignment (16 Feb 2025)
Lijie Liu, Tianxiang Ma, Bingchuan Li, et al.
Lijie Liu, Tianxiang Ma, Bingchuan Li, Zhuowei Chen, Jiawei Liu, Qian He, Xinglong Wu
- Multi-subject Open-set Personalization in Video Generation (10 Jan 2025)
Tsai-Shien Chen, Aliaksandr Siarohin, Willi Menapace, et al.
Tsai-Shien Chen, Aliaksandr Siarohin, Willi Menapace, Yuwei Fang, Kwot Sin Lee, Ivan Skorokhodov, Kfir Aberman, Jun-Yan Zhu, Ming-Hsuan Yang, Sergey Tulyakov
- ConceptMaster: Multi-Concept Video Customization on Diffusion Transformer Models Without Test-Time Tuning (8 Jan 2025)
Yuzhou Huang, Ziyang Yuan, Quande Liu, et al.
Yuzhou Huang, Ziyang Yuan, Quande Liu, Qiulin Wang, Xintao Wang, Ruimao Zhang, Pengfei Wan, Di Zhang, Kun Gai
- VideoMaker: Zero-shot Customized Video Generation with the Inherent Force of Video Diffusion Models (27 Dec 2024)
Tao Wu, Yong Zhang, Xiaodong Cun, et al.
Tao Wu, Yong Zhang, Xiaodong Cun, Zhongang Qi, Junfu Pu, Huanzhang Dou, Guangcong Zheng, Ying Shan, Xi Li
- Customcrafter: Customized Video Generation with Preserving Motion and Concept Composition Abilities (27 Dec 2024)
Tao Wu, Yong Zhang, Xintao Wang, et al.
Tao Wu, Yong Zhang, Xintao Wang, Xianpan Zhou, Guangcong Zheng, Zhongang Qi, Ying Shan, Xi Li
- CustomTTT: Motion and Appearance Customized Video Generation via Test-Time Training (20 Dec 2024)
Xiuli Bi, Jian Lu, Bo Liu, et al.
Xiuli Bi, Jian Lu, Bo Liu, Xiaodong Cun, Yong Zhang, Weisheng Li, Bin Xiao
- SUGAR: Subject-Driven Video Customization in a Zero-Shot Manner (13 Dec 2024)
Yufan Zhou, Ruiyi Zhang, Jiuxiang Gu, et al.
Yufan Zhou, Ruiyi Zhang, Jiuxiang Gu, Nanxuan Zhao, Jing Shi, Tong Sun
- DreamRunner: Fine-Grained Compositional Story-to-Video Generation with Retrieval-Augmented Motion Adaptation (25 Nov 2024)
Zun Wang, Jialu Li, Han Lin, et al.
Zun Wang, Jialu Li, Han Lin, Jaehong Yoon, Mohit Bansal
- VideoAlchemy: Open-set Personalization in Video Generation (15 Nov 2024)
Tsai-Shien Chen, Aliaksandr Siarohin, et al.
Tsai-Shien Chen, Aliaksandr Siarohin, Willi Menapace, Yuwei Fang, Ivan Skorokhodov, Jun-Yan Zhu, Kfir Aberman, Ming-Hsuan Yang, Sergey Tulyakov
- StoryAgent: Customized Storytelling Video Generation via Multi-Agent Collaboration (7 Nov 2024)
Panwen Hu, Jin Jiang, Jianqi Chen, et al.
Panwen Hu, Jin Jiang, Jianqi Chen, Mingfei Han, Shengcai Liao, Xiaojun Chang, Xiaodan Liang
- MotionBooth: Motion-Aware Customized Text-to-Video Generation (29 Oct 2024)
Jianzong Wu, Xiangtai Li, Yanhong Zeng, et al.
Jianzong Wu, Xiangtai Li, Yanhong Zeng, Jiangning Zhang, Qianyu Zhou, Yining Li, Yunhai Tong, Kai Chen
- Anim-Director: A Large Multimodal Model Powered Agent for Controllable Animation Video Generation (19 Aug 2024)
Yunxin Li, Haoyuan Shi, Baotian Hu, et al.
Yunxin Li, Haoyuan Shi, Baotian Hu, Longyue Wang, Jiashun Zhu, Jinyi Xu, Zhen Zhao, Min Zhang
- Still-Moving: Customized Video Generation Without Customized Video Data (11 Jul 2024)
Hila Chefer, Shiran Zada, Roni Paiss, Ariel Ephrat, et al.
Hila Chefer, Shiran Zada, Roni Paiss, Ariel Ephrat, Omer Tov, Michael Rubinstein, Lior Wolf, Tali Dekel, Tomer Michaeli, Inbar Mosseri
- VIMI: Grounding Video Generation through Multi-modal Instruction (8 Jul 2024)
Yuwei Fang, Willi Menapace, Aliaksandr Siarohin, et al.
Yuwei Fang, Willi Menapace, Aliaksandr Siarohin, Tsai-Shien Chen, Kuan-Chien Wang, Ivan Skorokhodov, Graham Neubig, Sergey Tulyakov
- Customvideo: Customizing Text-to-Video Generation with Multiple Subjects (22 May 2024)
Zhao Wang, Aoxue Li, et al.
Zhao Wang, Aoxue Li, Lingting Zhu, Yong Guo, Qi Dou, Zhenguo Li
- DisenStudio: Customized Multi-subject Text-to-Video Generation with Disentangled Spatial Control (21 May 2024)
Hong Chen, Xin Wang, Yipeng Zhang, et al.
Hong Chen, Xin Wang, Yipeng Zhang, Yuwei Zhou, Zeyang Zhang, Siao Tang, Wenwu Zhu
- Lumina-T2X: Transforming Text into Any Modality, Resolution, and Duration via Flow-based Large Diffusion Transformers (9 May 2024)
Peng Gao, Le Zhuo, Dongyang Liu, et al.
Peng Gao, Le Zhuo, Dongyang Liu, Ruoyi Du, Xu Luo, Longtian Qiu, Yuhang Zhang, Chen Lin, Rongjie Huang, Shijie Geng, Renrui Zhang, Junlin Xi, Wenqi Shao, Zhengkai Jiang, Tianshuo Yang, Weicai Ye, He Tong, Jingwen He, Yu Qiao, Hongsheng Li
- SubjectDrive: Scaling Generative Data in Autonomous Driving via Subject Control (28 Mar 2024)
Zheng Chen, Di Qiu, Rui Wang, et al.
Zheng Chen, Di Qiu, Rui Wang, Binyuan Huang, Yuqing Wen, Yucheng Zhao, Yaosi Hu, Yingfei Liu, Fan Jia, Weixin Mao, Tiancai Wang, Chi Zhang, Chang Wen Chen, Zhenzhong Chen, Xiangyu Zhang
- AesopAgent: Agent-driven Evolutionary System on Story-to-Video Production (12 Mar 2024)
Jiuniu Wang, Zehua Du, Yuyuan Zhao, et al.
Jiuniu Wang, Zehua Du, Yuyuan Zhao, Bo Yuan, Kexiang Wang, Jian Liang, Yaxi Zhao, Yihen Lu, Gengliang Li, Junlong Gao, Xin Tu, Zhenyu Guo
- VideoDrafter: Content-Consistent Multi-Scene Video Generation with LLM (2 Jan 2024)
Fuchen Long, Zhaofan Qiu, Ting Yao, et al.
Fuchen Long, Zhaofan Qiu, Ting Yao, Tao Mei
- Dreamvideo: Composing Your Dream Videos with Customized Subject and Motion (7 Dec 2023)
Yujie Wei, Shiwei Zhang, Zhiwu Qing, et al.
Yujie Wei, Shiwei Zhang, Zhiwu Qing, Hangjie Yuan, Zhiheng Liu, Yu Liu, Yingya Zhang, Jingren Zhou, Hongming Shan
- VideoBooth: Diffusion-based Video Generation with Image Prompts (1 Dec 2023)
Yuming Jiang, Tianxing Wu, Shuai Yang, et al.
Yuming Jiang, Tianxing Wu, Shuai Yang, Chenyang Si, Dahua Lin, Yu Qiao, Chen Change Loy, Ziwei Liu
- Videodreamer: Customized Multi-Subject Text-to-Video Generation with Disen-Mix Finetuning (2 Nov 2023)
Hong Chen, Xin Wang, Guanning Zeng, et al.
Hong Chen, Xin Wang, Guanning Zeng, Yipeng Zhang, Yuwei Zhou, Feilin Han, Wenwu Zhu
- Animate-A-Story: Storytelling with Retrieval-Augmented Video Generation (13 Jul 2023)
Yingqing He, Menghan Xia, Haoxin Chen, et al.
Yingqing He, Menghan Xia, Haoxin Chen, Xiaodong Cun, Yuan Gong, Jinbo Xing, Yong Zhang, Xintao Wang, Chao Weng, Ying Shan, Qifeng Chen
- TaleCrafter: Interactive Story Visualization with Multiple Characters (30 May 2023)
Yuan Gong, Youxin Pang, et al.
Yuan Gong, Youxin Pang, Xiaodong Cun, Menghan Xia, Yingqing He, Haoxin Chen, Longyue Wang, Yong Zhang, Xintao Wang, Ying Shan, Yujiu Yang
- Dreamix: Video Diffusion Models are General Video Editors (2 Feb 2023)
Eyal Molad, Eliahu Horwitz, et al.
Eyal Molad, Eliahu Horwitz, Dani Valevski, Alex Rav Acha, Yossi Matias, Yael Pritch, Yaniv Leviathan, Yedid Hoshen
🖼️ Image Control
- Frame Guidance: Training-Free Guidance for Frame-Level Control in Video Diffusion Models
Sangwon Jang, Taekyung Ki, Jaehyeong Jo, et al.
Sangwon Jang, Taekyung Ki, Jaehyeong Jo, Jaehong Yoon, Soo Ye Kim, Zhe Lin, Sung Ju Hwang
- DanceTogether! Identity-Preserving Multi-Person Interactive Video Generation
Junhao Chen, Mingjin Chen, Jianjin Xu, et al.
Junhao Chen, Mingjin Chen, Jianjin Xu, Xiang Li, Junting Dong, Mingze Sun, Puhua Jiang, Hongxiang Li, Yuhang Yang, Hao Zhao, Xiaoxiao Long, Ruqi Huang
- OmniVDiff: Omni Controllable Video Diffusion for Generation and Understanding (15 Apr 2025)
Dianbing Xi, Jiepeng Wang, Yuanzhi Liang, et al.
Xi Qiu, Yuchi Huo, Rui Wang, Chi Zhang, Xuelong Li
- HunyuanVideo: A Systematic Framework For Large Video Generative Models
Weijie Kong, Qi Tian, Zijian Zhang, et al. Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, Kathrina Wu, Qin Lin, Junkun Yuan, Yanxin Long, Aladdin Wang, Andong Wang, Changlin Li, Duojun Huang, Fang Yang, Hao Tan, Hongmei Wang, Jacob Song, Jiawang Bai, Jianbing Wu, Jinbao Xue, Joey Wang, Kai Wang, Mengyang Liu, Pengyu Li, Shuai Li, Weiyan Wang, Wenqing Yu, Xinchi Deng, Yang Li, Yi Chen, Yutao Cui, Yuanbo Peng, Zhentao Yu, Zhiyu He, Zhiyong Xu, Zixiang Zhou, Zunnan Xu, Yangyu Tao, Qinglin Lu, Songtao Liu, Dax Zhou, Hongfa Wang, Yong Yang, Di Wang, Yuhong Liu, Jie Jiang, Caesar Zhong
- Extrapolating and Decoupling Image-to-Video Generation Models: Motion Modeling is Easier Than You Think (2 Mar 2025)
[CVPR 2025] Jie Tian, Xiaoye Qu, Zhenyi Lu, et al. Wei Wei, Sichen Liu, Yu Cheng
- Adapting Image-to-Video Diffusion Models for Large-Motion Frame Interpolation (17 Feb 2025)
Luoxu Jin, Hiroshi Watanabe
- Generative Inbetweening: Adapting Image-to-Video Models for Keyframe Interpolation (12 Feb 2025)
[ICLR 2025] Xiaojuan Wang, Boyang Zhou, Brian Curless, et al. Ira Kemelmacher-Shlizerman, Aleksander Holynski, Steven M. Seitz
- Autoregressive Video Generation without Vector Quantization (9 Jan 2025)
[ICLR 2025] Haoge Deng, Ting Pan, Haiwen Diao, et al. Zhengxiong Luo, Yufeng Cui, Huchuan Lu, Shiguang Shan, Yonggang Qi, Xinlong Wang
- Through-The-Mask: Mask-based Motion Trajectories for Image-to-Video Generation (6 Jan 2025)
[CVPR 2025] Guy Yariv, Yuval Kirstain, Amit Zohar, et al. Shelly Sheynin, Yaniv Taigman, Yossi Adi, Sagie Benaim, Adam Polyak
- STIV: Scalable Text and Image Conditioned Video Generation (10 Dec 2024)
Zongyu Lin, Wei Liu, Chen Chen, et al. Jiasen Lu, Wenze Hu, Tsu-Jui Fu, Jesse Allardice, Zhengfeng Lai, Liangchen Song, Bowen Zhang, Cha Chen, Yiran Fei, Yifan Jiang, Lezhi Li, Yizhou Sun, Kai-Wei Chang, Yinfei Yang
- MotionStone: Decoupled Motion Intensity Modulation with Diffusion Transformer for Image-to-Video Generation (8 Dec 2024)
Shuwei Shi, Biao Gong, Xi Chen, et al. Dandan Zheng, Shuai Tan, Zizheng Yang, Yuyuan Li, Jingwen He, Kecheng Zheng, Jingdong Chen, Ming Yang, Yinqiang Zheng
- Lumiere: A Space-Time Diffusion Model for Video Generation
[SIGGRAPH Asia 2024] Omer Bar-Tal, Hila Chefer, Omer Tov, et al. Charles Herrmann, Roni Paiss, Shiran Zada, Ariel Ephrat, Junhwa Hur, Guanghui Liu, Amit Raj, Yuanzhen Li, Michael Rubinstein, Tomer Michaeli, Oliver Wang, Deqing Sun, Tali Dekel, Inbar Mosseri
- Identifying and Solving Conditional Image Leakage in Image-to-Video Diffusion Model (6 Nov 2024)
[NeurIPS 2024] Min Zhao, Hongzhou Zhu, Chendong Xiang, et al. Kaiwen Zheng, Chongxuan Li, Jun Zhu
- FrameBridge: Improving Image-to-Video Generation with Bridge Models (20 Oct 2024)
Yuji Wang, Zehua Chen, Xiaoyu Chen, et al. Jun Zhu, Jianfei Chen
- I4VGen: Image as Free Stepping Stone for Text-to-Video Generation
Xiefan Guo, Jinlin Liu, Miaomiao Cui, et al. Liefeng Bo, Di Huang
- DynamiCrafter: Animating Open-Domain Images with Video Diffusion Priors (1 Oct 2024)
[ECCV 2024] Jinbo Xing, Menghan Xia, Yong Zhang, et al. Haoxin Chen, Wangbo Yu, Hanyuan Liu, Gongye Liu, Xintao Wang, Ying Shan, Tien-Tsin Wong
- PhysGen: Rigid-Body Physics-Grounded Image-to-Video Generation (27 Sep 2024)
[ECCV 2024] Shaowei Liu, Zhongzheng Ren, Saurabh Gupta, et al. Shenlong Wang
- Structure and Content-Guided Video Synthesis with Diffusion Models (27 Sep 2024)
[ICCV 2023] Patrick Esser, Johnathan Chiu, Parmida Atighehchian, et al. Jonathan Granskog, Anastasis Germanidis
- DreamVideo: High-Fidelity Image-to-Video Generation with Image Retention and Text Guidance (16 Sep 2024)
[ICASSP 2025] Cong Wang, Jiaxi Gu, Panwen Hu, et al. Songcen Xu, Hang Xu, Xiaodan Liang
- Emu Video: Factorizing Text-to-Video Generation by Explicit Image Conditioning (2 Aug 2024)
[ECCV 2024] Rohit Girdhar, Mannat Singh, Andrew Brown, et al. Quentin Duval, Samaneh Azadi, Sai Saketh Rambhatla, Akbar Shah, Xi Yin, Devi Parikh, Ishan Misra
- MoVideo: Motion-Aware Video Generation with Diffusion Model (29 Jul 2024)
[ECCV 2024] Jingyun Liang, Yuchen Fan, Kai Zhang, et al. Radu Timofte, Luc Van Gool, Rakesh Ranjan
- I2V-Adapter: A General Image-to-Video Adapter for Diffusion Models (13 Jul 2024)
[ACM SIGGRAPH 2024] Xun Guo1, Mingwu Zheng, Liang Hou, et al. Yuan Gao, Yufan Deng, Pengfei Wan, Di Zhang, Yufan Liu, Weiming Hu, Zhengjun Zha, Haibin Huang, Chongyang Ma
- EasyAnimate: A High-Performance Long Video Generation Method based on Transformer Architecture (5 Jul 2024)
Jiaqi Xu, Xinyi Zou, Kunzhe Huang, et al. Yunkuo Chen, Bo Liu, MengLi Cheng, Xing Shi, Jun Huang
- ConsistI2V: Enhancing Visual Consistency for Image-to-Video Generation (1 Jul 2024)
[TMLR 2024] Weiming Ren, Huan Yang, Ge Zhang, et al. Cong Wei, Xinrun Du, Wenhao Huang, Wenhu Chen
- OmniTokenizer: A Joint Image-Video Tokenizer for Visual Generation (13 Jun 2024)
[NeurIPS 2024] Junke Wang, Yi Jiang, Zehuan Yuan, et al. Binyue Peng, Zuxuan Wu, Yu-Gang Jiang
- AID: Adapting Image2Video Diffusion Models for Instruction-guided Video Prediction (10 Jun 2024)
Zhen Xing, Qi Dai, Zejia Weng, et al. Zuxuan Wu, Yu-Gang Jiang
- TI2V-Zero: Zero-Shot Image Conditioning for Text-to-Video Diffusion Models (25 Apr 2024)
[CVPR 2024] Haomiao Ni, Bernhard Egger, Suhas Lohit, et al. Anoop Cherian, Ye Wang, Toshiaki Koike-Akino, Sharon X. Huang, Tim K. Marks
- TRIP: Temporal Residual Learning with Image Noise Prior for Image-to-Video Diffusion Models (25 Mar 2024)
[CVPR 2024] Zhongwei Zhang, Fuchen Long, Yingwei Pan, et al. Zhaofan Qiu, Ting Yao, Yang Cao, Tao Mei
- Follow-Your-Click: Open-domain Regional Image Animation via Short Prompts (13 Mar 2024)
Yue Ma, Yingqing He, Hongfa Wang, et al. Andong Wang, Chenyang Qi, Chengfei Cai, Xiu Li, Zhifeng Li, Heung-Yeung Shum, Wei Liu, Qifeng Chen
- AtomoVideo: High Fidelity Image-to-Video Generation (5 Mar 2024)
Litong Gong, Yiran Zhu, Weijie Li, et al. Xiaoyang Kang, Biao Wang, Tiezheng Ge, Bo Zheng
- Tuning-Free Noise Rectification for High Fidelity Image-to-Video Generation (5 Mar 2024)
Weijie Li, Litong Gong, Yiran Zhu, et al. Fanda Fan, Biao Wang, Tiezheng Ge, Bo Zheng
[![Code](https://img.shields.io/github/stars/alimama-creative/Noise-Rect