A parallel framework for population-based multi-agent reinforcement learning.
Key parameters and specifications
Key parameters and specifications
Parameter
Value
Language
Python
Category
Model Development
Subcategory
Architecture
Keywords
Autonomous vehicles, Multimodal
Stars
558
Forks
50
Contributors
1
Created date
2024-10-29
Updated date
2026-10-15
Stars in 1 day
0
Stars in 7 days
0
Rank
17,064
Previous rank
16,855
Rank change
-209
7-day stars rank
15,888
Previous 7-day stars rank
15,893
7-day stars rank change
5
Total stars rank
17,786
Previous total stars rank
17,752
Total stars rank change
-34
Country
China
README summary
Senna: Bridging Large Vision-Language Models and End-to-End Autonomous Driving Bo Jiang 1 , Shaoyu Chen 1 , Bencheng Liao 1 , Xingyu Zhang 2 , Wei Yin 2 , Qian Zhang 2 , Chang Huang 2 , Wenyu Liu 1 , Xinggang Wang 1,📧 1 Huazhong University of Science and Technology, 2 Horizon Robotics, 📧 corresponding author News Senna is accepted by IJCV 2026 🎉 ! Checkout our new work 🥰, Senna-2 arXiv paper released. We have released the code and weight of Senna-VLM, along with the
training and evaluation scripts. ☕️ Highlights Senna is an autonomous driving system that integrates a Large Vision-Language Model with an end-to-end model to improve planning safety, robustness and generalization. Senna achieves SOTA planning performance and demonstrates strong cross-scenario generalization and transferability. Getting Started Installtion Data Preparation We provide a script for generating QA data required for Senna training. 6-34b as the model for generating scene descriptions and planning explanations. 5 | 6 View | 128 | Hugging Face | Training
For Stage-1 Mix Pre-training: For Stage-2 Driving Fine-tuning and Stage-3 Planning Fine-tuning (full-parameter fine-tuning): For Stage-2 Driving Fine-tuning and Stage-3 Planning Fine-tuning (LoRA fine-tuning): In our experiments, we observed that full-parameter fine-tuning outperforms LoRA fine-tuning. Therefore, we recommend using full-parameter fine-tuning. However, if your machine has limited GPU memory (e. , only 24GB), you may consider using LoRA fine-tuning as an alternative
. Evaluation You can evaluate the accuracy of Senna meta-action planning using the script below. Visualization By running the visualization script below, you can overlay the predicted meta-actions and front-view scene descriptions onto the front-view image and save the results to the specified path. Qualitative Results Acknowledgments LLaVA, the codebase we built upon, we sincerely thank the contributors for their great work! Citation If you find Senna useful in your research or applications, please consider giving us
a star 🌟 and citing it by the following BibTeX entry.
Frequently asked questions
How many stars and forks are recorded?
hustvl/Senna has 558 stars and 50 forks.
What is the repository rank?
hustvl/Senna is ranked 17,064.
What language and category are recorded?
hustvl/Senna has 558 stars, uses Python and belongs to Model Development.
Which repository is closest in rank?
hustvl/Senna is compared with ignacio-rocco/detectorch, ranked 17,065 with 558 stars.
README
Senna: Bridging Large Vision-Language Models and End-to-End Autonomous Driving
[2026-03-16]: Checkout our new work 🥰, Senna-2 arXiv paper released.
[2024-12-08]: We have released the code and weight of Senna-VLM, along with the training and evaluation scripts.
[2024-10-29]: Senna arXiv paper released. Code/Models are coming soon. Please stay tuned! ☕️
Highlights
Senna is an autonomous driving system that integrates a Large Vision-Language Model with an end-to-end model to improve planning safety, robustness and generalization.
Senna achieves SOTA planning performance and demonstrates strong cross-scenario generalization and transferability.
We provide a script for generating QA data required for Senna training. The script uses LLaVA-v1.6-34b as the model for generating scene descriptions and planning explanations. You can run the script as follows:
For Stage-2 Driving Fine-tuning and Stage-3 Planning Fine-tuning (full-parameter fine-tuning):
sh train_tools/train_senna_llava.sh
For Stage-2 Driving Fine-tuning and Stage-3 Planning Fine-tuning (LoRA fine-tuning):
sh train_tools/train_senna_llava_lora.sh
In our experiments, we observed that full-parameter fine-tuning outperforms LoRA fine-tuning. Therefore, we recommend using full-parameter fine-tuning. However, if your machine has limited GPU memory (e.g., only 24GB), you may consider using LoRA fine-tuning as an alternative.
Evaluation
You can evaluate the accuracy of Senna meta-action planning using the script below.
sh eval_tools/senna_plan_cmd_eval_multi_img.sh
Visualization
By running the visualization script below, you can overlay the predicted meta-actions and front-view scene descriptions onto the front-view image and save the results to the specified path.
sh eval_tools/senna_plan_visualization.sh
Qualitative Results
Acknowledgments
LLaVA, the codebase we built upon, we sincerely thank the contributors for their great work!
Citation
If you find Senna useful in your research or applications, please consider giving us a star 🌟 and citing it by the following BibTeX entry.
@article{jiang2024senna,
title={Senna: Bridging Large Vision-Language Models and End-to-End Autonomous Driving},
author={Bo Jiang and Shaoyu Chen and Bencheng Liao and Xingyu Zhang and Wei Yin and Qian Zhang and Chang Huang and Wenyu Liu and Xinggang Wang},
year={2024},
eprint={2410.22313},
archivePrefix={arXiv},
primaryClass={cs.CV},
url={https://arxiv.org/abs/2410.22313},
}
