AI Rank
EN
← 开源
yule-BUAA

MergeLM

Codebase for Merging Language Models (ICML 2024)

Model DevelopmentFine-tuningPython
在 GitHub 打开
增长势头
+024 小时新增 Star0.0%
878
Star
53
Fork
+0
本周
2
贡献者
创建于 2023-11-06 · 更新于 2026-10-14 · 今日第 12965 名
主要开发者
eltociearyule-BUAA

yule-BUAA/MergeLM 排名第 12,965，有 878 个星标。

同类别比较

yule-BUAA/MergeLM 排名第 12,965，此处有 8 个同类条目；mystic123/tensorflow-yolo-v3 排名第 12,962。

同类别比较
名称排名指标
QData/spacetimeformer12,955
语言:
Jupyter Notebook
星标数:
879
分叉数:
201
简介:
Multivariate Time Series Forecasting with efficient Transformers. Code for the paper "Long-Range Transformers for Dynamic Spatiotemporal Forecasting."
dmlc/dmlc-core12,959
语言:
C++
星标数:
878
分叉数:
525
简介:
A common bricks library for building scalable and portable distributed machine learning.
lucidrains/RETRO-pytorch12,961
语言:
Python
星标数:
878
分叉数:
108
简介:
Implementation of RETRO, Deepmind's Retrieval based Attention net, in Pytorch
mystic123/tensorflow-yolo-v312,962
语言:
Python
星标数:
878
分叉数:
343
简介:
Implementation of YOLO v3 object detector in Tensorflow (TF-Slim)
google/hdrnet12,968
语言:
Python
星标数:
877
分叉数:
319
简介:
An implementation of 'Deep Bilateral Learning for Real-Time Image Enhancement', SIGGRAPH 2017
EpistasisLab/pmlb12,973
语言:
Python
星标数:
876
分叉数:
144
简介:
PMLB: A large, curated repository of benchmark datasets for evaluating supervised machine learning algorithms.
WooooDyy/AgentGym-RL12,975
语言:
Python
星标数:
876
分叉数:
89
简介:
Code and implementations for the paper "AgentGym-RL: Training LLM Agents for Long-Horizon Decision Making through Multi-Turn Reinforcement Learning" by Zhiheng Xi et al.
brownvc/R3GAN12,976
语言:
Python
星标数:
876
分叉数:
45
简介:
Code for NeurIPS 2024 paper - The GAN is dead; long live the GAN! A Modern Baseline GAN - by Huang et al.
关键参数与规格
关键参数与规格
参数值
语言Python
类别Model Development
子类别Fine-tuning
星标数878
分叉数53
贡献者2
创建日期2023-11-06
更新日期2026-10-14
1 天星标数0
7 天星标数0
排名12,965
上次排名12,862
排名变化-103
7 天星标排名13,694
上次 7 天星标排名10,549
7 天星标排名变化-3,145
总星标排名12,343
上次总星标排名12,319
总星标排名变化-24
国家China
README 摘录

Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunch This repository is built for the paper Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunch. 🔔 If you have any questions or suggestions, please feel free to let us know. You can directly email Le Yu using the email address yule@buaa. 💥 News 💥 🔥🔥🔥[**May 2, 2024**] Our paper is accepted at ICML 2024! The camera ready version is coming soon. 🔥🔥🔥[**February

9, 2024**] Special thanks to Sourab Mangrulkar for integrating our work into the huggingface/peft Project! 🔥🔥🔥[**January 28, 2024**] Our merged model supermario_v2 ranks first among 7B models on the Open LLM Leaderboard! 🔥🔥🔥[**December 4, 2023**] We appreciate Minhajul Hoque for sharing our work on Medium! 🔥🔥🔥[**November 29, 2023**] Special thanks to papersread. 🔥🔥🔥[**November 29, 2023**] We appreciate martyn for extending our work to Stable Diffusion models! 🔥🔥🔥[**November 27, 2023**] Special thanks to brucethemoose for applying our work on

the model on Hugging Face! 🔥🔥🔥[**November 26, 2023**] We appreciate cg123 for integrating our work into the mergekit Project! 🔥🔥🔥[**November 25, 2023**] Special thanks to fly51fly for sharing our work on Twitter! 🔥🔥🔥[**November 24, 2023**] We appreciate uukuguy for integrating our work into the Multi-LoRAs Project! 🔥🔥🔥[**November 23, 2023**] Special thanks to WizardLM for sharing our work on Twitter! 🔥🔥🔥[**November 21, 2023**] We appreciate PaperWeekly for sharing our work on WeChat and Zhihu! 🔥🔥🔥[**November 11, 2023**] Special thanks to

夕小瑶 for sharing our work on WeChat and Zhihu! 🔥🔥🔥[**November 6, 2023**] Our paper is available on arXiv, Papers With Code, and Hugging Face. Overview In this work, we uncover that Language Models (LMs), either encoder- or decoder-based, can **obtain new capabilities by assimilating the parameters of homologous models without the need for retraining or GPUs**. We introduce a novel operation called **DARE** to directly set most of (90% or even 99%) the delta parameters to zeros

without affecting the capabilities of SFT LMs. We sparsify delta parameters of multiple SFT homologous models with DARE as a **general preprocessing technique** and subsequently merge them into a single model by parameter averaging. The workflow is shown as follows, By conducting extensive experiments, we find that: 1. DARE is effective for SFT models whose delta parameter value ranges are relatively small (e. 005), being able to eliminate even 99\% delta parameters. Larger models can tolerate a higher proportion of

discarded parameters, indicating that SFT naturally learns an extremely sparse set of delta parameters, and nearly all abilities originate from the pre-trained LMs. DARE can merge multiple task-specific LMs into one LM with diverse abilities, which is able to possess the functionalities of all SFT models. For instance, the merger of WizardLM and WizardMath increases the GSM8K accuracy of WizardLM from 2. 3, maintaining its instruction-following capabilities while surpassing WizardMath's original 64. Language Models and Datasets We

conduct experiments on both encoder- and decoder-based LMs. For encoder-based LMs, we choose bert-base-uncased and roberta

常见问题

记录了多少星标和分叉？

yule-BUAA/MergeLM 有 878 个星标和 53 个分叉。

仓库排名是多少？

yule-BUAA/MergeLM 排名第 12,965。

记录的语言和类别是什么？

yule-BUAA/MergeLM 有 878 个星标，语言为 Python，类别为 Model Development。

哪个仓库排名最接近？

yule-BUAA/MergeLM 与 mystic123/tensorflow-yolo-v3 比较，后者排名第 12,962，有 878 个星标。

README

Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunch

This repository is built for the paper Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunch. 🔔 If you have any questions or suggestions, please feel free to let us know. You can directly email Le Yu using the email address yule@buaa.edu.cn or post an issue on this repository.

💥 News 💥

Overview

In this work, we uncover that Language Models (LMs), either encoder- or decoder-based, can obtain new capabilities by assimilating the parameters of homologous models without the need for retraining or GPUs.

  1. We introduce a novel operation called DARE to directly set most of (90% or even 99%) the delta parameters to zeros without affecting the capabilities of SFT LMs.
  2. We sparsify delta parameters of multiple SFT homologous models with DARE as a general preprocessing technique and subsequently merge them into a single model by parameter averaging.

The workflow is shown as follows,

By conducting extensive experiments, we find that:

  1. DARE is effective for SFT models whose delta parameter value ranges are relatively small (e.g., within 0.005), being able to eliminate even 99% delta parameters. Larger models can tolerate a higher proportion of discarded parameters, indicating that SFT naturally learns an extremely sparse set of delta parameters, and nearly all abilities originate from the pre-trained LMs. See (a) in the figure below.
  2. DARE can merge multiple task-specific LMs into one LM with diverse abilities, which is able to possess the functionalities of all SFT models. For instance, the merger of WizardLM and WizardMath increases the GSM8K accuracy of WizardLM from 2.2 to 66.3, maintaining its instruction-following capabilities while surpassing WizardMath's original 64.2 performance. See (b) in the figure below.

Language Models and Datasets

We conduct experiments on both encoder- and decoder-based LMs.

  • For encoder-based LMs, we choose bert-base-uncased and roberta-base as pre-trained backbones. Eight datasets from the GLUE benchmark are used, including CoLA, SST-2, MRPC, STS-B, QQP, MNLI, QNLI, and RTE.
  • For decoder-based LMs, we choose LLaMA, Llama 2, and Code Llama as pre-trained backbones. WizardLM, WizardMath, WizardCoder-Python, and Code Alpaca are used as fine-tuned models. We evaluate three tasks on five datasets: AlpacaEval (instruction-following), GSM8K and MATH (mathematical reasoning), and HumanEval and MBPP (code-generating).

Note that we provide GSM8K, MATH, and MBPP datasets in math_code_data/ folder, which are obtained from WizardLM repository. Other datasets can be automatically downloaded by our codes. For language models, you can download them either manually or by our codes.

You can also modify the cache_dir in the utils/load_config.py file to specify your own path to save datasets and models.

Model Merging Methods

We provide a well-coded implementation of five model merging methods in this repository, including Average Merging, Task Arithmetic, Fisher Merging, RegMean, and TIES-Merging. We also combine the proposed DARE with the above methods to facilitate the merging performance.

Environments

PyTorch 2.0.1, transformers 4.33.1, datasets 2.13.1, vllm 0.1.4, human_eval, numpy, and tqdm.

Executing Scripts for Encoder-based LMs

For encoder-based LMs, we first fine-tune them on the GLUE benchmark (support both single-task and multi-task settings), and then inference with them. We also provide scripts to merge encoder-based LMs with five model merging methods.

Scripts for Fine-Tuning on GLUE

  • Example of fine-tuning roberta-base on CoLA dataset under single-task setting:
python train_plms_glue.py --language_model_name roberta-base --dataset_name cola --learning_rate 1e-5 --num_runs 5
  • Example of fine-tuning roberta-base on CoLA and RTE datasets under multi-task setting:
python train_plms_glue.py --language_model_name roberta-base --dataset_name cola --multitask_training --auxiliary_dataset_name rte --learning_rate 1e-5 --num_runs 5

Scripts for Inference with DARE and Other Variants

  • Example of direct inference on roberta-base (drop rate 0.0):
python inference_plms_glue.py --language_model_name roberta-base --weight_mask_rate 0.0
  • Example of inference on roberta-base with DARE (drop rate 0.9):
python inference_plms_glue.py --language_model_name roberta-base --weight_mask_rate 0.9 --use_weight_rescale
  • Example of inference on roberta-base with DropOnly (drop rate 0.9):
python inference_plms_glue.py --language_model_name roberta-base --weight_mask_rate 0.9
  • Example of inference on roberta-base with magnitude-based pruning (drop rate 0.9):
python inference_plms_glue.py --language_model_name roberta-base --weight_mask_rate 0.9 --mask_strategy magnitude
  • Example of inference on roberta-base with masking fine-tuned parameters (drop rate 0.9):
python inference_plms_glue.py --language_model_name roberta-base --weight_mask_rate 0.9 --use_weight_rescale --weight_format finetuned_weight

Scripts for Merging Models

  • Example of merging pairwise fine-tuned roberta-base with Average Merging:
python merge_plms_glue.py --merging_method_name average_merging --language_model_name roberta-base
  • Example of merging pairwise fine-tuned roberta-base with Fisher Merging:
python merge_plms_glue.py --merging_method_name fisher_merging --normalize_fisher_weight --language_model_name roberta-base
  • Example of merging pairwise fine-tuned roberta-base with Average Merging and DARE:
python merge_plms_glue.py --merging_method_name mask_merging --use_weight_rescale --language_model_name roberta-base --mask_apply_method average_merging

Executing Scripts for Decoder-based LMs

Since the decoder-based LMs we use have already been fine-tuned, they can be directly utilized for inference. We also provide scripts to merge decoder-based LMs with two model merging methods (Average Merging and Task Arithmetic).

Scripts for Inference with DARE and Other Variants

  • Example of direct inference on WizardMath-7B-V1.0 on GSM8K (drop rate 0.0):
python inference_llms_instruct_math_code.py --dataset_name gsm8k --finetuned_model_name WizardMath-7B-V1.0 --tensor_parallel_size 1 --weight_mask_rate 0.0
  • Example of inference on WizardMath-7B-V1.0 on GSM8K with DARE (drop rate 0.9):
python inference_llms_instruct_math_code.py --dataset_name gsm8k --finetuned_model_name WizardMath-7B-V1.0 --tensor_parallel_size 1 --weight_mask_rate 0.9 --use_weight_rescale
  • Example of inference on WizardMath-7B-V1.0 on GSM8K with DropOnly (drop rate 0.9):
python inference_llms_instruct_math_code.py --dataset_name gsm8k --finetuned_model_name WizardMath-7B-V1.0 --tensor_parallel_size 1 --weight_mask_rate 0.9
  • Example of inference on WizardMath-7B-V1.0 on GSM8K with magnitude-based pruning (drop rate 0.9):
python inference_llms_instruct_math_code.py --dataset_name gsm8k --finetuned_model_name WizardMath-7B-V1.0 --tensor_parallel_size 1 --weight_mask_rate 0.9 --mask_strategy magnitude
  • Example of inference on WizardMath-7B-V1.0 on GSM8K with masking fine-tuned parameters (drop rate 0.9):
python inference_llms_instruct_math_code.py --dataset_name gsm8k --finetuned_model_name WizardMath-7B-V1.0 --tensor_parallel_size 1 --weight_mask_rate 0.9 --use_weight_rescale --weight_format finetuned_weight

Scripts for Merging Models

  • Example of merging WizardLM-13B-V1.2 and WizardMath-13B-V1.0 with Average Merging:
python merge_llms_instruct_math_code.py --merge_instruct --merge_math --merging_method_name average_merging --tensor_parallel_size 1
  • Example of merging WizardLM-13B-V1.2 and WizardMath-13B-V1.0 with Task Arithmetic:
python merge_llms_instruct_math_code.py --merge_instruct --merge_math --merging_method_name task_arithmetic --scaling_coefficient 1.0 --tensor_parallel_size 1
  • Example of merging WizardLM-13B-V1.2 and WizardMath-13B-V1.0 with Average Merging and DARE (drop rate 0.2):
python merge_llms_instruct_math_code.py --merge_instruct --merge_math --merging_method_name mask_merging --use_weight_rescale --weight_mask_rate 0.2 --mask_apply_method average_merging --tensor_parallel_size 1

❗Note 1: When merging decoder-based LMs, the number of GPUs we should allocate is equals to num_models_to_merge * tensor_parallel_size. For example, if we want to merge WizardLM-13B-V1.2 and WizardMath-13B-V1.0 with tensor_parallel_size == 1, then we should allocate 2 * 1 = 2 GPUs.

❗Note 2: If "AssertionError: data parallel group is already initialized" error is raised by vllm on your device, please try to run direct_inference_merged_llms_instruct_math_code.py with the corresponding setting. For example, if this error occurs when merging WizardLM-13B-V1.2 and WizardMath-13B-V1.0 with Average Merging and DARE (drop rate 0.2), please run the following command to evaluate on instruct- or math-related task

python direct_inference_merged_llms_instruct_math_code.py --merge_instruct --merge_math --merging_method_name mask_merging --use_weight_rescale --weight_mask_rate 0.2 --mask_apply_method average_merging --tensor_parallel_size 1 --evaluate_task instruct
python direct_inference_merged_llms_instruct_math_code.py --merge_instruct --merge_math --merging_method_name mask_merging --use_weight_rescale --weight_mask_rate 0.2 --mask_apply_method average_merging --tensor_parallel_size 1 --evaluate_task math

Evaluation Process for AlpacaEval, HumanEval and MBPP

For AlpacaEval, HumanEval and MBPP, our codes will store the generated files and please additionally run the following evaluation commands to get the final metrics.

  • For AlpacaEval: We use chatgpt_fn in alpaca_eval repository to compute the win rate. Firstly, please see alpaca_eval repository to install the environment. Then, if you want to evaluate the generated WizardLM-13B-V1.2_inference_mask_0.2_rescale_True.json file, please run
alpaca_eval --model_outputs ./save_gen_instruct_responses_results/alpaca_eval/WizardLM-13B-V1.2_inference_mask_0.2_rescale_True.json --annotators_config chatgpt_fn --name WizardLM-13B-V1.2_inference_mask_0.2_rescale_True
  • For HumanEval: Firstly, please see human-eval repository to install the environment. Then, if you want to evaluate the generated WizardCoder-Python-13B-V1.0_inference_mask_0.2_rescale_True.jsonl file, please run
evaluate_functional_correctness ./save_gen_codes_results/human_eval/WizardCoder-Python-13B-V1.0_inference_mask_0.2_rescale_True.jsonl
  • For MBPP: Firstly, please see bigcode-evaluation-harness repository to install the environment. Then, if you want to evaluate the generated WizardCoder-Python-13B-V1.0_inference_mask_0.2_rescale_True.jsonl file, please run
accelerate launch ./bigcode-evaluation-harness/main.py --tasks mbpp --allow_code_execution --load_generations_path ./save_gen_codes_results/mbpp/WizardCoder-Python-13B-V1.0_inference_mask_0.2_rescale_True.jsonl

Acknowledgments

We are grateful to the authors of WizardLM for making their project codes publicly available.

Citation

Please consider citing our paper when using this project.

@inproceedings{yu2024language,
  title={Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunch},
  author={Yu, Le and Yu, Bowen and Yu, Haiyang and Huang, Fei and Li, Yongbin},
  booktitle={International Conference on Machine Learning},
  year={2024},
  organization={PMLR}
}

Star History

排名相邻
同类项目