AI Rank
中文
← Open Source
SakanaAI

self-adaptive-llms

A Self-adaptation Framework🐙 that adapts LLMs for unseen tasks in real-time.

Model DevelopmentFine-tuningPython
Open on GitHub
Momentum
+0stars in 24 hours0.0%
1.23k
Stars
142
Forks
+0
This week
3
Contributors
Created 2024-12-02 · Updated 2026-10-16 · #11073 today
Top developers
floatingsundavidbegineltociear

SakanaAI/self-adaptive-llms is ranked 11,073 with 1,225 stars.

Same-category comparison

SakanaAI/self-adaptive-llms is ranked 11,073 among these 8 peers; google/prettytensor is ranked 11,069.

Same-category comparison
NameRankMetrics
huoyijie/AdvancedEAST11,054
Language:
Python
Stars:
1,229
Forks:
371
Description:
AdvancedEAST is an algorithm used for Scene image text detect, which is primarily based on EAST, and the significant improvement was also made, which make long text predictions more accurate.https://github.com/huoyijie/raspberrypi-car.
Genesis-Embodied-AI/RoboGen11,058
Language:
Python
Stars:
1,228
Forks:
108
Description:
A generative and self-guided robotic agent that endlessly propose and master new skills.
rohanpsingh/LearningHumanoidWalking11,061
Language:
Python
Stars:
1,228
Forks:
139
Description:
Training a humanoid robot for locomotion using Reinforcement Learning.
google/prettytensor11,069
Language:
Python
Stars:
1,226
Forks:
142
Description:
Pretty Tensor: Fluent Networks in TensorFlow.
Bolin97/GongBU11,077
Language:
Svelte
Stars:
1,224
Forks:
89
Description:
Paper accepted by CIKM 2024. Codes of GongBU, a LLM fine-tuning platform for domain-specific adaptation.
dee1024/pytorch-captcha-recognition11,084
Language:
Python
Stars:
1,223
Forks:
326
wagenaartje/neataptic11,086
Language:
JavaScript
Stars:
1,223
Forks:
286
Description:
:rocket: Blazing fast neuro-evolution & backpropagation for the browser and Node.js.
google-research/fixmatch11,087
Language:
Python
Stars:
1,222
Forks:
191
Description:
A simple method to perform semi-supervised learning with limited data.
Key parameters and specifications
Key parameters and specifications
ParameterValue
LanguagePython
CategoryModel Development
SubcategoryFine-tuning
Stars1,225
Forks142
Contributors3
Created date2024-12-02
Updated date2026-10-16
Stars in 1 day0
Stars in 7 days0
Rank11,073
Previous rank11,028
Rank change-45
7-day stars rank12,877
Previous 7-day stars rank12,892
7-day stars rank change15
Total stars rank9,535
Previous total stars rank9,529
Total stars rank change-6
CountryJapan
README summary

Transformer 2 : Self-adaptive LLMs 🐙 📚 [Paper] | 📄 [Blog] Self-adaptive large language models (LLMs) aim to solve the challenges posed by traditional fine-tuning methods, which are often computationally intensive and static in their ability to handle diverse tasks. We are excited to introduce Transformer², a novel self-adaptation framework that adapts LLMs for unseen tasks in real-time by selectively adjusting only the singular components of their weight matrices. During inference, Transformer² employs a two-pass mechanism: first, a dispatch

system identifies the task properties, and then task-specific "expert" vectors, trained using reinforcement learning, are dynamically mixed to obtain targeted behavior for the incoming prompt. Install Tasks Evaluator Usage We provide example scripts for both training and evaluation. Please change the argument in the provided script to choose among models and tasks Training Evaluation Prompt-based evaluation Classification experts can be loaded by specifying the CLS_EXPERT_PATH in the script. Few-shots evaluation Citation If you find **Transformer

^2** useful for your research, please cite using this BibTeX:

Frequently asked questions

How many stars and forks are recorded?

SakanaAI/self-adaptive-llms has 1,225 stars and 142 forks.

What is the repository rank?

SakanaAI/self-adaptive-llms is ranked 11,073.

What language and category are recorded?

SakanaAI/self-adaptive-llms has 1,225 stars, uses Python and belongs to Model Development.

Which repository is closest in rank?

SakanaAI/self-adaptive-llms is compared with google/prettytensor, ranked 11,069 with 1,226 stars.

README

Transformer2: Self-adaptive LLMs 🐙

📚 [Paper] | 📄 [Blog]

Self-adaptive large language models (LLMs) aim to solve the challenges posed by traditional fine-tuning methods, which are often computationally intensive and static in their ability to handle diverse tasks.

We are excited to introduce Transformer², a novel self-adaptation framework that adapts LLMs for unseen tasks in real-time by selectively adjusting only the singular components of their weight matrices. During inference, Transformer² employs a two-pass mechanism: first, a dispatch system identifies the task properties, and then task-specific "expert" vectors, trained using reinforcement learning, are dynamically mixed to obtain targeted behavior for the incoming prompt.

Installation

1. Clone the Repo

git clone https://github.com/SakanaAI/self-adaptive-llms
cd self-adaptive-llms

2. Install Libraries

conda create -n t2 python=3.11 -y
conda activate t2
pip install --upgrade pip
pip install -r requirements.txt

3. Install Tasks Evaluator

cd evaluation/fishfarm
pip install -e .

Usage

We provide example scripts for both training and evaluation.

Please change the argument in the provided script to choose among models and tasks

Training

bash scripts/train_task_expert.sh

Evaluation

Prompt-based evaluation

Classification experts can be loaded by specifying the CLS_EXPERT_PATH in the script.

bash scripts/eval_prompt_based.sh

Few-shots evaluation

bash scripts/eval_few_shot.sh

Citation

If you find Transformer^2 useful for your research, please cite using this BibTeX:

@misc{sun2025transformersquaredselfadaptivellms,
      title={Transformer-Squared: Self-adaptive LLMs}, 
      author={Qi Sun and Edoardo Cetin and Yujin Tang},
      year={2025},
      eprint={2501.06252},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2501.06252}, 
}
Nearby in the ranking
Similar items