A series of large language models trained from scratch by developers @01-ai.
Key parameters and specifications
Key parameters and specifications
Parameter
Value
Language
Python
Category
Models
Subcategory
Vision models
Stars
8,361
Forks
1,345
Contributors
0
Created date
2021-12-06
Updated date
2026-10-15
Stars in 1 day
-1
Stars in 7 days
0
Rank
18,433
Previous rank
7,292
Rank change
-11,141
7-day stars rank
11,666
Previous 7-day stars rank
9,033
7-day stars rank change
-2,633
Total stars rank
1,711
Previous total stars rank
1,710
Total stars rank change
-1
Country
USA
README summary
Masked Autoencoders: A PyTorch Implementation This is a PyTorch/GPU re-implementation of the paper Masked Autoencoders Are Scalable Vision Learners: The original implementation was in TensorFlow+TPU. This re-implementation is in PyTorch+GPU. This repo is a modification on the DeiT repo. Installation and preparation follow that repo. This repo is based on , for which a fix is needed to work with PyTorch 1. Catalog [x] Visualization demo [x] Pre-trained checkpoints + fine-tuning code [x] Pre-training
code Visualization demo Run our interactive visualization demo using Colab notebook (no GPU needed): Fine-tuning with pre-trained checkpoints The following table provides the pre-trained checkpoints used in the paper, converted from TF/TPU to PT/GPU: ViT-Base ViT-Large ViT-Huge pre-trained checkpoint download download download md5 8cad7c b8b06e 9bdbb0 The fine-tuning instruction is in FINETUNE. By fine-tuning these pre-trained models, we rank #1 in these classification tasks (detailed in the paper
): ViT-B ViT-L ViT-H ViT-H 448 prev best ImageNet-1K (no external data) 83. 1 following are evaluation of the same model weights (fine-tuned in original ImageNet-1K): ImageNet-Corruption (error rate) 51. 0 following are transfer learning by fine-tuning the pre-trained MAE on the target dataset: iNaturalists 2017 70. 0 Pre-training The pre-training instruction is in PRETRAIN. License This project is under the CC-BY-NC 4.
Frequently asked questions
How many stars and forks are recorded?
facebookresearch/mae has 8,361 stars and 1,345 forks.
What is the repository rank?
facebookresearch/mae is ranked 18,433.
What language and category are recorded?
facebookresearch/mae has 8,361 stars, uses Python and belongs to Models.
Which repository is closest in rank?
facebookresearch/mae is compared with LianjiaTech/BELLE, ranked 18,434 with 8,265 stars.
@Article{MaskedAutoencoders2021,
author = {Kaiming He and Xinlei Chen and Saining Xie and Yanghao Li and Piotr Doll{\'a}r and Ross Girshick},
journal = {arXiv:2111.06377},
title = {Masked Autoencoders Are Scalable Vision Learners},
year = {2021},
}
The original implementation was in TensorFlow+TPU. This re-implementation is in PyTorch+GPU.
This repo is a modification on the DeiT repo. Installation and preparation follow that repo.
This repo is based on timm==0.3.2, for which a fix is needed to work with PyTorch 1.8.1+.
Catalog
[x] Visualization demo
[x] Pre-trained checkpoints + fine-tuning code
[x] Pre-training code
Visualization demo
Run our interactive visualization demo using Colab notebook (no GPU needed):
Fine-tuning with pre-trained checkpoints
The following table provides the pre-trained checkpoints used in the paper, converted from TF/TPU to PT/GPU:
