A series of large language models trained from scratch by developers @01-ai
关键参数与规格
关键参数与规格
参数
值
语言
Python
类别
Models
子类别
Vision models
星标数
8,361
分叉数
1,345
贡献者
0
创建日期
2021-12-06
更新日期
2026-10-15
1 天星标数
-1
7 天星标数
0
排名
18,433
上次排名
7,292
排名变化
-11,141
7 天星标排名
11,666
上次 7 天星标排名
9,033
7 天星标排名变化
-2,633
总星标排名
1,711
上次总星标排名
1,710
总星标排名变化
-1
国家
USA
README 摘录
Masked Autoencoders: A PyTorch Implementation This is a PyTorch/GPU re-implementation of the paper Masked Autoencoders Are Scalable Vision Learners: The original implementation was in TensorFlow+TPU. This re-implementation is in PyTorch+GPU. This repo is a modification on the DeiT repo. Installation and preparation follow that repo. This repo is based on , for which a fix is needed to work with PyTorch 1. Catalog [x] Visualization demo [x] Pre-trained checkpoints + fine-tuning code [x] Pre-training
code Visualization demo Run our interactive visualization demo using Colab notebook (no GPU needed): Fine-tuning with pre-trained checkpoints The following table provides the pre-trained checkpoints used in the paper, converted from TF/TPU to PT/GPU: ViT-Base ViT-Large ViT-Huge pre-trained checkpoint download download download md5 8cad7c b8b06e 9bdbb0 The fine-tuning instruction is in FINETUNE. By fine-tuning these pre-trained models, we rank #1 in these classification tasks (detailed in the paper
): ViT-B ViT-L ViT-H ViT-H 448 prev best ImageNet-1K (no external data) 83. 1 following are evaluation of the same model weights (fine-tuned in original ImageNet-1K): ImageNet-Corruption (error rate) 51. 0 following are transfer learning by fine-tuning the pre-trained MAE on the target dataset: iNaturalists 2017 70. 0 Pre-training The pre-training instruction is in PRETRAIN. License This project is under the CC-BY-NC 4.
@Article{MaskedAutoencoders2021,
author = {Kaiming He and Xinlei Chen and Saining Xie and Yanghao Li and Piotr Doll{\'a}r and Ross Girshick},
journal = {arXiv:2111.06377},
title = {Masked Autoencoders Are Scalable Vision Learners},
year = {2021},
}
The original implementation was in TensorFlow+TPU. This re-implementation is in PyTorch+GPU.
This repo is a modification on the DeiT repo. Installation and preparation follow that repo.
This repo is based on timm==0.3.2, for which a fix is needed to work with PyTorch 1.8.1+.
Catalog
[x] Visualization demo
[x] Pre-trained checkpoints + fine-tuning code
[x] Pre-training code
Visualization demo
Run our interactive visualization demo using Colab notebook (no GPU needed):
Fine-tuning with pre-trained checkpoints
The following table provides the pre-trained checkpoints used in the paper, converted from TF/TPU to PT/GPU:
