AI Rank
中文
← Open Source
baaivision

tokenize-anything

[ECCV 2024] Tokenize Anything via Prompting.

ModelsVision modelsJupyter Notebook
Open on GitHub
Momentum
+0stars in 24 hours0.0%
600
Stars
27
Forks
+0
This week
3
Contributors
Created 2023-12-14 · Updated 2026-07-30 · #16298 today
Top developers
PhyscalXeltociearWXinlong

baaivision/tokenize-anything is ranked 16,298 with 600 stars.

Same-category comparison

baaivision/tokenize-anything is ranked 16,298 among these 8 peers; naver/anny is ranked 16,304.

Same-category comparison
NameRankMetrics
marl/openl316,206
Language:
Jupyter Notebook
Stars:
606
Forks:
66
Description:
OpenL3: Open-source deep audio and image embeddings.
google-research/lasertagger16,258
Language:
Python
Stars:
602
Forks:
92
facebookresearch/WSL-Images16,280
Language:
Python
Stars:
601
Forks:
62
Description:
Weakly Supervised Learning On Images.
shamangary/SSR-Net16,289
Language:
Python
Stars:
601
Forks:
146
Description:
[IJCAI18] SSR-Net: A Compact Soft Stagewise Regression Network for Age Estimation.
naver/anny16,304
Language:
Python
Stars:
600
Forks:
49
Description:
Anny, A Free and Interpretable Human Body Model for all ages, written in PyTorch.
ncbi-nlp/bluebert16,305
Language:
Python
Stars:
600
Forks:
80
Description:
BlueBERT, pre-trained on PubMed abstracts and clinical notes (MIMIC-III).
MIV-XJTU/JanusVLN16,308
Language:
Python
Stars:
599
Forks:
42
Description:
[ICLR2026] Official implementation for "JanusVLN: Decoupling Semantics and Spatiality with Dual Implicit Memory for Vision-Language Navigation".
d-li14/mobilenetv3.pytorch16,333
Language:
Python
Stars:
598
Forks:
130
Description:
74.3% MobileNetV3-Large and 67.2% MobileNetV3-Small model on ImageNet.
Key parameters and specifications
Key parameters and specifications
ParameterValue
LanguageJupyter Notebook
CategoryModels
SubcategoryVision models
KeywordsComputer vision
Stars600
Forks27
Contributors3
Created date2023-12-14
Updated date2026-07-30
Stars in 1 day0
Stars in 7 days0
Rank16,298
Previous rank16,088
Rank change-210
7-day stars rank15,456
Previous 7-day stars rank15,466
7-day stars rank change10
Total stars rank16,786
Previous total stars rank16,762
Total stars rank change-24
CountryChina
README summary

Tokenize Anything via Prompting Ting Pan 1,2* , Lulu Tang 2* , Xinlong Wang 2¶ , Shiguang Shan 1 1 ICT-CAS, 2 BAAI Equal Contribution, ¶ Project Lead [ ] [ ] We present **T**okenize **A**nything via **P**rompting, a unified and promptable model capable of simultaneously segmenting, recognizing, and captioning arbitrary regions, with flexible visual prompts (point, box and sketch). The model is trained with exhaustive segmentation masks sourced from SA-1B, coupled with semantic priors from a pre-trained EVA-CLIP with

5 billion parameters. 3 (for TextGeneration) gradio-image-prompter (for GradioApp, Install from URL) Installing Package Clone this repository to local disk and install: You can also install from the remote repository: Quick Start Development The **TAP** models can be used for diverse vision and language tasks. We adopt a modular design that decouples all components and predictors. As a best practice, implement your custom predictor and asynchronous pipeline as follows: See builtin examples (web-demo and evaluations) provided in

scripts for more details. Evaluation See Evaluation Guide for TAP-H. 1 Release Notes Three versions of the model are available with different image encoders. Use a longer pre-training and fine-tuning schedule (improved segmentation and caption performance). Apply weight decay for all bias parameters (avoid FP16 overflow in QK matmul). Sample point prompts from predicted mask instead of GT box during VG training. 1 model | (100% SA-1B, 180k), (VG, 50ep) | 4bdfb9 | 🤗 HF link | | **tap_vit_l** | ViT

-L TAP v1. 1 model | (100% SA-1B, 180k), (VG, 50ep) | c1d41f | 🤗 HF link | | **tap_vit_b** | ViT-B TAP v1. 1 model | (100% SA-1B, 180k), (VG, 50ep) | 707f80 | 🤗 HF link | V1. 0 Release Notes Two versions of the model are available with different image encoders. 0 model | (50% SA-1B, 90k), (VG, 25ep) | 03f8ec | 🤗 HF link | | **tap_vit_b** | ViT-B TAP v1. 0 model | (50% SA-1B, 90k), (VG, 25ep) | b45cbf | 🤗 HF link | Concept weights ***Note***: You

can generate these weights following the Concept Guide. 0 Citation Acknowledgement We thank the repositories: SAM, EVA, LLaMA, FlashAttention, Gradio, Detectron2 and CodeWithGPU.

Frequently asked questions

How many stars and forks are recorded?

baaivision/tokenize-anything has 600 stars and 27 forks.

What is the repository rank?

baaivision/tokenize-anything is ranked 16,298.

What language and category are recorded?

baaivision/tokenize-anything has 600 stars, uses Jupyter Notebook and belongs to Models.

Which repository is closest in rank?

baaivision/tokenize-anything is compared with naver/anny, ranked 16,304 with 600 stars.

README

Tokenize Anything via Prompting

Ting Pan1,2*,   Lulu Tang2*,   Xinlong Wang2¶,   Shiguang Shan1

1ICT-CAS,   2BAAI

  • Equal Contribution, ¶Project Lead

[Paper] [🤗 Demo]

We present Tokenize Anything via Prompting, a unified and promptable model capable of simultaneously segmenting, recognizing, and captioning arbitrary regions, with flexible visual prompts (point, box and sketch). The model is trained with exhaustive segmentation masks sourced from SA-1B, coupled with semantic priors from a pre-trained EVA-CLIP with 5 billion parameters.

Installation

Preliminaries

torch >= 2.1

flash-attn >= 2.3.3 (for TextGeneration)

gradio-image-prompter (for GradioApp, Install from URL)

Installing Package

Clone this repository to local disk and install:

cd tokenize-anything && pip install .

You can also install from the remote repository:

pip install git+ssh://git@github.com/baaivision/tokenize-anything.git

Quick Start

Development

The TAP models can be used for diverse vision and language tasks.

We adopt a modular design that decouples all components and predictors.

As a best practice, implement your custom predictor and asynchronous pipeline as follows:

from tokenize_anything import model_registry

with :
    model = model_registry[""](checkpoint="")
    results = (model, *args, **kwargs)

server.collect_results()

See builtin examples (web-demo and evaluations) provided in scripts for more details.

Inference

See Inference Guide.

See Concept Guide.

Evaluation

See Evaluation Guide for TAP-H.

See Evaluation Guide for TAP-L.

See Evaluation Guide for TAP-B.

Models

Model weights

V1.1 Release Notes

  • Three versions of the model are available with different image encoders.
  • Use a longer pre-training and fine-tuning schedule (improved segmentation and caption performance).
  • Apply weight decay for all bias parameters (avoid FP16 overflow in QK matmul).
  • Sample point prompts from predicted mask instead of GT box during VG training.
Model Description Schedule MD5 Weights
tap_vit_h ViT-H TAP v1.1 model (100% SA-1B, 180k), (VG, 50ep) 4bdfb9 🤗 HF link
tap_vit_l ViT-L TAP v1.1 model (100% SA-1B, 180k), (VG, 50ep) c1d41f 🤗 HF link
tap_vit_b ViT-B TAP v1.1 model (100% SA-1B, 180k), (VG, 50ep) 707f80 🤗 HF link

V1.0 Release Notes

  • Two versions of the model are available with different image encoders.
  • Original paper results.
Model Description Schedule MD5 Weights
tap_vit_l ViT-L TAP v1.0 model (50% SA-1B, 90k), (VG, 25ep) 03f8ec 🤗 HF link
tap_vit_b ViT-B TAP v1.0 model (50% SA-1B, 90k), (VG, 25ep) b45cbf 🤗 HF link

Concept weights

Note: You can generate these weights following the Concept Guide.

Concept Description Weights
Merged-2560 Merged concepts 🤗 HF link
LVIS-1203 LVIS concepts 🤗 HF link
COCO-80 COCO concepts 🤗 HF link

License

Apache License 2.0

Citation

@article{pan2023tap,
  title={Tokenize Anything via Prompting},
  author={Pan, Ting and Tang, Lulu and Wang, Xinlong and Shan, Shiguang},
  journal={arXiv preprint arXiv:2312.09128},
  year={2023}
}

Acknowledgement

We thank the repositories: SAM, EVA, LLaMA, FlashAttention, Gradio, Detectron2 and CodeWithGPU.

Nearby in the ranking
Similar items