AI Rank
中文
← Open Source
instadeepai

nucleotide-transformer

Foundation Models for Genomics & Transcriptomics.

ModelsDomain-specific modelsJupyter Notebook
Open on GitHub
Momentum
+0stars in 24 hours0.0%
926
Stars
103
Forks
+3
This week
11
Contributors
Created 2023-01-24 · Updated 2026-10-15 · #12594 today
Top developers
dallattbernardo-de-almeidaPBordesInstadeep

instadeepai/nucleotide-transformer is ranked 12,594 with 926 stars.

Same-category comparison

instadeepai/nucleotide-transformer is ranked 12,594 among these 8 peers; NVlabs/FasterViT is ranked 12,592.

Same-category comparison
NameRankMetrics
opendilab/LMDrive12,564
Language:
Jupyter Notebook
Stars:
931
Forks:
76
Description:
[CVPR 2024] LMDrive: Closed-Loop End-to-End Driving with Large Language Models.
nathanrs/tiny-diffusion12,582
Language:
Python
Stars:
929
Forks:
90
Description:
A character-level language diffusion model trained on Tiny Shakespeare.
facebookresearch/spiritlm12,587
Language:
Python
Stars:
928
Forks:
63
Description:
Inference code for the paper "Spirit-LM Interleaved Spoken and Written Language Model".
NVlabs/FasterViT12,592
Language:
Python
Stars:
926
Forks:
72
Description:
[ICLR 2024] Official PyTorch implementation of FasterViT: Fast Vision Transformers with Hierarchical Attention.
AMAP-ML/FluxText12,602
Language:
Python
Stars:
924
Forks:
33
Description:
Implementation of "FLUX-Text: A Simple and Advanced Diffusion Transformer Baseline for Scene Text Editing".
facebookresearch/mobile-vision12,613
Language:
Python
Stars:
922
Forks:
168
Description:
Mobile vision models and code.
Vchitect/Vchitect-2.012,621
Language:
Python
Stars:
921
Forks:
22
Description:
Vchitect-2.0: Parallel Transformer for Scaling Up Video Diffusion Models.
360CVGroup/FancyVideo12,628
Language:
Python
Stars:
920
Forks:
51
Description:
Video generation from text&image, 1st-gen.
Key parameters and specifications
Key parameters and specifications
ParameterValue
LanguageJupyter Notebook
CategoryModels
SubcategoryDomain-specific models
KeywordsBiomedical
Stars926
Forks103
Contributors11
Created date2023-01-24
Updated date2026-10-15
Stars in 1 day0
Stars in 7 days3
Rank12,594
Previous rank6,204
Rank change-6,390
7-day stars rank7,150
Previous 7-day stars rank7,179
7-day stars rank change29
Total stars rank11,808
Previous total stars rank11,787
Total stars rank change-21
CountryUK
README summary

AI Foundation Models for Genomics A hub for InstaDeep's cutting-edge deep learning models and research for genomics, originating from the Nucleotide Transformer and its evolutions. This is where we feature our collection of transformer-based genomic language models and innovative downstream applications. Our work in the genomics space began with **The Nucleotide Transformer**, developed in collaboration with Nvidia and TUM and trained on Cambridge-1, and has expanded to include projects like the **Agro Nucleotide Transformer** (in collaboration with

Google, trained on TPU-v4 accelerators), **SegmentNT**, and **ChatNT**. Our mission is to provide the scientific community with powerful, reproducible, and accessible tools to unlock new insights from biological sequences. This repository serves as the central place for sharing our models, inference code, pre-trained weights, and research contributions in the genomics domain, with explorations into future areas like single-cell transcriptomics. We are thrilled to open-source these works and provide the community with access to the code and

pre-trained weights for our diverse set of genomics language models and segmentation models. ✨ Featured Models & Research Evolutions This section highlights the key models and research directions from our team. Each entry provides a brief overview and links to detailed documentation, publications, and resources. *(Detailed code examples, setup for specific models, and in-depth figures are now located in their respective documentation pages within the folder. NTv3 uses a U-Net-like architecture with single-base tokenization to enable efficient

modeling of contexts up to 1 Mb at nucleotide resolution. Pretrained on 9 trillion base pairs from OpenGenome2 and post-trained on > 16,000 functional tracks from 24 animal and plant species, NTv3 achieves state-of-the-art accuracy for functional-track prediction and genome annotation across species. The model can be fine-tuned into a controllable generative model for designing enhancer sequences with specified activity levels, validated experimentally via STARR-seq. These models provide extremely accurate molecular phenotype prediction

compared to existing methods. *This family includes multiple variants (e. , 500M_human_ref, 2B5_1000G, NT-v2 series) which are detailed further in the specific documentation. AgroNT demonstrates state-of-the-art performance across several prediction tasks ranging from regulatory features, RNA processing, and gene expression in plants. SegmentNT, for instance, predicts 14 different classes of human genomic elements in sequences up to 30kb (generalizing to 50kbp) and demonstrates superior performance. This work investigates alternative tokenization strategies for genomic language

models and their impact on downstream performance and interpretability. sCT aims to learn rich representations from complex, high-dimensional single-cell datasets to improve various downstream analytical tasks. It learns biologically meaningful representations from large-scale transcriptomic profiles. It integrates bulk RNA-seq and DNA methylation data to generate powerful joint representations tailored for cancer-type classification and survival analysis. **Built on Strong Foundations:** Leveraging large-scale pre-training and diverse genomic datasets. **Cutting-Edge Research:** Incorporating the latest advancements

in deep learning for biological sequence analysis. **High Performance:** Designed and validated to achieve state-of-the-art results on

Frequently asked questions

How many stars and forks are recorded?

instadeepai/nucleotide-transformer has 926 stars and 103 forks.

What is the repository rank?

instadeepai/nucleotide-transformer is ranked 12,594.

What language and category are recorded?

instadeepai/nucleotide-transformer has 926 stars, uses Jupyter Notebook and belongs to Models.

Which repository is closest in rank?

instadeepai/nucleotide-transformer is compared with NVlabs/FasterViT, ranked 12,592 with 926 stars.

README

AI Foundation Models for Genomics

A hub for InstaDeep's cutting-edge deep learning models and research for genomics, originating from the Nucleotide Transformer and its evolutions.

🎯 Our Focus: Advancing Genomics with AI

Welcome to the InstaDeep AI for Genomics repository! This is where we feature our collection of transformer-based genomic language models and innovative downstream applications. Our work in the genomics space began with The Nucleotide Transformer, developed in collaboration with Nvidia and TUM and trained on Cambridge-1, and has expanded to include projects like the Agro Nucleotide Transformer (in collaboration with Google, trained on TPU-v4 accelerators), SegmentNT, and ChatNT.

Our mission is to provide the scientific community with powerful, reproducible, and accessible tools to unlock new insights from biological sequences. This repository serves as the central place for sharing our models, inference code, pre-trained weights, and research contributions in the genomics domain, with explorations into future areas like single-cell transcriptomics.

We are thrilled to open-source these works and provide the community with access to the code and pre-trained weights for our diverse set of genomics language models and segmentation models.

✨ Featured Models & Research Evolutions

This section highlights the key models and research directions from our team. Each entry provides a brief overview and links to detailed documentation, publications, and resources. (Detailed code examples, setup for specific models, and in-depth figures are now located in their respective documentation pages within the ./docs folder.)

🧬 The Nucleotide Transformer v3 (NTv3)

A unified multi-species foundation model that integrates representation learning, functional-track and genome-annotation prediction, and controllable sequence generation within a single efficient framework. NTv3 uses a U-Net-like architecture with single-base tokenization to enable efficient modeling of contexts up to 1 Mb at nucleotide resolution. Pretrained on 9 trillion base pairs from OpenGenome2 and post-trained on > 16,000 functional tracks from 24 animal and plant species, NTv3 achieves state-of-the-art accuracy for functional-track prediction and genome annotation across species. The model can be fine-tuned into a controllable generative model for designing enhancer sequences with specified activity levels, validated experimentally via STARR-seq.

🧬 The Nucleotide Transformer (NT)

Our foundational language models leverage DNA sequences from over 3,200 diverse human genomes and 850 genomes from a wide range of species. These models provide extremely accurate molecular phenotype prediction compared to existing methods. This family includes multiple variants (e.g., 500M_human_ref, 2B5_1000G, NT-v2 series) which are detailed further in the specific documentation.

🌾 Agro Nucleotide Transformer (AgroNT)

A novel foundational large language model trained on reference genomes from 48 plant species, with a predominant focus on crop species. AgroNT demonstrates state-of-the-art performance across several prediction tasks ranging from regulatory features, RNA processing, and gene expression in plants.

🧩 SegmentNT (& family: SegmentEnformer, SegmentBorzoi)

Segmentation models using transformer backbones (Nucleotide Transformers, Enformer, Borzoi) for predicting genomic elements at single-nucleotide resolution. SegmentNT, for instance, predicts 14 different classes of human genomic elements in sequences up to 30kb (generalizing to 50kbp) and demonstrates superior performance.

💬 ChatNT

A multimodal conversational agent designed with a deep understanding of DNA biological sequences, enabling interactive exploration and analysis of genomic data through natural language.

3️⃣ Codon-NT (Exploring 3-mer Tokenization)

A Nucleotide Transformer model variant trained on 3-mers (codons). This work investigates alternative tokenization strategies for genomic language models and their impact on downstream performance and interpretability.

🧬 Isoformer

A model designed for learning isoform-aware embeddings directly from RNA-seq data, enabling a deeper understanding of transcript-specific expression and regulation.

🔬 sCT (single-Cell Transformer)

Our foundational transformer model for single-cell and spatial transcriptomics data. sCT aims to learn rich representations from complex, high-dimensional single-cell datasets to improve various downstream analytical tasks.

🧪 BulkRNABert

BulkRNABert is a transformer-based, encoder-only foundation model designed for bulk RNA-seq data. It learns biologically meaningful representations from large-scale transcriptomic profiles.

🔗 MOJO (Multi-Omics JOint representation)

MOJO is a multimodal model designed learn embeddings of multi-omics data. It integrates bulk RNA-seq and DNA methylation data to generate powerful joint representations tailored for cancer-type classification and survival analysis.

💡 Why Choose InstaDeep's Genomic Models?

  • Built on Strong Foundations: Leveraging large-scale pre-training and diverse genomic datasets.
  • Cutting-Edge Research: Incorporating the latest advancements in deep learning for biological sequence analysis.
  • High Performance: Designed and validated to achieve state-of-the-art results on challenging genomic tasks.
  • Open and Accessible: We provide pre-trained weights, usage examples, and aim for easy integration into research workflows.
  • Collaborative Spirit: Developed with leading academic and industry partners.
  • Focused Expertise: Created by a dedicated team specializing in AI for genomics at InstaDeep.

🚀 Getting Started

To begin using models from this repository:

  1. Clone the repository:
    git clone https://github.com/instadeepai/nucleotide-transformer.git
cd nucleotide-transformer
  2. Set up your environment (virtual environment recommended):
    python -m venv .venv
source .venv/bin/activate # On Windows use `source .venv\Scripts\activate`
  3. Install the package and dependencies:
    pip install . # Installs the local package
# Or, for a general requirements file if you have one:
# pip install -r requirements.txt

For detailed instructions on individual models, including specific dependencies, downloading pre-trained weights, and Python usage examples, please refer to their dedicated documentation pages linked in the "Featured Models & Research Evolutions" section above (e.g., ./docs/nucleotide_transformer.md).

🤝 Community & Support

  • Questions & Bug Reports: Please use the GitHub Issues page.
  • Discussions: For broader discussions or questions, please use the GitHub Discussions tab (if enabled).
  • Stay Updated: Follow InstaDeep's official channels for announcements on new model releases and research updates.
Nearby in the ranking
Similar items