ast

YuanGongND/ast is ranked 10,195 among these 8 peers; thu-ml/unidiffuser is ranked 10,179.

AST: Audio Spectrogram Transformer News Introduction Citing Getting Started ESC-50 Recipe Speechcommands Recipe AudioSet Recipe Pretrained Models Use Pretrained Model For Downstream Tasks Contact News May, 2023: We have released demo for our audio large language model LTU (listen, think, and understand) that can do zero-shot audio classification and advanced reasoning. Try the online interactive demo **[[here]]( November, 2022: We decoupe and hyper-parameters by moving hyper-parameters from and to , so that it is easier to adapt

the scripts to new datasets. This might cause a bug, please report if you have any issue running any recipe. May, 2022: It was found that newer package has different behavior with older ones in SpecAugment and will cause a bug. March, 2022: We released a new preprint *CMKD: CNN/Transformer-Based Cross-Model Knowledge Distillation for Audio Classification*, where we proposed a knowledge distillation based method to further improve the AST model performance without changing its architecture. Feb, 2022

: The Self-Supervised AST (SSAST) code is released [[**here**]]( SSAST use self-supervised pretraining instead of supervised ImageNet pretraining, so it supports arbitrary patch shape and size (e. , a temperal frame and a square patch) with a good performance. Nov, 2021: The PSLA training pipeline used to train AST and baseline efficientnet model code is released [[**here**]]( It is a strong audio classification training pipeline that can be used for most deep learning models. Also, it has a one-click

FSD50K recipe that achieves SOTA 0. Introduction This repository contains the official implementation (in PyTorch) of the **Audio Spectrogram Transformer (AST)** proposed in the Interspeech 2021 paper AST: Audio Spectrogram Transformer (Yuan Gong, Yu-An Chung, James Glass). AST is the first **convolution-free, purely** attention-based model for audio classification which supports variable length input and can be applied to various tasks. We evaluate AST on various audio classification benchmarks, where it achieves new state-of-the-art results

of 0. For details, please refer to the paper and the ISCA SIGML talk. AST can be used with a few lines of code, and we also provide recipes to reproduce the SOTA results on AudioSet, ESC-50, and Speechcommands with almost one click. The AST model file is in , the recipes are in , when you run , it will call , which will then call and , which will then call . Citing Please cite our paper(s) if you find this repository

useful. The first paper proposes the Audio Spectrogram Transformer while the second paper describes the training pipeline that we applied on AST to achieve the new state-of-the-art on AudioSet. Clone or download this repository and set it as the working directory, create a virtual environment and install the dependencies. **Parameters:**\ : The number of classes (default: ). \ : The stride of patch spliting on the frequency dimension, for 16\*16 patchs, fstride=16 means no overlap, fstride=10 means overlap

of 6 (used in the paper). (default: )\ : The stride of patch spliting on the time dimension, for 16*16 patchs