Transformer-in-Vision

Recent Transformer-based CV and related works. Welcome to comment/contribute!

The transformer is now a basic component, adopted in nearly all AI models. Keep updated --> updated irregularly.

New Hope: LLM-in-Vision

Resource

• ChatGPT for Robotics: Design Principles and Model Abilities, [Paper], [Code]

• DIFFUSIONDB [Page], [Paper]

• LAION-5B [Page], [Paper]

• LAVIS [Page], [Paper]

• Imagen Video [Page], [Paper]

• Phenaki [Page], [Paper]

• DREAMFUSION [Page], [Paper]

• MAKE-A-VIDEO [Page], [Paper]

• Stable Difffusion [Page], [Paper]

• NUWA-Infinity [Page], [Paper]

• Parti [Page], [Code]

• Imagen [Page], [Paper]

• Gato: A Generalist Agent, [Paper]

• PaLM: Scaling Language Modeling with Pathways, [Paper]

• DALL·E 2 [Page], [Paper]

• SCENIC: A JAX Library for Computer Vision Research and Beyond, [Code]

• V-L joint learning study (with good tables): [METER], [Kaleido-BERT]

• Attention is all you need, [Paper]

• CLIP [Page], [Paper], [Code], [arXiv]

• DALL·E [Page], [Code], [Paper]

• huggingface/transformers

• Kyubyong/transformer, TF

• jadore801120/attention-is-all-you-need-pytorch, Torch

• krasserm/fairseq-image-captioning

• PyTorch Transformers Tutorials

• ictnlp/awesome-transformer

• basicv8vc/awesome-transformer

• dk-liang/Awesome-Visual-Transformer

• yuewang-cuhk/awesome-vision-language-pretraining-papers

Survey

• (arXiv 2023.2) TRANSFORMER-BASED SENSOR FUSION FOR AUTONOMOUS DRIVING: A SURVEY, [Paper], [Page]

• (arXiv 2023.2) Deep Learning for Video-Text Retrieval: a Review, [Paper]

• (arXiv 2023.2) Large-scale Multi-Modal Pre-trained Models: A Comprehensive Survey, [Paper]

• (arXiv 2023.2) Transformer-based Generative Adversarial Networks in Computer Vision: A Comprehensive Survey, [Paper]

• (arXiv 2023.2) Knowledge Distillation in Vision Transformers: A Critical Review, [Paper]

• (arXiv 2023.2) A Survey on Efficient Training of Transformers, [Paper]

• (arXiv 2023.1) ChatGPT is not all you need. A State of the Art Review of large Generative AI models, [Paper]

• (arXiv 2022.12) Transformers in Action Recognition: A Review on Temporal Modeling, [Paper]

• (arXiv 2022.11) Vision Transformers in Medical Imaging: A Review, [Paper]

• (arXiv 2022.11) A survey on knowledge-enhanced multimodal learning, [Paper]

• (arXiv 2022.10) Vision-Language Pre-training: Basics, Recent Advances, and Future Trends, [Paper]

• (arXiv 2022.10) A Survey on Graph Neural Networks and Graph Transformers in Computer Vision: A Task-Oriented Perspective, [Paper]

• (arXiv 2022.09) VISION TRANSFORMERS FOR ACTION RECOGNITION: A SURVEY, [Paper]

• (arXiv 2022.09) Transformers in Remote Sensing: A Survey, [Paper], [Code]

• (arXiv 2022.08) 3D Vision with Transformers: A Survey, [Paper], [Code]

• (arXiv 2022.08) A Survey on Masked Autoencoder for Self-supervised Learning in Vision and Beyond, [Paper]

• (arXiv 2022.07) Vision Transformers: State of the Art and Research Challenges, [Paper]

• (arXiv 2022.07) SELF-SUPERVISED LEARNING FOR VIDEOS: A SURVEY, [Paper]

• (arXiv 2022.06) Multimodal Learning with Transformers: A Survey, [Paper]

• (arXiv 2022.05) Vision Transformer: Vit and its Derivatives, [Paper]

• (arXiv 2022.05) Transformers in 3D Point Clouds: A Survey, [Paper]

• (arXiv 2022.04) Visual Attention Methods in Deep Learning: An In-Depth Survey, [Paper]

• (arXiv 2022.04) Vision-and-Language Pretrained Models: A Survey, [Paper]

• (arXiv 2022.03) A Roadmap for Big Model, [Paper]

• (arXiv 2022.03) Transformers Meet Visual Learning Understanding: A Comprehensive Review, [[Paper]](https://arxiv.org/pdf/2203.12944.pdf）

• (arXiv 2022.03) Recent Advances in Vision Transformer: A Survey and Outlook of Recent Work, [Paper], [Project]

• (arXiv 2022.02) A Survey of Vision-Language Pre-Trained Models, [Paper]

• (arXiv 2022.02) VLP: A Survey on Vision-Language Pre-training, [Paper]

• (arXiv 2022.02) Transformer for Graphs: An Overview from Architecture Perspective, [Paper]

• (arXiv 2022.01) Video Transformers: A Survey, [Paper]

• (arXiv 2021.11) ARE WE READY FOR A NEW PARADIGM SHIFT? A SURVEY ON VISUAL DEEP MLP, [Paper]

• (arXiv 2021.11) A Survey of Visual Transformers, [Paper]

• (arXiv 2021.09) Survey: Transformer based Video-Language Pre-training, [Paper]

• (arXiv 2021.06) A Survey of Transformers, [Paper]

• (arXiv 2021.06) Attention mechanisms and deep learning for machine vision: A survey of the state of the art, [Paper]

• (arXiv 2021.06) Pre-Trained Models: Past, Present and Future, [Paper]

• (arXiv 2021.05) Can Attention Enable MLPs To Catch Up With CNNs? [Paper]

• (arXiv 2021.03) A Practical Survey on Faster and Lighter Transformers, [Paper]

• (arXiv 2021.03) Perspectives and Prospects on Transformer Architecture for Cross-Modal Tasks with Language and Vision, [Paper]

• (arXiv 2021.01) A Survey on Visual Transformer, [Paper]

• (arXiv 2020.9) Efficient Transformers: A Survey, [Paper]

• (arXiv 2020.1) Transformers in Vision: A Survey, [Paper]

Recent Papers

2023.8

• (arXiv 2023.8) VL-PET: Vision-and-Language Parameter-Efficient Tuning via Granularity Control, [Paper], [Project]

2023.5

• (arXiv 2023.5) Understanding Gaussian Attention Bias of Vision Transformers Using Effective Receptive Fields, [Paper]

2023.3

• (arXiv 2023.3) Query-Dependent Video Representation for Moment Retrieval and Highlight Detection, [Paper], [Code]

2023.2

• (arXiv 2023.2) Open-domain Visual Entity Recognition: Towards Recognizing Millions of Wikipedia Entities, [Paper]

• (arXiv 2023.2) KS-DETR: Knowledge Sharing in Attention Learning for Detection Transformer, [Paper], [Code]

• (arXiv 2023.2) HUMAN MOTIONFORMER: TRANSFERRING HUMAN MOTIONS WITH VISION TRANSFORMERS, [Paper], [Code]

• (arXiv 2023.2) Aligning Text-to-Image Models using Human Feedback, [Paper]

• (arXiv 2023.2) Controlled and Conditional Text to Image Generation with Diffusion Prior, [Paper]

• (arXiv 2023.2) Can Pre-trained Vision and Language Models Answer Visual Information-Seeking Questions? [Paper], [Code]

• (arXiv 2023.2) OBJECT-CENTRIC VIDEO PREDICTION VIA DECOUPLING OF OBJECT DYNAMICS AND INTERACTIONS, [Paper], [Project]

• (arXiv 2023.2) Distribution Normalization: An “Effortless” Test-Time Augmentation for Contrastively Learned Visual-language Models, [Paper], [Code]

• (arXiv 2023.2) Teaching CLIP to Count to Ten, [Paper], [Project]

• (arXiv 2023.2) Designing an Encoder for Fast Personalization of Text-to-Image Models, [Paper], [Project]

• (arXiv 2023.2) Side Adapter Network for Open-Vocabulary Semantic Segmentation, [Paper], [Code]

• (arXiv 2023.2) Learning Visual Representations via Language-Guided Sampling, [Paper]

• (arXiv 2023.2) VoxFormer: Sparse Voxel Transformer for Camera-based 3D Semantic Scene Completion, [Paper], [Code]

• (arXiv 2023.2) Language-Driven Representation Learning for Robotics, [Paper], [Project]

• (arXiv 2023.2) A Convolutional Vision Transformer for Semantic Segmentation of Side-Scan Sonar Data, [Paper], [Code]

• (arXiv 2023.2) Lightweight Real-time Semantic Segmentation Network with Efficient Transformer and CNN, [Paper], [Code]

• (arXiv 2023.2) VIEWCO: DISCOVERING TEXT-SUPERVISED SEGMENTATION MASKS VIA MULTI-VIEW SEMANTIC CONSISTENCY, [Paper], [Code]

• (arXiv 2023.2) CertViT: Certified Robustness of Pre-Trained Vision Transformers, [Paper], [Code]

• (arXiv 2023.2) Paparazzi: A Deep Dive into the Capabilities of Language and Vision Models for Grounding Viewpoint Descriptions, [Paper]

• (arXiv 2023.2) MaskedKD: Efficient Distillation of Vision Transformers with Masked Images, [Paper]

• (arXiv 2023.2) A General Visual Representation Guided Framework with Global Affinity for Weakly Supervised Salient Object Detection, [Paper]

• (arXiv 2023.2) ViTA: A Vision Transformer Inference Accelerator for Edge Applications, [Paper]

• (arXiv 2023.2) Video Action Recognition Collaborative Learning with Dynamics via PSO-ConvNet Transformer, [Paper], [Code]

• (arXiv 2023.2) A Pilot Evaluation of ChatGPT and DALL-E 2 on Decision Making and Spatial Reasoning, [Paper]

• (arXiv 2023.2) StyLIP: Multi-Scale Style-Conditioned Prompt Learning for CLIP-based Domain Generalization, [Paper]

• (arXiv 2023.2) Meta Style Adversarial Training for Cross-Domain Few-Shot Learning, [Paper]

• (arXiv 2023.2) HYNETER: HYBRID NETWORK TRANSFORMER FOR OBJECT DETECTION, [Paper]

• (arXiv 2023.2) STOA-VLP: Spatial-Temporal Modeling of Object and Action for Video-Language Pre-training, [Paper]

• (arXiv 2023.2) Constraint and Union for Partially-Supervised Temporal Sentence Grounding, [Paper]

• (arXiv 2023.2) STB-VMM: Swin Transformer Based Video Motion Magnification, [Paper]

• (arXiv 2023.2) Fashion Image Retrieval with Multi-Granular Alignment, [Paper]

• (arXiv 2023.2) LayoutDiffuse: Adapting Foundational Diffusion Models for Layout-to-Image Generation, [Paper]

• (arXiv 2023.2) CK-Transformer: Commonsense Knowledge Enhanced Transformers for Referring Expression Comprehension, [Paper], [Code]

• (arXiv 2023.2) MaskSketch: Unpaired Structure-guided Masked Image Generation, [Paper]

• (arXiv 2023.2) Single Motion Diffusion, [Paper], [Code]

• (arXiv 2023.2) Tri-Perspective View for Vision-Based 3D Semantic Occupancy Prediction, [Paper], [Code]

• (arXiv 2023.2) ANSEL Photobot: A Robot Event Photographer with Semantic Intelligence, [Paper]

• (arXiv 2023.2) ForceFormer: Exploring Social Force and Transformer for Pedestrian Trajectory Prediction, [Paper]

• (arXiv 2023.2) Video Probabilistic Diffusion Models in Projected Latent Space, [Paper]

• (arXiv 2023.2) Dataset Interfaces: Diagnosing Model Failures Using Controllable Counterfactual Generation, [Paper], [Code]

• (arXiv 2023.2) Learning to Substitute Ingredients in Recipes, [Paper]

• (arXiv 2023.2) Energy Transformer, [Paper]

• (arXiv 2023.2) Efficiency 360: Efficient Vision Transformers, [Paper]

• (arXiv 2023.2) A-la-carte Prompt Tuning (APT): Combining Distinct Data Via Composable ` Prompting, [Paper]

• (arXiv 2023.2) Effective Data Augmentation With Diffusion Models, [Paper], [Project]

• (arXiv 2023.2) PRedItOR: Text Guided Image Editing with Diffusion Prior, [Paper]

• (arXiv 2023.2) TcGAN: Semantic-Aware and Structure-Preserved GANs with Individual Vision Transformer for Fast Arbitrary One-Shot Image Generation, [Paper]

• (arXiv 2023.2) Hierarchical Cross-modal Transformer for RGB-D Salient Object Detection, [Paper]

• (arXiv 2023.2) MINOTAUR: Multi-task Video Grounding From Multimodal Queries, [Paper]

• (arXiv 2023.2) Towards Efficient Visual Adaption via Structural Re-parameterization, [Paper], [Code]

• (arXiv 2023.2) Efficient 3D Object Reconstruction using Visual Transformers, [Paper]

• (arXiv 2023.2) Retrieval-augmented Image Captioning, [Paper]

• (arXiv 2023.2) Robust Human Motion Forecasting using Transformer-based Model, [Paper]

• (arXiv 2023.2) VQ3D: Learning a 3D-Aware Generative Model on ImageNet, [Paper], [Project]

• (arXiv 2023.2) UKnow: A Unified Knowledge Protocol for Common-Sense Reasoning and Vision-Language Pre-training, [Paper], [Code]

• (arXiv 2023.2) A THEORETICAL UNDERSTANDING OF SHALLOW VISION TRANSFORMERS: LEARNING, GENERALIZATION, AND SAMPLE COMPLEXITY, [Paper]

• (arXiv 2023.2) A Simple Zero-shot Prompt Weighting Technique to Improve Prompt Ensembling in Text-Image Models, [Paper]

• (arXiv 2023.2) Generalized Few-Shot Continual Learning with Contrastive Mixture of Adapters, [Paper], [Code]

• (arXiv 2023.2) Actional Atomic-Concept Learning for Demystifying Vision-Language Navigation, [Paper]

• (arXiv 2023.2) Towards Local Visual Modeling for Image Captioning, [Paper], [Code]

• (arXiv 2023.2) CLIP-RR: IMPROVED CLIP NETWORK FOR RELATION-FOCUSED CROSS-MODAL INFORMATION RETRIEVAL, [Paper]

• (arXiv 2023.2) Anticipating Next Active Objects for Egocentric Videos, [Paper], [Code]

• (arXiv 2023.2) UniAdapter: Unified Parameter-Efficient Transfer Learning for Cross-modal Modeling, [Paper], [Code]

• (arXiv 2023.2) TEAM DETR: GUIDE QUERIES AS A PROFESSIONAL TEAM IN DETECTION TRANSFORMERS, [Paper], [Code]

• (arXiv 2023.2) ConceptFusion: Open-set Multimodal 3D Mapping, [Paper], [Project]

• (arXiv 2023.2) Team Triple-Check at Factify 2: Parameter-Efficient Large Foundation Models with Feature Representations for Multi-Modal Fact Verification, [Paper], [Code]

• (arXiv 2023.2) PolyFormer: Referring Image Segmentation as Sequential Polygon Generation, [Paper]

• (arXiv 2023.2) Pose-Oriented Transformer with Uncertainty-Guided Refinement for 2D-to-3D Human Pose Estimation, [Paper]

• (arXiv 2023.2) TFormer: A Transmission-Friendly ViT Model for IoT Devices, [Paper], [Code]

• (arXiv 2023.2) Tri-Perspective View for Vision-Based 3D Semantic Occupancy Prediction, [Paper], [Code]

• (arXiv 2023.2) Adding Conditional Control to Text-to-Image Diffusion Models, [Paper], [Code]

• (arXiv 2023.2) Invariant Slot Attention: Object Discovery with Slot-Centric Reference Frames, [Paper]

• (arXiv 2023.2) IS MULTI-MODAL VISION SUPERVISION BENEFICIAL TO LANGUAGE? [Paper]

• (arXiv 2023.2) Data-Driven Stochastic Motion Evaluation and Optimization with Image by Spatially-Aligned Temporal Encoding, [Paper]

• (arXiv 2023.2) Scaling Vision Transformers to 22 Billion Parameters, [Paper]

• (arXiv 2023.2) Adapting Pre-trained Vision Transformers from 2D to 3D through Weight Inflation Improves Medical Image Segmentation, [Paper], [Code]

• (arXiv 2023.2) Mitigating Bias in Visual Transformers via Targeted Alignment, [Paper]

• (arXiv 2023.2) IH-ViT: Vision Transformer-based Integrated Circuit Appearance Defect Detection, [Paper]

• (arXiv 2023.2) Re-ViLM: Retrieval-Augmented Visual Language Model for Zero and Few-Shot Image Captioning, [Paper]

• (arXiv 2023.2) Learning by Asking for Embodied Visual Navigation and Task Completion, [Paper]

• (arXiv 2023.2) Reversible Vision Transformers, [Paper], [Code1], [Code2]

• (arXiv 2023.2) Neural Congealing: Aligning Images to a Joint Semantic Atlas, [Paper], [Project]

• (arXiv 2023.2) Adversarial Prompting for Black Box Foundation Models, [Paper]

• (arXiv 2023.2) Understanding Why ViT Trains Badly on Small Datasets: An Intuitive Perspective, [Paper], [Code]

• (arXiv 2023.2) CROSS-LAYER RETROSPECTIVE RETRIEVING VIA LAYER ATTENTION, [Paper], [Code]

• (arXiv 2023.2) Convolutional Neural Networks Trained to Identify Words Provide a Good Account of Visual Form Priming Effects, [Paper]

• (arXiv 2023.2) Zero-shot Generation of Coherent Storybook from Plain Text Story using Diffusion Models, [Paper]

• (arXiv 2023.2) OSRT: Omnidirectional Image Super-Resolution with Distortion-aware Transformer, [Paper]

• (arXiv 2023.2) Pic2Word: Mapping Pictures to Words for Zero-shot Composed Image Retrieval, [Paper], [Code]

• (arXiv 2023.2) SimCon Loss with Multiple Views for Text Supervised Semantic Segmentation, [Paper]

• (arXiv 2023.2) PhysFormer++: Facial Video-based Physiological Measurement with SlowFast Temporal Difference Transformer, [Paper]

• (arXiv 2023.2) Scaling Self-Supervised End-to-End Driving with Multi-View Attention Learning, [Paper]

• (arXiv 2023.2) HumanMAC: Masked Motion Completion for Human Motion Prediction, [Paper], [Project]

• (arXiv 2023.2) LAMPP: Language Models as Probabilistic Priors for Perception and Action, [Paper]

• (arXiv 2023.2) Zero-Shot Robot Manipulation from Passive Human Videos, [Paper], [Project]

• (arXiv 2023.2) MixFormer: End-to-End Tracking with Iterative Mixed Attention, [Paper], [Code]

• (arXiv 2023.2) LexLIP: Lexicon-Bottlenecked Language-Image Pre-Training for Large-Scale Image-Text Retrieval, [Paper]

• (arXiv 2023.2) V1T: large-scale mouse V1 response prediction using a Vision Transformer, [Paper]

• (arXiv 2023.2) AIM: ADAPTING IMAGE MODELS FOR EFFICIENT VIDEO ACTION RECOGNITION, [Paper], [Project]

• (arXiv 2023.2) KDEformer: Accelerating Transformers via Kernel Density Estimation, [Paper], [Code]

• (arXiv 2023.2) Semantic-Guided Image Augmentation with Pre-trained Models, [Paper]

• (arXiv 2023.2) X-ReID: Cross-Instance Transformer for Identity-Level Person Re-Identification, [Paper]

• (arXiv 2023.2) MOMA: Distill from Self-Supervised Teachers, [Paper]

• (arXiv 2023.2) Learning to Agree on Vision Attention for Visual Commonsense Reasoning, [Paper]

• (arXiv 2023.2) Efficient End-to-End Video Question Answering with Pyramidal Multimodal Transformer, [Paper], [Code]

• (arXiv 2023.2) LipFormer: Learning to Lipread Unseen Speakers based on Visual-Landmark Transformers, [Paper]

• (arXiv 2023.2) Oscillation-free Quantization for Low-bit Vision Transformers, [Paper]

• (arXiv 2023.2) Design Booster: A Text-Guided Diffusion Model for Image Translation with Spatial Layout Preservation, [Paper]

• (arXiv 2023.2) Contrast with Reconstruct: Contrastive 3D Representation Learning Guided by Generative Pretraining, [Paper], [Code]

• (arXiv 2023.2) Leaving Reality to Imagination: Robust Classification via Generated Datasets, [Paper], [Code]

• (arXiv 2023.2) CHiLS: Zero-Shot Image Classification with Hierarchical Label Sets, [Paper], [Code]

• (arXiv 2023.2) Zero-shot Image-to-Image Translation, [Paper], [Project]

• (arXiv 2023.2) Learning a Fourier Transform for Linear Relative Positional Encodings in Transformers, [Paper]

• (arXiv 2023.2) EXPLICIT BOX DETECTION UNIFIES END-TO-END MULTI-PERSON POSE ESTIMATION, [Paper], [Code]

• (arXiv 2023.2) CFFT-GAN: Cross-domain Feature Fusion Transformer for Exemplar-based Image Translation, [Paper]

• (arXiv 2023.2) DEVICE: DEpth and VIsual ConcEpts Aware Transformer for TextCaps, [Paper]

• (arXiv 2023.2) CVTNet: A Cross-View Transformer Network for Place Recognition Using LiDAR Data, [Paper], [Code]

• (arXiv 2023.2) DilateFormer: Multi-Scale Dilated Transformer for Visual Recognition, [Paper], [Code]

• (arXiv 2023.2) HDFormer: High-order Directed Transformer for 3D Human Pose Estimation, [Paper], [Code]

• (arXiv 2023.2) IC^3: Image Captioning by Committee Consensus, [Paper], [Code]

• (arXiv 2023.2) Boosting Low-Data Instance Segmentation by Unsupervised Pre-training with Saliency Prompt, [Paper]

• (arXiv 2023.2) QR-CLIP: Introducing Explicit Open-World Knowledge for Location and Time Reasoning, [Paper]

• (arXiv 2023.2) Vision Transformer-based Feature Extraction for Generalized Zero-Shot Learning, [Paper]

• (arXiv 2023.2) Multimodal Chain-of-Thought Reasoning in Language Models, [Paper], [Code]

• (arXiv 2023.2) CLIPood: Generalizing CLIP to Out-of-Distributions, [Paper]

• (arXiv 2023.2) Language Quantized AutoEncoders: Towards Unsupervised Text-Image Alignment, [Paper]

• (arXiv 2023.2) The geometry of hidden representations of large transformer models, [Paper]

• (arXiv 2023.2) Debiasing Vision-Language Models via Biased Prompts, [Paper], [Code]

• (arXiv 2023.2) COMPOSITIONAL PROMPT TUNING WITH MOTION CUES FOR OPEN-VOCABULARY VIDEO RELATION DETECTION, [Paper], [Code]

• (arXiv 2023.2) mPLUG-2: A Modularized Multi-modal Foundation Model Across Text, Image and Video, [Paper], [Code]

• (arXiv 2023.2) Transforming CLIP to an Open-vocabulary Video Model via Interpolated Weight Optimization, [Paper]

• (arXiv 2023.2) ADAPT: Action-aware Driving Caption Transformer, [Paper], [Code]

2023.1

• (arXiv 2023.1) AdaPoinTr: Diverse Point Cloud Completion with Adaptive Geometry-Aware Transformers, [Paper], [Code]

• (arXiv 2023.1) EXIF as Language: Learning Cross-Modal Associations Between Images and Camera Metadata, [Paper], [Project]

• (arXiv 2023.1) Head-Free Lightweight Semantic Segmentation with Linear Transformer, [Paper], [Code]

• (arXiv 2023.1) Geometry-biased Transformers for Novel View Synthesis, [Paper], [Project]

• (arXiv 2023.1) Continual Few-Shot Learning Using HyperTransformers, [Paper]

• (arXiv 2023.1) SEMPPL: PREDICTING PSEUDO-LABELS FOR BETTER CONTRASTIVE REPRESENTATIONS, [Paper]

• (arXiv 2023.1) Learning to Summarize Videos by Contrasting Clips, [Paper]

• (arXiv 2023.1) Guiding Text-to-Image Diffusion Model Towards Grounded Generation, [Paper], [Project]

• (arXiv 2023.1) Domain Expansion of Image Generators, [Paper], [Code]

• (arXiv 2023.1) Scene-centric vs. Object-centric Image-Text Cross-modal Retrieval: A Reproducibility Study, [Paper]

• (arXiv 2023.1) Tracr: Compiled Transformers as a Laboratory for Interpretability, [Paper], [Code]

• (arXiv 2023.1) CLIP the Gap: A Single Domain Generalization Approach for Object Detection, [Paper]

• (arXiv 2023.1) Text to Point Cloud Localization with Relation-Enhanced Transformer, [Paper]

• (arXiv 2023.1) GOHSP: A Unified Framework of Graph and Optimization-based Heterogeneous Structured Pruning for Vision Transformer, [Paper]

• (arXiv 2023.1) Toward Building General Foundation Models for Language, Vision, and Vision-Language Understanding Tasks, [Paper]

• (arXiv 2023.1) ViTs for SITS: Vision Transformers for Satellite Image Time Series, [Paper], [Code]

• (arXiv 2023.1) CLIP2Scene: Towards Label-efficient 3D Scene Understanding by CLIP, [Paper]

• (arXiv 2023.1) A Large-Scale Outdoor Multi-modal Dataset and Benchmark for Novel View Synthesis and Implicit Scene Reconstruction, [Paper], [Project]

• (arXiv 2023.1) USER: Unified Semantic Enhancement with Momentum Contrast for Image-Text Retrieval, [Paper], [Code]

• (arXiv 2023.1) SAT: Size-Aware Transformer for 3D Point Cloud Semantic Segmentation, [Paper]

• (arXiv 2023.1) Masked Visual Reconstruction in Language Semantic Space, [Paper], [Code]

• (arXiv 2023.1) Vision Learners Meet Web Image-Text Pairs, [Paper], [Code]

• (arXiv 2023.1) GLIGEN: Open-Set Grounded Text-to-Image Generation, [Paper], [Project]

• (arXiv 2023.1) Learning Customized Visual Models with Retrieval-Augmented Knowledge, [Paper], [Project]

• (arXiv 2023.1) UATVR: Uncertainty-Adaptive Text-Video Retrieval, [Paper]

• (arXiv 2023.1) Learning Aligned Cross-modal Representations for Referring Image Segmentation, [Paper]

• (arXiv 2023.1) T2M-GPT: Generating Human Motion from Textual Descriptions with Discrete Representations, [Paper], [Project]

• (arXiv 2023.1) DSVT: Dynamic Sparse Voxel Transformer with Rotated Sets, [Paper], [Code]

• (arXiv 2023.1) CMAE-V: Contrastive Masked Autoencoders for Video Action Recognition, [Paper]

• (arXiv 2023.1) Generating Templated Caption for Video Grounding, [Paper]

• (arXiv 2023.1) Dyna-DepthFormer: Multi-frame Transformer for Self-Supervised Depth Estimation in Dynamic Scenes, [Paper]

• (arXiv 2023.1) SwinDepth: Unsupervised Depth Estimation using Monocular Sequences via Swin Transformer and Densely Cascaded Network, [Paper]

• (arXiv 2023.1) CLIPTER: Looking at the Bigger Picture in Scene Text Recognition, [Paper]

• (arXiv 2023.1) Temporal Perceiving Video-Language Pre-training, [Paper]

• (arXiv 2023.1) Joint Representation Learning for Text and 3D Point Cloud, [Paper], [Code]

• (arXiv 2023.1) Effective End-to-End Vision Language Pretraining with Semantic Visual Loss, [Paper]

• (arXiv 2023.1) PTA-Det: Point Transformer Associating Point cloud and Image for 3D Object Detection, [Paper]

• (arXiv 2023.1) Face Recognition in the age of CLIP & Billion image datasets, [Paper]

• (arXiv 2023.1) HSTFormer: Hierarchical Spatial-Temporal Transformers for 3D Human Pose Estimation, [Paper], [Code]

• (arXiv 2023.1) Towards Models that Can See and Read, [Paper]

• (arXiv 2023.1) Embodied Agents for Efficient Exploration and Smart Scene Description, [Paper]

• (arXiv 2023.1) Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture, [Paper]

• (arXiv 2023.1) Revisiting the Spatial and Temporal Modeling for Few-shot Action Recognition, [Paper]

• (arXiv 2023.1) Multimodal Video Adapter for Parameter Efficient Video Text Retrieval, [Paper]

• (arXiv 2023.1) Self Supervision Does Not Help Natural Language Supervision at Scale, [Paper]

• (arXiv 2023.1) MULTI-TARGET MULTI-CAMERA VEHICLE TRACKING USING TRANSFORMER-BASED CAMERA LINK MODEL AND SPATIAL-TEMPORAL INFORMATION, [Paper]

• (arXiv 2023.1) ATMAN: Understanding Transformer Predictions Through Memory Efficient Attention Manipulation, [Paper]

• (arXiv 2023.1) DDS: Decoupled Dynamic Scene-Graph Generation Network, [Paper], [Code]

• (arXiv 2023.1) Visual Writing Prompts: Character-Grounded Story Generation with Curated Image Sequences, [Paper]

• (arXiv 2023.1) Image Memorability Prediction with Vision Transformers, [Paper]

• (arXiv 2023.1) HOLISTICALLY EXPLAINABLE VISION TRANSFORMERS, [Paper]

• (arXiv 2023.1) FlatFormer: Flattened Window Attention for Efficient Point Cloud Transformer, [Paper]

• (arXiv 2023.1) LEGO-Net: Learning Regular Rearrangements of Objects in Rooms, [Paper], [Project]

• (arXiv 2023.1) Zorro: the masked multimodal transformer, [Paper]

• (arXiv 2023.1) Towards Robust Video Instance Segmentation with Temporal-Aware Transformer, [Paper]

• (arXiv 2023.1) Learning Open-vocabulary Semantic Segmentation Models From Natural Language Supervision, [Paper], [Project]

• (arXiv 2023.1) Summarize the Past to Predict the Future: Natural Language Descriptions of Context Boost Multimodal Object Interaction Anticipation, [Paper], [Code]

• (arXiv 2023.1) Combined Use of Federated Learning and Image Encryption for Privacy-Preserving Image Classification with Vision Transformer, [Paper]

• (arXiv 2023.1) Slice Transformer and Self-supervised Learning for 6DoF Localization in 3D Point Cloud Maps, [Paper]

• (arXiv 2023.1) IMPROVING ACCURACY OF ZERO-SHOT ACTION RECOGNITION WITH HANDCRAFTED FEATURES, [Paper]

• (arXiv 2023.1) Learning to View: Decision Transformers for Active Object Detection, [Paper]

• (arXiv 2023.1) Visual Semantic Relatedness Dataset for Image Captioning, [Paper], [Code]

• (arXiv 2023.1) VERSATILE NEURAL PROCESSES FOR LEARNING IMPLICIT NEURAL REPRESENTATIONS, [Paper], [Code]

• (arXiv 2023.1) RangeViT: Towards Vision Transformers for 3D Semantic Segmentation in Autonomous Driving, [Paper], [Code]

• (arXiv 2023.1) Exploiting Optical Flow Guidance for Transformer-Based Video Inpainting, [Paper]

• (arXiv 2023.1) Image Super-Resolution using Efficient Striped Window Transformer, [Paper], [Code]

• (arXiv 2023.1) Out of Distribution Performance of State of Art Vision Model, [Paper], [Code]

• (arXiv 2023.1) Compact Transformer Tracker with Correlative Masked Modeling, [Paper], [Code]

• (arXiv 2023.1) Vision-Language Models Performing Zero-Shot Tasks Exhibit Gender-based Disparities, [Paper]

• (arXiv 2023.1) Cut and Learn for Unsupervised Object Detection and Instance Segmentation, [Paper], [Code]

• (arXiv 2023.1) Explaining Visual Biases as Words by Generating Captions, [Paper], [Code]

• (arXiv 2023.1) Revisiting Temporal Modeling for CLIP-based Image-to-Video Knowledge Transferring, [Paper], [Code]

• (arXiv 2023.1) Multi-video Moment Ranking with Multimodal Clue, [Paper]

• (arXiv 2023.1) SDF-FORMER: MONOCULAR SCENE RECONSTRUCTION WITH 3D SDF TRANSFORMERS, [Paper], [Project]

• (arXiv 2023.1) Grounding Language Models to Images for Multimodal Generation, [Paper]

• (arXiv 2023.1) Pseudo 3D Perception Transformer with Multi-level Confidence Optimization for Visual Commonsense Reasoning, [Paper]

• (arXiv 2023.1) A Modular Multi-stage Lightweight Graph Transformer Network for Human Pose and Shape Estimation from 2D Human Pose, [Paper]

• (arXiv 2023.1) Priors are Powerful: Improving a Transformer for Multi-camera 3D Detection with 2D Priors, [Paper]

• (arXiv 2023.1) UPop: Unified and Progressive Pruning for Compressing Vision-Language Transformers, [Paper]

• (arXiv 2023.1) Fairness-aware Vision Transformer via Debiased Self-Attention, [Paper]

• (arXiv 2023.1) Anchor-Based Adversarially Robust Zero-Shot Learning Driven by Language, [Paper]

• (arXiv 2023.1) Distilling Internet-Scale Vision-Language Models into Embodied Agents, [Paper]

• (arXiv 2023.1) 6-DoF Robotic Grasping with Transformer, [Paper]

• (arXiv 2023.1) Do Embodied Agents Dream of Pixelated Sheep?: Embodied Decision Making using Language Guided World Modelling, [Paper], [Project]

• (arXiv 2023.1) GALIP: Generative Adversarial CLIPs for Text-to-Image Synthesis, [Paper], [Code]

• (arXiv 2023.1) STAIR: Learning Sparse Text and Image Representation in Grounded Tokens, [Paper]

• (arXiv 2023.1) Aerial Image Object Detection With Vision Transformer Detector (ViTDet), [Paper]

• (arXiv 2023.1) Towards Vision Transformer Unrolling Fixed-Point Algorithm: a Case Study on Image Restoration, [Paper]

• (arXiv 2023.1) Debiased Fine-Tuning for Vision-language Models by Prompt Regularization, [Paper], [Code]

• (arXiv 2023.1) BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models, [Paper], [Code]

• (arXiv 2023.1) Tagging before Alignment: Integrating Multi-Modal Tags for Video-Text Retrieval, [Paper]

• (arXiv 2023.1) SEAFORMER: SQUEEZE-ENHANCED AXIAL TRANSFORMER FOR MOBILE SEMANTIC SEGMENTATION, [Paper], [Code]

• (arXiv 2023.1) Learning 6-DoF Fine-grained Grasp Detection Based on Part Affordance Grounding, [Paper], [Project]

• (arXiv 2023.1) Multimodal Event Transformer for Image-guided Story Ending Generation, [Paper]

• (arXiv 2023.1) Style-Aware Contrastive Learning for Multi-Style Image Captioning, [Paper]

• (arXiv 2023.1) 3DShape2VecSet: A 3D Shape Representation for Neural Fields and Generative Diffusion Models, [Paper]

• (arXiv 2023.1) Semi-Parametric Video-Grounded Text Generation, [Paper]

• (arXiv 2023.1) Robust Transformer with Locality Inductive Bias and Feature Normalization, [Paper]

• (arXiv 2023.1) LEVERAGING THE THIRD DIMENSION IN CONTRASTIVE LEARNING, [Paper]

• (arXiv 2023.1) Understanding Self-Supervised Pretraining with Part-Aware Representation Learning, [Paper]

• (arXiv 2023.1) Hypergraph Transformer for Skeleton-based Action Recognition, [Paper]

• (arXiv 2023.1) CPT-V: A Contrastive Approach to Post-Training Quantization of Vision Transformers, [Paper]

• (arXiv 2023.1) InstructPix2Pix: Learning to Follow Image Editing Instructions, [Paper], [Code]

• (arXiv 2023.1) OvarNet: Towards Open-vocabulary Object Attribute Recognition, [Paper], [Project]

• (arXiv 2023.1) DDS: Decoupled Dynamic Scene-Graph Generation Network, [Paper]

• (arXiv 2023.1) Token Transformer: Can class token help window-based transformer build better long-range interactions? [Paper]

• (arXiv 2023.1) Toward Building General Foundation Models for Language, Vision, and Vision-Language Understanding Tasks, [Paper]

• (arXiv 2023.1) Multimodal Inverse Cloze Task for Knowledge-based Visual Question Answering? [Paper], [Code]

• (arXiv 2023.1) FGAHOI: Fine-Grained Anchors for Human-Object Interaction Detection, [Paper], [Code]

• (arXiv 2023.1) Parallel Reasoning Network for Human-Object Interaction Detection, [Paper]

• (arXiv 2023.1) In Defense of Structural Symbolic Representation for Video Event-Relation Prediction, [Paper]

• (arXiv 2023.1) Scene Synthesis from Human Motion, [Paper], [Project]

2022.12

• (arXiv 2022.12) EVA: Exploring the Limits of Masked Visual Representation Learning at Scale, [Paper], [Code]

• (arXiv 2022.12) OneFormer: One Transformer to Rule Universal Image Segmentation, [Paper], [Code]

• (arXiv 2022.12) MMDialog: A Large-scale Multi-turn Dialogue Dataset Towards Multi-modal Open-domain Conversation, [Paper], [Project]

• (arXiv 2022.12) Why is Winoground Hard? Investigating Failures in Visuolinguistic Compositionality, [Paper], [Code]

• (arXiv 2022.12) Multimodal Information Bottleneck: Learning Minimal Sufficient Unimodal and Multimodal Representations, [Paper], [Code]

• (arXiv 2022.12) CLIP-FLOW: CONTRASTIVE LEARNING BY SEMISUPERVISED ITERATIVE PSEUDO LABELING FOR OPTICAL FLOW ESTIMATION, [Paper]

• (arXiv 2022.12) INSTRUCTION-FOLLOWING AGENTS WITH JOINTLY PRE-TRAINED VISION-LANGUAGE MODELS, [Paper], [Code]

• (arXiv 2022.12) MetaFormer Baselines for Vision, [Paper], [Code]

• (arXiv 2022.12) ViTCoD: Vision Transformer Acceleration via Dedicated Algorithm and Accelerator Co-Design, [Paper], [Code]

• (arXiv 2022.12) FROM PLAY TO POLICY: CONDITIONAL BEHAVIOR GENERATION FROM UNCURATED ROBOT DATA, [Paper], [Project]

• (arXiv 2022.12) Optimizing Prompts for Text-to-Image Generation, [Paper], [Code]

• (arXiv 2022.12) Attentive Mask CLIP, [Paper]

• (arXiv 2022.12) Rethinking Cooking State Recognition with Vision Transformers, [Paper]

• (arXiv 2022.12) Enhancing Multi-modal and Multi-hop Question Answering via Structured Knowledge and Unified Retrieval-Generation, [Paper], [Code]

• (arXiv 2022.12) MM-SHAP: A Performance-agnostic Metric for Measuring Multimodal Contributions in Vision and Language Models & Tasks, [Paper], [Code]

• (arXiv 2022.12) RepQ-ViT: Scale Reparameterization for Post-Training Quantization of Vision Transformers, [Paper]

• (arXiv 2022.12) WAVENHANCER: UNIFYING WAVELET AND TRANSFORMER FOR IMAGE ENHANCEMENT, [Paper]

• (arXiv 2022.12) AUTOENCODERS AS CROSS-MODAL TEACHERS: CAN PRETRAINED 2D IMAGE TRANSFORMERS HELP 3D REPRESENTATION LEARNING?, [Paper], [Code]

• (arXiv 2022.12) SceneGATE: Scene-Graph based co-Attention networks for TExt visual question answering, [Paper]

• (arXiv 2022.12) Emergent Analogical Reasoning in Large Language Models, [Paper]

• (arXiv 2022.12) Unleashing the Power of Visual Prompting At the Pixel Level, [Paper], [Code]

• (arXiv 2022.12) Does CLIP Bind Concepts? Probing Compositionality in Large Image Models, [Paper]

• (arXiv 2022.12) LayoutDETR: Detection Transformer Is a Good Multimodal Layout Designer, [Paper], [Code]

• (arXiv 2022.12) Towards Unsupervised Visual Reasoning: Do Off-The-Shelf Features Know How to Reason?, [Paper]

• (arXiv 2022.12) Benchmarking Spatial Relationships in Text-to-Image Generation, [Paper], [Project]

• (arXiv 2022.12) MetaCLUE: Towards Comprehensive Visual Metaphors Research, [Paper], [Project]

• (arXiv 2022.12) Tackling Ambiguity with Images: Improved Multimodal Machine Translation and Contrastive Evaluation, [Paper], [Code]

• (arXiv 2022.12) Cross-modal Attention Congruence Regularization for Vision-Language Relation Alignment, [Paper]

• (arXiv 2022.12) Does unsupervised grammar induction need pixels?, [Paper]

• (arXiv 2022.12) Hi-LASSIE: High-Fidelity Articulated Shape and Skeleton Discovery from Sparse Image Ensemble, [Paper]

• (arXiv 2022.12) MAViC: Multimodal Active Learning for Video Captioning, [Paper]

• (arXiv 2022.12) What Makes for Good Tokenizers in Vision Transformer? [Paper]

• (arXiv 2022.12) Not Just Pretty Pictures: Text-to-Image Generators Enable Interpretable Interventions for Robust Representations, [Paper], [Code]

• (arXiv 2022.12) Generalized Decoding for Pixel, Image, and Language, [Paper], [Project]

• (arXiv 2022.12) METEOR Guided Divergence for Video Captioning, [Paper], [Code]

• (arXiv 2022.12) SLGTFORMER: AN ATTENTION-BASED APPROACH TO SIGN LANGUAGE RECOGNITION, [Paper], [Code]

• (arXiv 2022.12) FROM IMAGES TO TEXTUAL PROMPTS: ZERO-SHOT VQA WITH FROZEN LARGE LANGUAGE MODELS, [Paper], [Code]

• (arXiv 2022.12) 3D Highlighter: Localizing Regions on 3D Shapes via Text Descriptions, [Paper], [Code]

• (arXiv 2022.12) Contrastive Language-Vision AI Models Pretrained on Web-Scraped Multimodal Data Exhibit Sexual Objectification Bias, [Paper]

• (arXiv 2022.12) Ultra-High-Definition Low-Light Image Enhancement: A Benchmark and Transformer-Based Method, [Paper], [Code]

• (arXiv 2022.12) Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generation, [Paper], [Project]

• (arXiv 2022.12) Beyond SOT: It’s Time to Track Multiple Generic Objects at Once, [Paper]

• (arXiv 2022.12) KNOWLEDGE-DRIVEN SCENE PRIORS FOR SEMANTIC AUDIO-VISUAL EMBODIED NAVIGATION, [Paper]

• (arXiv 2022.12) SegViT: Semantic Segmentation with Plain Vision Transformers, [Paper], [Code]

• (arXiv 2022.12) Open-Vocabulary Temporal Action Detection with Off-the-Shelf Image-Text Features, [Paper]

• (arXiv 2022.12) Point·E: A System for Generating 3D Point Clouds from Complex Prompts, [Paper], [Code]

• (arXiv 2022.12) Inductive Attention for Video Action Anticipation, [Paper]

• (arXiv 2022.12) Image-and-Language Understanding from Pixels Only, [Paper], [Code]

• (arXiv 2022.12) FlexiViT: One Model for All Patch Sizes, [Paper], [Code]

• (arXiv 2022.12) Unsupervised Object Localization: Observing the Background to Discover Objects, [Paper], [Code]

• (arXiv 2022.12) Vision Transformers are Parameter-Efficient Audio-Visual Learners, [Paper], [Project]

• (arXiv 2022.12) Full Contextual Attention for Multi-resolution Transformers in Semantic Segmentation, [Paper]

• (arXiv 2022.12) DETR4D: Direct Multi-View 3D Object Detection with Sparse Attention, [Paper]

• (arXiv 2022.12) Enhanced Training of Query-Based Object Detection via Selective Query Recollection, [Paper], [Code]

• (arXiv 2022.12) TEXT-GUIDED MASK-FREE LOCAL IMAGE RETOUCHING, [Paper]

• (arXiv 2022.12) Summary-Oriented Vision Modeling for Multimodal Abstractive Summarization, [Paper], [Code]

• (arXiv 2022.12) One-Shot Domain Adaptive and Generalizable Semantic Segmentation with Class-Aware Cross-Domain Transformers, [Paper]

• (arXiv 2022.12) ConQueR: Query Contrast Voxel-DETR for 3D Object Detection, [Paper]

• (arXiv 2022.12) Examining the Difference Among Transformers and CNNs with Explanation Methods, [Paper]

• (arXiv 2022.12) Find Someone Who: Visual Commonsense Understanding in Human-Centric Grounding, [Paper], [Code]

• (arXiv 2022.12) Dual-branch Cross-Patch Attention Learning for Group Affect Recognition, [Paper]

• (arXiv 2022.12) Cross-Modal Similarity-Based Curriculum Learning for Image Captioning, [Paper]

• (arXiv 2022.12) NLIP: Noise-robust Language-Image Pre-training, [Paper]

• (arXiv 2022.12) LidarCLIP or: How I Learned to Talk to Point Clouds, [Paper], [Code]

• (arXiv 2022.12) CLIPSEP: LEARNING TEXT-QUERIED SOUND SEPARATION WITH NOISY UNLABELED VIDEOS, [Paper]

• (arXiv 2022.12) Reproducible scaling laws for contrastive language-image learning, [Paper], [Code]

• (arXiv 2022.12) WHAT DO VISION TRANSFORMERS LEARN? A VISUAL EXPLORATION, [Paper]

• (arXiv 2022.12) Self-Play and Self-Describe: Policy Adaptation with Vision-Language Foundation Models, [Paper], [Project]

• (arXiv 2022.12) GPVIT: A HIGH RESOLUTION NON-HIERARCHICAL VISION TRANSFORMER WITH GROUP PROPAGATION, [Paper], [Code]

• (arXiv 2022.12) Learning 3D Representations from 2D Pre-trained Models via Image-to-Point Masked Autoencoders, [Paper], [Code]

• (arXiv 2022.12) Parallel Queries for Human-Object Interaction Detection, [Paper]

• (arXiv 2022.12) Structure-Guided Image Completion with Image-level and Object-level Semantic Discriminators, [Paper]

• (arXiv 2022.12) Localized Latent Updates for Fine-Tuning Vision-Language Models, [Paper]

• (arXiv 2022.12) CamoFormer: Masked Separable Attention for Camouflaged Object Detection, [Paper]

• (arXiv 2022.12) FastMIM: Expediting Masked Image Modeling Pre-training for Vision, [Paper], [Code]

• (arXiv 2022.12) OAMixer: Object-aware Mixing Layer for Vision Transformers, [Paper], [Code]

• (arXiv 2022.12) Doubly Right Object Recognition: A Why Prompt for Visual Rationales, [Paper]

• (arXiv 2022.12) RT-1: ROBOTICS TRANSFORMER FOR REAL-WORLD CONTROL AT SCALE, [Paper], [Project]

• (arXiv 2022.12) Egocentric Video Task Translation, [Paper]

• (arXiv 2022.12) ScanEnts3D: Exploiting Phrase-to-3D-Object Correspondences for Improved Visio-Linguistic Models in 3D Scenes, [Paper], [Project]

• (arXiv 2022.12) Curriculum Learning Meets Weakly Supervised Modality Correlation Learning, [Paper]

• (arXiv 2022.12) IMoS: Intent-Driven Full-Body Motion Synthesis for Human-Object Interactions, [Paper]

• (arXiv 2022.12) MultiAct: Long-Term 3D Human Motion Generation from Multiple Action Labels, [Paper]

• (arXiv 2022.12) A New Path: Scaling Vision-and-Language Navigation with Synthetic Instructions and Imitation Learning, [Paper]

• (arXiv 2022.12) Beyond Object Recognition: A New Benchmark towards Object Concept Learning, [Paper], [Project]

• (arXiv 2022.12) ViTPose+: Vision Transformer Foundation Model for Generic Body Pose Estimation, [Paper], [Code]

• (arXiv 2022.12) Structured Vision-Language Pretraining for Computational Cooking, [Paper]

• (arXiv 2022.12) MIME: Human-Aware 3D Scene Generation, [Paper], [Project]

• (arXiv 2022.12) OFASY S: A Multi-Modal Multi-Task Learning System for Building Generalist Models, [Paper], [Code]

• (arXiv 2022.12) Task Bias in Vision-Language Models, [Paper]

• (arXiv 2022.12) Multi-Concept Customization of Text-to-Image Diffusion, [Paper], [Code]

• (arXiv 2022.12) Few-View Object Reconstruction with Unknown Categories and Camera Poses, [Paper], [Project]

• (arXiv 2022.12) Masked Video Distillation: Rethinking Masked Feature Modeling for Self-supervised Video Representation Learning, [Paper], [Code]

• (arXiv 2022.12) Learning Video Representations from Large Language Models, [Paper], [Project]

• (arXiv 2022.12) Frozen CLIP Model is Efficient Point Cloud Backbone, [Paper]

• (arXiv 2022.12) DialogCC: Large-scale Multi-Modal Dialogue Dataset, [Paper], [Project]

• (arXiv 2022.12) Group Generalized Mean Pooling for Vision Transformer, [Paper]

• (arXiv 2022.12) LEARNING DOMAIN INVARIANT PROMPT FOR VISION-LANGUAGE MODELS, [Paper]

• (arXiv 2022.12) LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Models, [Paper]

• (arXiv 2022.12) Hyperbolic Contrastive Learning for Visual Representations beyond Objects, [Paper], [Code]

2022.11

• (arXiv 2022.11) Texts as Images in Prompt Tuning for Multi-Label Image Recognition, [Paper], [Code]

• (arXiv 2022.11) Tell Me What Happened: Unifying Text-guided Video Completion via Multimodal Masked Video Generation, [Paper]

• (arXiv 2022.11) InDiReCT: Language-Guided Zero-Shot Deep Metric Learning for Images, [Paper]

• (arXiv 2022.11) VoP: Text-Video Co-operative Prompt Tuning for Cross-Modal Retrieval, [Paper], [Code]

• (arXiv 2022.11) Completing point cloud from few points by Wasserstein GAN and Transformers, [Paper], [Code]

• (arXiv 2022.11) Integrally Pre-Trained Transformer Pyramid Networks, [Paper], [Code]

• (arXiv 2022.11) Data Augmentation Vision Transformer for Fine-grained Image Classification, [Paper]

• (arXiv 2022.11) DETRs with Collaborative Hybrid Assignments Training, [Paper], [Code]

• (arXiv 2022.11) Open-vocabulary Attribute Detection, [Paper], [Project]

• (arXiv 2022.11) Lite-Mono: A Lightweight CNN and Transformer Architecture for Self-Supervised Monocular Depth Estimation, [Paper], [Code]

• (arXiv 2022.11) Inversion-Based Creativity Transfer with Diffusion Models, [Paper], [Code]

• (arXiv 2022.11) CODA-Prompt: COntinual Decomposed Attention-based Prompting for Rehearsal-Free Continual Learning, [Paper]

• (arXiv 2022.11) SVFormer: Semi-supervised Video Transformer for Action Recognition, [Paper], [Code]

• (arXiv 2022.11) Generalizable Implicit Neural Representations via Instance Pattern Composers, [Paper]

• (arXiv 2022.11) Improving Visual-textual Sentiment Analysis by Fusing Expert Features, [Paper]

• (arXiv 2022.11) Self-Supervised Learning based on Heat Equation, [Paper]

• (arXiv 2022.11) Peekaboo: Text to Image Diffusion Models are Zero-Shot Segmentors, [Paper]

• (arXiv 2022.11) Paint by Example: Exemplar-based Image Editing with Diffusion Models, [Paper], [Code]

• (arXiv 2022.11) Human or Machine? Turing Tests for Vision and Language, [Paper], [Code]

• (arXiv 2022.11) Teach-DETR: Better Training DETR with Teachers, [Paper], [Code]

• (arXiv 2022.11) Conv2Former: A Simple Transformer-Style ConvNet for Visual Recognition, [Paper]

• (arXiv 2022.11) X^2-VLM: All-In-One Pre-trained Model For Vision-Language Tasks, [Paper], [Code]

• (arXiv 2022.11) Aligning Source Visual and Target Language Domains for Unpaired Video Captioning, [Paper]

• (arXiv 2022.11) On the Transferability of Visual Features in Generalized Zero-Shot Learning, [Paper], [Code]

• (arXiv 2022.11) Generalizable Industrial Visual Anomaly Detection with Self-Induction Vision Transformer, [Paper]

• (arXiv 2022.11) Transformer Based Multi-Grained Features for Unsupervised Person Re-Identification, [Paper], [Code]

• (arXiv 2022.11) Efficient Frequency Domain-based Transformers for High-Quality Image Deblurring, [Paper], [Code]

• (arXiv 2022.11) Event Transformer+. A multi-purpose solution for efficient event data processing, [Paper]

• (arXiv 2022.11) MagicPony: Learning Articulated 3D Animals in the Wild, [Paper], [Project]

• (arXiv 2022.11) Gated Class-Attention with Cascaded Feature Drift Compensation for Exemplar-free Continual Learning of Vision Transformers, [Paper], [Code]

• (arXiv 2022.11) Expectation-Maximization Contrastive Learning for Compact Video-and-Language Representations, [Paper], [Code]

• (arXiv 2022.11) N-Gram in Swin Transformers for Efficient Lightweight Image Super-Resolution, [Paper]

• (arXiv 2022.11) Robotic Skill Acquisition via Instruction Augmentation with Vision-Language Models, [Paper], [Code]

• (arXiv 2022.11) Peeling the Onion: Hierarchical Reduction of Data Redundancy for Efficient Vision Transformer Training, [Paper], [Code]

• (arXiv 2022.11) Unifying Vision-Language Representation Space with Single-tower Transformer, [Paper]

• (arXiv 2022.11) DeepSolo: Let Transformer Decoder with Explicit Points Solo for Text Spotting, [Paper]

• (arXiv 2022.11) Castling-ViT: Compressing Self-Attention via Switching Towards Linear-Angular Attention During Vision Transformer Inference, [Paper]

• (arXiv 2022.11) CL-CrossVQA: A Continual Learning Benchmark for Cross-Domain Visual Question Answering, [Paper]

• (arXiv 2022.11) Normal Transformer: Extracting Surface Geometry from LiDAR Points Enhanced by Visual Semantics, [Paper]

• (arXiv 2022.11) A Unified Model for Video Understanding and Knowledge Embedding with Heterogeneous Knowledge Graph Dataset, [Paper]

• (arXiv 2022.11) Efficient Video Representation Learning via Masked Video Modeling with Motion-centric Token Selection, [Paper]

• (arXiv 2022.11) DiffStyler: Controllable Dual Diffusion for Text-Driven Image Stylization, [Paper]

• (arXiv 2022.11) TORE: Token Reduction for Efficient Human Mesh Recovery with Transformer, [Paper]

• (arXiv 2022.11) Synthesizing Coherent Story with Auto-Regressive Latent Diffusion Models, [Paper], [Code]

• (arXiv 2022.11) Are Out-of-Distribution Detection Methods Reliable?, [Paper]

• (arXiv 2022.11) GLT-T: Global-Local Transformer Voting for 3D Single Object Tracking in Point Clouds, [Paper], [Code]

• (arXiv 2022.11) CROSS-MODAL CONTRASTIVE LEARNING FOR ROBUST REASONING IN VQA, [Paper], [Code]

• (arXiv 2022.11) LISA: Localized Image Stylization with Audio via Implicit Neural Representation, [Paper]

• (arXiv 2022.11) MagicVideo: Efficient Video Generation With Latent Diffusion Models, [Paper], [Code]

• (arXiv 2022.11) DreamArtist: Towards Controllable One-Shot Text-to-Image Generation via Contrastive Prompt-Tuning, [Paper]

• (arXiv 2022.11) Hybrid Transformer Based Feature Fusion for Self-Supervised Monocular Depth Estimation, [Paper]

• (arXiv 2022.11) Language in a Bottle: Language Model Guided Concept Bottlenecks for Interpretable Image Classification, [Paper]

• (arXiv 2022.11) Structure-Encoding Auxiliary Tasks for Improved Visual Representation in Vision-and-Language Navigation, [Paper]

• (arXiv 2022.11) You Need Multiple Exiting: Dynamic Early Exiting for Accelerating Unified Vision Language Model, [Paper]

• (arXiv 2022.11) Beyond Attentive Tokens: Incorporating Token Importance and Diversity for Efficient Vision Transformers, [Paper]

• (arXiv 2022.11) FlowLens: Seeing Beyond the FoV via Flow-guided Clip-Recurrent Transformer, [Paper], [Code]

• (arXiv 2022.11) PS-Transformer: Learning Sparse Photometric Stereo Network using Self-Attention Mechanism, [Paper]

• (arXiv 2022.11) On the Robustness, Generalization, and Forgetting of Shape-Texture Debiased Continual Learning, [Paper]

• (arXiv 2022.11) Vision Transformer with Super Token Sampling, [Paper], [Code]

• (arXiv 2022.11) Detect Only What You Specify : Object Detection with Linguistic Target, [Paper]

• (arXiv 2022.11) Visual Programming: Compositional visual reasoning without training, [Paper], [Project]

• (arXiv 2022.11) ClipCrop: Conditioned Cropping Driven by Vision-Language Model, [Paper]

• (arXiv 2022.11) SMAUG: Sparse Masked Autoencoder for Efficient Video-Language Pre-training, [Paper]

• (arXiv 2022.11) Blur Interpolation Transformer for Real-World Motion from Blur, [Paper]

• (arXiv 2022.11) Mean Shift Mask Transformer for Unseen Object Instance Segmentation, [Paper], [Code]

• (arXiv 2022.11) PointCLIP V2: Adapting CLIP for Powerful 3D Open-world Learning, [Paper], [Code]

• (arXiv 2022.11) Exploring Discrete Diffusion Models for Image Captioning, [Paper], [Code]

• (arXiv 2022.11) PERCEIVER-VL: Efficient Vision-and-Language Modeling with Iterative Latent Attention, [Paper], [Code]

• (arXiv 2022.11) Multitask Vision-Language Prompt Tuning, [Paper], [Code]

• (arXiv 2022.11) Teaching Structured Vision & Language Concepts to Vision & Language Models, [Paper]

• (arXiv 2022.11) WEIGHTED ENSEMBLE SELF-SUPERVISED LEARNING, [Paper]

• (arXiv 2022.11) BEVFormer v2: Adapting Modern Image Backbones to Bird’s-Eye-View Recognition via Perspective Supervision, [Paper]

• (arXiv 2022.11) Task Residual for Tuning Vision-Language Models, [Paper], [Code]

• (arXiv 2022.11) α DARTS Once More: Enhancing Differentiable Architecture Search by Masked Image Modeling, [Paper]

• (arXiv 2022.11) Delving into Transformer for Incremental Semantic Segmentation, [Paper]

• (arXiv 2022.11) DETRDistill: A Universal Knowledge Distillation Framework for DETR-families, [Paper]

• (arXiv 2022.11) PromptCap: Prompt-Guided Task-Aware Image Captioning, [Paper]

• (arXiv 2022.11) UNIFORMERV2: SPATIOTEMPORAL LEARNING BY ARMING IMAGE VITS WITH VIDEO UNIFORMER, [Paper], [Code]

• (arXiv 2022.11) Masked Reconstruction Contrastive Learning with Information Bottleneck Principle, [Paper]

• (arXiv 2022.11) Listen, denoise, action! Audio-driven motion synthesis with diffusion models, [Paper], [Project]

• (arXiv 2022.11) ConStruct-VL: Data-Free Continual Structured VL Concepts Learning, [Paper]

• (arXiv 2022.11) How to Fine-Tune Vision Models with SGD, [Paper]

• (arXiv 2022.11) Progressive Tree-Structured Prototype Network for End-to-End Image Captioning, [Paper], [Code]

• (arXiv 2022.11) CapEnrich: Enriching Caption Semantics for Web Images via Cross-modal Pre-trained Knowledge, [Paper], [Code]

• (arXiv 2022.11) Visual Commonsense-aware Representation Network for Video Captioning, [Paper], [Code]

• (arXiv 2022.11) Language Conditioned Spatial Relation Reasoning for 3D Object Grounding, [Paper], [Code]

• (arXiv 2022.11) HARDVS: Revisiting Human Activity Recognition with Dynamic Vision Sensors, [Paper], [Code]

• (arXiv 2022.11) Towards All-in-one Pre-training via Maximizing Multi-modal Mutual Information, [Paper], [Code]

• (arXiv 2022.11) Uni-Perceiver v2: A Generalist Model for Large-Scale Vision and Vision-Language Tasks, [Paper], [Code]

• (arXiv 2022.11) D^3ETR: Decoder Distillation for Detection Transformer, [Paper]

• (arXiv 2022.11) CAE v2: Context Autoencoder with CLIP Target, [Paper]

• (arXiv 2022.11) Cross-Modal Adapter for Text-Video Retrieval, [Paper], [Code]

• (arXiv 2022.11) TOKEN TURING MACHINES, [Paper]

• (arXiv 2022.11) WILL LARGE-SCALE GENERATIVE MODELS CORRUPT FUTURE DATASETS? [Paper], [Code]

• (arXiv 2022.11) Demystify Self-Attention in Vision Transformers from a Semantic Perspective: Analysis and Application, [Paper]

• (arXiv 2022.11) SATVSR: Scenario Adaptive Transformer for Cross Scenarios Video Super-Resolution, [Paper]

• (arXiv 2022.11) TransCC: Transformer-based Multiple Illuminant Color Constancy Using Multitask Learning, [Paper]

• (arXiv 2022.11) Stare at What You See: Masked Image Modeling without Reconstruction, [Paper], [Code]

• (arXiv 2022.11) HeatViT: Hardware-Efficient Adaptive Token Pruning for Vision Transformers, [Paper]

• (arXiv 2022.11) Cross-domain Federated Adaptive Prompt Tuning for CLIP, [Paper]

• (arXiv 2022.11) YORO - Lightweight End to End Visual Grounding, [Paper]

• (arXiv 2022.11) Knowledge Distillation for Detection Transformer with Consistent Distillation Points Sampling, [Paper]

• (arXiv 2022.11) BiViT: Extremely Compressed Binary Vision Transformer, [Paper]

• (arXiv 2022.11) ContextCLIP: Contextual Alignment of Image-Text pairs on CLIP visual representations, [Paper]

• (arXiv 2022.11) Zero-shot Image Captioning by Anchor-augmented Vision-Language Space Alignment, [Paper]

• (arXiv 2022.11) Seeing Beyond the Brain: Conditional Diffusion Model with Sparse Masked Modeling for Vision Decoding, [Paper], [Project]

• (arXiv 2022.11) Enhancing Few-Shot Image Classification with Cosine Transformer, [Paper], [Code]

• (arXiv 2022.11) SCOTCH and SODA: A Transformer Video Shadow Detection Framework, [Paper]

• (arXiv 2022.11) AU-Aware Vision Transformers for Biased Facial Expression Recognition, [Paper]

• (arXiv 2022.11) Fast Text-Conditional Discrete Denoising on Vector-Quantized Latent Spaces, [Paper], [Code]

• (arXiv 2022.11) Large-Scale Bidirectional Training for Zero-Shot Image Captioning, [Paper]

• (arXiv 2022.11) Grafting Pre-trained Models for Multimodal Headline Generation, [Paper]

• (arXiv 2022.11) CabViT: Cross Attention among Blocks for Vision Transformer, [Paper], [Code]

• (arXiv 2022.11) Composed Image Retrieval with Text Feedback via Multi-grained Uncertainty Regularization, [Paper]

• (arXiv 2022.11) SSGVS: Semantic Scene Graph-to-Video Synthesis, [Paper]

• (arXiv 2022.11) One-Time Model Adaptation to Heterogeneous Clients: An Intra-Client and Inter-Image Attention Design, [Paper]

• (arXiv 2022.11) An Improved End-to-End Multi-Target Tracking Method Based on Transformer Self-Attention, [Paper]

• (arXiv 2022.11) Zero-shot Visual Commonsense Immorality Prediction, [Paper], [Code]

• (arXiv 2022.11) Hyperbolic Cosine Transformer for LiDAR 3D Object Detection, [Paper]

• (arXiv 2022.11) Training a Vision Transformer from scratch in less than 24 hours with 1 GPU, [Paper], [Code]

• (arXiv 2022.11) ViTALiTy: Unifying Low-rank and Sparse Approximation for Vision Transformer Acceleration with a Linear Taylor Attention, [Paper]

• (arXiv 2022.11) SimOn: A Simple Framework for Online Temporal Action Localization, [Paper], [Code]

• (arXiv 2022.11) ERNIE-UNIX^2: A UNIFIED CROSS-LINGUAL CROSS-MODAL FRAMEWORK FOR UNDERSTANDING AND GENERATION, [Paper]

• (arXiv 2022.11) SG-Shuffle: Multi-aspect Shuffle Transformer for Scene Graph Generation, [Paper]

• (arXiv 2022.11) Understanding Cross-modal Interactions in V&L Models that Generate Scene Descriptions, [Paper]

• (arXiv 2022.11) VieCap4H - VLSP 2021: ObjectAoA - Enhancing performance of Object Relation Transformer with Attention on Attention for Vietnamese image captioning, [Paper]

• (arXiv 2022.11) Watching the News: Towards VideoQA Models that can Read, [Paper], [Project]

• (arXiv 2022.11) Efficient Joint Detection and Multiple Object Tracking with Spatially Aware Transformer, [Paper]

• (arXiv 2022.11) Demystify Transformers & Convolutions in Modern Image Deep Networks, [Paper], [Code]

• (arXiv 2022.11) InternImage: Exploring Large-Scale Vision Foundation Models with Deformable Convolutions, [Paper], [Code]

• (arXiv 2022.11) DEPTHFORMER: MULTIMODAL POSITIONAL ENCODINGS AND CROSS-INPUT ATTENTION FOR TRANSFORMER-BASED SEGMENTATION NETWORKS, [Paper]

• (arXiv 2022.11) Sequential Transformer for End-to-End Person Search, [Paper]

• (arXiv 2022.11) Prompting Large Pre-trained Vision-Language Models For Compositional Concept Learning, [Paper]

• (arXiv 2022.11) CASA: Category-agnostic Skeletal Animal Reconstruction, [Paper]

• (arXiv 2022.11) ViT-CX: Causal Explanation of Vision Transformers, [Paper]

• (arXiv 2022.11) Disentangling Content and Motion for Text-Based Neural Video Manipulation, [Paper]

• (arXiv 2022.11) Efficient Multi-order Gated Aggregation Network, [Paper]

• (arXiv 2022.11) CLOP: Video-and-Language Pre-Training with Knowledge Regularizations, [Paper]

• (arXiv 2022.11) MSMG-Net: Multi-scale Multi-grained Supervised Metworks for Multi-task Image Manipulation Detection and Localization, [Paper]

• (arXiv 2022.11) Understanding and Mitigating Overfitting in Prompt Tuning for Vision-Language Models, [Paper], [Code]

• (arXiv 2022.11) Zero-shot Video Moment Retrieval With Off-the-Shelf Models, [Paper]

• (arXiv 2022.11) Scaling Multimodal Pre-Training via Cross-Modality Gradient Harmonization, [Paper]

• (arXiv 2022.11) A Transformer Architecture for Online Gesture Recognition of Mathematical Expressions, [Paper]

• (arXiv 2022.11) Evaluating and Improving Factuality in Multimodal Abstractive Summarization, [Paper], [Code]

• (arXiv 2022.11) RCDPT: RADAR-CAMERA FUSION DENSE PREDICTION TRANSFORMER, [Paper]

• (arXiv 2022.11) Video Event Extraction via Tracking Visual States of Arguments, [Paper]

• (arXiv 2022.11) The Lottery Ticket Hypothesis for Vision Transformers, [Paper]

• (arXiv 2022.11) TEXTCRAFT: ZERO-SHOT GENERATION OF HIGHFIDELITY AND DIVERSE SHAPES FROM TEXT, [Paper]

• (arXiv 2022.11) PolyBuilding: Polygon Transformer for End-to-End Building Extraction, [Paper]

• (arXiv 2022.11) RETHINKING HIERARCHIES IN PRE-TRAINED PLAIN VISION TRANSFORMER, [Paper], [Code]

• (arXiv 2022.11) SAP-DETR: Bridging the Gap Between Salient Points and Queries-Based Transformer Detector for Fast Model Convergency, [Paper]

• (arXiv 2022.11) Could Giant Pretrained Image Models Extract Universal Representations? [Paper]

• (arXiv 2022.11) MAEDAY: MAE for few and zero shot AnomalY-Detection, [Paper], [Code]

• (arXiv 2022.11) Degenerate Swin to Win: Plain Window-based Transformer without Sophisticated Operations, [Paper]

• (arXiv 2022.11) Look Around and Refer: 2D Synthetic Semantics Knowledge Distillation for 3D Visual Grounding, [Paper], [Code]

• (arXiv 2022.11) SpaText: Spatio-Textual Representation for Controllable Image Generation, [Paper], [Project]

• (arXiv 2022.11) Learning 3D Scene Priors with 2D Supervision, [Paper], [Project]

• (arXiv 2022.11) PoET: Pose Estimation Transformer for Single-View, Multi-Object 6D Pose Estimation, [Paper], [Code]

• (arXiv 2022.11) Spatial-Spectral Transformer for Hyperspectral Image Denoising, [Paper], [Code]

• (arXiv 2022.11) Training Vision-Language Models with Less Bimodal Supervision, [Paper]

• (arXiv 2022.11) Text-Only Training for Image Captioning using Noise-Injected CLIP, [Paper], [Code]

• (arXiv 2022.11) Attention-based Neural Cellular Automata, [Paper]

• (arXiv 2022.11) eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoisers, [Paper], [Code]

• (arXiv 2022.11) Chinese CLIP: Contrastive Vision-Language Pretraining in Chinese, [Paper], [Code]

• (arXiv 2022.11) P^3OVD: Fine-grained Visual-Text Prompt-Driven Self-Training for Open-Vocabulary Object Detection, [Paper]

• (arXiv 2022.11) tSF: Transformer-based Semantic Filter for Few-Shot Learning, [Paper]

• (arXiv 2022.11) WITT: A WIRELESS IMAGE TRANSMISSION TRANSFORMER FOR SEMANTIC COMMUNICATIONS, [Paper], [Code]

• (arXiv 2022.11) Pair DETR: Contrastive Learning Speeds Up DETR Training, [Paper]

• (arXiv 2022.11) Interaction Visual Transformer for Egocentric Action Anticipation, [Paper]

• (arXiv 2022.11) UDE: A Unified Driving Engine for Human Motion Generation, [Paper], [Code]

• (arXiv 2022.11) Action-GPT: Leveraging Large-scale Language Models for Improved and Generalized Zero Shot Action Generation, [Paper], [Project]

• (arXiv 2022.11) Human or Machine? Turing Tests for Vision and Language, [Paper], [Code]

• (arXiv 2022.11) Knowledge Prompting for Few-shot Action Recognition, [Paper]

• (arXiv 2022.11) UPainting: Unified Text-to-Image Diffusion Generation with Cross-modal Guidance, [Paper], [Project]

• (arXiv 2022.11) LVP-M^3: Language-aware Visual Prompt for Multilingual Multimodal Machine Translation, [Paper]

• (arXiv 2022.11) PROCONTEXT: PROGRESSIVE CONTEXT TRANSFORMER FOR TRACKING, [Paper], [Code]

• (arXiv 2022.11) Video based Object 6D Pose Estimation using Transformers, [Paper], [Code]

• (arXiv 2022.11) S2WAT: Image Style Transfer via Hierarchical Vision Transformer using Strips Window Attention, [Paper], [Code]

• (arXiv 2022.11) Holistic Interaction Transformer Network for Action Detection, [Paper], [Code]

• (arXiv 2022.11) Learning and Retrieval from Prior Data for Skill-based Imitation Learning, [Paper], [Code]

• (arXiv 2022.11) SimpleClick: Interactive Image Segmentation with Simple Vision Transformers, [Paper], [Code]

• (arXiv 2022.11) TANGO: Text-driven Photorealistic and Robust 3D Stylization via Lighting Decomposition, [Paper], [Code]

• (arXiv 2022.11) CPL: Counterfactual Prompt Learning for Vision and Language Models, [Paper], [Code]

• (arXiv 2022.11) Plug-and-Play VQA: Zero-shot VQA by Conjoining Large Pretrained Models with Zero Training, [Paper]

• (arXiv 2022.11) Selective Query-guided Debiasing for Video Corpus Moment Retrieval, [Paper]

• (arXiv 2022.11) Scaling & Shifting Your Features: A New Baseline for Efficient Model Tuning, [Paper], [Code]

• (arXiv 2022.11) DENOISING MASKED AUTOENCODERS ARE CERTIFIABLE ROBUST VISION LEARNERS, [Paper], [Code]

• (arXiv 2022.11) Token-Label Alignment for Vision Transformers, [Paper], [Code]

• (arXiv 2022.11) CLIP-Fields: Weakly Supervised Semantic Fields for Robotic Memory, [Paper], [Code]

• (arXiv 2022.11) Multi-Scale Wavelet Transformer for Face Forgery Detection, [Paper]

• (arXiv 2022.11) CLIP-PAE: PROJECTION-AUGMENTATION EMBEDDING TO EXTRACT RELEVANT FEATURES FOR A DISENTANGLED, INTERPRETABLE, AND CONTROLLABLE TEXT-GUIDED IMAGE MANIPULATION, [Paper]

• (arXiv 2022.11) VISUAL PROMPT TUNING FOR TEST-TIME DOMAIN ADAPTATION, [Paper]

• (arXiv 2022.11) FastCLIPstyler: Optimisation-free Text-based Image Style Transfer Using Style Representations, [Paper]

• (arXiv 2022.11) PROGRESSIVE DENOISING MODEL FOR FINEGRAINED TEXT-TO-IMAGE GENERATION, [Paper]

• (arXiv 2022.11) DALL-E-Bot: Introducing Web-Scale Diffusion Models to Robotics, [Paper], [Project]

• (arXiv 2022.11) Decomposed Soft Prompt Guided Fusion Enhancing for Compositional Zero-Shot Learning, [Paper], [Code]

• (arXiv 2022.11) ACCURATE IMAGE RESTORATION WITH ATTENTION RETRACTABLE TRANSFORMER, [Paper], [Code]

• (arXiv 2022.11) Dilated Neighborhood Attention Transformer, [Paper], [Code]

• (arXiv 2022.11) Unified Loss of Pair Similarity Optimization for Vision-Language Retrieval, [Paper]

• (arXiv 2022.11) TVLT: Textless Vision-Language Transformer, [Paper], [Code]

2022.10

• (arXiv 2022.10) DiMBERT: Learning Vision-Language Grounded Representations with Disentangled Multimodal-Attention, [Paper]

• (arXiv 2022.10) TFORMER: 3D TOOTH SEGMENTATION IN MESH SCANS WITH GEOMETRY GUIDED TRANSFORMER, [Paper]

• (arXiv 2022.10) ON-THE-FLY OBJECT DETECTION USING STYLEGAN WITH CLIP GUIDANCE, [Paper]

• (arXiv 2022.10) Image-free Domain Generalization via CLIP for 3D Hand Pose Estimation, [Paper]

• (arXiv 2022.10) Temporal-Viewpoint Transportation Plan for Skeletal Few-shot Action Recognition, [Paper]

• (arXiv 2022.10) A SIMPLE, EFFICIENT AND SCALABLE CONTRASTIVE MASKED AUTOENCODER FOR LEARNING VISUAL REPRESENTATIONS, [Paper]

• (arXiv 2022.10) Time-rEversed diffusioN tEnsor Transformer: A new TENET of Few-Shot Object Detection, [Paper]

• (arXiv 2022.10) Foreign Object Debris Detection for Airport Pavement Images based on Self-supervised Localization and Vision Transformer, [Paper]

• (arXiv 2022.10) ViT-LSLA: Vision Transformer with Light Self-Limited-Attention, [Paper]

• (arXiv 2022.10) Generative Negative Text Replay for Continual Vision-Language Pretraining, [Paper]

• (arXiv 2022.10) PatchRot: A Self-Supervised Technique for Training Vision Transformers, [Paper]

• (arXiv 2022.10) MULTIMODAL TRANSFORMER DISTILLATION FOR AUDIO-VISUAL SYNCHRONIZATION, [Paper]

• (arXiv 2022.10) Multimodal Transformer for Parallel Concatenated Variational Autoencoders, [Paper]

• (arXiv 2022.10) Differentially Private CutMix for Split Learning with Vision Transformer, [Paper]

• (arXiv 2022.10) OHMG: ZERO-SHOT OPEN-VOCABULARY HUMAN MOTION GENERATION, [Paper]

• (arXiv 2022.10) VLT: Vision-Language Transformer and Query Generation for Referring Segmentation, [Paper]

• (arXiv 2022.10) PSFORMER: POINT TRANSFORMER FOR 3D SALIENT OBJECT DETECTION, [Paper]

• (arXiv 2022.10) GRAFTING VISION TRANSFORMERS, [Paper]

• (arXiv 2022.10) Generalization Differences between End-to-End and Neuro-Symbolic Vision-Language Reasoning Systems, [Paper]

• (arXiv 2022.10) FaD-VLP: Fashion Vision-and-Language Pre-training towards Unified Retrieval and Captioning, [Paper]

• (arXiv 2022.10) Masked Vision-Language Transformer in Fashion, [Paper], [Code]

• (arXiv 2022.10) Learning Variational Motion Prior for Video-based Motion Capture, [Paper]

• (arXiv 2022.10) Open-vocabulary Semantic Segmentation with Frozen Vision-Language Models, [Paper], [Code]

• (arXiv 2022.10) TEXT2MODEL: MODEL INDUCTION FOR ZERO-SHOT GENERALIZATION USING TASK DESCRIPTIONS, [Paper]

• (arXiv 2022.10) Learning Joint Representation of Human Motion and Language, [Paper]

• (arXiv 2022.10) ERNIE-ViLG 2.0: Improving Text-to-Image Diffusion Model with Knowledge-Enhanced Mixture-of-Denoising-Experts, [Paper]

• (arXiv 2022.10) MSF3DDETR: Multi-Sensor Fusion 3D Detection Transformer for Autonomous Driving, [Paper]

• (arXiv 2022.10) Li3DeTr: A LiDAR based 3D Detection Transformer, [[Paper]](Li3DeTr: A LiDAR based 3D Detection Transformer)

• (arXiv 2022.10) Masked Transformer for image Anomaly Localization, [Paper]

• (arXiv 2022.10) Discovering Design Concepts for CAD Sketches, [Paper]

• (arXiv 2022.10) Compressing And Debiasing Vision-Language Pre-Trained Models for Visual Question Answering, [Paper]

• (arXiv 2022.10) End-to-End Multimodal Representation Learning for Video Dialog, [Paper]

• (arXiv 2022.10) TPFNet: A Novel Text In-painting Transformer for Text Removal, [Paper], [Code]

• (arXiv 2022.10) IMU2CLIP: MULTIMODAL CONTRASTIVE LEARNING FOR IMU MOTION SENSORS FROM EGOCENTRIC VIDEOS AND TEXT NARRATIONS, [Paper]

• (arXiv 2022.10) Can Transformer Attention Spread Give Insights Into Uncertainty of Detected and Tracked Objects? [Paper]

• (arXiv 2022.10) SemFormer: Semantic Guided Activation Transformer for Weakly Supervised Semantic Segmentation, [Paper], [Code]

• (arXiv 2022.10) End-to-end Tracking with a Multi-query Transformer, [Paper]

• (arXiv 2022.10) Explicitly Increasing Input Information Density for Vision Transformers on Small Datasets, [Paper], [Code]

• (arXiv 2022.10) TAMFORMER: MULTI-MODAL TRANSFORMER WITH LEARNED ATTENTION MASK FOR EARLY INTENT PREDICTION, [Paper]

• (arXiv 2022.10) VISUAL ANSWER LOCALIZATION WITH CROSS-MODAL MUTUAL KNOWLEDGE TRANSFER, [Paper], [Code]

• (arXiv 2022.10) Visual Semantic Parsing: From Images to Abstract Meaning Representation, [Paper]

• (arXiv 2022.10) End-to-end Transformer for Compressed Video Quality Enhancement, [Paper]

• (arXiv 2022.10) PlanT: Explainable Planning Transformers via Object-Level Representations, [Paper], [Project]

• (arXiv 2022.10) Strong-TransCenter: Improved Multi-Object Tracking based on Transformers with Dense Representations, [Paper], [Code]

• (arXiv 2022.10) GliTr: Glimpse Transformers with Spatiotemporal Consistency for Online Action Prediction, [Paper]

• (arXiv 2022.10) VLC-BERT: Visual Question Answering with Contextualized Commonsense Knowledge, [Paper], [Code]

• (arXiv 2022.10) Learning by Hallucinating: Vision-Language Pre-training with Weak Supervision, [Paper]

• (arXiv 2022.10) Learning Explicit Object-Centric Representations with Vision Transformers, [Paper]

• (arXiv 2022.10) Abductive Action Inference, [Paper]

• (arXiv 2022.10) Minutiae-Guided Fingerprint Embeddings via Vision Transformers, [Paper], [Code]

• (arXiv 2022.10) 3DALL-E: Integrating Text-to-Image AI in 3D Design Workflows, [Paper]

• (arXiv 2022.10) COMPOSING ENSEMBLES OF PRE-TRAINED MODELS VIA ITERATIVE CONSENSUS, [Paper], [Code]

• (arXiv 2022.10) Do Vision-and-Language Transformers Learn Grounded Predicate-Noun Dependencies?, [Paper]

• (arXiv 2022.10) Boosting vision transformers for image retrieval, [Paper], [Code]

• (arXiv 2022.10) LiteVL: Efficient Video-Language Learning with Enhanced Spatial-Temporal Modeling, [Paper]

• (arXiv 2022.10) Fine-grained Semantic Alignment Network for Weakly Supervised Temporal Language Grounding, [Paper]

• (arXiv 2022.10) Face Pyramid Vision Transformer, [Paper], [Code]

• (arXiv 2022.10) Context-Enhanced Stereo Transformer, [Paper], [Code]

• (arXiv 2022.10) CRT-6D: Fast 6D Object Pose Estimation with Cascaded Refinement Transformers, [Paper], [Code]

• (arXiv 2022.10) Rethinking Learning Approaches for Long-Term Action Anticipation, [Paper], [Code]

• (arXiv 2022.10) Extending Phrase Grounding with Pronouns in Visual Dialogues, [Paper], [Code]

• (arXiv 2022.10) Accumulated Trivial Attention Matters in Vision Transformers on Small Datasets, [Paper], [Code]

• (arXiv 2022.10) Transformers For Recognition In Overhead Imagery: A Reality Check, [Paper]

• (arXiv 2022.10) Anticipative Feature Fusion Transformer for Multi-Modal Action Anticipation, [Paper], [Code]

• (arXiv 2022.10) UIA-ViT: Unsupervised Inconsistency-Aware Method based on Vision Transformer for Face Forgery Detection, [Paper]

• (arXiv 2022.10) LCPFormer: Towards Effective 3D Point Cloud Analysis via Local Context Propagation in Transformers, [Paper]

• (arXiv 2022.10) Towards Real-Time Text2Video via CLIP-Guided, Pixel-Level Optimization, [Paper], [Code]

• (arXiv 2022.10) Language-free Training for Zero-shot Video Grounding, [Paper], [Code]

• (arXiv 2022.10) Foreground Guidance and Multi-Layer Feature Fusion for Unsupervised Object Discovery with Transformers, [Paper], [Code]

• (arXiv 2022.10) Towards Unifying Reference Expression Generation and Comprehension, [Paper]

• (arXiv 2022.10) Robust Self-Supervised Learning with Lie Groups, [Paper]

• (arXiv 2022.10) VIOLA: Imitation Learning for Vision-Based Manipulation with Object Proposal Priors, [Paper], [Code]

• (arXiv 2022.10) VTC: Improving Video-Text Retrieval with User Comments, [Paper], [Project]

• (arXiv 2022.10) SOLVING REASONING TASKS WITH A SLOT TRANSFORMER, [Paper], [Code]

• (arXiv 2022.10) Prompting through Prototype: A Prototype-based Prompt Learning on Pretrained Vision-Language Models, [Paper]

• (arXiv 2022.10) Grounded Video Situation Recognition, [Paper], [Project]

• (arXiv 2022.10) Single Image Super-Resolution Using Lightweight Networks Based on Swin Transformer, [Paper]

• (arXiv 2022.10) Visual Spatial Description: Controlled Spatial-Oriented Image-to-Text Generation, [Paper], [Code]

• (arXiv 2022.10) MovieCLIP: Visual Scene Recognition in Movies, [Paper]

• (arXiv 2022.10) PointTAD: Multi-Label Temporal Action Detection with Learnable Query Points, [Paper], [Code]

• (arXiv 2022.10) TOWARDS SUSTAINABLE SELF-SUPERVISED LEARNING, [Paper]

• (arXiv 2022.10) Visual-Semantic Contrastive Alignment for Few-Shot Image Classification, [Paper]

• (arXiv 2022.10) i-MAE: ARE LATENT REPRESENTATIONS IN MASKED AUTOENCODERS LINEARLY SEPARABLE? [Paper], [Code]

• (arXiv 2022.10) 2nd Place Solution to ECCV 2022 Challenge: Transformer-based Action recognition in hand-object interacting scenarios, [Paper]

• (arXiv 2022.10) 1st Place Solution to ECCV 2022 Challenge on HBHA: Transformer-based Global 3D Hand Pose Estimation in Two Hands Manipulating Objects Scenarios, [Paper]

• (arXiv 2022.10) DALLE-2 is Seeing Double: Flaws in Word-to-Concept Mapping in Text2Image Models, [Paper]

• (arXiv 2022.10) CLIP-Driven Fine-grained Text-Image Person Re-identification, [Paper]

• (arXiv 2022.10) Dense but Efficient VideoQA for Intricate Compositional Reasoning, [Paper]

• (arXiv 2022.10) Multi-view Gait Recognition based on SiameseVisionTransformer, [Paper]

• (arXiv 2022.10) TOIST: Task Oriented Instance Segmentation Transformer with Noun-Pronoun Distillation, [Paper], [Code]

• (arXiv 2022.10) CroCo: Self-Supervised Pre-training for 3D Vision Tasks by Cross-View Completion, [Paper], [Project]

• (arXiv 2022.10) A Unified View of Masked Image Modeling, [Paper], [Code]

• (arXiv 2022.10) Cross-Modal Fusion Distillation for Fine-Grained Sketch-Based Image Retrieval, [Paper], [Code]

• (arXiv 2022.10) BOAT: Bilateral Local Attention Vision Transformer, [Paper]

• (arXiv 2022.12) TOKEN MERGING: YOUR VIT BUT FASTER, [Paper], [Code]

• (arXiv 2022.10) Using Language to Extend to Unseen Domains, [Paper], [Code]

• (ar