awesome-offline-rl

This is a collection of research and review papers for offline reinforcement learning (offline rl). Feel free to star and fork.

Maintainers:

• Haruka Kiyohara (Cornell University)
• Yuta Saito (Hanjuku-kaso Co., Ltd. / Cornell University)

We are looking for more contributors and maintainers! Please feel free to pull requests.

format:
- [title](paper link) [links]
  - author1, author2, and author3. arXiv/conferences/journals/, year.

For any questions, feel free to contact: hk844@cornell.edu

Table of Contents

• Papers
  • Review/Survey/Position Papers
    • Offline RL
    • Off-Policy Evaluation and Learning
    • Related Reviews
  • Offline RL: Theory/Methods
  • Offline RL: Benchmarks/Experiments
  • Offline RL: Applications
  • Off-Policy Evaluation and Learning: Theory/Methods
    • Off-Policy Evaluation: Contextual Bandits
    • Off-Policy Evaluation: Reinforcement Learning
    • Off-Policy Learning
  • Off-Policy Evaluation and Learning: Benchmarks/Experiments
  • Off-Policy Evaluation and Learning: Applications
• Open Source Software/Implementations
• Blog/Podcast
  • Blog
  • Podcast
• Related Workshops
• Tutorials/Talks/Lectures

Papers

Review/Survey/Position Papers

Offline RL

• Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback
  • Stephen Casper, Xander Davies, Claudia Shi, Thomas Krendl Gilbert, Jérémy Scheurer, Javier Rando, Rachel Freedman, Tomasz Korbak, David Lindner, Pedro Freire, Tony Wang, Samuel Marks, Charbel-Raphaël Segerie, Micah Carroll, Andi Peng, Phillip Christoffersen, Mehul Damani, Stewart Slocum, Usman Anwar, Anand Siththaranjan, Max Nadeau, Eric J. Michaud, Jacob Pfau, Dmitrii Krasheninnikov, Xin Chen, Lauro Langosco, Peter Hase, Erdem Bıyık, Anca Dragan, David Krueger, Dorsa Sadigh, and Dylan Hadfield-Menell. arXiv, 2023.
• A Survey on Offline Model-Based Reinforcement Learning
  • Haoyang He. arXiv, 2023.
• Foundation Models for Decision Making: Problems, Methods, and Opportunities
  • Sherry Yang, Ofir Nachum, Yilun Du, Jason Wei, Pieter Abbeel, Dale Schuurmans. arXiv, 2023.
• A Survey on Offline Reinforcement Learning: Taxonomy, Review, and Open Problems
  • Rafael Figueiredo Prudencio, Marcos R. O. A. Maximo, and Esther Luna Colombini. arXiv, 2022.
• Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems
  • Sergey Levine, Aviral Kumar, George Tucker, and Justin Fu. arXiv, 2020.

Off-Policy Evaluation and Learning

• A Review of Off-Policy Evaluation in Reinforcement Learning
  • Masatoshi Uehara, Chengchun Shi, and Nathan Kallus. arXiv, 2022.

Related Reviews

• On the Opportunities and Challenges of Offline Reinforcement Learning for Recommender Systems
  • Xiaocong Chen, Siyu Wang, Julian McAuley, Dietmar Jannach, and Lina Yao. arXiv, 2023.
• Understanding Reinforcement Learning Algorithms: The Progress from Basic Q-learning to Proximal Policy Optimization
  • Mohamed-Amine Chadi and Hajar Mousannif. arXiv, 2023.
• Offline Evaluation for Reinforcement Learning-based Recommendation: A Critical Issue and Some Alternatives
  • Romain Deffayet, Thibaut Thonet, Jean-Michel Renders, and Maarten de Rijke. arXiv, 2023.
• A Survey on Transformers in Reinforcement Learning
  • Wenzhe Li, Hao Luo, Zichuan Lin, Chongjie Zhang, Zongqing Lu, and Deheng Ye. arXiv, 2023.
• Deep Reinforcement Learning: Opportunities and Challenges
  • Yuxi Li. arXiv, 2022.
• A Survey on Model-based Reinforcement Learning
  • Fan-Ming Luo, Tian Xu, Hang Lai, Xiong-Hui Chen, Weinan Zhang, and Yang Yu. arXiv, 2022.
• Survey on Fair Reinforcement Learning: Theory and Practice
  • Pratik Gajane, Akrati Saxena, Maryam Tavakol, George Fletcher, and Mykola Pechenizkiy. arXiv, 2022.
• Accelerating Offline Reinforcement Learning Application in Real-Time Bidding and Recommendation: Potential Use of Simulation
  • Haruka Kiyohara, Kosuke Kawakami, and Yuta Saito. arXiv, 2021.
• A Survey of Generalisation in Deep Reinforcement Learning
  • Robert Kirk, Amy Zhang, Edward Grefenstette, and Tim Rocktäschel. arXiv, 2021.

Offline RL: Theory/Methods

• Value-Aided Conditional Supervised Learning for Offline RL
  • Jeonghye Kim, Suyoung Lee, Woojun Kim, and Youngchul Sung. arXiv, 2024.
• Towards an Information Theoretic Framework of Context-Based Offline Meta-Reinforcement Learning
  • Lanqing Li, Hai Zhang, Xinyu Zhang, Shatong Zhu, Junqiao Zhao, and Pheng-Ann Heng. arXiv, 2024.
• DiffStitch: Boosting Offline Reinforcement Learning with Diffusion-based Trajectory Stitching
  • Guanghe Li, Yixiang Shan, Zhengbang Zhu, Ting Long, and Weinan Zhang. arXiv, 2024.
• Deep autoregressive density nets vs neural ensembles for model-based offline reinforcement learning
  • Abdelhakim Benechehab, Albert Thomas, and Balázs Kégl. arXiv, 2024.
• Context-Former: Stitching via Latent Conditioned Sequence Modeling
  • Ziqi Zhang, Jingzehua Xu, Zifeng Zhuang, Jinxin Liu, and Donglin wang. arXiv, 2024.
• Adversarially Trained Actor Critic for offline CMDPs
  • Honghao Wei, Xiyue Peng, Xin Liu, and Arnob Ghosh. arXiv, 2024.
• Optimistic Model Rollouts for Pessimistic Offline Policy Optimization
  • Yuanzhao Zhai, Yiying Li, Zijian Gao, Xudong Gong, Kele Xu, Dawei Feng, Ding Bo, and Huaimin Wang. arXiv, 2024.
• Solving Continual Offline Reinforcement Learning with Decision Transformer
  • Kaixin Huang, Li Shen, Chen Zhao, Chun Yuan, and Dacheng Tao. arXiv, 2024.
• MoMA: Model-based Mirror Ascent for Offline Reinforcement Learning
  • Mao Hong, Zhiyue Zhang, Yue Wu, and Yanxun Xu. arXiv, 2024.
• Reframing Offline Reinforcement Learning as a Regression Problem
  • Prajwal Koirala and Cody Fleming. arXiv, 2024.
• Efficient Two-Phase Offline Deep Reinforcement Learning from Preference Feedback
  • Yinglun Xu and Gagandeep Singh. arXiv, 2024.
• Policy-regularized Offline Multi-objective Reinforcement Learning
  • Qian Lin, Chao Yu, Zongkai Liu, and Zifan Wu. arXiv, 2024.
• Differentiable Tree Search in Latent State Space
  • Dixant Mittal and Wee Sun Lee. arXiv, 2024.
• Learning from Sparse Offline Datasets via Conservative Density Estimation
  • Zhepeng Cen, Zuxin Liu, Zitong Wang, Yihang Yao, Henry Lam, and Ding Zhao. ICLR, 2024.
• Safe Offline Reinforcement Learning with Feasibility-Guided Diffusion Model
  • Yinan Zheng, Jianxiong Li, Dongjie Yu, Yujie Yang, Shengbo Eben Li, Xianyuan Zhan, and Jingjing Liu. ICLR, 2024.
• PDiT: Interleaving Perception and Decision-making Transformers for Deep Reinforcement Learning
  • Hangyu Mao, Rui Zhao, Ziyue Li, Zhiwei Xu, Hao Chen, Yiqun Chen, Bin Zhang, Zhen Xiao, Junge Zhang, and Jiangjin Yin. AAMAS, 2024.
• Critic-Guided Decision Transformer for Offline Reinforcement Learning
  • Yuanfu Wang, Chao Yang, Ying Wen, Yu Liu, and Yu Qiao. AAAI, 2024.
• CUDC: A Curiosity-Driven Unsupervised Data Collection Method with Adaptive Temporal Distances for Offline Reinforcement Learning
  • Chenyu Sun, Hangwei Qian, and Chunyan Miao. AAAI, 2024.
• Neural Network Approximation for Pessimistic Offline Reinforcement Learning
  • Di Wu, Yuling Jiao, Li Shen, Haizhao Yang, and Xiliang Lu. AAAI, 2024.
• A Perspective of Q-value Estimation on Offline-to-Online Reinforcement Learning
  • Yinmin Zhang, Jie Liu, Chuming Li, Yazhe Niu, Yaodong Yang, Yu Liu, and Wanli Ouyang. AAAI, 2024.
• The Generalization Gap in Offline Reinforcement Learning
  • Ishita Mediratta, Qingfei You, Minqi Jiang, and Roberta Raileanu. arXiv, 2023.
• Decoupling Meta-Reinforcement Learning with Gaussian Task Contexts and Skills
  • Hongcai He, Anjie Zhu, Shuang Liang, Feiyu Chen, and Jie Shao. arXiv, 2023.
• MICRO: Model-Based Offline Reinforcement Learning with a Conservative Bellman Operator
  • Xiao-Yin Liu, Xiao-Hu Zhou, Guo-Tao Li, Hao Li, Mei-Jiang Gui, Tian-Yu Xiang, De-Xing Huang, and Zeng-Guang Hou. arXiv, 2023.
• Model-Based Epistemic Variance of Values for Risk-Aware Policy Optimization
  • Carlos E. Luis, Alessandro G. Bottero, Julia Vinogradska, Felix Berkenkamp, and Jan Peters. arXiv, 2023.
• Using Curiosity for an Even Representation of Tasks in Continual Offline Reinforcement Learning
  • Pankayaraj Pathmanathan, Natalia Díaz-Rodríguez, and Javier Del Ser. arXiv, 2023.
• Projected Off-Policy Q-Learning (POP-QL) for Stabilizing Offline Reinforcement Learning
  • Melrose Roderick, Gaurav Manek, Felix Berkenkamp, and J. Zico Kolter. arXiv, 2023.
• Offline Data Enhanced On-Policy Policy Gradient with Provable Guarantees
  • Yifei Zhou, Ayush Sekhari, Yuda Song, and Wen Sun. arXiv, 2023.
• Switch Trajectory Transformer with Distributional Value Approximation for Multi-Task Reinforcement Learning
  • Qinjie Lin, Han Liu, and Biswa Sengupta. arXiv, 2023.
• Hierarchical Decision Transformer
  • André Correia and Luís A. Alexandre. arXiv, 2023.
• Prompt-Tuning Decision Transformer with Preference Ranking
  • Shengchao Hu, Li Shen, Ya Zhang, and Dacheng Tao. arXiv, 2023.
• Context Shift Reduction for Offline Meta-Reinforcement Learning
  • Yunkai Gao, Rui Zhang, Jiaming Guo, Fan Wu, Qi Yi, Shaohui Peng, Siming Lan, Ruizhi Chen, Zidong Du, Xing Hu, Qi Guo, Ling Li, and Yunji Chen. arXiv, 2023.
• Uni-O4: Unifying Online and Offline Deep Reinforcement Learning with Multi-Step On-Policy Optimization
  • Kun Lei, Zhengmao He, Chenhao Lu, Kaizhe Hu, Yang Gao, and Huazhe Xu. arXiv, 2023.
• Score Models for Offline Goal-Conditioned Reinforcement Learning
  • Harshit Sikchi, Rohan Chitnis, Ahmed Touati, Alborz Geramifard, Amy Zhang, and Scott Niekum. arXiv, 2023.
• Offline RL with Observation Histories: Analyzing and Improving Sample Complexity
  • Joey Hong, Anca Dragan, and Sergey Levine. arXiv, 2023.
• Expressive Modeling Is Insufficient for Offline RL: A Tractable Inference Perspective
  • Xuejie Liu, Anji Liu, Guy Van den Broeck, and Yitao Liang. arXiv, 2023.
• Rethinking Decision Transformer via Hierarchical Reinforcement Learning
  • Yi Ma, Chenjun Xiao, Hebin Liang, and Jianye Hao. arXiv, 2023.
• Unleashing the Power of Pre-trained Language Models for Offline Reinforcement Learning
  • Ruizhe Shi, Yuyao Liu, Yanjie Ze, Simon S. Du, and Huazhe Xu. arXiv, 2023.
• GOPlan: Goal-conditioned Offline Reinforcement Learning by Planning with Learned Models
  • Mianchu Wang, Rui Yang, Xi Chen, and Meng Fang. arXiv, 2023.
• SERA: Sample Efficient Reward Augmentation in offline-to-online Reinforcement Learning
  • Ziqi Zhang, Xiao Xiong, Zifeng Zhuang, Jinxin Liu, and Donglin Wang. arXiv, 2023.
• Bridging Distributionally Robust Learning and Offline RL: An Approach to Mitigate Distribution Shift and Partial Data Coverage
  • Kishan Panaganti, Zaiyan Xu, Dileep Kalathil, and Mohammad Ghavamzadeh. arXiv, 2023.
• Guided Data Augmentation for Offline Reinforcement Learning and Imitation Learning
  • Nicholas E. Corrado, Yuxiao Qu, John U. Balis, Adam Labiosa, and Josiah P. Hanna. arXiv, 2023.
• CROP: Conservative Reward for Model-based Offline Policy Optimization
  • Hao Li, Xiao-Hu Zhou, Xiao-Liang Xie, Shi-Qi Liu, Zhen-Qiu Feng, Xiao-Yin Liu, Mei-Jiang Gui, Tian-Yu Xiang, De-Xing Huang, Bo-Xian Yao, and Zeng-Guang Hou. arXiv, 2023.
• Towards Robust Offline Reinforcement Learning under Diverse Data Corruption
  • Rui Yang, Han Zhong, Jiawei Xu, Amy Zhang, Chongjie Zhang, Lei Han, and Tong Zhang. arXiv, 2023.
• Offline Retraining for Online RL: Decoupled Policy Learning to Mitigate Exploration Bias
  • Max Sobol Mark, Archit Sharma, Fahim Tajwar, Rafael Rafailov, Sergey Levine, and Chelsea Finn. arXiv, 2023.
• Boosting Continuous Control with Consistency Policy
  • Yuhui Chen, Haoran Li, and Dongbin Zhao. arXiv, 2023.
• Planning to Go Out-of-Distribution in Offline-to-Online Reinforcement Learning
  • Trevor McInroe, Stefano V. Albrecht, and Amos Storkey. arXiv, 2023.
• Reward-Consistent Dynamics Models are Strongly Generalizable for Offline Reinforcement Learning
  • Fan-Ming Luo, Tian Xu, Xingchen Cao, and Yang Yu. arXiv, 2023.
• DiffCPS: Diffusion Model based Constrained Policy Search for Offline Reinforcement Learning
  • Longxiang He, Linrui Zhang, Junbo Tan, and Xueqian Wang. arXiv, 2023.
• Self-Confirming Transformer for Locally Consistent Online Adaptation in Multi-Agent Reinforcement Learning
  • Tao Li, Juan Guevara, Xinghong Xie, and Quanyan Zhu. arXiv, 2023.
• Learning to Reach Goals via Diffusion
  • Vineet Jain and Siamak Ravanbakhsh. arXiv, 2023.
• Decision ConvFormer: Local Filtering in MetaFormer is Sufficient for Decision Making
  • Jeonghye Kim, Suyoung Lee, Woojun Kim, and Youngchul Sung. arXiv, 2023.
• Consistency Models as a Rich and Efficient Policy Class for Reinforcement Learning
  • Zihan Ding and Chi Jin. arXiv, 2023.
• Pessimistic Nonlinear Least-Squares Value Iteration for Offline Reinforcement Learning
  • Qiwei Di, Heyang Zhao, Jiafan He, and Quanquan Gu. arXiv, 2023.
• Reasoning with Latent Diffusion in Offline Reinforcement Learning
  • Siddarth Venkatraman, Shivesh Khaitan, Ravi Tej Akella, John Dolan, Jeff Schneider, and Glen Berseth. arXiv, 2023.
• Hundreds Guide Millions: Adaptive Offline Reinforcement Learning with Expert Guidance
  • Qisen Yang, Shenzhi Wang, Qihang Zhang, Gao Huang, and Shiji Song. arXiv, 2023.
• Towards Robust Offline-to-Online Reinforcement Learning via Uncertainty and Smoothness
  • Xiaoyu Wen, Xudong Yu, Rui Yang, Chenjia Bai, and Zhen Wang. arXiv, 2023.
• Robust Offline Reinforcement Learning -- Certify the Confidence Interval
  • Jiarui Yao and Simon Shaolei Du. arXiv, 2023.
• Stackelberg Batch Policy Learning
  • Wenzhuo Zhou and Annie Qu. arXiv, 2023.
• H2O+: An Improved Framework for Hybrid Offline-and-Online RL with Dynamics Gaps
  • Haoyi Niu, Tianying Ji, Bingqi Liu, Haocheng Zhao, Xiangyu Zhu, Jianying Zheng, Pengfei Huang, Guyue Zhou, Jianming Hu, and Xianyuan Zhan. arXiv, 2023.
• Q-Transformer: Scalable Offline Reinforcement Learning via Autoregressive Q-Functions
  • Yevgen Chebotar, Quan Vuong, Alex Irpan, Karol Hausman, Fei Xia, Yao Lu, Aviral Kumar, Tianhe Yu, Alexander Herzog, Karl Pertsch, Keerthana Gopalakrishnan, Julian Ibarz, Ofir Nachum, Sumedh Sontakke, Grecia Salazar, Huong T Tran, Jodilyn Peralta, Clayton Tan, Deeksha Manjunath, Jaspiar Singht, Brianna Zitkovich, Tomas Jackson, Kanishka Rao, Chelsea Finn, and Sergey Levine. arXiv, 2023.
• DOMAIN: MilDly COnservative Model-BAsed OfflINe Reinforcement Learning
  • Xiao-Yin Liu, Xiao-Hu Zhou, Xiao-Liang Xie, Shi-Qi Liu, Zhen-Qiu Feng, Hao Li, Mei-Jiang Gui, Tian-Yu Xiang, De-Xing Huang, and Zeng-Guang Hou. arXiv, 2023.
• Guided Online Distillation: Promoting Safe Reinforcement Learning by Offline Demonstration
  • Jinning Li, Xinyi Liu, Banghua Zhu, Jiantao Jiao, Masayoshi Tomizuka, Chen Tang, and Wei Zhan. arXiv, 2023.
• Equivariant Data Augmentation for Generalization in Offline Reinforcement Learning
  • Cristina Pinneri, Sarah Bechtle, Markus Wulfmeier, Arunkumar Byravan, Jingwei Zhang, William F. Whitney, and Martin Riedmiller. arXiv, 2023.
• Reasoning with Latent Diffusion in Offline Reinforcement Learning
  • Siddarth Venkatraman, Shivesh Khaitan, Ravi Tej Akella, John Dolan, Jeff Schneider, and Glen Berseth. arXiv, 2023.
• Hundreds Guide Millions: Adaptive Offline Reinforcement Learning with Expert Guidance
  • Qisen Yang, Shenzhi Wang, Qihang Zhang, Gao Huang, and Shiji Song. arXiv, 2023.
• Multi-Objective Decision Transformers for Offline Reinforcement Learning
  • Abdelghani Ghanem, Philippe Ciblat, and Mounir Ghogho. arXiv, 2023.
• AlphaStar Unplugged: Large-Scale Offline Reinforcement Learning
  • Michaël Mathieu, Sherjil Ozair, Srivatsan Srinivasan, Caglar Gulcehre, Shangtong Zhang, Ray Jiang, Tom Le Paine, Richard Powell, Konrad Żołna, Julian Schrittwieser, David Choi, Petko Georgiev, Daniel Toyama, Aja Huang, Roman Ring, Igor Babuschkin, Timo Ewalds, Mahyar Bordbar, Sarah Henderson, Sergio Gómez Colmenarejo, Aäron van den Oord, Wojciech Marian Czarnecki, Nando de Freitas, and Oriol Vinyals. arXiv, 2023.
• Exploiting Generalization in Offline Reinforcement Learning via Unseen State Augmentations
  • Nirbhay Modhe, Qiaozi Gao, Ashwin Kalyan, Dhruv Batra, Govind Thattai, and Gaurav Sukhatme. arXiv, 2023.
• PASTA: Pretrained Action-State Transformer Agents
  • Raphael Boige, Yannis Flet-Berliac, Arthur Flajolet, Guillaume Richard, and Thomas Pierrot. arXiv, 2023.
• Towards A Unified Agent with Foundation Models
  • Norman Di Palo, Arunkumar Byravan, Leonard Hasenclever, Markus Wulfmeier, Nicolas Heess, and Martin Riedmiller. arXiv, 2023.
• Goal-Conditioned Predictive Coding as an Implicit Planner for Offline Reinforcement Learning
  • Zilai Zeng, Ce Zhang, Shijie Wang, and Chen Sun. arXiv, 2023.
• Offline Reinforcement Learning with Imbalanced Datasets
  • Li Jiang, Sijie Chen, Jielin Qiu, Haoran Xu, Wai Kin Chan, and Zhao Ding. arXiv, 2023.
• LLQL: Logistic Likelihood Q-Learning for Reinforcement Learning
  • Outongyi Lv, Bingxin Zhou, and Yu Guang Wang. arXiv, 2023.
• Elastic Decision Transformer
  • Yueh-Hua Wu, Xiaolong Wang, and Masashi Hamaya. arXiv, 2023.
• Prioritized Trajectory Replay: A Replay Memory for Data-driven Reinforcement Learning
  • Jinyi Liu, Yi Ma, Jianye Hao, Yujing Hu, Yan Zheng, Tangjie Lv, and Changjie Fan. arXiv, 2023.
• Is RLHF More Difficult than Standard RL?
  • Yuanhao Wang, Qinghua Liu, and Chi Jin. arXiv, 2023.
• Supervised Pretraining Can Learn In-Context Reinforcement Learning
  • Jonathan N. Lee, Annie Xie, Aldo Pacchiano, Yash Chandak, Chelsea Finn, Ofir Nachum, and Emma Brunskill. arXiv, 2023.
• Fighting Uncertainty with Gradients: Offline Reinforcement Learning via Diffusion Score Matching
  • H.J. Terry Suh, Glen Chou, Hongkai Dai, Lujie Yang, Abhishek Gupta, and Russ Tedrake. arXiv, 2023.
• Safe Reinforcement Learning with Dead-Ends Avoidance and Recovery
  • Xiao Zhang, Hai Zhang, Hongtu Zhou, Chang Huang, Di Zhang, Chen Ye, and Junqiao Zhao. arXiv, 2023.
• CLUE: Calibrated Latent Guidance for Offline Reinforcement Learning
  • Jinxin Liu, Lipeng Zu, Li He, and Donglin Wang. arXiv, 2023.
• Harnessing Mixed Offline Reinforcement Learning Datasets via Trajectory Weighting
  • Zhang-Wei Hong, Pulkit Agrawal, Rémi Tachet des Combes, and Romain Laroche.
• Beyond OOD State Actions: Supported Cross-Domain Offline Reinforcement Learning
  • Jinxin Liu, Ziqi Zhang, Zhenyu Wei, Zifeng Zhuang, Yachen Kang, Sibo Gai, and Donglin Wang. arXiv, 2023.
• A Primal-Dual-Critic Algorithm for Offline Constrained Reinforcement Learning
  • Kihyuk Hong, Yuhang Li, and Ambuj Tewari. arXiv, 2023.
• HIPODE: Enhancing Offline Reinforcement Learning with High-Quality Synthetic Data from a Policy-Decoupled Approach
  • Shixi Lian, Yi Ma, Jinyi Liu, Yan Zheng, and Zhaopeng Meng. arXiv, 2023.
• Ensemble-based Offline-to-Online Reinforcement Learning: From Pessimistic Learning to Optimistic Exploration
  • Kai Zhao, Yi Ma, Jinyi Liu, Yan Zheng, and Zhaopeng Meng. arXiv, 2023.
• In-Sample Policy Iteration for Offline Reinforcement Learning
  • Xiaohan Hu, Yi Ma, Chenjun Xiao, Yan Zheng, and Zhaopeng Meng. arXiv, 2023.
• Instructed Diffuser with Temporal Condition Guidance for Offline Reinforcement Learning
  • Jifeng Hu, Yanchao Sun, Sili Huang, SiYuan Guo, Hechang Chen, Li Shen, Lichao Sun, Yi Chang, and Dacheng Tao. arXiv, 2023.
• Offline Prioritized Experience Replay
  • Yang Yue, Bingyi Kang, Xiao Ma, Gao Huang, Shiji Song, and Shuicheng Yan. arXiv, 2023.
• Delphic Offline Reinforcement Learning under Nonidentifiable Hidden Confounding
  • Alizée Pace, Hugo Yèche, Bernhard Schölkopf, Gunnar Rätsch, and Guy Tennenholtz. arXiv, 2023.
• Offline Meta Reinforcement Learning with In-Distribution Online Adaptation
  • Jianhao Wang, Jin Zhang, Haozhe Jiang, Junyu Zhang, Liwei Wang, and Chongjie Zhang. arXiv, 2023.
• Diffusion Model is an Effective Planner and Data Synthesizer for Multi-Task Reinforcement Learning
  • Haoran He, Chenjia Bai, Kang Xu, Zhuoran Yang, Weinan Zhang, Dong Wang, Bin Zhao, and Xuelong Li. arXiv, 2023.
• Reinforcement Learning with Human Feedback: Learning Dynamic Choices via Pessimism
  • Zihao Li, Zhuoran Yang, and Mengdi Wang. arXiv, 2023.
• MADiff: Offline Multi-agent Learning with Diffusion Models
  • Zhengbang Zhu, Minghuan Liu, Liyuan Mao, Bingyi Kang, Minkai Xu, Yong Yu, Stefano Ermon, and Weinan Zhang. arXiv, 2023.
• Provable Offline Reinforcement Learning with Human Feedback
  • Wenhao Zhan, Masatoshi Uehara, Nathan Kallus, Jason D. Lee, and Wen Sun. arXiv, 2023.
• Think Before You Act: Decision Transformers with Internal Working Memory
  • Jikun Kang, Romain Laroche, Xindi Yuan, Adam Trischler, Xue Liu, and Jie Fu. arXiv, 2023.
• Distributionally Robust Optimization Efficiently Solves Offline Reinforcement Learning
  • Yue Wang, Yuting Hu, Jinjun Xiong, and Shaofeng Zou. arXiv, 2023.
• Offline Primal-Dual Reinforcement Learning for Linear MDPs
  • Germano Gabbianelli, Gergely Neu, Nneka Okolo, and Matteo Papini. arXiv, 2023.
• Federated Offline Policy Learning with Heterogeneous Observational Data
  • Aldo Gael Carranza and Susan Athey. arXiv, 2023.
• Offline Reinforcement Learning with Additional Covering Distributions
  • Chenjie Mao. arXiv, 2023.
• Reward-agnostic Fine-tuning: Provable Statistical Benefits of Hybrid Reinforcement Learning
  • Gen Li, Wenhao Zhan, Jason D. Lee, Yuejie Chi, and Yuxin Chen. arXiv, 2023.
• Stackelberg Decision Transformer for Asynchronous Action Coordination in Multi-Agent Systems
  • Bin Zhang, Hangyu Mao, Lijuan Li, Zhiwei Xu, Dapeng Li, Rui Zhao, and Guoliang Fan. arXiv, 2023.
• Federated Ensemble-Directed Offline Reinforcement Learning
  • Desik Rengarajan, Nitin Ragothaman, Dileep Kalathil, and Srinivas Shakkottai. arXiv, 2023.
• IDQL: Implicit Q-Learning as an Actor-Critic Method with Diffusion Policies
  • Philippe Hansen-Estruch, Ilya Kostrikov, Michael Janner, Jakub Grudzien Kuba, and Sergey Levine. arXiv, 2023.
• Using Offline Data to Speed-up Reinforcement Learning in Procedurally Generated Environments
  • Alain Andres, Lukas Schäfer, Esther Villar-Rodriguez, Stefano V.Albrecht, Javier Del Ser. arXiv, 2023.
• Reinforcement Learning from Passive Data via Latent Intentions [website]
  • Dibya Ghosh, Chethan Bhateja, and Sergey Levine. arXiv, 2023.
• Uncertainty-driven Trajectory Truncation for Model-based Offline Reinforcement Learning
  • Junjie Zhang, Jiafei Lyu, Xiaoteng Ma, Jiangpeng Yan, Jun Yang, Le Wan, and Xiu Li. arXiv, 2023.
• RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignment
  • Hanze Dong, Wei Xiong, Deepanshu Goyal, Rui Pan, Shizhe Diao, Jipeng Zhang, Kashun Shum, and Tong Zhang. arXiv, 2023.
• Batch Quantum Reinforcement Learning
  • Maniraman Periyasamy, Marc Hölle, Marco Wiedmann, Daniel D. Scherer, Axel Plinge, and Christopher Mutschler. arXiv, 2023.
• Accelerating exploration and representation learning with offline pre-training
  • Bogdan Mazoure, Jake Bruce, Doina Precup, Rob Fergus, and Ankit Anand. arXiv, 2023.
• On Context Distribution Shift in Task Representation Learning for Offline Meta RL
  • Chenyang Zhao, Zihao Zhou, and Bin Liu. arXiv, 2023.
• Optimal Goal-Reaching Reinforcement Learning via Quasimetric Learning
  • Tongzhou Wang, Antonio Torralba, Phillip Isola, and Amy Zhang. arXiv, 2023.
• Learning Excavation of Rigid Objects with Offline Reinforcement Learning
  • Shiyu Jin, Zhixian Ye, and Liangjun Zhang. arXiv, 2023.
• Goal-conditioned Offline Reinforcement Learning through State Space Partitioning
  • Mianchu Wang, Yue Jin, and Giovanni Montana. arXiv, 2023.
• Merging Decision Transformers: Weight Averaging for Forming Multi-Task Policies
  • Daniel Lawson and Ahmed H. Qureshi. arXiv, 2023.
• Deploying Offline Reinforcement Learning with Human Feedback
  • Ziniu Li, Ke Xu, Liu Liu, Lanqing Li, Deheng Ye, and Peilin Zhao. arXiv, 2023.
• Synthetic Experience Replay
  • Cong Lu, Philip J. Ball, and Jack Parker-Holder. arXiv, 2023.
• ENTROPY: Environment Transformer and Offline Policy Optimization
  • Pengqin Wang, Meixin Zhu, and Shaojie Shen. arXiv, 2023.
• Graph Decision Transformer
  • Shengchao Hu, Li Shen, Ya Zhang, and Dacheng Tao. arXiv, 2023.
• Selective Uncertainty Propagation in Offline RL
  • Sanath Kumar Krishnamurthy, Tanmay Gangwani, Sumeet Katariya, Branislav Kveton, and Anshuka Rangi. arXiv, 2023.
• Off-the-Grid MARL: a Framework for Dataset Generation with Baselines for Cooperative Offline Multi-Agent Reinforcement Learning
  • Claude Formanek, Asad Jeewa, Jonathan Shock, and Arnu Pretorius. arXiv, 2023.
• Skill Decision Transformer
  • Shyam Sudhakaran and Sebastian Risi. arXiv, 2023.
• Guiding Online Reinforcement Learning with Action-Free Offline Pretraining
  • Deyao Zhu, Yuhui Wang, Jürgen Schmidhuber, and Mohamed Elhoseiny. arXiv, 2023.
• SaFormer: A Conditional Sequence Modeling Approach to Offline Safe Reinforcement Learning
  • Qin Zhang, Linrui Zhang, Haoran Xu, Li Shen, Bowen Wang, Yongzhe Chang, Xueqian Wang, Bo Yuan, and Dacheng Tao. arXiv, 2023.
• APAC: Authorized Probability-controlled Actor-Critic For Offline Reinforcement Learning
  • Jing Zhang, Chi Zhang, Wenjia Wang, and Bing-Yi Jing. arXiv, 2023.
• Designing an offline reinforcement learning objective from scratch
  • Gaon An, Junhyeok Lee, Xingdong Zuo, Norio Kosaka, Kyung-Min Kim, and Hyun Oh Song. arXiv, 2023.
• Behaviour Discriminator: A Simple Data Filtering Method to Improve Offline Policy Learning
  • Qiang Wang, Robert McCarthy, David Cordova Bulens, Kevin McGuinness, Noel E. O'Connor, Francisco Roldan Sanchez, and Stephen J. Redmond. arXiv, 2023.
• Learning to View: Decision Transformers for Active Object Detection
  • Wenhao Ding, Nathalie Majcherczyk, Mohit Deshpande, Xuewei Qi, Ding Zhao, Rajasimman Madhivanan, and Arnie Sen. arXiv, 2023.
• Risk Sensitive Dead-end Identification in Safety-Critical Offline Reinforcement Learning
  • Taylor W. Killian, Sonali Parbhoo, and Marzyeh Ghassemi. arXiv, 2023.
• Value Enhancement of Reinforcement Learning via Efficient and Robust Trust Region Optimization
  • Chengchun Shi, Zhengling Qi, Jianing Wang, and Fan Zhou. arXiv, 2023.
• Contextual Conservative Q-Learning for Offline Reinforcement Learning
  • Ke Jiang, Jiayu Yao, and Xiaoyang Tan. arXiv, 2023.
• Offline Policy Optimization in RL with Variance Regularizaton
  • Riashat Islam, Samarth Sinha, Homanga Bharadhwaj, Samin Yeasar Arnob, Zhuoran Yang, Animesh Garg, Zhaoran Wang, Lihong Li, and Doina Precup. arXiv, 2023.
• Transformer in Transformer as Backbone for Deep Reinforcement Learning
  • Hangyu Mao, Rui Zhao, Hao Chen, Jianye Hao, Yiqun Chen, Dong Li, Junge Zhang, and Zhen Xiao. arXiv, 2023.
• SPQR: Controlling Q-ensemble Independence with Spiked Random Model for Reinforcement Learning
  • Dohyeok Lee, Seungyub Han, Taehyun Cho, and Jungwoo Lee. NeurIPS, 2023.
• Revisiting the Minimalist Approach to Offline Reinforcement Learning
  • Denis Tarasov, Vladislav Kurenkov, Alexander Nikulin, and Sergey Kolesnikov. NeurIPS, 2023.
• Constrained Policy Optimization with Explicit Behavior Density for Offline Reinforcement Learning
  • Jing Zhang, Chi Zhang, Wenjia Wang, and Bingyi Jing. NeurIPS, 2023.
• Supported Value Regularization for Offline Reinforcement Learning
  • Yixiu Mao, Hongchang Zhang, Chen Chen, Yi Xu, and Xiangyang Ji. NeurIPS, 2023.
• Conservative State Value Estimation for Offline Reinforcement Learning
  • Liting Chen, Jie Yan, Zhengdao Shao, Lu Wang, Qingwei Lin, Saravan Rajmohan, Thomas Moscibroda, and Dongmei Zhang. NeurIPS, 2023.
• Understanding and Addressing the Pitfalls of Bisimulation-based Representations in Offline Reinforcement Learning
  • Hongyu Zang, Xin Li, Leiji Zhang, Yang Liu, Baigui Sun, Riashat Islam, Remi Tachet des Combes, and Romain Laroche. NeurIPS, 2023.
• Adversarial Model for Offline Reinforcement Learning
  • Mohak Bhardwaj, Tengyang Xie, Byron Boots, Nan Jiang, and Ching-An Cheng. NeurIPS, 2023.
• Percentile Criterion Optimization in Offline Reinforcement Learning
  • Cyrus Cousins, Elita Lobo, Marek Petrik, and Yair Zick. NeurIPS, 2023.
• Importance Weighted Actor-Critic for Optimal Conservative Offline Reinforcement Learning
  • Hanlin Zhu, Paria Rashidinejad, and Jiantao Jiao. NeurIPS, 2023.
• HIQL: Offline Goal-Conditioned RL with Latent States as Actions
  • Seohong Park, Dibya Ghosh, Benjamin Eysenbach, and Sergey Levine. NeurIPS, 2023.
• Recovering from Out-of-sample States via Inverse Dynamics in Offline Reinforcement Learning
  • Ke Jiang, Jia-Yu Yao, and Xiaoyang Tan. NeurIPS, 2023.
• Offline RL with Discrete Proxy Representations for Generalizability in POMDPs
  • Pengjie Gu, Xinyu Cai, Dong Xing, Xinrun Wang, Mengchen Zhao, and Bo An. NeurIPS, 2023.
• Offline Multi-Agent Reinforcement Learning with Implicit Global-to-Local Value Regularization
  • Xiangsen Wang, Haoran Xu, Yinan Zheng, and Xianyuan Zhan. NeurIPS, 2023.
• Bi-Level Offline Policy Optimization with Limited Exploration
  • Wenzhuo Zhou. NeurIPS, 2023.
• Provably (More) Sample-Efficient Offline RL with Options
  • Xiaoyan Hu and Ho-fung Leung. NeurIPS, 2023.
• Double Pessimism is Provably Efficient for Distributionally Robust Offline Reinforcement Learning: Generic Algorithm and Robust Partial Coverage
  • Jose Blanchet, Miao Lu, Tong Zhang, and Han Zhong. NeurIPS, 2023.
• AlberDICE: Addressing Out-Of-Distribution Joint Actions in Offline Multi-Agent RL via Alternating Stationary Distribution Correction Estimation
  • Daiki E. Matsunaga, Jongmin Lee, Jaeseok Yoon, Stefanos Leonardos, Pieter Abbeel, and Kee-Eung Kim. NeurIPS, 2023.
• Budgeting Counterfactual for Offline RL
  • Yao Liu, Pratik Chaudhari, and Rasool Fakoor. NeurIPS, 2023.
• Efficient Diffusion Policies for Offline Reinforcement Learning
  • Bingyi Kang, Xiao Ma, Chao Du, Tianyu Pang, and Shuicheng Yan. NeurIPS, 2023.
• Cal-QL: Calibrated Offline RL Pre-Training for Efficient Online Fine-Tuning
  • Mitsuhiko Nakamoto, Yuexiang Zhai, Anikait Singh, Max Sobol Mark, Yi Ma, Chelsea Finn, Aviral Kumar, and Sergey Levine. NeurIPS, 2023.
• Policy Finetuning in Reinforcement Learning via Design of Experiments using Offline Data
  • Ruiqi Zhang and Andrea Zanette. NeurIPS, 2023.
• Offline Minimax Soft-Q-learning Under Realizability and Partial Coverage
  • Masatoshi Uehara, Nathan Kallus, Jason D. Lee, and Wen Sun. NeurIPS, 2023.
• Provably Efficient Offline Reinforcement Learning in Regular Decision Processes
  • Roberto Cipollone, Anders Jonsson, Alessandro Ronca, and Mohammad Sadegh Talebi. NeurIPS, 2023.
• Provably Efficient Offline Goal-Conditioned Reinforcement Learning with General Function Approximation and Single-Policy Concentrability
  • Hanlin Zhu and Amy Zhang. NeurIPS, 2023.
• On Sample-Efficient Offline Reinforcement Learning: Data Diversity, Posterior Sampling and Beyond
  • Thanh Nguyen-Tang and Raman Arora. NeurIPS, 2023.
• Conservative Offline Policy Adaptation in Multi-Agent Games
  • Chengjie Wu, Pingzhong Tang, Jun Yang, Yujing Hu, Tangjie Lv, Changjie Fan, and Chongjie Zhang. NeurIPS, 2023.
• Look Beneath the Surface: Exploiting Fundamental Symmetry for Sample-Efficient Offline RL
  • Peng Cheng, Xianyuan Zhan, Zhihao Wu, Wenjia Zhang, Shoucheng Song, Han Wang, Youfang Lin, and Li Jiang. NeurIPS, 2023.
• Survival Instinct in Offline Reinforcement Learning
  • Anqi Li, Dipendra Misra, Andrey Kolobov, and Ching-An Cheng. NeurIPS, 2023.
• Learning from Visual Observation via Offline Pretrained State-to-Go Transformer
  • Bohan Zhou, Ke Li, Jiechuan Jiang, and Zongqing Lu. NeurIPS, 2023.
• Design from Policies: Conservative Test-Time Adaptation for Offline Policy Optimization
  • Jinxin Liu, Hongyin Zhang, Zifeng Zhuang, Yachen Kang, Donglin Wang, and Bin Wang. NeurIPS, 2023.
• Learning to Influence Human Behavior with Offline Reinforcement Learning
  • Joey Hong, Anca Dragan, and Sergey Levine. NeurIPS, 2023.
• Residual Q-Learning: Offline and Online Policy Customization without Value
  • Chenran Li, Chen Tang, Haruki Nishimura, Jean Mercat, Masayoshi Tomizuka, Wei Zhan. NeurIPS, 2023.
• Train Once, Get a Family: State-Adaptive Balances for Offline-to-Online Reinforcement Learning
  • Shenzhi Wang, Qisen Yang, Jiawei Gao, Matthieu Gaetan Lin, Hao Chen, Liwei Wu, Ning Jia, Shiji Song, and Gao Huang. NeurIPS, 2023.
• Beyond Uniform Sampling: Offline Reinforcement Learning with Imbalanced Datasets
  • Zhang-Wei Hong, Aviral Kumar, Sathwik Karnik, Abhishek Bhandwaldar, Akash Srivastava, Joni Pajarinen, Romain Laroche, Abhishek Gupta, and Pulkit Agrawal. NeurIPS, 2023.
• Understanding, Predicting and Better Resolving Q-Value Divergence in Offline-RL
  • Yang Yue, Rui Lu, Bingyi Kang, Shiji Song, and Gao Huang. NeurIPS, 2023.
• Corruption-Robust Offline Reinforcement Learning with General Function Approximation
  • Chenlu Ye, Rui Yang, Quanquan Gu, and Tong Zhang. NeurIPS, 2023.
• Learning to Modulate pre-trained Models in RL
  • Thomas Schmied, Markus Hofmarcher, Fabian Paischer, Razvan Pascanu, and Sepp Hochreiter. NeurIPS, 2023.
• Counterfactual Conservative Q Learning for Offline Multi-agent Reinforcement Learning
  • Jianzhun Shao, Yun Qu, Chen Chen, Hongchang Zhang, and Xiangyang Ji. NeurIPS, 2023.
• One Risk to Rule Them All: A Risk-Sensitive Perspective on Model-Based Offline Reinforcement Learning
  • Marc Rigter, Bruno Lacerda, and Nick Hawes. NeurIPS, 2023.
• Goal-Conditioned Predictive Coding for Offline Reinforcement Learning
  • Zilai Zeng, Ce Zhang, Shijie Wang, and Chen Sun. NeurIPS, 2023.
• Mutual Information Regularized Offline Reinforcement Learning
  • Xiao Ma, Bingyi Kang, Zhongwen Xu, Min Lin, and Shuicheng Yan. NeurIPS, 2023.
• Offline RL With Heteroskedastic Datasets and Support Constraints
  • Anikait Singh, Aviral Kumar, Quan Vuong, Yevgen Chebotar, and Sergey Levine. NeurIPS, 2023.
• Offline Reinforcement Learning with Differential Privacy
  • Dan Qiao and Yu-Xiang Wang. NeurIPS, 2023.
• Accountability in Offline Reinforcement Learning: Explaining Decisions with a Corpus of Examples
  • Hao Sun, Alihan Hüyük, Daniel Jarrett, and Mihaela van der Schaar. NeurIPS, 2023.
• Reining Generalization in Offline Reinforcement Learning via Representation Distinction
  • Yi Ma, Hongyao Tang, Dong Li, and Zhaopeng Meng. NeurIPS, 2023.
• VOCE: Variational Optimization with Conservative Estimation for Offline Safe Reinforcement Learning
  • Jiayi Guan, Guang Chen, Jiaming Ji, Long Yang, ao zhou, Zhijun Li, and changjun jiang. NeurIPS, 2023.
• SafeDICE: Offline Safe Imitation Learning with Non-Preferred Demonstrations
  • Youngsoo Jang, Geon-Hyeong Kim, Jongmin Lee, Sungryull Sohn, Byoungjip Kim, Honglak Lee, and Moontae Lee. NeurIPS, 2023.
• Hierarchical Diffusion for Offline Decision Making
  • Wenhao Li, Xiangfeng Wang, Bo Jin, and Hongyuan Zha. ICML, 2023.
• MAHALO: Unifying Offline Reinforcement Learning and Imitation Learning from Observations
  • Anqi Li, Byron Boots, and Ching-An Cheng. ICML, 2023.
• Safe Offline Reinforcement Learning with Real-Time Budget Constraints
  • Qian Lin, Bo Tang, Zifan Wu, Chao Yu, Shangqin Mao, Qianlong Xie, Xingxing Wang, and Dong Wang. ICML, 2023.
• Near-optimal Conservative Exploration in Reinforcement Learning under Episode-wise Constraints
  • Donghao Li, Ruiquan Huang, Cong Shen, and Jing Yang. ICML, 2023.
• A Connection between One-Step Regularization and Critic Regularization in Reinforcement Learning
  • Benjamin Eysenbach, Matthieu Geist, Sergey Levine, and Ruslan Salakhutdinov. ICML, 2023.
• Anti-Exploration by Random Network Distillation
  • Alexander Nikulin, Vladislav Kurenkov, Denis Tarasov, and Sergey Kolesnikov. ICML, 2023.
• Optimal Goal-Reaching Reinforcement Learning via Quasimetric Learning
  • Tongzhou Wang, Antonio Torralba, Phillip Isola, and Amy Zhang. ICML, 2023.
• PASTA: Pessimistic Assortment Optimization
  • Juncheng Dong, Weibin Mo, Zhengling Qi, Cong Shi, Ethan X Fang, and Vahid Tarokh. ICML, 2023.
• Contrastive Energy Prediction for Exact Energy-Guided Diffusion Sampling in Offline Reinforcement Learning
  • Cheng Lu, Huayu Chen, Jianfei Chen, Hang Su, Chongxuan Li, and Jun Zhu. ICML, 2023.
• Supported Trust Region Optimization for Offline Reinforcement Learning
  • Yixiu Mao, Hongchang Zhang, Chen Chen, Yi Xu, and Xiangyang Ji. ICML, 2023.
• Principled Offline RL in the Presence of Rich Exogenous Information
  • Riashat Islam, Manan Tomar, Alex Lamb, Yonathan Efroni, Hongyu Zang, Aniket Rajiv Didolkar, Dipendra Misra, Xin Li, Harm van Seijen, Remi Tachet des Combes, and John Langford. ICML, 2023.
• Efficient Online Reinforcement Learning with Offline Data
  • Philip J. Ball, Laura Smith, Ilya Kostrikov, and Sergey Levine. ICML, 2023.
• Boosting Offline Reinforcement Learning with Action Preference Query
  • Qisen Yang, Shenzhi Wang, Matthieu Gaetan Lin, Shiji Song, and Gao Huang. ICML, 2023.
• Model-based Offline Reinforcement Learning with Count-based Conservatism
  • Byeongchan Kim and Min-hwan Oh. ICML, 2023.
• Constrained Decision Transformer for Offline Safe Reinforcement Learning
  • Zuxin Liu, Zijian Guo, Yihang Yao, Zhepeng Cen, Wenhao Yu, Tingnan Zhang, and Ding Zhao. ICML, 2023.
• Model-Bellman Inconsistency for Model-based Offline Reinforcement Learning
  • Yihao Sun, Jiaji Zhang, Chengxing Jia, Haoxin Lin, Junyin Ye, and Yang Yu. ICML, 2023.
• Provably Efficient Offline Reinforcement Learning with Perturbed Data Sources
  • Chengshuai Shi, Wei Xiong, Cong Shen, and Jing Yang. ICML, 2023.
• What is Essential for Unseen Goal Generalization of Offline Goal-conditioned RL?
  • Rui Yang, Yong Lin, Xiaoteng Ma, Hao Hu, Chongjie Zhang, and Tong Zhang. ICML, 2023.
• Policy Regularization with Dataset Constraint for Offline Reinforcement Learning
  • Yuhang Ran, Yi-Chen Li, Fuxiang Zhang, Zongzhang Zhang, and Yang Yu. ICML, 2023.
• MetaDiffuser: Diffusion Model as Conditional Planner for Offline Meta-RL
  • Fei Ni, Jianye Hao, Yao Mu, Yifu Yuan, Yan Zheng, Bin Wang, and Zhixuan Liang. ICML, 2023.
• Distance Weighted Supervised Learning for Offline Interaction Data
  • Joey Hejna, Jensen Gao, and Dorsa Sadigh. ICML, 2023.
• Masked Trajectory Models for Prediction, Representation, and Control
  • Philipp Wu, Arjun Majumdar, Kevin Stone, Yixin Lin, Igor Mordatch, Pieter Abbeel, and Aravind Rajeswaran. ICML, 2023.
• Contrastive Energy Prediction for Exact Energy-Guided Diffusion Sampling in Offline Reinforcement Learning
  • Cheng Lu, Huayu Chen, Jianfei Chen, Hang Su, Chongxuan Li, and Jun Zhu. ICML, 2023.
• Bayesian Reparameterization of Reward-Conditioned Reinforcement Learning with Energy-based Models
  • Wenhao Ding, Tong Che, Ding Zhao, and Marco Pavone. ICML, 2023.
• Warm-Start Actor-Critic: From Approximation Error to Sub-optimality Gap
  • Hang Wang, Sen Lin, and Junshan Zhang. ICML, 2023.
• Future-conditioned Unsupervised Pretraining for Decision Transformer
  • Zhihui Xie, Zichuan Lin, Deheng Ye, Qiang Fu, Wei Yang, and Shuai Li. ICML, 2023.
• PAC-Bayesian Offline Contextual Bandits With Guarantees
  • Otmane Sakhi, Nicolas Chopin, and Pierre Alquier. ICML, 2023.
• Q-learning Decision Transformer: Leveraging Dynamic Programming for Conditional Sequence Modelling in Offline RL
  • Taku Yamagata, Ahmed Khalil, and Raul Santos-Rodriguez. ICML, 2023.
• Jump-Start Reinforcement Learning [website]
  • Ikechukwu Uchendu, Ted Xiao, Yao Lu, Banghua Zhu, Mengyuan Yan, Joséphine Simon, Matthew Bennice, Chuyuan Fu, Cong Ma, Jiantao Jiao, Sergey Levine, and Karol Hausman. ICML, 2023.
• Learning Temporally AbstractWorld Models without Online Experimentation
  • Benjamin Freed, Siddarth Venkatraman, Guillaume Adrien Sartoretti, Jeff Schneider, and Howie Choset. ICML, 2023.
• A Framework for Adapting Offline Algorithms to Solve Combinatorial Multi-Armed Bandit Problems with Bandit Feedback
  • Guanyu Nie, Yididiya Y Nadew, Yanhui Zhu, Vaneet Aggarwal, and Christopher John Quinn. ICML, 2023.
• Revisiting the Linear-Programming Framework for Offline RL with General Function Approximation
  • Asuman Ozdaglar, Sarath Pattathil, Jiawei Zhang, and Kaiqing Zhang. ICML, 2023.
• Semi-Supervised Offline Reinforcement Learning with Action-Free Trajectories
  • Qinqing Zheng, Mikael Henaff, Brandon Amos, and Aditya Grover. ICML, 2023.
• Actor-Critic Alignment for Offline-to-Online Reinforcement Learning
  • Zishun Yu and Xinhua Zhang. ICML, 2023.
• Leveraging Offline Data in Online Reinforcement Learning
  • Andrew Wagenmaker and Aldo Pacchiano. ICML, 2023.
• Offline Reinforcement Learning with Closed-Form Policy Improvement Operators
  • Jiachen Li, Edwin Zhang, Ming Yin, Qinxun Bai, Yu-Xiang Wang, and William Yang Wang. ICML, 2023.
• Offline Learning in Markov Games with General Function Approximation
  • Yuheng Zhang, Yu Bai, and Nan Jiang. ICML, 2023.
• Offline Meta Reinforcement Learning with In-Distribution Online Adaptation
  • Jianhao Wang, Jin Zhang, Haozhe Jiang, Junyu Zhang, Liwei Wang, and Chongjie Zhang. ICML, 2023.
• Scaling Pareto-Efficient Decision Making Via Offline Multi-Objective RL
  • Baiting Zhu, Meihua Dang, and Aditya Grover. ICLR, 2023.
• Confidence-Conditioned Value Functions for Offline Reinforcement Learning
  • Joey Hong, Aviral Kumar, and Sergey Levine. ICLR, 2023.
• Offline Q-Learning on Diverse Multi-Task Data Both Scales And Generalizes [website]
  • Aviral Kumar, Rishabh Agarwal, Xinyang Geng, George Tucker, and Sergey Levine. ICLR, 2023.
• Is Conditional Generative Modeling all you need for Decision-Making? [website]
  • Anurag Ajay, Yilun Du, Abhi Gupta, Joshua Tenenbaum, Tommi Jaakkola, and Pulkit Agrawal. ICLR, 2023
• Offline RL with No OOD Actions: In-Sample Learning via Implicit Value Regularization
  • Haoran Xu, Li Jiang, Jianxiong Li, Zhuoran Yang, Zhaoran Wang, Victor Wai Kin Chan, and Xianyuan Zhan. ICLR, 2023.
• Extreme Q-Learning: MaxEnt RL without Entropy
  • Divyansh Garg, Joey Hejna, Matthieu Geist, and Stefano Ermon. ICLR, 2023.
• Dichotomy of Control: Separating What You Can Control from What You Cannot
  • Mengjiao Yang, Dale Schuurmans, Pieter Abbeel, and Ofir Nachum. ICLR, 2023.
• From Play to Policy: Conditional Behavior Generation from Uncurated Robot Data
  • Zichen Jeff Cui, Yibin Wang, Nur Muhammad Mahi Shafiullah, and Lerrel Pinto. ICLR, 2023.
• VIPeR: Provably Efficient Algorithm for Offline RL with Neural Function Approximation
  • Thanh Nguyen-Tang and Raman Arora. ICLR, 2023.
• Optimal Conservative Offline RL with General Function Approximation via Augmented Lagrangian
  • Paria Rashidinejad, Hanlin Zhu, Kunhe Yang, Stuart Russell, and Jiantao Jiao. ICLR, 2023.
• The In-Sample Softmax for Offline Reinforcement Learning
  • Chenjun Xiao, Han Wang, Yangchen Pan, Adam White, and Martha White. ICLR, 2023.
• VIP: Towards Universal Visual Reward and Representation via Value-Implicit Pre-Training [website] [code]
  • Yecheng Jason Ma, Shagun Sodhani, Dinesh Jayaraman, Osbert Bastani, Vikash Kumar, and Amy Zhang. ICLR, 2023.
• Does Zero-Shot Reinforcement Learning Exist?
  • Ahmed Touati, Jérémy Rapin, and Yann Ollivier. ICLR, 2023.
• Behavior Prior Representation learning for Offline Reinforcement Learning
  • Hongyu Zang, Xin Li, Jie Yu, Chen Liu, Riashat Islam, Remi Tachet Des Combes, and Romain Laroche. ICLR, 2023.
• Mind the Gap: Offline Policy Optimization for Imperfect Rewards
  • Jianxiong Li, Xiao Hu, Haoran Xu, Jingjing Liu, Xianyuan Zhan, Qing-Shan Jia, and Ya-Qin Zhang. ICLR, 2023.
• Offline Congestion Games: How Feedback Type Affects Data Coverage Requirement
  • Haozhe Jiang, Qiwen Cui, Zhihan Xiong, Maryam Fazel, and Simon S. Du. ICLR, 2023.
• User-Interactive Offline Reinforcement Learning
  • Phillip Swazinna, Steffen Udluft, and Thomas Runkler. ICLR, 2023.
• Discovering Generalizable Multi-agent Coordination Skills from Multi-task Offline Data
  • Fuxiang Zhang, Chengxing Jia, Yi-Chen Li, Lei Yuan, Yang Yu, and Zongzhang Zhang. ICLR, 2023.
• Hybrid RL: Using Both Offline and Online Data Can Make RL Efficient [code]
  • Yuda Song, Yifei Zhou, Ayush Sekhari, J. Andrew Bagnell, Akshay Krishnamurthy, and Wen Sun. ICLR, 2023.
• Harnessing Mixed Offline Reinforcement Learning Datasets via Trajectory Weighting
  • Zhang-Wei Hong, Pulkit Agrawal, Remi Tachet des Combes, and Romain Laroche. ICLR, 2023.
• Efficient Offline Policy Optimization with a Learned Model
  • Zichen Liu, Siyi Li, Wee Sun Lee, Shuicheng Yan, and Zhongwen Xu. ICLR, 2023.
• Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learning
  • Zhendong Wang, Jonathan J Hunt, and Mingyuan Zhou. ICLR, 2023.
• When Data Geometry Meets Deep Function: Generalizing Offline Reinforcement Learning
  • Jianxiong Li, Xianyuan Zhan, Haoran Xu, Xiangyu Zhu, Jingjing Liu, and Ya-Qin Zhang. ICLR, 2023.
• In-sample Actor Critic for Offline Reinforcement Learning
  • Hongchang Zhang, Yixiu Mao, Boyuan Wang, Shuncheng He, Yi Xu, and Xiangyang Ji. ICLR, 2023.
• Value Memory Graph: A Graph-Structured World Model for Offline Reinforcement Learning
  • Deyao Zhu, Li Erran Li, and Mohamed Elhoseiny. ICLR, 2023.
• Conservative Bayesian Model-Based Value Expansion for Offline Policy Optimization
  • Jihwan Jeong, Xiaoyu Wang, Michael Gimelfarb, Hyunwoo Kim, Baher Abdulhai, and Scott Sanner. ICLR, 2023.
• Offline Reinforcement Learning via High-Fidelity Generative Behavior Modeling
  • Huayu Chen, Cheng Lu, Chengyang Ying, Hang Su, and Jun Zhu. ICLR, 2023.
• Offline Reinforcement Learning with Differentiable Function Approximation is Provably Efficient
  • Ming Yin, Mengdi Wang, and Yu-Xiang Wang. ICLR, 2023.
• Nearly Minimax Optimal Offline Reinforcement Learning with Linear Function Approximation: Single-Agent MDP and Markov Game
  • Wei Xiong, Han Zhong, Chengshuai Shi, Cong Shen, Liwei Wang, and Tong Zhang. ICLR, 2023.
• Pessimism in the Face of Confounders: Provably Efficient Offline Reinforcement Learning in Partially Observable Markov Decision Processes
  • Miao Lu, Yifei Min, Zhaoran Wang, and Zhuoran Yang. ICLR, 2023.
• Hyper-Decision Transformer for Efficient Online Policy Adaptation
  • Mengdi Xu, Yuchen Lu, Yikang Shen, Shun Zhang, Ding Zhao, and Chuang Gan. ICLR, 2023.
• Efficient Planning in a Compact Latent Action Space
  • Zhengyao Jiang, Tianjun Zhang, Michael Janner, Yueying Li, Tim Rocktäschel, Edward Grefenstette, and Yuandong Tian. ICLR, 2023.
• Preference Transformer: Modeling Human Preferences using Transformers for RL [website]
  • Changyeon Kim, Jongjin Park, Jinwoo Shin, Honglak Lee, Pieter Abbeel, and Kimin Lee. ICLR, 2023.
• Behavior Proximal Policy Optimization
  • Zifeng Zhuang, Kun Lei, Jinxin Liu, Donglin Wang, and Yilang Guo. ICLR, 2023.
• Provably Efficient Neural Offline Reinforcement Learning via Perturbed Rewards
  • Thanh Nguyen-Tang and Raman Arora. ICLR, 2023.
• The Provable Benefits of Unsupervised Data Sharing for Offline Reinforcement Learning
  • Hao Hu, Yiqin Yang, Qianchuan Zhao, and Chongjie Zhang. ICLR, 2023.
• Decision Transformer under Random Frame Dropping
  • Kaizhe Hu, Ray Chen Zheng, Yang Gao, and Huazhe Xu. ICLR, 2023.
• Policy Expansion for Bridging Offline-to-Online Reinforcement Learning
  • Haichao Zhang, We Xu, and Haonan Yu. ICLR, 2023.
• Finetuning Offline World Models in the Real World
  • Yunhai Feng, Nicklas Hansen, Ziyan Xiong, Chandramouli Rajagopalan, and Xiaolong Wang. CoRL, 2023.
• On the Sample Complexity of Vanilla Model-Based Offline Reinforcement Learning with Dependent Samples
  • Mustafa O. Karabag and Ufuk Topcu. AAAI, 2023.
• Adaptive Policy Learning for Offline-to-Online Reinforcement Learning
  • Han Zheng, Xufang Luo, Pengfei Wei, Xuan Song, Dongsheng Li, and Jing Jiang. AAAI, 2023.
• Safe Policy Improvement for POMDPs via Finite-State Controllers
  • Thiago D. Simão, Marnix Suilen, and Nils Jansen. AAAI, 2023.
• Behavior Estimation from Multi-Source Data for Offline Reinforcement Learning
  • Guoxi Zhang and Hisashi Kashima. AAAI, 2023.
• On Instance-Dependent Bounds for Offline Reinforcement Learning with Linear Function Approximation
  • Thanh Nguyen-Tang, Ming Yin, Sunil Gupta, Svetha Venkatesh, and Raman Arora. AAAI, 2023.
• Contrastive Example-Based Control
  • Kyle Hatch, Benjamin Eysenbach, Rafael Rafailov, Tianhe Yu, Ruslan Salakhutdinov, Sergey Levine, and Chelsea Finn. LDC, 2023.
• Curriculum Offline Reinforcement Learning
  • Yuanying Cai, Chuheng Zhang, Hanye Zhao, Li Zhao, and Jiang Bian. AAMAS. 2023.
• Offline Reinforcement Learning with On-Policy Q-Function Regularization
  • Laixi Shi, Robert Dadashi, Yuejie Chi, Pablo Samuel Castro, and Matthieu Geist. ECML, 2023.
• Model-based Offline Policy Optimization with Adversarial Network
  • Junming Yang, Xingguo Chen, Shengyuan Wang, and Bolei Zhang. ECAI, 2023.
• Efficient experience replay architecture for offline reinforcement learning
  • Longfei Zhang, Yanghe Feng, Rongxiao Wang, Yue Xu, Naifu Xu, Zeyi Liu, and Hang Du. RIA, 2023.
• Automatic Trade-off Adaptation in Offline RL
  • Phillip Swazinna, Steffen Udluft, and Thomas Runkler. ESANN, 2023.
• Offline Robot Reinforcement Learning with Uncertainty-Guided Human Expert Sampling
  • Ashish Kumar and Ilya Kuzovkin. arXiv, 2022.
• Latent Variable Representation for Reinforcement Learning
  • Tongzheng Ren, Chenjun Xiao, Tianjun Zhang, Na Li, Zhaoran Wang, Sujay Sanghavi, Dale Schuurmans, and Bo Dai. arXiv, 2022.
• Learning From Good Trajectories in Offline Multi-Agent Reinforcement Learning
  • Qi Tian, Kun Kuang, Furui Liu, and Baoxiang Wang. arXiv, 2022.
• State-Aware Proximal Pessimistic Algorithms for Offline Reinforcement Learning
  • Chen Chen, Hongyao Tang, Yi Ma, Chao Wang, Qianli Shen, Dong Li, and Jianye Hao. arXiv, 2022.
• Masked Autoencoding for Scalable and Generalizable Decision Making
  • Fangchen Liu, Hao Liu, Aditya Grover, and Pieter Abbeel. arXiv, 2022.
• Improving TD3-BC: Relaxed Policy Constraint for Offline Learning and Stable Online Fine-Tuning
  • Alex Beeson and Giovanni Montana. arXiv, 2022.
• Q-Ensemble for Offline RL: Don't Scale the Ensemble, Scale the Batch Size
  • Alexander Nikulin, Vladislav Kurenkov, Denis Tarasov, Dmitry Akimov, and Sergey Kolesnikov. arXiv, 2022.
• Let Offline RL Flow: Training Conservative Agents in the Latent Space of Normalizing Flows
  • Dmitriy Akimov, Vladislav Kurenkov, Alexander Nikulin, Denis Tarasov, and Sergey Kolesnikov. arXiv, 2022.
• Model-based Trajectory Stitching for Improved Offline Reinforcement Learning
  • Charles A. Hepburn and Giovanni Montana. arXiv, 2022.
• Offline Reinforcement Learning with Adaptive Behavior Regularization
  • Yunfan Zhou, Xijun Li, and Qingyu Qu. arXiv, 2022.
• Contextual Transformer for Offline Meta Reinforcement Learning
  • Runji Lin, Ye Li, Xidong Feng, Zhaowei Zhang, Xian Hong Wu Fung, Haifeng Zhang, Jun Wang, Yali Du, and Yaodong Yang. arXiv, 2022.
• Wall Street Tree Search: Risk-Aware Planning for Offline Reinforcement Learning
  • Dan Elbaz, Gal Novik, and Oren Salzman. arXiv, 2022.
• ARMOR: A Model-based Framework for Improving Arbitrary Baseline Policies with Offline Data
  • Tengyang Xie, Mohak Bhardwaj, Nan Jiang, and Ching-An Cheng. arXiv, 2022.
• Contrastive Value Learning: Implicit Models for Simple Offline RL
  • Bogdan Mazoure, Benjamin Eysenbach, Ofir Nachum, and Jonathan Tompson. arXiv, 2022.
• Optimistic Curiosity Exploration and Conservative Exploitation with Linear Reward Shaping
  • Hao Sun, Lei Han, Rui Yang, Xiaoteng Ma, Jian Guo, and Bolei Zhou. arXiv, 2022.
• Optimal Conservative Offline RL with General Function Approximation via Augmented Lagrangian
  • Paria Rashidinejad, Hanlin Zhu, Kunhe Yang, Stuart Russell, and Jiantao Jiao. ICLR, 2023.
• Agent-Controller Representations: Principled Offline RL with Rich Exogenous Information
  • Riashat Islam, Manan Tomar, Alex Lamb, Yonathan Efroni, Hongyu Zang, Aniket Didolkar, Dipendra Misra, Xin Li, Harm van Seijen, Remi Tachet des Combes, and John Langford. arXiv, 2022.
• Provable Safe Reinforcement Learning with Binary Feedback
  • Andrew Bennett, Dipendra Misra, and Nathan Kallus. arXiv, 2022.
• Learning on the Job: Self-Rewarding Offline-to-Online Finetuning for Industrial Insertion of Novel Connectors from Vision
  • Ashvin Nair, Brian Zhu, Gokul Narayanan, Eugen Solowjow, and Sergey Levine. arXiv, 2022.
• Implicit Offline Reinforcement Learning via Supervised Learning
  • Alexandre Piche, Rafael Pardinas, David Vazquez, Igor Mordatch, and Chris Pal. arXiv, 2022.
• Robust Offline Reinforcement Learning with Gradient Penalty and Constraint Relaxation
  • Chengqian Gao, Ke Xu, Liu Liu, Deheng Ye, Peilin Zhao, and Zhiqiang Xu. arXiv, 2022.
• Boosting Offline Reinforcement Learning via Data Rebalancing
  • Yang Yue, Bingyi Kang, Xiao Ma, Zhongwen Xu, Gao Huang, and Shuicheng Yan. arXiv, 2022.
• ConserWeightive Behavioral Cloning for Reliable Offline Reinforcement Learning [code]
  • Tung Nguyen, Qinqing Zheng, and Aditya Grover. arXiv, 2022.
• State Advantage Weighting for Offline RL
  • Jiafei Lyu, Aicheng Gong, Le Wan, Zongqing Lu, and Xiu Li. arXiv, 2022.
• Blessing from Experts: Super Reinforcement Learning in Confounded Environments
  • Jiayi Wang, Zhengling Qi, and Chengchun Shi. arXiv, 2022.
• DCE: Offline Reinforcement Learning With Double Conservative Estimates
  • Chen Zhao, Kai Xing Huang, and Chun Yuan. arXiv, 2022.
• On the Opportunities and Challenges of using Animals Videos in Reinforcement Learning
  • Vittorio Giammarino. arXiv, 2022.
• Offline Reinforcement Learning with Instrumental Variables in Confounded Markov Decision Processes
  • Zuyue Fu, Zhengling Qi, Zhaoran Wang, Zhuoran Yang, Yanxun Xu, and Michael R. Kosorok. arXiv, 2022.
• Exploiting Reward Shifting in Value-Based Deep RL
  • Hao Sun, Lei Han, Rui Yang, Xiaoteng Ma, Jian Guo, and Bolei Zhou. arXiv, 2022.
• Distributionally Robust Offline Reinforcement Learning with Linear Function Approximation
  • Xiaoteng Ma, Zhipeng Liang, Li Xia, Jiheng Zhang, Jose Blanchet, Mingwen Liu, Qianchuan Zhao, and Zhengyuan Zhou. arXiv, 2022.
• C^2:Co-design of Robots via Concurrent Networks Coupling Online and Offline Reinforcement Learning
  • Ci Chen, Pingyu Xiang, Haojian Lu, Yue Wang, and Rong Xiong. arXiv, 2022.
• Strategic Decision-Making in the Presence of Information Asymmetry: Provably Efficient RL with Algorithmic Instruments
  • Mengxin Yu, Zhuoran Yang, and Jianqing Fan. arXiv, 2022.
• Distributionally Robust Model-Based Offline Reinforcement Learning with Near-Optimal Sample Complexity
  • Laixi Shi and Yuejie Chi. arXiv, 2022.
• AdaCat: Adaptive Categorical Discretization for Autoregressive Models
  • Qiyang Li, Ajay Jain, and Pieter Abbeel. arXiv, 2022.
• Branch Ranking for Efficient Mixed-Integer Programming via Offline Ranking-based Policy Learning
  • Zeren Huang, Wenhao Chen, Weinan Zhang, Chuhan Shi, Furui Liu, Hui-Ling Zhen, Mingxuan Yuan, Jianye Hao, Yong Yu, and Jun Wang. arXiv, 2022.
• Offline Reinforcement Learning at Multiple Frequencies [webpage]
  • Kaylee Burns, Tianhe Yu, Chelsea Finn, and Karol Hausman. arXiv, 2022.
• General Policy Evaluation and Improvement by Learning to Identify Few But Crucial States
  • Francesco Faccio, Aditya Ramesh, Vincent Herrmann, Jean Harb, and Jürgen Schmidhuber. arXiv, 2022.
• Behavior Transformers: Cloning k modes with one stone
  • Nur Muhammad Mahi Shafiullah, Zichen Jeff Cui, Ariuntuya Altanzaya, and Lerrel Pinto. arXiv, 2022.
• Contrastive Learning as Goal-Conditioned Reinforcement Learning
  • Benjamin Eysenbach, Tianjun Zhang, Ruslan Salakhutdinov, and Sergey Levine. arXiv, 2022.
• Federated Offline Reinforcement Learning
  • Doudou Zhou, Yufeng Zhang, Aaron Sonabend-W, Zhaoran Wang, Junwei Lu, and Tianxi Cai. arXiv, 2022.
• Provable Benefit of Multitask Representation Learning in Reinforcement Learning
  • Yuan Cheng, Songtao Feng, Jing Yang, Hong Zhang, and Yingbin Liang. arXiv, 2022
• Provably Efficient Offline Reinforcement Learning with Trajectory-Wise Reward
  • Tengyu Xu and Yingbin Liang. arXiv, 2022.
• Model-Based Reinforcement Learning Is Minimax-Optimal for Offline Zero-Sum Markov Games
  • Yuling Yan, Gen Li, Yuxin Chen, and Jianqing Fan. arXiv, 2022.
• Offline Reinforcement Learning with Causal Structured World Models
  • Zheng-Mao Zhu, Xiong-Hui Chen, Hong-Long Tian, Kun Zhang, and Yang Yu. arXiv, 2022.
• Incorporating Explicit Uncertainty Estimates into Deep Offline Reinforcement Learning
  • David Brandfonbrener, Remi Tachet des Combes, and Romain Laroche. arXiv, 2022.
• Know Your Boundaries: The Necessity of Explicit Behavioral Cloning in Offline RL
  • Wonjoon Goo and Scott Niekum. arXiv, 2022.
• Byzantine-Robust Online and Offline Distributed Reinforcement Learning
  • Yiding Chen, Xuezhou Zhang, Kaiqing Zhang, Mengdi Wang, and Xiaojin Zhu. arXiv, 2022.
• Model Generation with Provable Coverability for Offline Reinforcement Learning
  • Chengxing Jia, Hao Yin, Chenxiao Gao, Tian Xu, Lei Yuan, Zongzhang Zhang, and Yang Yu. arXiv, 2022.
• You Can't Count on Luck: Why Decision Transformers Fail in Stochastic Environments
  • Keiran Paster, Sheila McIlraith, and Jimmy Ba. arXiv, 2022.
• Multi-Game Decision Transformers
  • Kuang-Huei Lee, Ofir Nachum, Mengjiao Yang, Lisa Lee, Daniel Freeman, Winnie Xu, Sergio Guadarrama, Ian Fischer, Eric Jang, Henryk Michalewski, and Igor Mordatch. arXiv, 2022.
• Hierarchical Planning Through Goal-Conditioned Offline Reinforcement Learning
  • Jinning Li, Chen Tang, Masayoshi Tomizuka, and Wei Zhan. arXiv, 2022.
• Distance-Sensitive Offline Reinforcement Learning
  • Jianxiong Li, Xianyuan Zhan, Haoran Xu, Xiangyu Zhu, Jingjing Liu, and Ya-Qin Zhang. arXiv, 2022.
• No More Pesky Hyperparameters: Offline Hyperparameter Tuning for RL
  • Han Wang, Archit Sakhadeo, Adam White, James Bell, Vincent Liu, Xutong Zhao, Puer Liu, Tadashi Kozuno, Alona Fyshe, and Martha White. arXiv, 2022.
• How to Spend Your Robot Time: Bridging Kickstarting and Offline Reinforcement Learning for Vision-based Robotic Manipulation
  • Alex X. Lee, Coline Devin, Jost Tobias Springenberg, Yuxiang Zhou, Thomas Lampe, Abbas Abdolmaleki, and Konstantinos Bousmalis. arXiv, 2022.
• Offline Visual Representation Learning for Embodied Navigation
  • Karmesh Yadav, Ram Ramrakhya, Arjun Majumdar, Vincent-Pierre Berges, Sachit Kuhar, Dhruv Batra, Alexei Baevski, and Oleksandr Maksymets. arXiv, 2022.
• Towards Flexible Inference in Sequential Decision Problems via Bidirectional Transformers
  • Micah Carroll, Jessy Lin, Orr Paradise, Raluca Georgescu, Mingfei Sun, David Bignell, Stephanie Milani, Katja Hofmann, Matthew Hausknecht, Anca Dragan, and Sam Devlin. arXiv, 2022.
• BATS: Best Action Trajectory Stitching
  • Ian Char, Viraj Mehta, Adam Villaflor, John M. Dolan, Jeff Schneider. arXiv, 2022.
• Settling the Sample Complexity of Model-Based Offline Reinforcement Learning
  • Gen Li, Laixi Shi, Yuxin Chen, Yuejie Chi, and Yuting Wei. arXiv, 2022.
• PAnDR: Fast Adaptation to New Environments from Offline Experiences via Decoupling Policy and Environment Representations
  • Tong Sang, Hongyao Tang, Yi Ma, Jianye Hao, Yan Zheng, Zhaopeng Meng, Boyan Li, and Zhen Wang. arXiv, 2022.
• Offline Reinforcement Learning Under Value and Density-Ratio Realizability: the Power of Gaps
  • Jinglin Chen and Nan Jiang. arXiv, 2022.
• Meta Reinforcement Learning for Adaptive Control: An Offline Approach
  • Daniel G. McClement, Nathan P. Lawrence, Johan U. Backstrom, Philip D. Loewen, Michael G. Forbes, and R. Bhushan Gopaluni. arXiv, 2022.
• The Efficacy of Pessimism in Asynchronous Q-Learning
  • Yuling Yan, Gen Li, Yuxin Chen, and Jianqing Fan. arXiv, 2022.
• Reinforcement Learning for Linear Quadratic Control is Vulnerable Under Cost Manipulation
  • Yunhan Huang and Quanyan Zhu. arXiv, 2022.
• A Regularized Implicit Policy for Offline Reinforcement Learning
  • Shentao Yang, Zhendong Wang, Huangjie Zheng, Yihao Feng, and Mingyuan Zhou. arXiv, 2022.
• Reinforcement Learning in Possibly Nonstationary Environments [code]
  • Mengbing Li, Chengchun Shi, Zhenke Wu, and Piotr Fryzlewicz. arXiv, 2022.
• Statistically Efficient Advantage Learning for Offline Reinforcement Learning in Infinite Horizons
  • Chengchun Shi, Shikai Luo, Hongtu Zhu, and Rui Song. arXiv, 2022.
• VRL3: A Data-Driven Framework for Visual Deep Reinforcement Learning
  • Che Wang, Xufang Luo, Keith Ross, and Dongsheng Li. arXiv, 2022.
• Retrieval-Augmented Reinforcement Learning
  • Anirudh Goyal, Abram L. Friesen, Andrea Banino, Theophane Weber, Nan Rosemary Ke, Adria Puigdomenech Badia, Arthur Guez, Mehdi Mirza, Ksenia Konyushkova, Michal Valko, Simon Osindero, Timothy Lillicrap, Nicolas Heess, and Charles Blundell. arXiv, 2022.
• Online Decision Transformer
  • Qinqing Zheng, Amy Zhang, and Aditya Grover. arXiv, 2022.
• Transferred Q-learning
  • Elynn Y. Chen, Michael I. Jordan, and Sai Li. arXiv, 2022.
• Settling the Communication Complexity for Distributed Offline Reinforcement Learning
  • Juliusz Krysztof Ziomek, Jun Wang, and Yaodong Yang. arXiv, 2022.
• Offline Reinforcement Learning with Realizability and Single-policy Concentrability
  • Wenhao Zhan, Baihe Huang, Audrey Huang, Nan Jiang, and Jason D. Lee. arXiv, 2022.
• Rethinking Goal-conditioned Supervised Learning and Its Connection to Offline RL
  • Rui Yang, Yiming Lu, Wenzhe Li, Hao Sun, Meng Fang, Yali Du, Xiu Li, Lei Han, and Chongjie Zhang. arXiv, 2022.
• Stochastic Gradient Descent with Dependent Data for Offline Reinforcement Learning
  • Jing Dong and Xin T. Tong. arXiv, 2022.
• Can Wikipedia Help Offline Reinforcement Learning?
  • Machel Reid, Yutaro Yamada, and Shixiang Shane Gu. arXiv, 2022.
• MOORe: Model-based Offline-to-Online Reinforcement Learning
  • Yihuan Mao, Chao Wang, Bin Wang, and Chongjie Zhang. arXiv, 2022.
• Operator Deep Q-Learning: Zero-Shot Reward Transferring in Reinforcement Learning
  • Ziyang Tang, Yihao Feng, and Qiang Liu. arXiv, 2022.
• Importance of Empirical Sample Complexity Analysis for Offline Reinforcement Learning
  • Samin Yeasar Arnob, Riashat Islam, and Doina Precup. arXiv, 2022.
• Single-Shot Pruning for Offline Reinforcement Learning
  • Samin Yeasar Arnob, Riyasat Ohib, Sergey Plis, and Doina Precup. arXiv, 2022.
• Monte Carlo Augmented Actor-Critic for Sparse Reward Deep Reinforcement Learning from Suboptimal Demonstrations [website] [code]
  • Albert Wilcox, Ashwin Balakrishna, Jules Dedieu, Wyame Benslimane, Daniel S. Brown, and Ken Goldberg. NeurIPS, 2022.
• Data-Driven Offline Decision-Making via Invariant Representation Learning
  • Han Qi, Yi Su, Aviral Kumar, and Sergey Levine. NeurIPS, 2022.
• Bellman Residual Orthogonalization for Offline Reinforcement Learning
  • Andrea Zanette, and Martin J. Wainwright. NeurIPS, 2022.
• A Near-Optimal Primal-Dual Method for Off-Policy Learning in CMDP
  • Fan Chen, Junyu Zhang, and Zaiwen Wen. NeurIPS, 2022.
• RORL: Robust Offline Reinforcement Learning via Conservative Smoothing
  • Rui Yang, Chenjia Bai, Xiaoteng Ma, Zhaoran Wang, Chongjie Zhang, and Lei Han. NeurIPS, 2022.
• On Gap-dependent Bounds for Offline Reinforcement Learning
  • Xinqi Wang, Qiwen Cui, and Simon S. Du. NeurIPS, 2022.
• Provably Efficient Offline Multi-agent Reinforcement Learning via Strategy-wise Bonus
  • Qiwen Cui and Simon S. Du. NeurIPS, 2022.
• Supported Policy Optimization for Offline Reinforcement Learning
  • Jialong Wu, Haixu Wu, Zihan Qiu, Jianmin Wang, and Mingsheng Long. NeurIPS, 2022.
• When to Trust Your Simulator: Dynamics-Aware Hybrid Offline-and-Online Reinforcement Learning
  • Haoyi Niu, Shubham Sharma, Yiwen Qiu, Ming Li, Guyue Zhou, Jianming Hu, and Xianyuan Zhan. NeurIPS, 2022.
• Why So Pessimistic? Estimating Uncertainties for Offline RL through Ensembles, and Why Their Independence Matters
  • Seyed Kamyar Seyed Ghasemipour, Shixiang Shane Gu, and Ofir Nachum. NeurIPS, 2022.
• When does return-conditioned supervised learning work for offline reinforcement learning?
  • David Brandfonbrener, Alberto Bietti, Jacob Buckman, Romain Laroche, and Joan Bruna. NeurIPS, 2022.
• Pessimism for Offline Linear Contextual Bandits using ℓp Confidence Sets
  • Gene Li, Cong Ma, and Nathan Srebro. NeurIPS, 2022.
• RAMBO-RL: Robust Adversarial Model-Based Offline Reinforcement Learning
  • Marc Rigter, Bruno Lacerda, and Nick Hawes. NeurIPS, 2022.
• When is Offline Two-Player Zero-Sum Markov Game Solvable?
  • Qiwen Cui, and Simon S. Du. NeurIPS, 2022.
• Robust Reinforcement Learning using Offline Data
  • Kishan Panaganti, Zaiyan Xu, Dileep Kalathil, and Mohammad Ghavamzadeh. NeurIPS, 2022.
• Bidirectional Learning for Offline Infinite-width Model-based Optimization
  • Can Chen, Yingxue Zhang, Jie Fu, Xue Liu, and Mark Coates. NeurIPS, 2022.
• Mildly Conservative Q-Learning for Offline Reinforcement Learning
  • Jiafei Lyu, Xiaoteng Ma, Xiu Li, and Zongqing Lu. NeurIPS, 2022.
• Bootstrapped Transformer for Offline Reinforcement Learning
  • Kerong Wang, Hanye Zhao, Xufang Luo, Kan Ren, Weinan Zhang, and Dongsheng Li. NeurIPS, 2022.
• LobsDICE: Offline Learning from Observation via Stationary Distribution Correction Estimation
  • Geon-Hyeong Kim, Jongmin Lee, Youngsoo Jang, Hongseok Yang, and Kee-Eung Kim. NeurIPS, 2022.
• Latent-Variable Advantage-Weighted Policy Optimization for Offline RL
  • Xi Chen, Ali Ghadirzadeh, Tianhe Yu, Yuan Gao, Jianhao Wang, Wenzhe Li, Bin Liang, Chelsea Finn, and Chongjie Zhang. NeurIPS, 2022.
• Double Check Your State Before Trusting It: Confidence-Aware Bidirectional Offline Model-Based Imagination
  • Jiafei Lyu, Xiu Li, and Zongqing Lu. NeurIPS, 2022.
• Improving Zero-shot Generalization in Offline Reinforcement Learning using Generalized Similarity Functions
  • Bogdan Mazoure, Ilya Kostrikov, Ofir Nachum, and Jonathan Tompson. NeurIPS, 2022.
• Offline Goal-Conditioned Reinforcement Learning via f-Advantage Regression
  • Yecheng Jason Ma, Jason Yan, Dinesh Jayaraman, and Osbert Bastani. NeurIPS, 2022.
• Dual Generator Offline Reinforcement Learning
  • Quan Vuong, Aviral Kumar, Sergey Levine, and Yevgen Chebotar. NeurIPS, 2022.
• MoCoDA: Model-based Counterfactual Data Augmentation
  • Silviu Pitis, Elliot Creager, Ajay Mandlekar, and Animesh Garg. NeurIPS, 2022.
• A Policy-Guided Imitation Approach for Offline Reinforcement Learning [code]
  • Haoran Xu, Li Jiang, Jianxiong Li, and Xianyuan Zhan. NeurIPS, 2022.
• A Unified Framework for Alternating Offline Model Training and Policy Learning
  • Shentao Yang, Shujian Zhang, Yihao Feng, and Mingyuan Zhou. NeurIPS, 2022.
• Model-Based Offline Reinforcement Learning with Pessimism-Modulated Dynamics Belief
  • Kaiyang Guo, Yunfeng Shao, and Yanhui Geng. NeurIPS, 2022.
• S2P: State-conditioned Image Synthesis for Data Augmentation in Offline Reinforcement Learning
  • Daesol Cho, Dongseok Shim, and H. Jin Kim. NeurIPS, 2022.
• ASPiRe:Adaptive Skill Priors for Reinforcement Learning
  • Mengda Xu, Manuela Veloso, and Shuran Song. NeurIPS, 2022.
• Skills Regularized Task Decomposition for Multi-task Offline Reinforcement Learning
  • Minjong Yoo, Sangwoo Cho, and Honguk Woo. NeurIPS, 2022.
• Offline Multi-Agent Reinforcement Learning with Knowledge Distillation
  • Wei-Cheng Tseng, Tsun-Hsuan Wang, Yen-Chen Lin, and Phillip Isola. NeurIPS, 2022.
• Shadow Knowledge Distillation: Bridging Offline and Online Knowledge Transfer
  • Lujun Li and Zhe Jin. NeurIPS, 2022.
• Addressing Optimism Bias in Sequence Modeling for Reinforcement Learning
  • Adam Villaflor, Zhe Huang, Swapnil Pande, John Dolan, and Jeff Schneider. ICML, 2022.
• Offline RL Policies Should be Trained to be Adaptive
  • Dibya Ghosh, Anurag Ajay, Pulkit Agrawal, and Sergey Levine. ICML, 2022.
• Adversarially Trained Actor Critic for Offline Reinforcement Learning
  • Ching-An Cheng, Tengyang Xie, Nan Jiang, and Alekh Agarwal. ICML, 2022.
• Pessimistic Minimax Value Iteration: Provably Efficient Equilibrium Learning from Offline Datasets
  • Han Zhong, Wei Xiong, Jiyuan Tan, Liwei Wang, Tong Zhang, Zhaoran Wang, and Zhuoran Yang. ICML, 2022.
• How to Leverage Unlabeled Data in Offline Reinforcement Learning
  • Tianhe Yu, Aviral Kumar, Yevgen Chebotar, Karol Hausman, Chelsea Finn, and Sergey Levine. ICML, 2022.
• Plan Better Amid Conservatism: Offline Multi-Agent Reinforcement Learning with Actor Rectification
  • Ling Pan, Longbo Huang, Tengyu Ma, and Huazhe Xu. ICML, 2022.
• Learning Pseudometric-based Action Representations for Offline Reinforcement Learning
  • Pengjie Gu, Mengchen Zhao, Chen Chen, Dong Li, Jianye Hao, and Bo An. ICML, 2022.
• Offline Meta-Reinforcement Learning with Online Self-Supervision
  • Vitchyr H. Pong, Ashvin Nair, Laura Smith, Catherine Huang, and Sergey Levine. ICML, 2022.
• Versatile Offline Imitation from Observations and Examples via Regularized State-Occupancy Matching
  • Yecheng Jason Ma, Andrew Shen, Dinesh Jayaraman, and Osbert Bastani. ICML, 2022.
• Constrained Offline Policy Optimization
  • Nicholas Polosky, Bruno C. Da Silva, Madalina Fiterau, and Jithin Jagannath. ICML, 2022.
• Discriminator-Weighted Offline Imitation Learning from Suboptimal Demonstrations
  • Haoran Xu, Xianyuan Zhan, Honglei Yin, and Huiling Qin. ICML, 2022.
• Provably Efficient Offline Reinforcement Learning for Partially Observable Markov Decision Processes
  • Hongyi Guo, Qi Cai, Yufeng Zhang, Zhuoran Yang, and Zhaoran Wang. ICML, 2022.
• Pessimistic Q-Learning for Offline Reinforcement Learning: Towards Optimal Sample Complexity
  • Laixi Shi, Gen Li, Yuting Wei, Yuxin Chen, and Yuejie Chi. ICML, 2022.
• Efficient Reinforcement Learning in Block MDPs: A Model-free Representation Learning Approach
  • Xuezhou Zhang, Yuda Song, Masatoshi Uehara, Mengdi Wang, Alekh Agarwal, and Wen Sun. ICML, 2022.
• Prompting Decision Transformer for Few-Shot Policy Generalization
  • Mengdi Xu, Yikang Shen, Shun Zhang, Yuchen Lu, Ding Zhao, Joshua B. Tenenbaum, and Chuang Gan. ICML, 2022.
• Regularizing a Model-based Policy Stationary Distribution to Stabilize Offline Reinforcement Learning
  • Shentao Yang, Yihao Feng, Shujian Zhang, and Mingyuan Zhou. ICML, 2022.
• On the Role of Discount Factor in Offline Reinforcement Learning
  • Hao Hu, Yiqin Yang, Qianchuan Zhao, and Chongjie Zhang. ICML, 2022.
• Koopman Q-learning: Offline Reinforcement Learning via Symmetries of Dynamics
  • Matthias Weissenbacher, Samarth Sinha, Animesh Garg, and Yoshinobu Kawahara. ICML, 2022.
• Representation Learning for Online and Offline RL in Low-rank MDPs [video]
  • Masatoshi Uehara, Xuezhou Zhang, and Wen Sun. ICLR, 2022.
• Pessimistic Model-based Offline Reinforcement Learning under Partial Coverage [video]
  • Masatoshi Uehara and Wen Sun. ICLR, 2022.
• Revisiting Design Choices in Model-Based Offline Reinforcement Learning
  • Cong Lu, Philip J. Ball, Jack Parker-Holder, Michael A. Osborne, and Stephen J. Roberts. ICLR, 2022.
• DR3: Value-Based Deep Reinforcement Learning Requires Explicit Regularization
  • Aviral Kumar, Rishabh Agarwal, Tengyu Ma, Aaron Courville, George Tucker, and Sergey Levine. ICLR, 2022.
• COptiDICE: Offline Constrained Reinforcement Learning via Stationary Distribution Correction Estimation
  • Jongmin Lee, Cosmin Paduraru, Daniel J. Mankowitz, Nicolas Heess, Doina Precup, Kee-Eung Kim, and Arthur Guez. ICLR, 2022.
• POETREE: Interpretable Policy Learning with Adaptive Decision Trees
  • Alizée Pace, Alex J. Chan, and Mihaela van der Schaar. ICLR, 2022.
• Planning in Stochastic Environments with a Learned Model
  • Ioannis Antonoglou, Julian Schrittwieser, Sherjil Ozair, Thomas K Hubert, and David Silver. ICLR, 2022.
• Offline Reinforcement Learning with Value-based Episodic Memory
  • Xiaoteng Ma, Yiqin Yang, Hao Hu, Qihan Liu, Jun Yang, Chongjie Zhang, Qianchuan Zhao, and Bin Liang. ICLR, 2022.
• When Should We Prefer Offline Reinforcement Learning Over Behavioral Cloning?
  • Aviral Kumar, Joey Hong, Anikait Singh, and Sergey Levine. ICLR, 2022.
• Learning Value Functions from Undirected State-only Experience [website] [code]
  • Matthew Chang, Arjun Gupta, and Saurabh Gupta. ICLR, 2022.
• Rethinking Goal-Conditioned Supervised Learning and Its Connection to Offline RL
  • Rui Yang, Yiming Lu, Wenzhe Li, Hao Sun, Meng Fang, Yali Du, Xiu Li, Lei Han, and Chongjie Zhang. ICLR, 2022.
• Offline Reinforcement Learning with Implicit Q-Learning
  • Ilya Kostrikov, Ashvin Nair, and Sergey Levine. ICLR, 2022.
• RvS: What is Essential for Offline RL via Supervised Learning?
  • Scott Emmons, Benjamin Eysenbach, Ilya Kostrikov, and Sergey Levine. ICLR, 2022.
• Pareto Policy Pool for Model-based Offline Reinforcement Learning
  • Yijun Yang, Jing Jiang, Tianyi Zhou, Jie Ma, and Yuhui Shi. ICLR, 2022.
• CrowdPlay: Crowdsourcing Human Demonstrations for Offline Learning
  • Matthias Gerstgrasser, Rakshit Trivedi, and David C. Parkes. ICLR, 2022.
• COPA: Certifying Robust Policies for Offline Reinforcement Learning against Poisoning Attacks
  • Fan Wu, Linyi Li, Chejian Xu, Huan Zhang, Bhavya Kailkhura, Krishnaram Kenthapadi, Ding Zhao, and Bo Li. ICLR, 2022.
• DARA: Dynamics-Aware Reward Augmentation in Offline Reinforcement Learning
  • Jinxin Liu, Hongyin Zhang, and Donglin Wang. ICLR, 2022.
• Near-optimal Offline Reinforcement Learning with Linear Representation: Leveraging Variance Information with Pessimism
  • Ming Yin, Yaqi Duan, Mengdi Wang, and Yu-Xiang Wang. ICLR, 2022.
• Pessimistic Bootstrapping for Uncertainty-Driven Offline Reinforcement Learning
  • Chenjia Bai, Lingxiao Wang, Zhuoran Yang, Zhihong Deng, Animesh Garg, Peng Liu, and Zhaoran Wang. ICLR, 2022.
• Offline Neural Contextual Bandits: Pessimism, Optimization and Generalization
  • Thanh Nguyen-Tang, Sunil Gupta, A.Tuan Nguyen, and Svetha Venkatesh. ICLR, 2022.
• Generalized Decision Transformer for Offline Hindsight Information Matching [website]
  • Hiroki Furuta, Yutaka Matsuo, and Shixiang Shane Gu. ICLR, 2022.
• Model-Based Offline Meta-Reinforcement Learning with Regularization
  • Sen Lin, Jialin Wan, Tengyu Xu, Yingbin Liang, and Junshan Zhang. ICLR, 2022.
• AW-Opt: Learning Robotic Skills with Imitation and Reinforcement at Scale [website]
  • Yao Lu, Karol Hausman, Yevgen Chebotar, Mengyuan Yan, Eric Jang, Alexander Herzog, Ted Xiao, Alex Irpan, Mohi Khansari, Dmitry Kalashnikov, and Sergey Levine. CoRL, 2022.
• Dealing with the Unknown: Pessimistic Offline Reinforcement Learning
  • Jinning Li, Chen Tang, Masayoshi Tomizuka, and Wei Zhan. CoRL, 2022.
• You Only Evaluate Once: a Simple Baseline Algorithm for Offline RL
  • Wonjoon Goo and Scott Niekum. CoRL, 2022.
• S4RL: Surprisingly Simple Self-Supervision for Offline Reinforcement Learning
  • Samarth Sinha and Animesh Garg. CoRL, 2022.
• A Workflow for Offline Model-Free Robotic Reinforcement Learning [website]
  • Aviral Kumar, Anikait Singh, Stephen Tian, Chelsea Finn, and Sergey Levine. CoRL, 2022.
• Beyond Pick-and-Place: Tackling Robotic Stacking of Diverse Shapes [blog] [video] [code]
  • Alex X. Lee, Coline Devin, Yuxiang Zhou, Thomas Lampe, Konstantinos Bousmalis, Jost Tobias Springenberg, Arunkumar Byravan, Abbas Abdolmaleki, Nimrod Gileadi, David Khosid, Claudio Fantacci, Jose Enrique Chen, Akhil Raju, Rae Jeong, Michael Neunert, Antoine Laurens, Stefano Saliceti, Federico Casarini, Martin Riedmiller, Raia Hadsell, and Francesco Nori. CoRL, 2022.
• Finetuning from Offline Reinforcement Learning: Challenges, Trade-offs and Practical Solutions
  • Yicheng Luo, Jackie Kay, Edward Grefenstette, and Marc Peter Deisenroth. RLDM, 2022.
• Offline Reinforcement Learning with Representations for Actions
  • Xingzhou Lou, Qiyue Yin, Junge Zhang, Chao Yu, Zhaofeng He, Nengjie Cheng, and Kaiqi Huang. Information Sciences, 2022.
• Towards Off-Policy Learning for Ranking Policies with Logged Feedback
  • Teng Xiao and Suhang Wang. AAAI, 2022.
• Safe Offline Reinforcement Learning Through Hierarchical Policies
  • Shaofan Liu and Shiliang Sun. PAKDD, 2022.
• TD3 with Reverse KL Regularizer for Offline Reinforcement Learning from Mixed Datasets
  • Yuanying Cai, Chuheng Zhang, Li Zhao, Wei Shen, Xuyun Zhang, Lei Song, Jiang Bian, Tao Qin, and Tieyan Liu. ICDM, 2022.
• Sample Complexity of Offline Reinforcement Learning with Deep ReLU Networks
  • Thanh Nguyen-Tang, Sunil Gupta, Hung Tran-The, and Svetha Venkatesh. arXiv, 2021.
• Model Selection in Batch Policy Optimization
  • Jonathan N. Lee, George Tucker, Ofir Nachum, and Bo Dai. arXiv, 2021.
• Learning Contraction Policies from Offline Data
  • Navid Rezazadeh, Maxwell Kolarich, Solmaz S. Kia, and Negar Mehr. arXiv, 2021.
• CoMPS: Continual Meta Policy Search
  • Glen Berseth, Zhiwei Zhang, Grace Zhang, Chelsea Finn, Sergey Levine. arXiv, 2021.
• MESA: Offline Meta-RL for Safe Adaptation and Fault Tolerance
  • Michael Luo, Ashwin Balakrishna, Brijen Thananjeyan, Suraj Nair, Julian Ibarz, Jie Tan, Chelsea Finn, Ion Stoica, and Ken Goldberg. arXiv, 2021.
• Offline Pre-trained Multi-Agent Decision Transformer: One Big Sequence Model Conquers All StarCraftII Tasks
  • Linghui Meng, Muning Wen, Yaodong Yang, Chenyang Le, Xiyun Li, Weinan Zhang, Ying Wen, Haifeng Zhang, Jun Wang, and Bo Xu. arXiv, 2021.
• Policy Gradient and Actor-Critic Learning in Continuous Time and Space: Theory and Algorithms
  • Yanwei Jia and Xun Yu Zhou. arXiv, 2021.
• Offline Reinforcement Learning: Fundamental Barriers for Value Function Approximation [video]
  • Dylan J. Foster, Akshay Krishnamurthy, David Simchi-Levi, and Yunzong Xu. arXiv, 2021.
• UMBRELLA: Uncertainty-Aware Model-Based Offline Reinforcement Learning Leveraging Planning
  • Christopher Diehl, Timo Sievernich, Martin Krüger, Frank Hoffmann, and Torsten Bertran. arXiv, 2021.
• Exploiting Action Impact Regularity and Partially Known Models for Offline Reinforcement Learning
  • Vincent Liu, James Wright, and Martha White. arXiv, 2021.
• Batch Reinforcement Learning from Crowds
  • Guoxi Zhang and Hisashi Kashima. arXiv, 2021.
• SCORE: Spurious COrrelation REduction for Offline Reinforcement Learning
  • Zhihong Deng, Zuyue Fu, Lingxiao Wang, Zhuoran Yang, Chenjia Bai, Zhaoran Wang, and Jing Jiang. arXiv, 2021.
• Safely Bridging Offline and Online Reinforcement Learning
  • Wanqiao Xu, Kan Xu, Hamsa Bastani, and Osbert Bastani. arXiv, 2021.
• Efficient Robotic Manipulation Through Offline-to-Online Reinforcement Learning and Goal-Aware State Information
  • Jin Li, Xianyuan Zhan, Zixu Xiao, and Guyue Zhou. arXiv, 2021.
• Value Penalized Q-Learning for Recommender Systems
  • Chengqian Gao, Ke Xu, and Peilin Zhao. arXiv, 2021.
• Offline Reinforcement Learning with Soft Behavior Regularization
  • Haoran Xu, Xianyuan Zhan, Jianxiong Li, and Honglei Yin. arXiv, 2021.
• Planning from Pixels in Environments with Combinatorially Hard Search Spaces
  • Marco Bagatella, Mirek Olšák, Michal Rolínek, and Georg Martius. arXiv, 2021.
• StARformer: Transformer with State-Action-Reward Representations
  • Jinghuan Shang and Michael S. Ryoo. arXiv, 2021.
• Offline RL With Resource Constrained Online Deployment [code]
  • Jayanth Reddy Regatti, Aniket Anand Deshmukh, Frank Cheng, Young Hun Jung, Abhishek Gupta, and Urun Dogan. arXiv, 2021.
• Lifelong Robotic Reinforcement Learning by Retaining Experiences [website]
  • Annie Xie and Chelsea Finn. arXiv, 2021.
• Dual Behavior Regularized Reinforcement Learning
  • Chapman Siu, Jason Traish, and Richard Yi Da Xu. arXiv, 2021.
• DCUR: Data Curriculum for Teaching via Samples with Reinforcement Learning [website] [code]
  • Daniel Seita, Abhinav Gopal, Zhao Mandi, and John Canny. arXiv, 2021.
• DROMO: Distributionally Robust Offline Model-based Policy Optimization
  • Ruizhen Liu, Dazhi Zhong, and Zhicong Chen. arXiv, 2021.
• Implicit Behavioral Cloning
  • Pete Florence, Corey Lynch, Andy Zeng, Oscar Ramirez, Ayzaan Wahid, Laura Downs, Adrian Wong, Johnny Lee, Igor Mordatch, and Jonathan Tompson. arXiv, 2021.
• Reducing Conservativeness Oriented Offline Reinforcement Learning
  • Hongchang Zhang, Jianzhun Shao, Yuhang Jiang, Shuncheng He, and Xiangyang Ji. arXiv, 2021.
• Policy Gradients Incorporating the Future
  • David Venuto, Elaine Lau, Doina Precup, and Ofir Nachum. arXiv, 2021.
• Offline Decentralized Multi-Agent Reinforcement Learning
  • Jiechuan Jiang and Zongqing Lu. arXiv, 2021.
• OPAL: Offline Preference-Based Apprenticeship Learning [website]
  • Daniel Shin and Daniel S. Brown. arXiv, 2021.
• Constraints Penalized Q-Learning for Safe Offline Reinforcement Learning
  • Haoran Xu, Xianyuan Zhan, and Xiangyu Zhu. arXiv, 2021.
• Where is the Grass Greener? Revisiting Generalized Policy Iteration for Offline Reinforcement Learning
  • Lionel Blondé and Alexandros Kalousis. arXiv, 2021.
• The Least Restriction for Offline Reinforcement Learning
  • Zizhou Su. arXiv, 2021.
• Offline-to-Online Reinforcement Learning via Balanced Replay and Pessimistic Q-Ensemble
  • Seunghyun Lee, Younggyo Seo, Kimin Lee, Pieter Abbeel, and Jinwoo Shin. arXiv, 2021.
• Causal Reinforcement Learning using Observational and Interventional Data
  • Maxime Gasse, Damien Grasset, Guillaume Gaudron, and Pierre-Yves Oudeyer. arXiv, 2021.
• On the Sample Complexity of Batch Reinforcement Learning with Policy-Induced Data
  • Chenjun Xiao, Ilbin Lee, Bo Dai, Dale Schuurmans, and Csaba Szepesvari. arXiv, 2021.
• Behavioral Priors and Dynamics Models: Improving Performance and Domain Transfer in Offline RL [website]
  • Catherine Cang, Aravind Rajeswaran, Pieter Abbeel, and Michael Laskin. arXiv, 2021.
• On Multi-objective Policy Optimization as a Tool for Reinforcement Learning
  • Abbas Abdolmaleki, Sandy H. Huang, Giulia Vezzani, Bobak Shahriari, Jost Tobias Springenberg, Shruti Mishra, Dhruva TB, Arunkumar Byravan, Konstantinos Bousmalis, Andras Gyorgy, Csaba Szepesvari, Raia Hadsell, Nicolas Heess, and Martin Riedmiller. arXiv, 2021.
• Offline Reinforcement Learning as Anti-Exploration
  • Shideh Rezaeifar, Robert Dadashi, Nino Vieillard, Léonard Hussenot, Olivier Bachem, Olivier Pietquin, and Matthieu Geist. arXiv, 2021.
• Corruption-Robust Offline Reinforcement Learning
  • Xuezhou Zhang, Yiding Chen, Jerry Zhu, and Wen Sun. arXiv, 2021.
• Offline Inverse Reinforcement Learning
  • Firas Jarboui and Vianney Perchet. arXiv, 2021.
• Heuristic-Guided Reinforcement Learning
  • Ching-An Cheng, Andrey Kolobov, and Adith Swaminathan. arXiv, 2021.
• Reinforcement Learning as One Big Sequence Modeling Problem
  • Michael Janner, Qiyang Li, and Sergey Levine. arXiv, 2021.
• Decision Transformer: Reinforcement Learning via Sequence Modeling
  • Lili Chen, Kevin Lu, Aravind Rajeswaran, Kimin Lee, Aditya Grover, Michael Laskin, Pieter Abbeel, Aravind Srinivas, and Igor Mordatch. arXiv, 2021.
• Model-Based Offline Planning with Trajectory Pruning
  • Xianyuan Zhan, Xiangyu Zhu, and Haoran Xu. arXiv, 2021.
• InferNet for Delayed Reinforcement Tasks: Addressing the Temporal Credit Assignment Problem
  • Markel Sanz Ausin, Hamoon Azizsoltani, Song Ju, Yeo Jin Kim, and Min Chi. arXiv, 2021.
• Infinite-Horizon Offline Reinforcement Learning with Linear Function Approximation: Curse of Dimensionality and Algorithm [video]
  • Lin Chen, Bruno Scherrer, and Peter L. Bartlett. arXiv, 2021.
• MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scale [website]
  • Dmitry Kalashnikov, Jacob Varley, Yevgen Chebotar, Benjamin Swanson, Rico Jonschkowski, Chelsea Finn, Sergey Levine, and Karol Hausman. arXiv, 2021.
• Distributional Offline Continuous-Time Reinforcement Learning with Neural Physics-Informed PDEs (SciPhy RL for DOCTR-L)
  • Igor Halperin. arXiv, 2021.
• Regularized Behavior Value Estimation
  • Caglar Gulcehre, Sergio Gómez Colmenarejo, Ziyu Wang, Jakub Sygnowski, Thomas Paine, Konrad Zolna, Yutian Chen, Matthew Hoffman, Razvan Pascanu, and Nando de Freitas. arXiv, 2021.
• Improved Context-Based Offline Meta-RL with Attention and Contrastive Learning
  • Lanqing Li, Yuanhao Huang, and Dijun Luo. arXiv, 2021.
• Instrumental Variable Value Iteration for Causal Offline Reinforcement Learning
  • Luofeng Liao, Zuyue Fu, Zhuoran Yang, Mladen Kolar, and Zhaoran Wang. arXiv, 2021.
• GELATO: Geometrically Enriched Latent Model for Offline Reinforcement Learning
  • Guy Tennenholtz, Nir Baram, and Shie Mannor. arXiv, 2021.
• MUSBO: Model-based Uncertainty Regularized and Sample Efficient Batch Optimization for Deployment Constrained Reinforcement Learning
  • DiJia Su, Jason D. Lee, John M. Mulvey, and H. Vincent Poor. arXiv, 2021.
• Continuous Doubly Constrained Batch Reinforcement Learning
  • Rasool Fakoor, Jonas Mueller, Pratik Chaudhari, and Alexander J. Smola. arXiv, 2021.
• Q-Value Weighted Regression: Reinforcement Learning with Limited Data
  • Piotr Kozakowski, Łukasz Kaiser, Henryk Michalewski, Afroz Mohiuddin, and Katarzyna Kańska. arXiv, 2021.
• Finite Sample Analysis of Minimax Offline Reinforcement Learning: Completeness, Fast Rates and First-Order Efficiency
  • Masatoshi Uehara, Masaaki Imaizumi, Nan Jiang, Nathan Kallus, Wen Sun, and Tengyang Xie. arXiv, 2021.
• Fast Rates for the Regret of Offline Reinforcement Learning [video]
  • Yichun Hu, Nathan Kallus, and Masatoshi Uehara. arXiv, 2021.
• Safe Policy Learning through Extrapolation: Application to Pre-trial Risk Assessment [video]
  • Eli Ben-Michael, D. James Greiner, Kosuke Imai, and Zhichao Jiang.
• Weighted Model Estimation for Offline Model-based Reinforcement Learning
  • Toru Hishinuma and Kei Senda. NeurIPS, 2021.
• A Minimalist Approach to Offline Reinforcement Learning
  • Scott Fujimoto and Shixiang Shane Gu. NeurIPS, 2021.
• Conservative Offline Distributional Reinforcement Learning
  • Yecheng Jason Ma, Dinesh Jayaraman, and Osbert Bastani. NeurIPS, 2021.
• Pessimism Meets Invariance: Provably Efficient Offline Mean-Field Multi-Agent RL
  • Minshuo Chen, Yan Li, Ethan Wang, Zhuoran Yang, Zhaoran Wang, and Tuo Zhao. NeurIPS, 2021.
• Believe What You See: Implicit Constraint Approach for Offline Multi-Agent Reinforcement Learning
  • Yiqin Yang, Xiaoteng Ma, Chenghao Li, Zewu Zheng, Qiyuan Zhang, Gao Huang, Jun Yang, and Qianchuan Zhao. NeurIPS, 2021.
• Provable Benefits of Actor-Critic Methods for Offline Reinforcement Learning
  • Andrea Zanette, Martin J. Wainwright, and Emma Brunskill. NeurIPS, 2021.
• Multi-Objective SPIBB: Seldonian Offline Policy Improvement with Safety Constraints in Finite MDPs
  • Harsh Satija, Philip S. Thomas, Joelle Pineau, and Romain Laroche. NeurIPS, 2021.
• Offline Reinforcement Learning as One Big Sequence Modeling Problem
  • Michael Janner, Qiyang Li, and Sergey Levine. NeurIPS, 2021.
• Bridging Offline Reinforcement Learning and Imitation Learning: A Tale of Pessimism [video]
  • Paria Rashidinejad, Banghua Zhu, Cong Ma, Jiantao Jiao, and Stuart Russell. NeurIPS, 2021.
• Offline Reinforcement Learning with Reverse Model-based Imagination
  • Jianhao Wang, Wenzhe Li, Haozhe Jiang, Guangxiang Zhu, Siyuan Li, and Chongjie Zhang. NeurIPS, 2021.
• Offline Meta Reinforcement Learning -- Identifiability Challenges and Effective Data Collection Strategies
  • Ron Dorfman, Idan Shenfeld, and Aviv Tamar. NeurIPS, 2021.
• Nearly Horizon-Free Offline Reinforcement Learning
  • Tongzheng Ren, Jialian Li, Bo Dai, Simon S. Du, and Sujay Sanghavi. NeurIPS, 2021.
• Conservative Data Sharing for Multi-Task Offline Reinforcement Learning
  • Tianhe Yu, Aviral Kumar, Yevgen Chebotar, Karol Hausman, Sergey Levine, and Chelsea Finn. NeurIPS, 2021.
• Online and Offline Reinforcement Learning by Planning with a Learned Model
  • Julian Schrittwieser, Thomas Hubert, Amol Mandhane, Mohammadamin Barekatain, Ioannis Antonoglou, and David Silver. NeurIPS, 2021.
• Policy Finetuning: Bridging Sample-Efficient Offline and Online Reinforcement Learning
  • Tengyang Xie, Nan Jiang, Huan Wang, Caiming Xiong, and Yu Bai. NeurIPS, 2021.
• Offline RL Without Off-Policy Evaluation
  • David Brandfonbrener, William F. Whitney, Rajesh Ranganath, and Joan Bruna. NeurIPS, 2021.
• Offline Model-based Adaptable Policy Learning
  • Xiong-Hui Chen, Yang Yu, Qingyang Li, Fan-Ming Luo, Zhiwei Tony Qin, Shang Wenjie, and Jieping Ye. NeurIPS, 2021.
• COMBO: Conservative Offline Model-Based Policy Optimization
  • Tianhe Yu, Aviral Kumar, Rafael Rafailov, Aravind Rajeswaran, Sergey Levine, and Chelsea Finn. NeurIPS, 2021.
• PerSim: Data-Efficient Offline Reinforcement Learning with Heterogeneous Agents via Personalized Simulators
  • Anish Agarwal, Abdullah Alomar, Varkey Alumootil, Devavrat Shah, Dennis Shen, Zhi Xu, and Cindy Yang. NeurIPS, 2021.
• Near-Optimal Offline Reinforcement Learning via Double Variance Reduction
  • Ming Yin, Yu Bai, and Yu-Xiang Wang. NeurIPS, 2021.
• Bellman-consistent Pessimism for Offline Reinforcement Learning [video]
  • Tengyang Xie, Ching-An Cheng, Nan Jiang, Paul Mineiro, and Alekh Agarwal. NeurIPS, 2021.
• The Difficulty of Passive Learning in Deep Reinforcement Learning
  • Georg Ostrovski, Pablo Samuel Castro, and Will Dabney. NeurIPS, 2021.
• Uncertainty-Based Offline Reinforcement Learning with Diversified Q-Ensemble
  • Gaon An, Seungyong Moon, Jang-Hyun Kim, and Hyun Oh Song. NeurIPS, 2021.
• Towards Instance-Optimal Offline Reinforcement Learning with Pessimism
  • Ming Yin and Yu-Xiang Wang. NeurIPS, 2021.
• EMaQ: Expected-Max Q-Learning Operator for Simple Yet Effective Offline and Online RL
  • Seyed Kamyar Seyed Ghasemipour, Dale Schuurmans, and Shixiang Shane Gu. ICML, 2021.
• Actionable Models: Unsupervised Offline Reinforcement Learning of Robotic Skills [website]
  • Yevgen Chebotar, Karol Hausman, Yao Lu, Ted Xiao, Dmitry Kalashnikov, Jake Varley, Alex Irpan, Benjamin Eysenbach, Ryan Julian, Chelsea Finn, and Sergey Levine. ICML, 2021.
• Is Pessimism Provably Efficient for Offline RL? [video]
  • Ying Jin, Zhuoran Yang, and Zhaoran Wang. ICML, 2021.
• Representation Matters: Offline Pretraining for Sequential Decision Making
  • Mengjiao Yang and Ofir Nachum. ICML, 2021.
• Offline Reinforcement Learning with Pseudometric Learning
  • Robert Dadashi, Shideh Rezaeifar, Nino Vieillard, Léonard Hussenot, Olivier Pietquin, and Matthieu Geist. ICML, 2021.
• Augmented World Models Facilitate Zero-Shot Dynamics Generalization From a Single Offline Environment
  • Philip J. Ball, Cong Lu, Jack Parker-Holder, and Stephen Roberts. ICML, 2021.
• Offline Contextual Bandits with Overparameterized Models
  • David Brandfonbrener, William F. Whitney, Rajesh Ranganath and Joan Bruna. ICML, 2021.
• Risk Bounds and Rademacher Complexity in Batch Reinforcement Learning
  • Yaqi Duan, Chi Jin, and Zhiyuan Li. ICML, 2021.
• Offline Reinforcement Learning with Fisher Divergence Critic Regularization
  • Ilya Kostrikov, Jonathan Tompson, Rob Fergus, and Ofir Nachum. ICML, 2021.
• OptiDICE: Offline Policy Optimization via Stationary Distribution Correction Estimation
  • Jongmin Lee, Wonseok Jeon, Byung-Jun Lee, Joelle Pineau, and Kee-Eung Kim. ICML, 2021.
• Uncertainty Weighted Actor-Critic for Offline Reinforcement Learning
  • Yue Wu, Shuangfei Zhai, Nitish Srivastava, Joshua Susskind, Jian Zhang, Ruslan Salakhutdinov, and Hanlin Goh. ICML, 2021.
• Vector Quantized Models for Planning
  • Sherjil Ozair, Yazhe Li, Ali Razavi, Ioannis Antonoglou, Aäron van den Oord, and Oriol Vinyals. ICML, 2021.
• Exponential Lower Bounds for Batch Reinforcement Learning: Batch RL can be Exponentially Harder than Online RL [video]
  • Andrea Zanette. ICML, 2021.
• Instabilities of Offline RL with Pre-Trained Neural Representation
  • Ruosong Wang, Yifan Wu, Ruslan Salakhutdinov, and Sham M. Kakade. ICML, 2021.
• Offline Meta-Reinforcement Learning with Advantage Weighting
  • Eric Mitchell, Rafael Rafailov, Xue Bin Peng, Sergey Levine, and Chelsea Finn. ICML, 2021.
• Model-Based Offline Planning [video]
  • Arthur Argenson and Gabriel Dulac-Arnold. ICLR, 2021.
• Batch Reinforcement Learning Through Continuation Method
  • Yijie Guo, Shengyu Feng, Nicolas Le Roux, Ed Chi, Honglak Lee, and Minmin Chen. ICLR, 2021.
• Model-Based Visual Planning with Self-Supervised Functional Distances
  • Stephen Tian, Suraj Nair, Frederik Ebert, Sudeep Dasari, Benjamin Eysenbach, Chelsea Finn, and Sergey Levine. ICLR, 2021.
• Deployment-Efficient Reinforcement Learning via Model-Based Offline Optimization
  • Tatsuya Matsushima, Hiroki Furuta, Yutaka Matsuo, Ofir Nachum, and Shixiang Gu. ICLR, 2021.
• Efficient Fully-Offline Meta-Reinforcement Learning via Distance Metric Learning and Behavior Regularization
  • Lanqing Li, Rui Yang, and Dijun Luo. ICLR, 2021.
• DeepAveragers: Offline Reinforcement Learning by Solving Derived Non-Parametric MDPs
  • Aayam Kumar Shrestha, Stefan Lee, Prasad Tadepalli, and Alan Fern. ICLR, 2021.
• What are the Statistical Limits of Offline RL with Linear Function Approximation? [video]
  • Ruosong Wang, Dean Foster, and Sham M. Kakade. ICLR, 2021.
• Reset-Free Lifelong Learning with Skill-Space Planning [website]
  • Kevin Lu, Aditya Grover, Pieter Abbeel, and Igor Mordatch. ICLR, 2021.
• Risk-Averse Offline Reinforcement Learning
  • Núria Armengol Urpí, Sebastian Curi, and Andreas Krause. ICLR, 2021.
• Finite-Sample Regret Bound for Distributionally Robust Offline Tabular Reinforcement Learning
  • Zhengqing Zhou, Zhengyuan Zhou, Qinxun Bai, Linhai Qiu, Jose Blanchet, and Peter Glynn. AISTATS, 2021.
• Exploration by Maximizing Rényi Entropy for Reward-Free RL Framework
  • Chuheng Zhang, Yuanying Cai, Longbo Huang, and Jian Li. AAAI, 2021.
• Efficient Self-Supervised Data Collection for Offline Robot Learning
  • Shadi Endrawis, Gal Leibovich, Guy Jacob, Gal Novik, Aviv Tamar. ICRA, 2021.
• Boosting Offline Reinforcement Learning with Residual Generative Modeling
  • Hua Wei, Deheng Ye, Zhao Liu, Hao Wu, Bo Yuan, Qiang Fu, Wei Yang, and Zhenhui (Jessie)Li. IJCAI, 2021.
• BRAC+: Improved Behavior Regularized Actor Critic for Offline Reinforcement Learning
  • Chi Zhang, Sanmukh Rao Kuppannagari, and Viktor K Prasanna. ACML, 2021.
• Behavior Constraining in Weight Space for Offline Reinforcement Learning
  • Phillip Swazinna, Steffen Udluft, Daniel Hein, and Thomas Runkler. ESANN, 2021.
• Finite-Sample Analysis For Decentralized Batch Multi-Agent Reinforcement Learning With Networked Agents
  • Kaiqing Zhang, Zhuoran Yang, Han Liu, Tong Zhang, and Tamer Başar. IEEE T AUTOMATIC CONTROL, 2021.
• Can Active Sampling Reduce Causal Confusion in Offline Reinforcement Learning?
  • Gunshi Gupta, Tim G. J. Rudner, Rowan Thomas McAllister, Adrien Gaidon, and Yarin Gal. CLeaR, 2021.
• Reinforcement Learning via Fenchel-Rockafellar Duality [software]
  • Ofir Nachum and Bo Dai. arXiv, 2020.
• AWAC: Accelerating Online Reinforcement Learning with Offline Datasets [website] [code] [blog]
  • Ashvin Nair, Abhishek Gupta, Murtaza Dalal, and Sergey Levine. arXiv, 2020.
• Sparse Feature Selection Makes Batch Reinforcement Learning More Sample Efficient
  • Botao Hao, Yaqi Duan, Tor Lattimore, Csaba Szepesvári, and Mengdi Wang. arXiv, 2020.
• A Variant of the Wang-Foster-Kakade Lower Bound for the Discounted Setting
  • Philip A