Publications

Publications and Recent Pre-Prints

A complete list of publications (Nov 2022) can be found in my CV or in my Google scholar.

• Value-Based Deep Reinforcement Learning Requires Explicit Regularization
  Aviral Kumar, Rishabh Agarwal, Aaron Courville, Tengyu Ma, George Tucker, Sergey Levine [Paper] [Talk]
  Oral Presentation at the Deep Reinforcement Learning Workshop at NeurIPS 2021 (6/140 papers)
  Oral Presentation at the Overparameterization Workshop at ICML 2021 (6/45 papers)
  Oral Presentation at RL for Real Life Workshop at ICML 2021 (12/90 papers)
  International Conference on Learning Representations (ICLR), 2022 (Spotlight Presentation)

• A Workflow for Offline Model-Free Robotic Reinforcement Learning\ Aviral Kumar^, Anikait Singh^, Stephen Tian, Chelsea Finn, Sergey Levine [Paper] [Talk]
  Conference on Robot Learning (CoRL), 2021 (Oral Presentation)

• Offline Q-Learning on Diverse Multi-Task Data Scales and Generalizes
  Aviral Kumar, Rishabh Agarwal, Xinyang Geng, George Tucker, Sergey Levine [arXiv]
  Oral Presentation at the Deep Reinforcement Learning Workshop at NeurIPS 2022 (6/196 papers)

• Data-Driven Offline Optimization for Architecting Hardware Accelerators [Paper] [Talk] [Blog]
  Aviral Kumar^, Amir Yazdanbakhsh^, Milad Hashemi, Kevin Swersky, Sergey Levine
  International Conference on Learning Representations (ICLR), 2022

• Pre-Training for Robots: How Offline RL Enables Learning New Tasks from a Handful of Trials
  Aviral Kumar^, Anikait Singh^, Frederik Ebert^*, Yanlai Yang, Chelsea Finn, Sergey Levine [arXiv]

• How to Leverage Unlabeled Data in Offline Reinforcement Learning [Paper] [Talk]
  Tianhe Yu^, Aviral Kumar^, Yevgen Chebotar, Chelsea Finn, Karol Hausman, Sergey Levine
  International Conference on Machine Learning (ICML), 2022

• Should I Run Offline Reinforcement Learning or Behavioral Cloning?
  Aviral Kumar^, Joey Hong^, Anikait Singh, Sergey Levine [Paper] [Blog]
  International Conference on Learning Representations (ICLR), 2022

• Conservative Data-Sharing for Multi-Task Offline Reinforcement Learning [Paper]
  Tianhe Yu^, Aviral Kumar^, Yevgen Chebotar, Karol Hausman, Sergey Levine, Chelsea Finn\ Neural Information Processing Systems (NeurIPS), 2021.

• Conservative Objective Models for Effective Offline Model-Based Optimization [Paper] [Blog]
  Brandon Trabucco^, Aviral Kumar^, Xinyang Geng, Sergey Levine
  International Conference on Machine Learning (ICML), 2021.

• Implicit Under-Parameterization Inhibts Data-Efficient Deep Reinforcement Learning
  Aviral Kumar^, Rishabh Agarwal^, Dibya Ghosh, Sergey Levine [arXiv]
  International Conference on Learning Representations (ICLR), 2021

• OPAL: Offline Primitive Discovery For Accelerating Reinforcement Learning
  Anurag Ajay, Aviral Kumar, Pulkit Agarwal, Sergey Levine, Ofir Nachum [arXiv]
  International Conference on Learning Representations (ICLR), 2021

• Conservative Safety Crtics for Exploration
  Homanaga Bharadhwaj, Aviral Kumar, Nicholas Rhinehart, Sergey Levine, Florian Shkruti, Animesh Garg [arXiv]
  International Conference on Learning Representations (ICLR), 2021

• COG: Connecting New Skills to Past Experience with Offline Reinforcement Learning
  Avi Singh, Albert Yu, Jonathan Yang, Jesse Zhang, Aviral Kumar, Sergey Levine
  [arXiv] [website] (October 2020)
  4th Conference on Robot Learning (CoRL), 2020\

• Conservative Q-Learning for Offline Reinforcement Learning
  Aviral Kumar, Aurick Zhou, George Tucker, Sergey Levine
  [arXiv] [website] [Blog]
  Neural Information Processing Systems (NeurIPS), 2020\

• DisCor: Corrective Feedback in Reinforcement Learning via Distribution Correction
  Aviral Kumar, Abhishek Gupta, Sergey Levine [arXiv] [BAIR Blog] (March 2020)
  Neural Information Processing Systems (NeurIPS) (Spotlight Presentation, 3% acceptance rate), 2020\

• Model Inversion Networks for Model-Based Optimization
  Aviral Kumar, Sergey Levine [arXiv] (December 2019)
  Neural Information Processing Systems (NeurIPS), 2020\

• One Solution is Not All You Need: Few-Shot Extrapolation Via Structured MaxEnt RL
  Saurabh Kumar, Aviral Kumar, Sergey Levine, Chelsea Finn [arXiv] (October 2020)
  Neural Information Processing Systems (NeurIPS), 2020\

• Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reduction
  Aviral Kumar^, Justin Fu^, George Tucker, Sergey Levine
  Neural Information Processing Systems (NeurIPS), 2019
  [Paper] [Project Page] [BAIR Blog] \

• Diagnosing Bottlenecks in Deep Q-Learning Algorithms
  Justin Fu^, Aviral Kumar^, Matthew Soh, Sergey Levine
  International Conference on Machine Learning (ICML) 2019 [Paper] (* Equal Contribution)\

• Graph Normalizing Flows
  Jenny Liu*, Aviral Kumar*, Jimmy Ba, Jamie Kiros, Kevin Swersky
  Neural Information Processing Systems (NeurIPS), 2019
  [Paper] [Project Page] (* Equal Contribution)

• Trainable Calibration Measures for Neural Networks from Kernel Mean Embeddings
  Aviral Kumar, Sunita Sarawagi, Ujjwal Jain
  35th International Conference on Machine Learning (ICML) 2018 [Main Paper]

• GRevnet: Improving Graph Neural Networks with Reversible Computation
  Aviral Kumar, Jimmy Ba, Jamie Kiros, Kevin Swersky
  NeuRIPS 2018 Relational Representation Learning Workshop [Paper]

• Feudal Learning for Large Discrete Action Spaces with Recursive Substructure
  Aviral Kumar, Kevin Swersky, Geoffrey Hinton
  Hierarchical Reinforcement Learning Workshop, NIPS 2017
  [Main Paper]

Tutorials

• Offline Reinforcement Learning: Tutorial, Review and Perspectives on Open Problems
  Sergey Levine, Aviral Kumar, George Tucker, Justin Fu
  [arXiv] (May 2020)\

Older Pre-Prints

• Datasets for Data-Driven Deep Reinforcement Learning
  Justin Fu, Aviral Kumar, Ofir Nachum, George Tucker, Sergey Levine
  [arXiv] [Benchmark] (April 2020)
  At RL for Real Life Virtual Conference, 2020.\

• Reward-Condtioned Policies
  Aviral Kumar, Xue Bin Peng, Sergey Levine [arXiv] (December 2019)\

• Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learning
  Xue Bin Peng, Aviral Kumar, Grace Zhang, Sergey Levine [Project Page] (October 2019)