Hello! I am a Research Scientist at Google DeepMind, based in Mountain View. I finished my Ph.D. from UC Berkeley in September 2023, and will start as an Assistant Professor in the Computer Science (CSD) and Machine Learning (MLD) departments at Carnegie Mellon University (CMU) in Fall 2024. I am interested in a number of topics (the most recent ones include):
- reinforcement learning (RL) algorithms that work reliably at scale [recent works: stop regressing, ICML 2024; scaling offline RL; ICLR 2023; value vs policy, arXiv 2024; online fine-tuning, NeurIPS 2023]
- intersection of foundation models and decision making / reinforcement learning (both on using foundation models for decision making and developing decision-making and RL tools for building better foundation models) [recent works: ArCHer, ICML 2024; RL + synthetic data => 8x, arXiv 2024; preference fine-tuning, ICML 2024; promptable representations]
- robotic learning and control [recent works: SuSIE, ICLR 2024; V-PTR, ICRA 2024]
- Applications [recent works: DigiRL, arXiv 2024; Promoter design, arXiv 2024]
[CV (a bit out of date)], [Google Scholar], [Twitter]
My research goal is to develop a “tool-box” to solve a variety of decision-making problems reliably, robustly and effectively in the real world. Towards this goal, I am interested in developing techniques and algorithms for sequential decision-making. My past work has focused on developing methods for running reinforcement learning (RL) on static datasets and understanding and addressing challenges in using RL with deep neural networks. I am interested in taking my methods all the way to the real world and have been studying many applications (and I will continue to look out for interesting real-world applications!) – please check out our recent works on offline decision-making for chip design, real robot pre-training, and computational design! On-going works on crystal structure optimization in computational chemistry and promoter design in computational biology to come out soon. If you want to learn more about offline RL, please check out our NeurIPS tutorial.