Statistically Efficient Offline Reinforcement Learning and Causal Machine Learning

Speaker

Masatoshi Uehara, Cornell

Despite the remarkable achievements of reinforcement learning (RL) in the realm of gaming, such as AlphaGo and OpenAI Five, its implementation in scientific domains, such as economics and medicine, remains limited. This limitation is partly due to the costly and potentially hazardous nature of conducting experiments that involve human interaction. To address this challenge, statistically efficient offline RL, which enables sequential decision-making in a sample-efficient manner using offline data, is crucial. During this presentation, I will elaborate on my research in this area, with a primary focus on our “double minimax RL framework.” This framework satisfies several desiderata such as (1) it can integrate any rich function approximation such as deep neural networks, (2) it is statistically efficient, and (3) it enables us to carry out statistical inference. For the remainder of this presentation, I will elaborate on the extension of the double minimax RL framework to address a wider range of data-driven inverse problems. These problems encompass several significant problems, including instrumental variable regression in economics, causal inference in the presence of unmeasured confounders, and offline policy evaluation in partially observable MDPs.

Bio: Masatoshi Uehara is a third-year Ph.D. student at Cornell CS, advised by Nathan Kallus. He previously received a bachelor’s degree in applied mathematics and computer science from the University of Tokyo and a master’s of science in statistics at Harvard University. His research is at the intersection of reinforcement learning, causal ML, and Econ ML. He has won two scholarships, awarded to the most outstanding students from Japan, during his Ph.D. program. His works have been selected as spotlight/oral papers (2–5%) in top machine learning conferences such as ICML, NeurIPS, and ICLR.

In-Person seminars will be held at Mason Lab 211, 9 Hillhouse Avenue with the option of virtual participation 
3:30pm -   Pre-talk meet and greet teatime - Dana House, 24 Hillhouse Avenue 
Masatoshi Uehara’s website