From robotics to autonomous vehicles, machine learning agents deployed in the physical world (“physical agents”) promise to revolutionize endeavors ranging from manufacturing to agriculture to domestic labor. In this talk, we will develop mathematical foundations, from the ground up, for how to carry out this vision. We will begin our investigation by examining linear dynamical systems, a simple and fundamental model of the interaction between a physical agent and its environment. We prove mathematically that simple exploration attains optimal performance for some of both the simplest and the most complex learning problems in this class. The above finding, while powerful, strongly motivates moving past linear dynamics as a mathematical testbed for understanding learning with physical agents. 

Hence, we turn to providing mathematical guarantees for a setting of real-world importance that does not fit the linear mold: behavior cloning. Behavior cloning — teaching a robot to imitate from example demonstrations — lies at the heart of many of today’s most promising robot learning endeavors due to its intuitive data collection and simplicity. Though it can work incredibly well, we still do not have a clear understanding of what circumstances ensure its success. Bringing together the flexibility of generative models with key intuitions arising from the study of linear control,  we introduce a framework for behavior cloning that enables an agent to imitate nearly arbitrary behavior with provable guarantees, even when the dynamics governing the agent and environments interaction are nonlinear. We conclude by outlining ongoing work and future steps towards building out the mathematical and conceptual tooling for understanding the next steps towards general, capable and flexible physical agents. 

Bio: Max Simchowitz is a postdoctoral researcher in the Robot Locomotion Group at MIT CSAIL. He studies the theoretical foundations of machine learning problems with a sequential or dynamical component; he currently focuses on robotics and out-of-distribution learning, and with past work ranging broadly across control, reinforcement learning, optimization and algorithmic fairness. He received his PhD from University of California, Berkeley in 2021 under Ben Recht and Michael I. Jordan, and his work has been recognized with an ICML 2018 Best Paper Award, ICML 2022 Outstanding Paper Award, and RSS 2023 Best Paper Finalist designation.