Distribution shifts—when the training distribution does not match the test distribution—pose significant challenges for machine learning models deployed in the wild. In this talk, we will discuss how using additional human supervision, beyond just labels, can help to train models to “think the right way” and be more robust to changing settings. First, we will show how using intermediate concepts, e.g., teaching a model to recognize osteoarthritis by first identifying clinical concepts such as bone spurs, can improve model generalization as well as enable humans to interact with, and correct, the model at test time. Next, we will discuss using techniques from distributionally robust optimization that take the opposite tack and can train models that avoid latching onto human-specified spurious correlations (like image background) and can consequently generalize better. Finally, we will introduce WILDS, a benchmark for in-the-wild distribution shifts that can be used to develop and evaluate general-purpose algorithms for building robust models.

This is joint work with Shiori Sagawa, Thao Nguyen, Yew Siang Tang, Henrik Marklund, Sergey Levine, Chelsea Finn, Percy Liang, and many other collaborators.

Bio:

Pang Wei Koh is a fifth-year CS PhD student at Stanford, advised by Percy Liang. His research has been recognized by a best paper award at ICML; a top 10 papers of the year award at RECOMB; and a Facebook PhD fellowship. His work has also appeared in Nature and Cell, and has been featured in The New York Times, The Washington Post, The Telegraph, MIT Technology Review, and other news outlets. Before starting his PhD, Pang Wei was the third employee at Coursera, where he served as Director of Partnerships.