Monday, October 30, 2017 - 4:15pm to 5:30pm
Stern School of Business, New York University
Statistical Inference for Model Parameters with Stochastic Gradient Descent
In this talk, we investigate the problem of statistical inference of the true model parameters based on stochastic gradient descent (SGD). To this end, we propose two consistent estimators of the asymptotic covariance of the average iterate from SGD: (1) an intuitive plug-in estimator and (2) a computationally more efficient batch-means estimator, which only uses the iterates from SGD. As the SGD process forms a time-inhomogeneous Markov chain, our batch-means estimator with carefully chosen increasing batch sizes generalizes the classical batch-means estimator designed for time-homogenous Markov chains. Both proposed estimators allow us to construct asymptotically exact confidence intervals and hypothesis tests. We further discuss an extension to conducting inference based on SGD for high-dimensional linear regression. Bio: Xi Chen is an assistant professor at Department of Information, Operations, and Management Sciences at Stern School of Business at New York University. Before that, he was a Postdoc in the group of Prof. Michael Jordan at UC Berkeley. He obtained his Ph.D. from the Machine Learning Department at Carnegie Mellon University (CMU); and his Masters degree in Operations Research from the Tepper School of Business at CMU. He studies sequential analysis and multi-armed bandits, high-dimensional statistics, and shape-restricted nonparametric regression. He also studies operations research/management problems, such as the data-driven revenue management. He received Simons-Berkeley Research Fellowship, Google Faculty Research Award, Adobe Data Science Research Award, and was featured in 2017 Forbes list of “30 Under30 in Science”.
Statistics & Data Science, 24 Hillhouse Avenue, New Haven