Given sequentially observed data, anytime-valid methods guarantee valid inference at arbitrary stopping times, as opposed to pre-specified sample sizes, thereby allowing the experimenter to stop experiments early. In this talk, I will present two recent advances in the emerging field of sequential anytime-valid inference (SAVI).

First, consider two forecasters, each making a prediction for a sequence of events over time. How can we rigorously compare these forecasters as the events unfold, while avoiding restrictive assumptions such as stationarity? I will address this question by designing a novel inference procedure for estimating the time-varying difference in mean forecast scores. The procedure utilizes confidence sequences, which are sequences of confidence intervals that are anytime-valid and can be continuously monitored over time. I will demonstrate applications of this approach to real-world sports and weather forecasters.

Next, given a composite null hypothesis over sequentially observed data (e.g., whether high-volatility days are random in a financial time series), consider two or more testing procedures that are powerful against different alternatives. How can we combine these procedures so that we can leverage their collective statistical power, particularly when the procedures are valid under different information sets (i.e., filtrations)? This general question arises in various sequential inference problems, such as randomness testing and multi-step forecast evaluation. I will introduce a simple solution that allows us to combine arbitrary sequential tests that are based on e-processes—the SAVI notion of statistical evidence—across different filtrations. 

Speaker bio: Dr. Yo Joong “YJ” Choe is a Postdoctoral Scholar at the University of Chicago’s Data Science Institute. He received a joint Ph.D. in Statistics and Machine Learning (ML) from Carnegie Mellon University in 2023. His research interests include: (1) reliable evaluation of black-box forecasters, with a focus on sequential anytime-valid inference (SAVI) methods, and (2) causal approaches to enhancing the transparency of large language models (LLMs). His work is published in top journals, such as Operations Research, as well as major ML conferences, such as NeurIPS, ICML, and ICLR. Previously, Dr. Choe was a Research Scientist at Kakao and Kakao Brain, where he developed models and resources for practical natural language processing (NLP) tasks.