Toward anytime valid inference without likelihoods

Music City Center 

Approximately valid inference is available with and without likelihoods, e.g., using the asymptotic normality of maximum likelihood estimators and M-estimators more generally. In certain applications, however, there is a need for methods which are exactly (not just asymptotically) valid in an anytime sense; that is, methods that offer frequentist error rate guarantees even if the sample size depends on the observed data. Likelihood-based methods are now available that achieve this goal of anytime validity, but what about cases when the data analyst is not willing to specify a likelihood? In such cases, the relevant unknowns often take the form of risk (expected loss) minimizers. In this talk, I'll start by describing the growth-rate optimal e-process that offers anytime valid and efficient inference on risk minimizers. This optimal e-process depends on certain features that are typically unknown, so a data-driven construction is needed. For this, I'll develop a generalized universal inference framework that mimics the optimal e-process and show some efficiency results and numerical illustrations. To date, generalized universal inference has only been demonstrated to achieve anytime validity empirically in simulations, and I'll highlight the challenges in proving this theoretically.