Optimal transport in Statistics and Machine Learning

In this talk, we revisit the classical theory of variance bounds for unbiased estimators and develop an analogous framework for a different notion of estimator stability, which we term sensitivity. Roughly speaking, the sensitivity of an estimator quantifies how much its output changes when its inputs are perturbed slightly. In the same way that classical inequalities like the Cramer-Rao lower bound for the variance of unbiased estimators can be interpreted geometrically via the Fisher-Rao geometry, our notion of sensitivity admits analogous inequalities and theory, only that now they arise from Wasserstein geometry. We will discuss this Wasserstein-Cramer-Rao lower bound and its associated notion of Wasserstein Fisher information (which are closely related to notions introduced by Li and Zhao, 2023), introduce the notion of Wasserstein exponential families and their defining properties, and introduce the concept of Wasserstein MLEs. In particular, we will discuss that the Wasserstein MLE is, generically, asymptotically efficient for sensitivity, in the sense that it achieves the corresponding Wasserstein-Cramer-Rao lower bound. Our theory reveals new optimality properties for existing estimators and, in other cases, reveals entirely new estimators.

This is joint work with Adam Quinn Jaffe and Bodhisattva Sen.
 

This talk will describe an adaptive Langevin diffusion sampling procedure for empirical Bayes learning of the prior distribution in a random effects linear model. The procedure may be implemented either parametrically or nonparametrically, and both forms may be motivated from a Wasserstein gradient flow approach to maximizing the marginal log-likelihood. I will discuss some basic consistency results for the (possibly nonparametric) maximum likelihood estimator in this problem, and then describe an exact asymptotic analysis of the learning dynamics in a setting of i.i.d. random design using dynamical mean-field theory.

Joint work with Leying Guan, Yandi Shen, Yihong Wu,. Justin Ko, Bruno Loureiro, and Yue M. Lu. 

In the statistical problem of denoising, Bayes and empirical Bayes methods can "overshrink" their output relative to the latent variables of interest. This work is focused on constrained denoising problems which mitigate such phenomena. At the oracle level, i.e., when the latent variable distribution is assumed known, we apply tools from the theory of optimal transport to characterize the solution to (i) variance-constrained, (ii) distribution-constrained, and (iii) general-constrained denoising problems. At the empirical level, i.e., when the latent variable distribution is not known, we use empirical Bayes methodology to estimate these oracle denoisers. Our approach is modular, and transforms any suitable (unconstrained) EB denoiser into a constrained EB denoiser. We prove explicit rates of convergence for our proposed methodologies, which both extend and sharpen existing asymptotic results that have previously considered only variance constraints. We apply our methodology in two applications: one in astronomy concerning the relative chemical abundances in a large catalog of red-clump stars, and one in baseball concerning minor- and major league batting skill for rookie players. 

The Gromov–Wasserstein (GW) problem, rooted in optimal transport theory, considers optimal alignment of metric-measure (mm) spaces that minimizes pairwise distance distortion. As such, it matches up the mm spaces' internal structures and offers a powerful framework for aligning heterogeneous datasets. Despite its broad applicability, a statistical and computational GW theory has started to develop only recently, with the derivation of sharp empirical convergence rates and algorithms for approximate computation subject to non-asymptotic convergence guarantees. In this work, we develop the first limit laws for empirical GW distances and describe consistent resampling schemes. We treat three important settings of GW alignment: (i) discrete-to-discrete (unregularized); (ii) semi-discrete (unregularized); and (iii) general distributions under moment constraints with entropic regularization. In particular, in the discrete case, we show that the limit can be expressed as a solution to a linear program with random Gaussian coefficients, which can be efficiently simulated. This enables inference with the GW distance, which we leverage for an application to graph-isomorphism testing.