Thursday, Aug 8: 10:30 AM - 12:20 PM
1648
Topic-Contributed Paper Session
Oregon Convention Center
Room: CC-B119
Statistical learning approaches are indispensable components to accelerate scientific discovery in the age of artificial intelligence. Statistical theory and methods have been increasingly used for predicting costly computer simulations of integral and differential equations, and inversely estimating system parameters from correlated experimental or field observations, including images, spatio-temporal and functional data. An advantage of a statistical or probabilistic model is the availability of internal uncertainty quantification, which has become the basis of various modern machine learning algorithms, such as adaptive designs for active learning and Bayesian optimization.
Various interesting problems, including high-dimensional parameter space, complex data structure, and highly nonlinear maps, emerge from the scientific theory and mathematical models. These problems require more efficient and scalable statistical approaches for prediction and estimation. In recent years, substantial progress has been made toward solving these challenges through new statistical theory and algorithms in dynamical systems, graphs, functions and manifolds. These advances enable a wide range of applications in science and engineering, ranging from image analysis to climate modeling. This session will bring together experts to share their latest findings in theory and computation, and to outline future directions of statistical learning and uncertainty quantification approaches for complex systems, making a timely contribution to the 2024 JSM program.
The tentative titles of five speakers are as follows.
1. Yun Yang (University of Illinois Urbana-Champaign, Associate professor): Minimizing Convex Functionals over Space of Probability Measures via KL Divergence Gradient Flow
2. Chih-Li Sung (Michigan State University, Assistant Professor): Functional-Input Gaussian Processes with Applications to Inverse Scattering Problem
3. Jiaoyang Huang (University of Pennsylvania, Assistant Professor): Fisher-Rao Gradient Flow: Geodesic Convexity and Functional Inequality
4. Mengyang Gu (University of California, Santa Barbara, Assistant Professor): Fast Ab Initio Uncertainty Quantification and Data Inversion for Dynamical Systems
5. Omar Al-Ghattas (University of Chicago, PhD Student): High and Infinite-Dimensional Analysis of Ensemble Kalman Methods
Applied
Yes
Main Sponsor
Uncertainty Quantification in Complex Systems Interest Group
Co Sponsors
Section on Bayesian Statistical Science
Section on Physical and Engineering Sciences
Presentations
Motivated by the computation of the non-parametric maximum likelihood estimator (NPMLE) and the Bayesian posterior in statistics, this paper explores the problem of convex optimization over the space of all probability distributions. We introduce an implicit scheme, called the implicit KL proximal descent (IKLPD) algorithm, for discretizing a continuous-time gradient flow relative to the KullbackLeibler divergence for minimizing a convex target functional. We show that IKLPD converges to a global optimum at a polynomial rate from any initialization; moreover, if the objective functional is strongly convex relative to the KL divergence, for example, when the target functional itself is a KL divergence as in the context of Bayesian posterior computation, IKLPD exhibits globally exponential convergence. Computationally, we propose a numerical method based on normalizing flow to realize IKLPD. Conversely, our numerical method can also be viewed as a new approach that sequentially trains a normalizing flow for minimizing a convex functional with a strong theoretical guarantee.
Speaker
Yun Yang, University of Illinois Urbana-Champaign
Surrogate modeling based on Gaussian processes (GPs) has received increasing attention in the analysis of complex problems in science and engineering. Despite extensive studies on GP modeling, the developments for functional inputs are scarce. Motivated by an inverse scattering problem in which functional inputs representing the support and material properties of the scatterer are involved in the partial differential equations, a new class of kernel functions for functional inputs is introduced for GPs. Based on the proposed GP models, the asymptotic convergence properties of the resulting mean squared prediction errors are derived and the finite sample performance is demonstrated by numerical examples. In the application to inverse scattering, a surrogate model is constructed with functional inputs, and a new Bayesian framework is introduced to recover the reflective index of an inhomogeneous isotropic scattering region of interest for a given far-field pattern.
Sampling a probability distribution with an unknown normalization constant is a fundamental problem in computational science and engineering. This task may be cast as an optimization problem over all probability measures, where an initial distribution evolves dynamically towards the desired minimizer (the target distribution) via gradient flows. By choosing different metrics for the gradient flow, different algorithms with different convergence properties arise.
In this talk, I will focus on the Fisher-Rao metric, which is known to be the unique metric (up to scaling) that is diffeomorphism invariant.
Unlike the Wasserstein metric, a significant challenge arises from the absence of geodesic convexity under the Fisher-Rao metric for common energy functionals such as the Kullback-Leibler (KL) divergence. I will present a novel functional inequality for Fisher-Rao gradient flow. This leads to a uniform exponential rate of convergence for the gradient flow associated with KL-divergence, as well as for large families of f-divergences.
Estimating parameters from data is a fundamental problem, which is customarily done by minimizing a loss function between a model and observed statistics. In this talk, we discuss another paradigm termed the ab initio uncertainty quantification (AIUQ) method, for improving loss-minimization estimation in two steps. In step one, we define a probabilistic generative model from the beginning of data processing and show the equivalence between loss-minimization estimation and a statistical estimator. In step two, we develop better models or estimators, such as the maximum marginal likelihood or Bayesian estimators to improve estimation. To illustrate, we introduce two approaches to estimate dynamical systems, one in Fourier analysis of microscopy videos, and the other in inversely estimating the particle interaction kernel from trajectory. In the first approach, we show that differential dynamic microscopy, a scattering-based analysis tool that extracts dynamical information of microscopy videos, is equivalent to fitting Fourier temporal auto-covariance based on a latent factor model we constructed. We derived likelihood-based inference and accelerated the computation thousands of times by utilizing the generalized Schur algorithm for Toeplitz covariances. In the second approach, we developed a new method called the inverse Kalman filter which enables accurate matrix-vector multiplication between a covariance matrix from a dynamic linear model and any real-valued vector with a linear computational cost. These new approaches outline a wide range of applications, such as probing optically dense systems, automated determination of gelation time, and estimating cellular interaction for fibroblasts on liquid crystalline substrates.
Speaker
Mengyang Gu, University of California-Santa Barbara
Many modern algorithms for inverse problems and data assimilation rely on ensemble Kalman updates to blend prior predictions with observed data. Ensemble Kalman methods often perform well with a small ensemble size, which is essential in applications where generating each particle is costly due to high or infinite dimensionality of the state. This talk will describe a novel non-asymptotic and dimension free analysis of ensemble Kalman updates that rigorously explains why a small ensemble size suffices if the prior covariance has moderate effective dimension due to fast spectrum decay or approximate sparsity. We present our theory in a unified framework, comparing several implementations of ensemble Kalman updates that use perturbed observations, square root filtering, and localization.