Frontiers in Models Based on Gaussian Process Priors: Asymptotics, Applications, and Scalability

1089 

Invited Paper Session 

Sanvesh Srivastava (1), Terrance Savitsky (2), Cheng Li (3), Nan Wu (4), Michael Zhang (5), Annie Sauer (6)

(1) University of Iowa, Iowa City, (2) US Bureau of Labor Statistics, Washington DC, (3) National University of Singapore, Singapore, (4) University of Texas at Dallas, Dallas, (5) University of Hong Kong, Hong Kong, (6) North Carolina State University, Raleigh

Terrance Savitsky  
US Bureau of Labor Statistics

Sanvesh Srivastava  
University of Iowa

Cheng Li  
National University of Singapore

Nan Wu  
University of Texas at Dallas

Michael Zhang  
University of Hong Kong

Annie Booth  
North Carolina State University

Gaussian process (GP) priors are a prominent tool for nonparametric regression, widely known for their adaptability and flexibility. Despite their importance, the computational costs of posterior computations in GP-based models, particularly those with non-Gaussian likelihoods, have historically limited their application in automated systems. Furthermore, existing theoretical results focus on estimating high-dimensional density or regression functions under sparsity-type assumptions. Fundamental results, such as posterior consistency and contraction rates, have yet to be fully explored for GPs in settings like inferring covariance parameters or manifolds from noisy high-dimensional covariates.

Progress has been made through approximations like sparse GPs and graph-based GPs. Still, more work is needed to enable scalable and theoretically grounded GP models for realistic high-dimensional data. This session highlights recent results that address these challenges, offering innovative solutions and expanding the applicability of GP priors to new domains while retaining their strengths. The four speakers in the session focus on the following critical areas:

1. Fixed-Domain Asymptotics (Cheng Li): Li's talk addresses the challenges of estimating Matern covariance function parameters in spatial GP regression models. Key results include the non-estimability of specific covariance function parameters and posterior contraction rates for the estimable ones.
2. Inferring Manifolds from Noisy Covariates (Nan Wu): Wu's talk presents a new algorithm for manifold reconstruction via GP priors and related posterior consistency results. The main novelty of this work is that it allows the interpolation of the estimated manifold between fitted data points and turns a global manifold reconstruction problem into a local regression problem.
3. Non-linear Latent Variable Models (Michael Zhang): Zhang's talk broadens the application scope of GP-based latent variable models to those with non-Gaussian likelihoods. The key idea of this work is to leverage the flexibility of random Fourier features for scalable posterior computations.
4. Scalable Algorithms for Fitting Deep-GPs (Annie Booth): Deep GPs (DGPs) perform functional composition, in which intermediate GP layers warp the original inputs, providing flexibility to model non-stationary dynamics. Booth's talk expands the capabilities of Bayesian DGP posterior inference by using the Vecchia approximation, allowing linear computational scaling without compromising accuracy or uncertainty quantification.

The session's organization is such that the first talk sets the session's tone by introducing the abstract setting of GP regression, which the last three speakers leverage. Building on the general setup, the second talk introduces GP extensions that handle covariates with a near manifold structure. This work strikes a balance between theory and computation. Finally, scalable posterior computations are the focus of the last two talks. Specifically, Zhang's talk develops scalable extensions of GP-based latent variable models for non-Gaussian likelihood. Booth's talk leads to the session's climax, where novel Vecchia-type algorithms are introduced for fitting large-scale deep GP models and estimating posterior uncertainty. This session brings together these cutting-edge advances and critical insights into pushing GPs forward into new theory, computation, and application frontiers.

International Indian Statistical Association ³

International Society for Bayesian Analysis (ISBA) ²

Section on Bayesian Statistical Science ¹