17: Scalable density regression using logistic Gaussian processes: a generalized variational approach

Thomas M. Menino Convention & Exhibition Center 

We propose a scalable framework for conditional density regression based on logistic Gaussian processes. A barrier to scalable computation for these models has traditionally been the need to calculate observation dependent normalizing constants by numerical integration. We avoid this by using generalized Bayesian inference, replacing the negative log-likelihood with a Hyvärinen score. This score-based approach depends only on derivatives of the response's log density, eliminating the need to compute any normalizing constants. However, the required Gaussian process computations can still be computationally expensive. We address this using sparse inducing point variational approximations, making our method scalable to large datasets. Our nonparametric prior can be centered on an existing parametric model. The nonparametric corrections provide interpretable diagnostics that reveal inadequacies in the base model, such as missing covariate-dependent heteroscedasticity and skewness in the response distribution. We demonstrate good predictive performance and scalability through simulated and real examples, including a large spatio-temporal temperature dataset.

Density regression

Generalized Bayesian inference

Variational inference

Logistic Gaussian pro-
cess

Misspecification 

Section on Bayesian Statistical Science