Scalable Fully Bayesian Framework for Gaussian Process Model with Built-in Input Dimension Reduction

Thomas M. Menino Convention & Exhibition Center 

Gaussian process models(GP) are widely used for surrogate modeling because they provide flexible regression with uncertainty quantification, but their use is limited by high-dimensional inputs and cubic computational cost. Most existing methods address dimension reduction(DR) and scalability separately, often through multi-stage or approximate procedures that weaken uncertainty propagation. We propose a unified, fully Bayesian framework that jointly performs input DR and GP modeling within a single hierarchical model. DR is achieved through an orthonormal projection matrix with priors on the Stiefel manifold, enabling coherent posterior learning via Hamiltonian Monte Carlo with geodesic dynamics. We extend this framework to Deep GP with built-in DR to model complex nonlinear systems. To scale inference to large datasets, we incorporate Vecchia sparse covariance approximations, reducing computational complexity from cubic to near-linear while preserving predictive accuracy and calibrated uncertainty. Extensive numerical studies show the Bayesian method with Vecchia scaling gives better predictions and more reliable uncertainty, providing a robust alternative to existing methods.

Bayesian inference

Dimension reduction

Deep Gaussian processes

Hamiltonian Monte Carlo

Uncertainty quantification

Vecchia approximation 

Uncertainty Quantification in Complex Systems Interest Group