Supervised Dimension Reduction for Regression Models with High-Dimensional Output

Music City Center 

Regression models with high-dimensional response are increasingly ubiquitous across various domains, including computer experiments with high-dimensional output. Current methodology involves compressing the response using Unsupervised Dimension Reduction (UnsuperDR) techniques such as Singular Value Decomposition (SVD), and training regression models to predict the compressed values. We implement a novel Supervised Dimension Reduction (SuperDR) approach to infer an optimal linear compression within a comprehensive statistical model to simultaneously compress and predict high-dimensional response variables. Leveraging recent advances in SuperDR for linear models and regression modeling for multivariate output, our approach alternates between estimating a compressed regression model and an expansion matrix, theoretically converging to an optimal solution. Our framework is agnostic to the chosen regression model, as demonstrated by our implementation with Polynomial Chaos Expansion and Random Forests regression. We compare the effectiveness of SuperDR against the state-of-the-art UnsuperDR framework.

Supervised Dimension Reduction

High-Dimensional Response

Nonlinear Regression 

Section on Statistical Learning and Data Science