Scalable Multi-Response Learning with Effect Decomposition

Thomas M. Menino Convention & Exhibition Center 

We study high-dimensional multiple sparse linear regression in settings where the coefficients for different responses can exhibit shared support. Existing $\ell_1/\ell_q$ regularization approaches can exploit this structure but often require complex joint optimization over multiple penalties and tuning over a two-dimensional regularization grid, limiting scalability to modern data settings. Leveraging recent advances in scalable group-lasso penalty optimization, we propose a decomposable hybrid regularization model and show its equivalence to a single weighted group-lasso problem, enabling efficient computation and reducing parameter tuning complexity. We further provide a framework for interpreting across-response heterogenity and study conditions under which the resulting coefficient estimates are identifiable, connecting uniqueness of the decomposition to interpretability. We present empirical results to compare the performance and scalability of our proposed method with existing regularization approaches.

Multiple sparse regression

Variable selection

High dimensional statistics

Multitask learning