A Class of Non-separable Penalty Functions for Bayesian Lasso-like Regression

Music City Center 

Non-separable penalty functions are often used in regression modeling to enforce group sparsity structure, reduce the influence of unusual features, and improve estimation and prediction by providing a more realistic match between model and data. From a Bayesian perspective, such penalty functions correspond to a lack of (conditional) prior independence among the regression coefficients. We describe a class of prior distributions for regression coefficients that generates non-separable penalty functions. The priors have connections to L1-norm penalization and the Bayesian lasso (BL) and elastic net (BEN) regression models. The regularization properties of the class of priors can be understood both by studying its tunable parameters directly and via the connections to BL and BEN regression. We discuss full Bayesian inference under these priors and variable selection via Bayes factors and posterior model probabilities. Inference and prediction under the class of priors is shown to perform competitively under a range of example data structures.

Bayesian elastic net

Bayesian lasso

Penalized regression 

Section on Bayesian Statistical Science