Fractional Ridge Regression: A New Perspective on Shrinkage Regression and Variable Selection

Music City Center 

l_p norm penalization, notably the Lasso, has become a standard technique, extending shrinkage regression to subset selection. Despite aiming for oracle properties and consistent estimation, existing Lasso-derived methods still rely on shrinkage toward a null model, necessitating careful tuning parameter selection and yielding stepwise variable selection. This research introduces Fractional Ridge Regression (Fridge), a novel generalization of the Lasso penalty that penalizes only a fraction of the coefficients. Critically, Fridge shrinks the model toward a non-null model of a prespecified target size, even under extreme regularization. By selectively penalizing coefficients associated with less important variables, Fridge aims to reduce bias, improve performance relative to the Lasso, and offer more intuitive model interpretation while retaining certain advantages of best subset selection.

Shrinkage Regression

Regularization

Variable Selection

Sparse Modeling 

Section on Statistical Learning and Data Science