32: Robustness of OLS, Ridge, Lasso, and Elastic Net in Presence of Outliers: Simulation and Application

Music City Center 

In linear regression models, multicollinearity often results in unstable and unreliable parameter estimates. Ridge regression, a biased estimation technique, is commonly used to mitigate this issue and produce more reliable estimates of regression coefficients. Several estimators have been developed to select the optimal ridge parameter. This study focuses on the top 16 estimators from the 366 evaluated by Mermi et al. (2024), along with seven additional estimators introduced over time. These 23 estimators were compared to Ordinary Least Squares (OLS), Elastic-Net (EN), Lasso, and generalized ridge (GR) regression to evaluate their performance across different levels of multicollinearity. Simulated data, both with and without outliers, and various parametric conditions were used for the comparisons. The results indicated that certain ridge regression estimators perform reliably with small sample sizes and high correlations without outliers. However, some estimators performed better when outliers were present due to small sample sizes and increased variance. GR, EN, and Lasso were robust with large datasets, except with substantial outliers and high variance.

MSE

Multicollinearity

Ridge regression

Lasso

Elastic net

OLS 

Section on Statistical Computing