A Statistical Test for SWAG: Assessing the Rashomon Effect in Sparse Model Selection

Music City Center 

The Rashomon effect, introduced by Leo Breiman, highlights the existence of multiple plausible models explaining the same problem. The Sparse Wrapper Algorithm (SWAG) addresses this effect in high-dimensional settings by selecting sets of strong yet sparse models. However, as a heuristic method, its effectiveness requires formal validation. We propose a statistical testing framework to assess whether SWAG extracts informative models. Instead of testing individual models, we leverage the SWAG network and apply graph-theoretical tools to evaluate its informativeness. By comparing SWAG networks under null and alternative hypotheses, we examine whether selected models concentrate around significant variables through the use of network measures to quantify these differences. Using a bootstrap approach to approximate test statistic distributions, we confirm that entropies of eigenvector centrality and variable frequency are two strong candidates that differentiate the SWAG network under the null and the alternative hypotheses. This framework offers a foundation for inference on Rashomon sets and for interpreting model diversity.

SWAG

Multi-model selection

Statistical testing

Rashomon effect

Sparse models 

Section on Statistical Learning and Data Science