More of Less: A Rashomon Algorithm for Sparse Model Sets

Music City Center 

The traditional statistical/machine-learning paradigm generally seeks a single best model for prediction and interpretation. However, the Rashomon Effect, introduced by Leo Breiman, challenges this by highlighting how multiple equally good predictive models can exist for the same problem. This has significant implications for interpretation, usability, variable importance, and replicability. The collection of such models within a function class is called the Rashomon set, and recent research has focused on identifying and analyzing these sets. Motivated by sparse latent representations in high-dimensional problems, we propose a heuristic method that finds sets of sparse models with strong predictive power. Using a greedy forward-search, the algorithm builds progressively larger models by leveraging good low-dimensional ones. These sparse models, maintaining near-equal performance to common reference models (i.e. Rashomon Sets), can be connected into networks that provide deeper insights into variable interactions and into how latent variables contribute to the Rashomon Effect.

SWAG: Sparse Wrapper Algorithm

Latent Representations

Replication Crisis

High-Dimensional Problems

Variable Importance

Explainable AI 

Section on Statistical Learning and Data Science