03. All Models Are Wrong, but a Set of them is Useful

Prediction performance, in practical settings, is mostly achieved by one sin-
gle (complex) model which nevertheless can lack in interpretability. However,
a recent idea has developed pushing to find a set of equally-performing mod-
els instead of a single one, called the Rashomon set. In this direction, the
Sparse Wrapper Algorithm (SWAG) is a recently proposed multi-model selection
method consisting in a greedy algorithm that combines screening and wrapper
approaches to create a set of low-dimensional models with good predictive power
using a learning mechanism of choice. As a result of its modelling flexibility,
practitioners can pick the model that best reflects their needs and/or domain
expertise without losing accuracy. In addition, the SWAG can deal with many
problematic features in the data such as missing values, outliers, collinearity and
others. Finally, the set of SWAG models can be used to construct a network
that highlights the intensity and the direction of attribute interaction from a
broader and more insightful perspective. We highlight how this method delivers
important results for decison-makers in fields such as genomics, engineering and
neurology.

SWAG

Rashomon Set

Interpretability

Multi-Model Inference

Desicion-Making