A model-agnostic ensemble framework with built-in LOCO feature importance inference

Music City Center 

Interpretability and reliability are crucial desiderata when machine learning is applied in critical applications. However, generating interpretations and uncertainty quantifications for black-box ML models often costs significant extra computation and held-out data. In this talk, I will introduce a novel ensemble framework where one can simultaneously train a predictive model and gives uncertainty quantification for its interpretation, in the form of leave-one-covariate-out (LOCO) feature importance. This framework is almost model-agnostic, can be applied with any base model, for regression or classification tasks. Most notably, it avoids model-refitting and data-splitting, and hence there is no extra cost, computationally and statistically, for uncertainty quantification. To ensure the inference validity without data splitting, we address a number of challenges by leveraging the stability of the ensemble training process. I will discuss some broad connection of this work to selective inference, and other model-agnostic feature importance inference methods. I will also demonstrate the framework via some real benchmark datasets.

Feature interaction, random ensembles, uncertainty quantification