Mixed Integer Programming for Feature Selection in Scalar-on-Function Regression

Thomas M. Menino Convention & Exhibition Center 

Feature selection is a critical challenge in model selection, particularly for functional data, where appropriate statistical methodologies remain underdeveloped. This study investigates the application of Mixed Integer Programming (MIP) combined with information criteria for best feature subset selection in scalar-on-function regression (i.e., regression models where predictors are curves). Utilizing the computational power of an optimization tool uniquely allows us to employ combinatorics in feature selection, identifying the true best subset of features by comparing which minimizes the residuals the most. Transforming the functional regression problem into a classic linear model framework with grouped variables allows the use of model selection criteria such as Bayesian Information Criterion (BIC), in combination with MIP. In simulation studies, we compared our MIP method to alternative approaches and found that it consistently identifies truly active features, while not overselecting inactive features.

Functional Data Analysis

Mixed Integer Programming

Model Selection 

Caucus for Women in Statistics and Data Science