Inference on the Significance of Modalities in Multimodal Generalized Linear Models

Music City Center 

Multimodal statistical models have gained much attention in recent years, yet there lacks rigorous statistical inference tools for inferring the significance of a single modality within a multimodal model. This inference problem is particularly challenging in high-dimensional multimodal models. In high-dimensional multimodal generalized linear models, we propose a novel entropy-based metric, called the Expected Relative Entropy (ERE), to quantify the information gain of one modality in addition to all other modalities in the model. We then propose a deviance-based statistic to estimate the ERE. We prove that the deviance-based statistic is consistent with the ERE and derive its asymptotic distribution, which enables the calculation of confidence intervals and p-values to assess the significance of a given modality. We numerically evaluate the empirical performance of our proposed inference tool on various high-dimensional multimodal generalized linear models and demonstrate its good performance. We also apply our method to a multimodal neuroimaging dataset to demonstrate its capability to infer the significance of imaging modalities, which is crucial for neuroscience studies.

High-dimensional inference

Multimodal data,

Relative Entropy

Sure Independence Screening 

Biometrics Section