Multistudy Multimodal Pretraining and Transfer Learning

Thomas M. Menino Convention & Exhibition Center 

Machine learning has become integral to biomedical research, yet its success often depends on large, high-quality datasets that are rarely available. Transfer learning (TL) offers a principled way to leverage information from related large-scale datasets to improve prediction in a target study. However, existing TL methods are typically designed for single-view settings, limiting their use with complex multiview data collected across multiple cohorts. To address this gap, we propose a Multistudy Multimodal TL framework that enables integration-aware knowledge transfer across studies and modalities. The framework employs pretraining using cooperative learning followed by fine-tuning, yielding both general and context-specific predictions. Simulation studies and multimodal analyses from cancer immunotherapy and inflammatory bowel disease cohorts demonstrate that our method improves predictive accuracy and cross-study generalization. Our method thus establishes a principled and scalable foundation for cross-study and cross-modality transfer learning, substantially outperforming published methods in estimation and prediction. An open-source implementation is publicly available.

Transfer learning

Cooperative Learning

Multimodal Integration

Multistudy

Pretraining

Regularization