Prototype-driven fusion of pathology and spatial transcriptomics for interpretable survival prediction

Thomas M. Menino Convention & Exhibition Center 

Integrating pathology images with spatial transcriptomics provides comprehensive insights into the tissue microenvironment. However, a significant challenge remains in translating these multi-modal data into clinical outcome predictions at the cohort level. While existing spatial transcriptomics (ST) tools excel at within-sample analysis and cross-modal gene expression imputation, there remains a lack of a unified framework for direct, cohort-level prognostication. Here,  we introduce PathoSpatial, an interpretable multimodal framework that integrates co-registered whole slide images (WSIs) and ST to learn spatially informed prognostic representations. PathoSpatial uses task-guided prototype learning within a hierarchical, multi-level expert architecture, decoupling unsupervised modality-specific prototype discovery from supervised cross-modal prototype aggregation. This design improves discriminative capability while preserving biological interpretability. Evaluated on a large breast cancer ST cohort, PathoSpatial delivers the best overall performance on survival outcome prediction, consistently surpassing or matching leading uni- and multimodal methods. Post-hoc prototype interpretation and molecular risk decomposition provide quantitative, biologically grounded explanations and highlight clinically actionable determinants of prognosis, advancing scalable multimodal integration for next-generation computational pathology.