Semi-Supervised Transformer for Rare Disease Phenotyping

1442 

Contributed Abstract 

Poster 

Kimberly Greco (1), Zongxin Yang (2), Mengyan Li (3), Tianxi Cai (1)

(1) Harvard University, N/A, (2) Harvard Medical School, N/A, (3) Bentley University, N/A

Zongxin Yang  
Harvard Medical School

Mengyan Li  
Bentley University

Tianxi Cai  
Harvard University

Kimberly Greco  
Harvard University

Kimberly Greco  
Harvard University

Training robust phenotyping algorithms for rare disease research is challenging due to the scarcity of labeled data and biases in label sources. Gold-standard labels from registries and expert chart review ensure accuracy but suffer from selection bias and high ascertainment costs, while electronic health record (EHR)-derived labels encompass a broader patient population but introduce noise. To address these challenges, we propose a semi-supervised transformer framework that integrates gold-standard labels with iteratively updated silver-standard labels from structured EHR data to train a flexible disease classifier. Initial medical code embeddings are learned from EHR co-occurrence patterns, then refined and aggregated into patient-level representations via a multi-layer transformer that leverages self-attention to dynamically capture long-range dependencies across a patient's medical history. Validated on EHR data from Boston Children's Hospital, our model significantly improves downstream phenotype classification, patient clustering, and disease progression prediction over baseline methods, offering a scalable and generalizable solution for phenotyping in data-limited settings.

Semi-Supervised Learning|Transformers|Phenotyping|Electronic Health Records|Rare Diseases| Machine Learning

Section on Statistical Learning and Data Science

Machine Learning