Transporting Treatment Effects from Randomized Trials to EHR Populations using Propensity Score Predictive Inference

Thomas M. Menino Convention & Exhibition Center 

Randomized controlled trials provide strong internal validity, but they often lack external validity when attempting to extend results to a pre-specified external target population. This limitation, known as transportability, arises when the distribution of effect modifiers differs between the trial and target populations of interest. To address this challenge, we develop a novel interaction-based Propensity Score Predictive Inference (PSPI) method that emphasizes the central role of sampling propensity scores for sampling-induced effect heterogeneity, combined with flexible outcome models. We introduce two PSPI variants for estimating average treatment effects and potential outcomes across treatment groups by incorporating a natural cubic spline of the propensity score and modeling high-dimensional covariates using Bayesian Additive Regression Trees. We also introduce a balancing transformation to improve the distributional alignment of propensity scores between trial and target populations. Simulation studies show that PSPI models outperform existing methods, achieving lower root mean squared errors and near-nominal coverage rates, particularly in settings with small sample sizes, imbalanced treatment groups, or covariate shift between trial participants and the target population. We further demonstrate the utility of our approach by transporting the effect of a depression care intervention from the CODIACS Vanguard trial to trial-eligible post-acute coronary syndrome patients with elevated depressive symptoms in the All of Us electronic health records.

Bayesian Additive Regression Trees

Electronic Health Records

Generalizability

Propensity Score

Population average treatment effects

Randomized Trials