A Cautionary Note for Borrowing External Controls in Clinical Trials

Thomas M. Menino Convention & Exhibition Center 

Leveraging external control (EC) data can improve efficiency in estimating causal effects in clinical trials, but the use of non-randomized data complicates statistical inference. Propensity score (PS)-based adjustments are commonly used to remove confounding bias under ignorability assumptions, yet the variance behavior of the resulting inferential statistics is less studied, and naive application can lead to erratic type-I error rates. This paper investigates the behavior of test statistics for commonly used procedures after PS-based EC borrowing, identifies the reasons for misbehaved type-I errors, and proposes remedies for PS matching and weighting in single-arm and hybrid randomized controlled trial (RCT) designs. Extensive simulation studies demonstrate that the proposed remedies provide adequate variance estimation and recover type-I error rates close to the nominal level. Based on these results, we offer recommendations for practitioners to ensure valid causal effect estimation when borrowing ECs and illustrate the methods using a real RCT.

External data borrowing

matching

weighting

bootstrap

type-I error 

Biopharmaceutical Section