Evaluating and Testing for Actionable Treatment Effect Heterogeneity

Music City Center 

Developing tools for estimating heterogeneous treatment effects (HTE) has been an area of active research in recent years. While these tools have proven to be useful in many contexts, a concern when deploying such methods is the degree to which incorporating HTE into a prediction model provides an advantage over methods which do not allow for treatment effect variation. To address this, we propose a procedure which evaluates the extent to which an HTE model provides a predictive advantage by targeting the gain in predictive performance from using a flexible predictive model incorporating HTE versus a similar alternative model which that is constrained to not allow variation in treatment effect. By drawing upon recent work on nested cross-validation techniques for prediction error inference, we generate confidence intervals for this measure of gain in predictive performance which allows one to calculate the level at which one is confident
of a substantial HTE-modeling gain in prediction - a quantity which we refer to as the h-value. Our procedure is generic and can be used to assess the benefit of modeling HTE for any method that incorporates treatment effect variation.

interaction

model comparison

precision medicine

resampling 

Biometrics Section