Nonparametric Bayesian Q-learning for optimization of dynamic treatment regimes

Oregon Convention Center 

Existing methods for estimation of dynamic treatment regimes are limited to intention-to-treat analyses--which estimate the effect of randomization to a particular treatment regime without considering the compliance behavior of patients. In this article, we propose a novel nonparametric Bayesian Q-learning approach to construct optimal sequential treatment regimes that adjust for partial compliance. We consider the popular potential compliance framework, where some potential compliances are latent and need to be imputed. The key challenge is learning the joint distribution of the potential compliances, which we accomplish using a Dirichlet process mixture model. Our approach provides two kinds of treatment regimes: (1) conditional regimes that depend on the potential compliance values; and (2) marginal regimes where the potential compliances are marginalized. Extensive simulation studies highlight the usefulness of our method compared to intention-to-treat analyses. We apply our method on the Adaptive Treatment for Alcohol and Cocaine Dependence Study (ENGAGE), where the goal is to construct optimal treatment regimes to engage patients in therapy.

Dirichlet process mixture

Endogeneity

Gaussian copula

Intention-to-treat analysis 

Biometrics Section