15: Flexible Individualized Treatment Strategies in Micro Randomized Trials with Binary Rewards

Music City Center 

Micro-randomized trials (MRTs) are often used in mHealth studies to assess app-based interventions. Participants are randomized to receive treatment at a series of decision points, traditionally using the same rule across individuals. Several recent MRTs utilize Thompson Sampling (TS), a reinforcement learning algorithm, to build individualized treatment strategies that optimize delivery with respect to a reward. Treatment may interact with several contextual features, but estimation of models in this setting can be unreliable. This is especially difficult with a binary reward where complete separation often occurs, even with a large sample and few features. We present an approach to balance algorithmic flexibility and computational cost in the context of a binary reward that (1) uses partial pooling and weakly informative priors that apply more shrinkage to higher-order interactions and (2) considers the amount of information available in the data when defining a model. Our approach is useful in MRTs where the TS algorithm must be automated. We demonstrate the empirical utility of our method in a digital twin of an ongoing MRT study, LowSalt4Life, compared to logical alternatives.

Mobile health

Micro-randomized trials

Clinical trials

Reinforcement learning

Individualized treatment 

Biometrics Section