Data driven experiment designs for policy learning and evaluation

Oregon Convention Center 

This talk considers procedures for experiments with multiple treatment conditions, in which the experimenter wishes to use the experimental data to learn a policy for assigning treatment in the future: determining which of multiple treatments performed best, on average (best arm identification), or learning a contextual assignment regime, allowing that different treatment conditions may be best for different subgroups of the population with different covariate values (contextual policy learning). The problem requires the experimental planner to determine both experimental treatment allocation procedures, and assignment recommendation procedures using experimental data. Additionally, the experimenter may also like to not only learn, but obtain estimates of mean response under the learned policy. I discuss design considerations and applications to contextual settings where predictive contexts are known and unknown.