Step on that Rake! A Weighting Approach to Handle Complex Randomization in Natural Experiments

Music City Center 

Randomized control trials eliminate confounding and reduce selection bias, requiring simple comparisons to estimate ATEs; however, in the real world, unequal group sizes, unbalanced covariates, and practical difficulties complicate implementation. Analytic strategies often employ statistical controls to adjust for such complications. Instead, we describe our use of weights to adjust for differential ratios across study sites and randomization blocks from a standardized lottery process that allocates housing. Each residential building held its own lottery following the same protocol; however, each building comprised different types of units that each had their own eligibility criteria and corresponding differences in supply relative to demand that produced the treatment (those offered housing) and control (those not offered housing) groups. Applying iterative proportional fitting, or raking, we create weights to address overlapping and intersecting unit type eligibility. Utilizing data at T2, we demonstrate how this plays out for analytic purposes and compare our approach and various other strategies and the differences in ATE that result.

Randomized control trial

natural experiment

weighting

raking 

Government Statistics Section