Testing Random Effects for Binomial Data: Minimax Goodness-of-Fit Testing and Meta-analyses

Music City Center 

In many modern scientific investigations, researchers conduct numerous small-scale studies with few participants. Since individual participant outcomes can be difficult to interpret, combining data across studies via random effects has become standard practice for drawing broader scientific conclusions. In this talk, we introduce an optimal methodology for testing properties of random effects arising from binomial counts. Using the minimax framework, we characterize how the worst-case power of the best Goodness-of-fit test depends on the number of studies and participants. Interestingly, the optimal test is related to a debiased version of Pearson's chi-squared test.

We then turn to meta-analyses, where a central question is to determine whether multiple studies agree on a treatment's effectiveness before pooling all data. We show how the difficulty of this problem depends on the underlying effect size and demonstrate that a debiased version of Cochran's chi-squared test is minimax-optimal. Finally, we illustrate how the proposed methodology improves the construction of p-values and confidence intervals for assessing the safety of drugs associated with rare adverse outcomes.

hypothesis testing

meta-analysis

local minimax

critical separation

Wasserstein distance

homogeneity testing 

IMS