Testing Random Effects for Binomial Data: Minimax Goodness-of-Fit Testing and Meta-analyses

2152 

Contributed Abstract 

Paper 

Lucas Kania (1), Sivaraman Balakrishnan (1), Larry Wasserman (1)

(1) Carnegie Mellon University, N/A

Sivaraman Balakrishnan  
Carnegie Mellon University

Larry Wasserman  
Carnegie Mellon University

Lucas Kania  
Carnegie Mellon University

Lucas Kania  
Carnegie Mellon University

In many modern scientific investigations, researchers conduct numerous small-scale studies with few participants. Since individual participant outcomes can be difficult to interpret, combining data across studies via random effects has become standard practice for drawing broader scientific conclusions. In this talk, we introduce an optimal methodology for testing properties of random effects arising from binomial counts. Using the minimax framework, we characterize how the worst-case power of the best Goodness-of-fit test depends on the number of studies and participants. Interestingly, the optimal test is related to a debiased version of Pearson's chi-squared test.

We then turn to meta-analyses, where a central question is to determine whether multiple studies agree on a treatment's effectiveness before pooling all data. We show how the difficulty of this problem depends on the underlying effect size and demonstrate that a debiased version of Cochran's chi-squared test is minimax-optimal. Finally, we illustrate how the proposed methodology improves the construction of p-values and confidence intervals for assessing the safety of drugs associated with rare adverse outcomes.

hypothesis testing|meta-analysis|local minimax|critical separation|Wasserstein distance|homogeneity testing

IMS

Statistical Theory