Online Testing of Grouped Hypotheses

Music City Center 

Recently there has been a growing interest in "online" testing of hypotheses, where the hypotheses are generated sequentially, potentially over an infinite period. Online testing procedures make real time decisions for each hypothesis, before future hypotheses are available, with the goal of controlling an overall error measure related to the False Discovery Rate at every decision point. We consider an online testing problem where at every time point, a group of hypotheses is obtained, and such groups are obtained sequentially, possibly indefinitely. Testing of such grouped hypotheses involves combining online testing procedures with offline procedures that leverage the grouping structure of the hypotheses. Our proposed online grouped testing method is based on the local false discovery rate, and inspired from the online procedure proposed by Gang, Sun and Wang (2021), and grouped hypotheses testing procedure proposed by Sarkar and Zhao (2022). This talk will introduce our method, discuss the role of alpha investment, a key strategy for controlled allocation of significance level to each test, and theoretical guarantee of control on an overall error measure while optimizing power.

Multiple Hypotheses Testing

Online Hypotheses Testing

Grouped Testing

Local FDR 

IMS