Consonant closed testing with optimality guarantees

Music City Center 

A central goal in designing clinical trials is to find the test that maximizes power (or equivalently minimizes required sample size) for finding a false null hypothesis while controlling the type I error rate. When there is more than one test, such as in clinical trials with multiple endpoints, the issues of optimal design and optimal procedures become more complex. We address the question of how such optimal tests should be defined and how they can be found. We present a complete solution for deriving optimal procedures for two hypotheses, which have desired monotonicity properties, and are computationally simple. For some of the optimization formulations this yields optimal procedures that are identical to existing procedures, such as Hommel's procedure or the procedure of Bittman et al. (2009), while for other cases it yields completely novel and more powerful procedures than existing ones. For settings with more than two hypotheses, we introduce a bottom-up approach that builds on the closure principle and our optimal two-hypothesis solution to construct new, powerful procedures for familywise error rate control. We demonstrate the advantages of our methods through extensive simulations and real-data applications, highlighting their improved power over existing approaches.

Joint work with Rajesh Karmakar, Abba Krieger, and Saharon Rosset

Family wise error

Most powerful test

Multiple endpoints

Strong control.