Optimally adaptive test for high dimensional hypotheses via minimax deficiency

Music City Center 

The detection boundary is a tool for power evaluation of a high dimensional test, which provides a binary phase transition of power in terms of signal density and strength. However, it cannot separate the $L_{2}$ and higher criticism (HC) tests under dense signals, and the $L_{\infty}$ and HC tests under highly sparse signals as they share the same detection boundary. This paper proposes minimax relative deficiency and minimax absolute deficiency as sharper measures for power evaluation than the detection boundary, and develop an adaptive testing procedure by combining three basic tests via a power enhancement. The proposed test is robust to the unknown signal density and strength with sharp optimal relative deficiency and nearly optimal absolute deficiency over the whole signal density regime. A full comparison of the proposed test with the existing methods is provided using the minimax deficiency measures. Simulation studies and a real data application to climate change analysis are conducted to evaluate the proposed test and demonstrate its superiority.

deficiency

detection boundary

high dimensionality

minimax optimality

power enhancement 

IMS