Higher-criticism for sparse multi-stream change-point detection

Music City Center 

We present a statistical procedure based on higher criticism (HC) to address the sparse multi-stream (or multi-sensor) quickest change-point detection problem. Namely, we aim to detect a potential change in the distribution of many data streams at some unknown time. If a change occurs, only a few streams are affected, whereas the identity of the affected streams is unknown. The HC-based procedure involves testing for a change point in individual streams and combining multiple tests using higher criticism. Relying on the HC thresholding mechanism, the procedure also indicates a set of streams suspected to be affected by the change.

We demonstrate the effectiveness of the HC-based method compared to other methods through extensive numerical evaluations. Additionally, we provide a theoretical analysis under a sparse heteroscedastic normal change-point model. We establish an information-theoretic detection delay lower bound when individual tests are based on the likelihood ratio or the generalized likelihood ratio statistics and show that the delay of the HC-based method converges in distribution to this bound. In the special case of constant variance, our bounds coincide with known results in (Chan, 2017).

This is a joint work with Tingnan Gong and Yao Xie.

Change-point detection

Higher criticism

p-values