30: Choosing a Significance Level for Sequential Hypothesis Testing in Online Change Point Detection

Music City Center 

Change point detection (CPD) is the process of finding distributional changes in time-ordered data streams. When CPD is applied repeatedly as data is collected, this is referred to as online CPD. Online CPD is employed in smartphone-based digital phenotyping studies in psychiatric populations to detect changes in behavior as they occur so a patient's care team can intervene to prevent a potential adverse event. By repeatedly performing CPD hypothesis tests in this way, a multiple testing problem quickly emerges. Furthermore, since consecutive tests incorporate a subset of the same days, the test statistics will be highly correlated. Thus, multiple testing methodology that assume independence or even allow for a small degree of correlation are not appropriate. In settings where we want to minimize false alarms while maintaining adequate sensitivity, it is important to use an appropriate significance level for each test that accounts for this high degree of serial correlation. We present an approach that adjusts for sequential hypothesis testing in the online CPD setting and demonstrate its effectiveness in a digital phenotyping study of patients with mood affective disorders.

Change point detection

Sequential hypothesis testing

Digital phenotyping 

Mental Health Statistics Section