Aggregating Dependent Signals with Heavy-Tailed Combination Tests

Music City Center 

Combining dependent p-values poses a long-standing challenge in statistical inference, particularly when aggregating findings from multiple methods to enhance signal detection. Recently, p-value combination tests based on regularly varying-tailed distributions, such as the Cauchy combination test and harmonic mean p-value, have attracted attention for their robustness to unknown dependence. This paper provides a theoretical and empirical evaluation of these methods under an asymptotic regime where the number of p-values is fixed and the global test significance level approaches zero. We examine two types of dependence among the p-values. First, when p-values are pairwise asymptotically independent, such as with bivariate normal test statistics with no perfect correlation, we prove that these combination tests are asymptotically valid. However, they become equivalent to the Bonferroni test as the significance level tends to zero for both one-sided and two-sided p-values. Empirical investigations suggest that this equivalence can emerge at moderately small significance levels. Second, under pairwise asymptotic dependence, such as with bivariate t-distributed test statistics, these combination tests can remain valid with certain choices of heavy-tailed distributions and exhibit notable power gains over Bonferroni, even as the significance level diminishes.

Cauchy combination test

Dependent p-values combination

Quasi-asymptotic independence

t-copula 

IMS