Nonparametric Empirical Bayes and Selective Inference

Music City Center 

Consider a multi-population inference problem where it is of interest to estimate the mean of the population with the highest observed sample average. The usual confidence interval does not work in this case -- offering increasingly lower coverage than the nominal value when the total number of populations gets larger. This phenomenon is often referred to as the Winner's Curse. Various modifications have been proposed to adjust for the selection step. We show that interval procedures that guarantee nominal coverage conditional on the selection event typically have infinite expected length. This result motivates us to consider empirical Bayesian solutions which offer coverage guarantees only on average over some parameter subspace. Nonparametric empirical Bayesian solutions are shown to generally offer good coverage with high precision but can perform poorly when one population is very different from all others -- a clear violation of the underlying exchangeability assumption. We conclude with further mitigation strategies and discuss their frequentist and Bayesian interpretations.

Selective inference

Winner's curse

Infinite length confidence intervals

Hierarchical Bayes

Empirical Bayes

Predictive recursion