Valid F-screening in linear regression

Thomas M. Menino Convention & Exhibition Center 

Suppose that a data analyst wishes to report the results of a least squares linear regression only if the overall null hypothesis is rejected. This practice, which I refer to as F-screening (since the overall null hypothesis is typically tested using an F-statistic), is in fact common practice across a number of applied fields. Unfortunately, it poses a problem: standard guarantees for the inferential outputs of linear regression, such as Type 1 error control of hypothesis tests and nominal coverage of confidence intervals, hold unconditionally, but fail to hold conditional on rejection of the overall null hypothesis. In this talk, I will present an inferential toolbox for the coefficients in a least squares model that are valid conditional on rejection of the overall null hypothesis. In particular, I will construct a selective p-value that leads to tests that are consistent and control the selective Type 1 error, i.e., the Type 1 error conditional on having rejected the overall null hypothesis. Furthermore, I will derive an expression for the Fisher information about the coefficients resulting from the proposed approach, and compare this to the Fisher information that results from an alternative approach that relies on sample splitting. Finally, I will investigate the proposed approach in simulation and via re-analysis of a dataset from the epidemiological literature.

selective inference

linear models

p-values

ANOVA