05 Choosing an indifference zone for second-generation p-values to maximize the statistical properties

Oregon Convention Center 

Second-generation p-values (SGPVs) were proposed to address the imperfections of classical p-values. They maintain the favorable properties of classical p-values while emphasizing scientific relevance to expand their utility, functionality, and applicability. They report evidence in favor of the alternative, in favor of the null hypothesis, or neither (inconclusive); they automatically incorporate an adjustment for multiple comparisons; they have lower false discovery rates; and they are easier to interpret. The most crucial component of a SGPV analysis is choosing the indifference zone. In practice, this is not easily done, as statisticians and collaborators do not always agree. We explore how choosing different indifference zones affects the SGPVs' statistical properties, and we propose allowing the indifference zone to 'shrink' in cases of collaborator uncertainty. We demonstrate that when an uncertain but wide small sample indifference zone is identified, shrinking it balances the errors between behaviors of a fixed zone and a point null. This trade-off leads to improved communication between statisticians and collaborators when planning an SGPV analysis.