Nonparametric understanding of parametric tests

Oregon Convention Center 

One argument against statistical tests, which have come under intense criticism recently, is that the null hypothesis is never true ("all models are wrong but some are useful"), and therefore it is not informative to reject it.

Given a (parametric) test, a general nonparametric space of distributions can be split up into distributions for which the rejection probability is either (a) smaller (or equal) or (b) larger than the nominal test level. These constitute the "effective null hypothesis" and "effective alternative" of the test. When tests are applied, normally there is an informal research hypothesis, which would be translated into a set of statistical models. This set can be called the "interpretative null hypothesis" (or "interpretative alternative" depending on how the test problem is formulated). Understanding whether a statistical test is appropriate in such a situation amounts to understanding how the effective hypotheses relate to the interpretative hypotheses. This is essentially different from the question whether the test's model assumptions hold, which is not required to apply it.

Foundations of statistics

Frequentism

Statistical tests 

International Statistical Institute