Frequentist vs. Bayesian? – Comparing Two Trinomial Proportions in Small Sample Cases

Music City Center 

Classical hypothesis tests implicitly require that an ordering be defined on the elements of a sample space. Colloquially, this ordering is used to describe the "extremity" of a sample element in the direction of the alternative hypothesis. More extreme elements provide greater evidence for the alternative hypothesis and against the null hypothesis. In many applications, especially those described in introductory statistics courses, the notion of extremity is quite natural. However, in other cases, constructing a reasonable ordering of the sample space is far less straightforward. We present such an example in the context of studying rater disagreement in evaluating nursing home patient care needs. The trinomial sample space for these data was difficult to order in a way that was both consistent with the classical statistical practice and needed modification. Ultimately, Bayesian methods were employed to construct a reasonable ordering for use with a classical test of hypotheses.

Bayesian inference

classical inference

p-values

extremity

sample space ordering 

Section on Statistics and Data Science Education