09. Developing, Validating, and Analyzing an Assessment that Includes Interactions Among Learning Objectives Related to Confidence Intervals

Hypothesis testing is typically presented as a rote multi-step procedure. This training has cultivated the current mindset held by many disciplines in that only statistically significant results are meaningful. With a recent desire to shift away from the dichotomous formal decision framework, interval estimation is of increasing importance. Confidence intervals are also susceptible to being treated in a dichotomous way if the manner in which they are presented focuses solely on whether the null value falls in the interval. As such, it is insufficient for just the presentation of the statistical findings to shift from a p-value to a confidence interval; the mindset must shift as well from that of desiring statistical significance to that of desiring statistical transparency of results. If students will be encouraged in the foreseeable future to communicate their findings using interval estimates, then it is imperative that statistics instructors have the means necessary to assess students' conceptual understanding of confidence intervals.

This study serves as a prototype for how to develop, validate, and analyze an instrument that includes interaction effects among learning outcomes. In this study, we take an innovative approach to assessment development by employing a fractional factorial design to highlight the interactions among key learning objectives related to confidence intervals. We use qualitative think-aloud interviews to validate the instrument and to identify students' epistemic understandings about confidence intervals. We collect data from participating large-enrollment introductory statistics students at Penn State to measure students' statistical literacy surrounding confidence intervals upon the conclusion of an introductory statistics course using simulation-based inference methods. This research could inform the findings of previous studies on statistical literacy that include confidence intervals as one of many topics being assessed.

Mid-Level

Knowledge

Women in Statistics and Data Science 2024