Exploring Best Practices for Inferential Analysis of Paradata-Derived Statistics

Thomas M. Menino Convention & Exhibition Center 

Paradata from internet surveys can yield information about respondent behavior that is useful in evaluating questionnaire performance. Because paradata (e.g., timestamps, browser information, and user actions) are not designed with analysis in mind but rather to facilitate instrument functionality, output files can be complex and ill-structured for analytic purposes. For instance, a respondent's paradata record for the time spent interacting with the questionnaire may be hundreds or thousands of lines long, depending on how many actions occurred each session.

Researchers have adapted to the unwieldy nature of paradata analysis; however, it is often limited to descriptive comparisons without the rigor of statistical inference. The atypical data structure can introduce challenges to statistical best practices, complicating the calculation of variance estimates that serve inferential analysis of paradata-derived statistics. In this paper, the authors apply nonparametric methods to these metrics using data from a nationally representative government survey. These results may better inform researchers in maximizing the utility of this valuable data source.

Paradata

Inference

Nonparametric 

Government Statistics Section