Using a non-response survey to assess the risk of bias in large-scale educational assessments

Music City Center 

Large-scale educational assessments (LSA) are widely used to derive probabilistic knowledge about the distribution of certain characteristics or outcomes in an educational system. This knowledge is usually used to inform various aspects of education policy in both national and international contexts. A fundamental difficulty these studies face in justifying their conclusions is the inferential problems posed by non-response, which raises concerns about sampling and non-sampling errors in the inference.
We propose using a "non-response survey" (NRS) to investigate the risk of non-sampling errors in the inferences drawn by these studies. The NRS collects information on a subset of variables that are deemed central to the study and is applied to sampled units that did not participate in the LSA. To assess the risk of non-sampling errors, the distribution of each key variable is compared between the information gathered by the LSA and the NRS. Besides discussing the methodological merits of such an analysis, we present the results of a feasibility study in which we conduct an NRS in two countries within the context of the OECD's Teaching and Learning International Study (TALIS).

Non-response

Non-response bias

Total survey error

Identifiability of population parameters

Large-scale educational assessment 

Survey Research Methods Section