Title: Integration of Concepts, Methodology and Empirical Results on Biases from Incomplete Data in Survey and Non-Survey Information Sources
Author: John L. Eltinge, United States Census Bureau
[email protected] Key words: auxiliary data; data quality; incomplete frame coverage; total survey error model; total uncertainty analysis; unit, wave and item survey nonresponse
Abstract:
This paper reviews and integrates the wide range of literature on concepts, methodology and empirical results related to biases from incomplete data in survey and non-survey information sources. Two areas receive principal attention.
The first area focuses on analyses of incomplete-data biases as such, and on related mitigation efforts. This includes work with incomplete-data patterns arising from:
- unit, wave and item nonresponse in sample surveys; and
- problems with administrative records and other organic-data sources used to develop and refine survey frames, weighting and imputation procedures, and also used as direct inputs for production of statistical information
The discussion of incomplete-data bias places special emphasis on:
- availability, costs and limitations of auxiliary data used for evaluation of biases;
- development and evaluation of models used for those evaluations; and
- reporting of empirical results from those evaluations
The second area focuses on integration of nonresponse bias into a broader context, including:
- Comparison of the magnitudes of incomplete-data biases with the magnitudes of other components of total survey error models, e.g., measurement error and modeling error
- Quantitative and qualitative assessment of the ways in which incomplete-data biases, and related mitigation efforts, may affect multiple dimensions of data quality, e.g., accuracy; comparability; temporal and cross-sectional granularity; interpretability; and relevance
- Evaluation of the impact of incomplete-data bias on the value delivered to stakeholders through a specified suite of statistical information products
- Transparent and actionable communication with stakeholders regarding the above-mentioned concepts and empirical results