Improving the Estimates from Web Panel Surveys Using Probability Reference Surveys and Multiple Imputation

Thomas M. Menino Convention & Exhibition Center 

To improve the timeliness of data products, survey researchers and practitioners have increasingly
used web surveys and alike to collect information for population health research and dissemination. From the statistical inferential perspective, some of these web panel surveys may have a lack of a well-defined probability sampling structure while other web panel surveys may be probability-based, but may still be subject to high nonresponse and/or coverage errors. Certain statistical adjustments are therefore needed to make proper inferences using such web panel surveys. With a high-quality reference probability survey available, one popular adjustment approach is to create pseudoweights that properly "weight" the web panel survey samples back to the target population underlying the reference survey in order to produce population-weighted estimates of the target of interest. When the variable of interest is collected in the web survey but not in the reference survey, the analytical question can also be framed as a missing data problem. Thus we propose to multiply impute the missing variable on the reference survey combining the information from both data sources. Through various comparisons, we demonstrate that results from different imputation and pseudoweighting models can be compared and used to understand features of the data to aid the analysis. We illustrate the main features and performance of the multiple imputation strategy using a simulation study. We also present a real data analysis based on the web-based Research and Development Survey and the National Health Interview Survey.

Nonprobability survey

Nonresponse error

Missingness mechanism

Multiple imputation

Propensity score

Pseudoweights