Bias Evaluation for Web Health Surveys, A Sensitivity Analysis Approach

Oregon Convention Center 

Survey researchers have increasingly used web surveys to collect information for population health research and dissemination. These data are often referred to as nonprobability samples due to the lack of a well-defined probability sampling structure. Certain statistical adjustments are therefore needed to make proper inferences using web surveys. One popular adjustment approach is to create pseudo weights that properly ``weight'' the web survey samples back to the target population using a reference survey. Li et al. (2022) showed that it is crucial to include variables that are both related to the outcome of interest and the sample selection into web surveys ("confounders" in epidemiology) in the weighting adjustment. In practice, however, it can be challenging to ensure that all important confounders are included in the data collection and subsequent weighting adjustment. Therefore, a plausible strategy for evaluation is to conduct a sensitivity analysis based on the web survey estimates when certain confounders are excluded. We illustrate this idea using public-use data from the from the Research and Development Survey and National Health Interview Survey.