Optimizing Data Collection Interventions to Balance Cost and Quality in a Sequential Multimode Surve

Music City Center 

Responsive and adaptive designs have emerged as a framework for targeting and reallocating resources during the data collection period in order to improve survey data collection efficiency. Here, we report on the implementation and evaluation of a responsive design experiment in the National Survey of College Graduates that optimizes the cost-quality tradeoff by minimizing a function of data collection costs and the root mean squared error of a key survey measure, self-reported salary. At three points during the data collection process, we predict outcomes and costs for remaining non-respondents and combine with data from respondents to optimize effort on remaining cases with respect to cost and root mean squared error (RMSE) of mean self-reported salary This process allowed us to reduce data collection costs by nearly 10%, without a statistically or practically significant increase in the RMSE of mean salary or decrease in the unweighted response rate. This experiment demonstrates the potential for these types of designs to more effectively target data collection resources in order to reach survey quality goals.

Responsive design

National Survey of College Graduates

Posterior predictive distribution 

Survey Research Methods Section