Survey Costs: What Measures Are Used in Survey Methodology and Statistics Research?

Thomas M. Menino Convention & Exhibition Center 

Survey costs have long been acknowledged as one of the most important constraints on survey designs. Despite this importance, survey costs and their correlates remain largely hidden. When costs are discussed, different measures are used and inputs into the cost measures are inconsistently defined, leading to difficulties in comparing costs across studies. The goal of this paper is to examine how costs are defined in published survey methodological and statistical articles. To do so, this paper content analyzes over 1500 articles in four journals dedicated solely to survey methodology and statistics research. Costs for each of these papers are coded according to the categories of monetary and nonmonetary costs, the cost metric, the data source for costs, and whether the costs reflect fixed or variable costs. Only 10 percent of these articles report any cost information or include costs in a statistical formula. Of this 10 percent, about 95 percent examine variable costs, but there is little similarity in other aspects of cost measurement, including whether the costs are estimated or observed in data, the units of analysis for costs (e.g., total, costs per sampled unit, costs per complete), or whether costs are reported for a single part of a study design or summed over multiple components. Nine common measures of nonmonetary costs are identified, and almost 30 inputs into monetary costs are used in different combinations across studies. Although standardized cost measures may be developed in the future, for now, researchers should clearly define cost measurements and consider analyses of multiple studies within organizations or across partnering organizations to identify associations between costs, survey design features, and error indicators.

Survey Costs

Data collection

Variable costs 

Survey Research Methods Section