Utilizing Data from an Incomplete Sample to Supplement the Probability-Based U.S. PIAAC Cycle II

Oregon Convention Center 

Incomplete survey data can arise when there are unexpected disruptions to data collection, resulting in a sample that is a product of the probability-based sample design, the non-probabilistic mechanism that determined which sampled cases were worked, and nonresponse. In this paper, we describe a method used in the U.S. PIAAC Survey for combining incomplete survey data with complete survey data. The sample design consisted of a core national sample and a state-based supplemental sample. Data collection for the state supplement was halted less than halfway into the data collection period, before interviewers had visited all areas. Although the core sample was sufficient for national estimates, including the partial data from the state supplement could help improve small area estimates and psychometric modeling. We combined the two samples by using a composite weighting technique, where the compositing factor was based on effective sample size to reflect variance and a Kolmogorov-Smirnov statistic to reflect potential bias. As an evaluation, we compared survey estimates, variances, and measures of association using the resulting weights compared to those for the weighted core sample.