What's the Weight? Estimating Controlled Outcome Differences in Complex Surveys for Health Disparities Research

Oregon Convention Center 

A basic descriptive question in statistics asks whether there are differences in mean outcomes between groups based on levels of a discrete covariate (e.g., racial disparities in health outcomes, differences in opinion based on political party identification, heterogeneity in educational outcomes for students in urban vs. rural school districts, etc). When this categorical covariate of interest is correlated with other factors related to the outcome, however, direct comparisons may lead to erroneous estimates and invalid inferential conclusions without appropriate adjustment. Propensity score methods to adjust for such confounding are broadly employed with observational data as a tool to achieve covariate balance, but implementing them in settings with complex survey weights remains a relatively less researched question, in particular, when the survey weights may also depend on the group variable of interest. In this work, we focus on the specific case when sample selection depends on the grouping covariate of interest. We propose identification formulas to properly estimate the average controlled difference (ACD) in outcomes between groups, with appropriate weighting for covariate imbalance and generalizability. Via extensive simulation, we show that our proposed methods outperform traditional analytic approaches, with less bias in estimating the ACD, lower mean squared error, and close to nominal coverage rates, particularly in our setting of interest. We present the motivation for these methods and results using data from the National Health and Nutrition Examination Survey (NHANES), investigating the interplay of race and social determinants of health when our interest lies in estimating racial differences in mean telomere length. The NHANES sampling scheme and corresponding survey weights depend on self-reported race. We build a "propensity for race" to properly adjust for other social determinants while characterizing the controlled race effect on telomere length. We find that evidence of racial differences in telomere length between Black and White individuals attenuates after accounting for available socioeconomic status variables in NHANES and after utilizing appropriate propensity score and survey weighting techniques. It is our hope that this work will not only further our understanding of appropriate survey design considerations in this framework, but also make the task of analyzing such data in similar settings more accessible to researchers. Software for these methods have been implemented in a publicly-available R package.

Confounding Bias

Average Controlled Difference

Complex Surveys

NHANES

Racial Disparities

Telomere Length 

Section on Statistics in Epidemiology