Multivariate Weighted Multilevel Modeling of Multi-Domain Survey Scales in a Prospective Study

Thomas M. Menino Convention & Exhibition Center 

Multiple outcomes from the same sample are often analyzed separately, ignoring correlations between outcomes. Multivariate multilevel methods (MM) that explicitly model multiple outcomes simultaneously may be more effective, but are rarely used, particularly in complex surveys. We applied MM to evaluate an intervention using an 8-domain survey assessing population health management capabilities of health centers (HCs). Data were nested at three levels: 310 staff across 32 HCs, surveyed at two time points. To address unequal selection probabilities, we used weighted MM (WMM). Four WMMs were specified under varying assumptions about correlation structures across domains and data levels with robust standard errors estimated. Model performance was assessed using information criteria, absolute fit, and prediction measures. All WMMs outperformed combined univariate models. Models accounting for HC- or staff-level correlations demonstrated superior fit and predictive performance, while the most complex model showed evidence of overfitting and high computational burden. The findings highlight the advantages of WMM and demonstrate the first application of a three-level WMM for 8 outcomes.

Multi-domain survey

Weighted multilevel model

Multivariate weighted multilevel model

Multivariate longitudinal data 

Survey Research Methods Section