Study design features improve the replicability in brain-wide association studies

Music City Center 

Several recent studies have raised concerns about the replicability of brain-wide association studies (BWAS). Here, we perform analyses and meta-analyses of a robust effect size index using 63 longitudinal and cross-sectional MRI studies (77,695 total scans) to demonstrate that optimizing study design is an important way to improve standardized effect sizes and replicability in BWAS. A meta-analysis of brain volume associations with age indicates that BWAS with larger covariate variance have larger effect size estimates and that the longitudinal studies we examined have systematically larger standardized effect sizes than cross-sectional studies. Analyzing age effects on global and regional brain measures in the Lifespan Brain Chart Consortium, we show that modifying longitudinal study design to increase between-subject variability and adding a single additional longitudinal measurement per subject improves effect sizes. However, evaluating these longitudinal sampling schemes on cognitive, psychopathology, and demographic associations with structural and functional brain outcome measures in the Adolescent Brain and Cognitive Development dataset shows that longitudinal studies can, counterintuitively, be detrimental to replicability. We demonstrate that the benefit of conducting longitudinal studies depends on the strengths of the between- and within-subject associations of the brain and non-brain measures. Explicitly modeling between- and within-subject effects avoids conflating the effects and allows optimizing effect sizes for them separately. These findings underscore the importance of considering design features in BWAS and emphasize that increasing sample size is not the only approach to improve the replicability of BWAS.

Replicability

Brain-behavior association study

effect size