Effective data visualizations for large meta-analyses: Evidence from a randomized survey experiment

Oregon Convention Center 

Meta-analysis, a statistical method for synthesizing effect sizes across studies, aims to provide a robust summary of evidence that can facilitate better evidence-based decision-making by policy-makers or practitioners. However, common meta-analytic visualizations such as forest plots rely on statistical conventions that may be unfamiliar to decision-makers. We argue that incorporating empirical evidence from cognitive science into the design of meta-analytic visualizations will improve lay persons' comprehension of the evidence. Prior work showed that an alternative design - the Meta-Analytic Rain Cloud (MARC) plot - is more effective than existing visualizations for communicating to lay audiences, at least for small meta-analyses (k = 5 studies). The present research proposes adjustments to the MARC plot and conducts a statistical cognition experiment to examine whether the advantages of the MARC plot persist in larger meta-analyses (k = 10, 20, 50, 100). Of all visualization types, the adjusted MARC plot had the best performance, offering a 1.03 sd improvement over a bar plot and a 0.36 sd improvement over a forest plot (p < 0.05, adjusting for multiple comparisons).

meta-analysis

data visualization

survey experiment

statistical cognition 

Survey Research Methods Section