Monday, Aug 5: 2:00 PM - 3:50 PM
1308
Invited Paper Session
Oregon Convention Center
Room: CC-B112
Data derived from electronic health records, surveys, insurance claims, and other administrative sources are increasingly used to generate so-called "real-world evidence" about treatment and policy decisions in health systems. Such data can provide more insight into how healthcare is delivered "on the ground" than traditional designed studies, but generating trustworthy evidence from these data requires careful consideration of data completeness, quality, and judicious statistical methods to reduce biases due to confounding, misclassification, and sampling.
While health systems and government regulatory bodies have begun to recognize the usefulness of real-world data, enthusiasm has been tempered by disagreements between the findings of real-world studies and randomized trials. For example, a recent paper aiming to predict the findings of a randomized trial of a new diabetes drug based on real-world data generated significant controversy when its estimate of the drug's efficacy on some outcomes was markedly lower than the trial's published results. There is also concern that decisions based on evidence derived from systems which already generate disparate outcomes across population subgroups may reinforce and perhaps even worsen health disparities.
To improve the trustworthiness of real-world evidence, there is a need to adapt existing methods to handle the scale and complexity of real-world data and to create novel approaches to address new questions. This invited session will highlight the work of field leading methodological researchers who actively apply their work to improve health care decision-making using real-world data.
Applied
Yes
Main Sponsor
Health Policy Statistics Section
Co Sponsors
Caucus for Women in Statistics
ENAR
Society for Medical Decision Making
Presentations
Covariate adjustment can improve precision in analyzing randomized clinical trials. In May 2023, FDA issued a final guidance for industry for covariate adjustment in randomized trials for drugs and biological products. However, there are still many challenges and open questions in implementing covariate adjustment in practice, e.g. missing data, stratified experiment, cluster randomized trials, and lack of software. In this talk, we review several important recent advancements in the theory, methods, and software that address these barriers.
Learning health systems use internal data and experience to promote continuous improvement and innovation in health care delivery. Three key questions often underlie many tasks in building an effective learning health system: 'who is most in need of intervention?', 'what interventions work?' and 'what interventions work best for which individuals?'. In this talk we explore some examples of how statistical advances in risk prediction, causal inference, and subgroup identification can aid in these three key tasks. We provide an example of they have been used to help tailor intervention decisions in a large academic health system and discuss some key challenges to progress in a learning health system.
Data from population-based surveys can be used to inform public policy related to mental health, but insights derived from these data are often limited as the underlying research tends to focus on only one or a few variables at a time. Here, we demonstrate the utility of explainable machine learning approaches when applied to survey data from school-aged youth in Canada and the United States. First, we accessed data for 11,000 participants (ages 10-11) from the Adolescent Brain and Cognitive Development (ABCD) Study. Using gradient-boosted trees paired with SHapely Additive exPlanations (SHAP), we ranked over 50 bio-psycho-social factors by their importance in explaining variability in problematic behaviors and symptoms. Second, we applied the same methodology to over 20,000 participants (ages 13-18) from the 2019 and 2023 cycles of the Ontario Student Drug Use and Health Survey (OSDUHS). Our goal is to rank and characterize the associations of a broad set of "Whole Person" factors with mental wellbeing in youth. This information can be used by a range of stakeholders, including scientists, policymakers, and the general public, to prioritize areas of focus moving ahead.
Our ongoing work aims to build a framework for evaluating the social impact of algorithms in health care before they are deployed, rather than after they have been implemented and harms have occurred. We leverage advances in statistical machine learning and mathematical decision science modeling as well as qualitative research to prioritize equity considerations in our causal network model. Groups marginalized by the health care system already experience lower life expectancy and poorer health outcomes—algorithms could more frequently be contributing to improving health equity rather than harming it.