Program and Policy Evaluation: The Role of the Statistician

Elizabeth Eisenhauer Chair
Westat
 
Ruth Etzioni Panelist
Fred Hutchinson Cancer Research Center
 
Michael Baiocchi Panelist
 
Ping Yu Panelist
Booz Allen Hamilton, Inc
 
Debra Rog Panelist
Westat
 
Nandita Mitra Panelist
University of Pennsylvania
 
Kristin Linn Panelist
University of Pennsylvania
 
Elizabeth Eisenhauer Organizer
Westat
 
Tuesday, Aug 6: 2:00 PM - 3:50 PM
1315 
Invited Panel Session 
Oregon Convention Center 
Room: CC-F150 
To facilitate evidence-based decision making, research teams must communicate the nuances of statistical analyses in program and policy evaluation. The necessary involvement of statisticians in program design and evaluation was recognized as far back as the 1950s, and contemporary statisticians in this domain must balance analytic rigor and clear communication with non-statistical audiences. As articulated by Daughty in 1959, "The statistician's role in program design is by no means limited to the production and interpretation of statistical data. Of even greater significance is the point of view which the statistician has in attacking his problem…" This panel discussion brings together statistical and evaluation practitioners who have embraced rigorous analysis and effective communication within the evaluation domain. Panelists will provide examples of designing and communicating statistical studies in program and policy evaluation and discuss challenges and opportunities in this role.
In close partnership with Brazilian federal prosecutors, Michael Baiocchi (Associate Professor, Department of Epidemiology and Population Health, Stanford University) and his lab have developed a set of anti-labor trafficking algorithms; he will briefly discuss the development of these complex algorithms but will focus on evaluation of the use and performance of these algorithms in complex, fast-paced, high-consequence situations. Ruth Etzioni (Professor and Rosalie and Harold Rea Brown Chair, Program in Biostatistics, Fred Hutch Cancer Center) will describe statistical and simulation modeling to estimate the frequency of overdiagnosis associated with cancer screening, a key driver of cancer early detection policy. Ping Yu (Chief Growth Officer, Arbor Research) will discuss how he and his colleagues integrate traditional statistical models with artificial intelligence/machine learning (AI/ML) to design and conduct healthcare program evaluation, for example, using time series clustering to select comparison groups in experimental or quasi-experimental studies and using social network analysis and AI/ML to assess program impact. Debra Rog (Vice President for Social Policy and Economics Research, Westat) will discuss the role of the statistician in mixed methods studies which combine qualitative and quantitative methods for evaluation. Nandita Mitra (Professor of Biostatistics, University of Pennsylvania) will share her lab's developments of causal inference methods to evaluate nutritional excise taxes under complex real-world challenges such as spillover and heterogeneity. She will also share her thoughts on demonstrating causality in a criminal justice case. Kristin Linn (Assistant Professor of Biostatistics, University of Pennsylvania) will describe the design and analysis of a large, pragmatic cluster randomized trial comparing the effectiveness of implementation strategies to ensure delivery of a secure firearm storage intervention to parents/caregivers during annual pediatric well-child visits at large health systems in Colorado and Michigan.

Applied

Yes

Main Sponsor

Health Policy Statistics Section

Co Sponsors

Scientific and Public Affairs Advisory Committee
Section on Statistical Consulting
Social Statistics Section