Program and Policy Evaluation: The Role of the Statistician

Oregon Convention Center 

Room: CC-F150 

To facilitate evidence-based decision making, research teams must communicate the nuances of statistical analyses in program and policy evaluation. The necessary involvement of statisticians in program design and evaluation was recognized as far back as the 1950s, and contemporary statisticians in this domain must balance analytic rigor and clear communication with non-statistical audiences. As articulated by Daughty in 1959, "The statistician's role in program design is by no means limited to the production and interpretation of statistical data. Of even greater significance is the point of view which the statistician has in attacking his problem…" This panel discussion brings together statistical and evaluation practitioners who have embraced rigorous analysis and effective communication within the evaluation domain. Panelists will provide examples of designing and communicating statistical studies in program and policy evaluation and discuss challenges and opportunities in this role.
In close partnership with Brazilian federal prosecutors, Michael Baiocchi (Associate Professor, Department of Epidemiology and Population Health, Stanford University) and his lab have developed a set of anti-labor trafficking algorithms; he will briefly discuss the development of these complex algorithms but will focus on evaluation of the use and performance of these algorithms in complex, fast-paced, high-consequence situations. Ruth Etzioni (Professor and Rosalie and Harold Rea Brown Chair, Program in Biostatistics, Fred Hutch Cancer Center) will describe statistical and simulation modeling to estimate the frequency of overdiagnosis associated with cancer screening, a key driver of cancer early detection policy. Ping Yu (Chief Growth Officer, Arbor Research) will discuss how he and his colleagues integrate traditional statistical models with artificial intelligence/machine learning (AI/ML) to design and conduct healthcare program evaluation, for example, using time series clustering to select comparison groups in experimental or quasi-experimental studies and using social network analysis and AI/ML to assess program impact. Debra Rog (Vice President for Social Policy and Economics Research, Westat) will discuss the role of the statistician in mixed methods studies which combine qualitative and quantitative methods for evaluation. Nandita Mitra (Professor of Biostatistics, University of Pennsylvania) will share her lab's developments of causal inference methods to evaluate nutritional excise taxes under complex real-world challenges such as spillover and heterogeneity. She will also share her thoughts on demonstrating causality in a criminal justice case. Kristin Linn (Assistant Professor of Biostatistics, University of Pennsylvania) will describe the design and analysis of a large, pragmatic cluster randomized trial comparing the effectiveness of implementation strategies to ensure delivery of a secure firearm storage intervention to parents/caregivers during annual pediatric well-child visits at large health systems in Colorado and Michigan.

Yes

Health Policy Statistics Section

Scientific and Public Affairs Advisory Committee

Section on Statistical Consulting

Social Statistics Section