Abstract Number:
1315
Submission Type:
Invited Panel Session
Participants:
Elizabeth Eisenhauer (1), Ping Yu (3), Lauren Forrow (4), Erika Liliedahl (1), Lucy D'Agostino McGowan (5), Ruth Etzioni (6), Michael Baiocchi (1), Andrew Gelman (2)
Institutions:
(1) N/A, N/A, (2) Columbia University, N/A, (3) Booz Allen Hamilton, Inc, N/A, (4) Mathematica Policy Research, N/A, (5) Wake Forest University, N/A, (6) Fred Hutchinson Cancer Research Center, N/A
Chair:
Panelist(s):
Session Organizer:
Session Description:
Given the persistent relevance of causal claims in public discourse, communicating the nuances of statistical analyses for program and policy evaluation is paramount. While the underappreciated involvement of statisticians throughout program design and evaluation was recognized as far back as 1959[1], contemporary statisticians must now wear the dual hats of rigorous analysis innovators and interpreters for audiences less versed in statistics. As articulated by Daughty in 1959, "The statistician's role in program design is by no means limited to the production and interpretation of statistical data. Of even greater significance is the point of view which the statistician has in attacking his problem…" This panel discussion brings together statistical practitioners who have embraced dual facets of technical analysis and effective communication within the evaluation domain. Andrew Gelman will lead the discussion, where panelists will provide examples of designing and communicating statistical studies in program and policy evaluation and discuss challenges and opportunities in this role.
Lauren Forrow and colleagues have used Bayesian methods to better align estimands with policy questions of interest, for example, reframing program effects in probabilistic terms and estimating programs' effects on health disparities. Ruth B. Etzioni will discuss overdiagnosis in cancer early detection and, more generally, breast and prostate cancer screening. In close partnership with Brazilian federal prosecutors, Michael Baiocchi and his lab have developed a set of anti-labor trafficking algorithms; he will briefly discuss the development of these complex algorithms but mostly will focus on evaluation of the use and performance of these algorithms as they are used in complex, fast-paced, high-consequence situations. Ping Yu will discuss how he and his colleagues integrate traditional statistical models with artificial intelligence/machine learning (AI/ML) in designing and conducting healthcare program evaluation, for example, how time series clustering can be used to select comparison groups in experimental or quasi-experimental studies and how social network analysis together with AI/ML can be harnessed to capture the complex interconnections within the data to assess program impact. Lucy D'Agostino McGowan will discuss design principles of data analysis, defining the roles of data analysis "producers" and "consumers" (or stakeholders) and propose a general framework for determining alignment between the two. Erika Liliedahl (invited) will offer a whole-of-government perspective on the critical role that statistical methods play in program evaluations designed to estimate impacts or outcomes of government programs, policies, or operations, and how this requires statisticians and program evaluators to work collectively across disciplines to develop rigorous evidence that answers priority learning questions.
[1] Doughty JH (1959) Program Design and Evaluation: The Role of the Statistician. Canadian Journal of Public Health/Revue Canadienne de Sante'e Publique 50.10: 436-441.
Sponsors:
Health Policy Statistics Section 1
Section on Statistical Consulting 2
Social Statistics Section 3
Theme:
Statistics and Data Science: Informing Policy and Countering Misinformation
Yes
Applied
Yes
Estimated Audience Size
Large (150-275)
I have read and understand that JSM participants must abide by the Participant Guidelines.
Yes
I understand and have communicated to my proposed speakers that JSM participants must register and pay the appropriate registration fee by June 1, 2024. The registration fee is nonrefundable.
I understand