Synthesis of Ingram Olkin Forum on Statistical Challenges in the Analysis of Police Use of Force

Ima Placeholder Chair
ASA-Placeholder Record
Claire Kelling Organizer
Wednesday, Aug 7: 8:30 AM - 10:20 AM
Invited Paper Session 
Oregon Convention Center 
Room: CC-252 
Excessive use of force by police is an urgent problem of concern to sociologists, statisticians, policymakers, and the general public. Issues with data quality, processing of unstructured data, tradeoffs between data access and privacy concerns, statistical challenges in analyzing fairness, and other topics have been highlighted as specific areas of concern. In addition, the methodologies used to analyze police use of force have also been extremely varied and results are often incompatible or rely on implausible unstated assumptions. In November 2023, we will host an in-person Ingram Olkin Forum (IOF) workshop where we hope to shed light not only on unique approaches and perspectives in these areas, but also to develop and suggest steps that statisticians and researchers should consider to gain a deeper understanding of these issues and provide better evidence to inform reform efforts. During the Invited Paper session, the Organizing Committee members will debrief on the findings of the Ingram Olkin Forum and discuss next steps for research in the areas that are the main focus of the forum.



Main Sponsor

Social Statistics Section

Co Sponsors

Committee on Law and Justice Statistics
Justice Equity Diversity and Inclusion Outreach Group


Data Access and Spatial Privacy Considerations in Policing

Modern classes of statistical models rely on the availability of detailed information associated with policing events. In practice, it is often difficult to obtain quality data, especially from small-town police departments. After data is obtained, we must consider how/if that data can be released to the public more broadly. In policing location data, on use of force incidents for example, uncertainty is introduced for many reasons such as privacy-preserving efforts, geocoding algorithms, and data-gathering mechanisms. First, we discuss various data-collection initiatives and legal processes (such as FOIA) that statisticians may consider to collect and/or access policing data. Next, we introduce methods to analyze messy/incomplete spatial policing data. We conclude with areas to consider for future statistical research, brought through the recent National Institute of Statistical Science forum. 


Claire Kelling

Engaging Communities on Statistical Research in the Analysis of Police Use of Force

Interdisciplinary research in the realm of police use of force possesses the unique potential to unite stakeholders from academia, government, community organizations, and law enforcement. This talk delves into our endeavors and insights derived from the Ingram Olkin Forum, aptly titled "Statistical Challenges in the Analysis of Police Use of Force." Drawing from our experiences and interactions with forum participants, we have formulated a comprehensive set of guidelines. These guidelines are designed to foster investment from all stakeholders in the research findings and, most importantly, to steer the research outputs towards creating a measurable societal impact. 


Greg Lanzalotto, University of Pennsylvania

Statistical Issues in Analyzing Misconduct in Policing

Data collection on police-civilian encounters is generally designed for law enforcement purposes, rather than statistical analysis of police misconduct. This gap poses challenges for policymakers and applied researchers seeking to draw reliable inferences from this data. A recent National Institute of Statistical Science forum brought together statisticians and researchers to discuss the state of the field and next steps. We first review key data limitations such as selection and mismeasurement—i.e. omission of encounters where officers choose not to detain civilians and inaccuracies in officer accounts of civilian attributes/behavior. We then formally define estimands that quantify the amount of police force that is (1) discriminatory, (2) unjustified under department policy, or (3) outlying compared to the levels typically used by other officers. We review how common practices in applied research lead to statistical bias in estimation of these quantities. Finally, we discuss how recently proposed estimators and partial identification procedures address some of these concerns, and we highlight a number of open questions and important directions where statisticians can contribute. 


Dean Knox, University of Pennsylvania

Unstructured Data in Policing: Prospects and Challenges

Key information about police incidents is often found not in standardized, structured databases, but in unstructured sources, especially in written or typed documents, reports, transcripts, and administrative forms. At the recent Ingram Olkin Forum on "Statistical Challenges in the Analysis of Police Use of Force", researchers and practitioners explored challenges of and approaches to incorporating these data sources into analyses. We look at the sources of unstructured data collections along with examples of public records requests for document collections. We then review scalable approaches to file management for large collections, including duplicate detection. From there we highlight key technical tasks when processing unstructured collections and review existing tools and approaches to them. We finish by discussing how to evaluate the quality of structured data extracted from unstructured collections. 


Tarak Shah, Human Rights Data Analysis Group