Synthesis of Ingram Olkin Forum on Statistical Challenges in the Analysis of Police Use of Force

Modern classes of statistical models rely on the availability of detailed information associated with policing events. In practice, it is often difficult to obtain quality data, especially from small-town police departments. After data is obtained, we must consider how/if that data can be released to the public more broadly. In policing location data, on use of force incidents for example, uncertainty is introduced for many reasons such as privacy-preserving efforts, geocoding algorithms, and data-gathering mechanisms. First, we discuss various data-collection initiatives and legal processes (such as FOIA) that statisticians may consider to collect and/or access policing data. Next, we introduce methods to analyze messy/incomplete spatial policing data. We conclude with areas to consider for future statistical research, brought through the recent National Institute of Statistical Science forum. 

Interdisciplinary research in the realm of police use of force possesses the unique potential to unite stakeholders from academia, government, community organizations, and law enforcement. This talk delves into our endeavors and insights derived from the Ingram Olkin Forum, aptly titled "Statistical Challenges in the Analysis of Police Use of Force." Drawing from our experiences and interactions with forum participants, we have formulated a comprehensive set of guidelines. These guidelines are designed to foster investment from all stakeholders in the research findings and, most importantly, to steer the research outputs towards creating a measurable societal impact. 

Data collection on police-civilian encounters is generally designed for law enforcement purposes, rather than statistical analysis of police misconduct. This gap poses challenges for policymakers and applied researchers seeking to draw reliable inferences from this data. A recent National Institute of Statistical Science forum brought together statisticians and researchers to discuss the state of the field and next steps. We first review key data limitations such as selection and mismeasurement—i.e. omission of encounters where officers choose not to detain civilians and inaccuracies in officer accounts of civilian attributes/behavior. We then formally define estimands that quantify the amount of police force that is (1) discriminatory, (2) unjustified under department policy, or (3) outlying compared to the levels typically used by other officers. We review how common practices in applied research lead to statistical bias in estimation of these quantities. Finally, we discuss how recently proposed estimators and partial identification procedures address some of these concerns, and we highlight a number of open questions and important directions where statisticians can contribute. 

Key information about police incidents is often found not in standardized, structured databases, but in unstructured sources, especially in written or typed documents, reports, transcripts, and administrative forms. At the recent Ingram Olkin Forum on "Statistical Challenges in the Analysis of Police Use of Force", researchers and practitioners explored challenges of and approaches to incorporating these data sources into analyses. We look at the sources of unstructured data collections along with examples of public records requests for document collections. We then review scalable approaches to file management for large collections, including duplicate detection. From there we highlight key technical tasks when processing unstructured collections and review existing tools and approaches to them. We finish by discussing how to evaluate the quality of structured data extracted from unstructured collections.