Scrutinizing AI Classification Performance Using Neutral Zones

Music City Center 

As Artificial Intelligence (AI) becomes more ubiquitous, it makes sense to think about what exactly we mean by artificial intelligence. As Statistics is generally considered the science of data, it makes sense that we think of statistical thinking in AI. However, the conundrum is that there are not as many papers looking at statistical thinking in AI, more in the deployment of AI to data problems. A seminal paper by Yu and Kumbier (2017) introduced the P-Q-R-S framework-Population, Question, Representation, and Scrutiny-as essential components in deploying AI systems. This presentation focuses on the S component, exploring neutral zones as a method for managing ambiguity in classifications. Specifically, we examine the framework proposed by Jeske and Smith (2018), which introduced neutral zones for LDA and QDA, enabling control over FPR and FNR in ambiguous cases. Additionally, we will discuss the challenges of constructing neutral zones for non-model-based methods, such as KNN and ANN, where output is limited to class probabilities. Through this exploration, we aim to highlight the importance of statistical scrutiny in enhancing the reliability and interpretability of AI system.

Neutral zones, classification 

Caucus for Women in Statistics