Incorporating Inclconlsuve Outcomes in Error Rate Estimation with Applications in Forensic Science

Oregon Convention Center 

Binary decision-making occurs in many areas of science and policy; e.g., medicine (tumor present or absent), forensics (ID or exclusion), finance (good or bad credit risk), and agriculture (healthy or diseased plant). Lab or field studies may be conducted to assess the error rates in such binary decision-making processes (e.g., proficiency tests for radiologists or latent print examiners). In such tests, a true outcome is known (e.g., latent print and file print did or did not come from the same source), but study outcomes allow three responses (e.g., ``same,'' ``different,'' ``inconclusive''). Many forensic science articles report such studies' results by completely ignoring inconclusive decisions, which can artificially increase the apparent error rate. In this talk, we propose a weighting scheme to incorporate inconclusive decisions into error rates stratified by latent print quality. Additionally, we propose that Standardization can be used to compare error rates across labs and studies.

error rates

inconclusive decisions

standardization

small sample size

quality

forensic science 

Survey Research Methods Section