What is Analytic Fluency? A Thematic Content Analysis of Interviews with Expert Data Analysts

2328 

Contributed Abstract 

Paper 

Matthew Vanaman (1), Roger Peng (2)

(1) University of Texas at Austin, N/A, (2) University of Texas, Austin, N/A

Roger Peng  
University of Texas, Austin

Matthew Vanaman  
University of Texas at Austin

Matthew Vanaman  
University of Texas at Austin

It is common sense that data should be analyzed well rather than badly. Despite this, the actual criteria by which we judge the quality of an analysis are opaque, intuitive, and heavily influenced by the uncertain standards of disciplinary norms, routines, or subjective judgments of what "feels right" or "seems off". This lack of explicit criteria is problematic not just for analysts facing real challenges in their work, but also for hiring, program evaluation, and teaching. Indeed, many analysts report that their training left them unprepared for the challenges faced in real-world analytic settings. To better understand what good analysis looks like, we conducted a qualitative study using grounded theory methodology in a sample of highly experienced analysts from diverse professional backgrounds. Our aim was to more explicitly identify the content of what we call analytic fluency, or the "soft skills" of data analysis used in real-world settings. Our analysis uncovered 5 rich, higher-order themes (i.e., families of skills) along with 11 lower-order sub-themes. We present these findings and consider their implications for data analysis practice.

analytic fluency|data analysis practice|psychology of data analysis|data analysis expertise|industrial-organizational psychology|data science

Quality and Productivity Section

Statistical Consulting and Skills Development