Evaluating the Measurement of Household Expectations with Audio Recordings and Machine Learning

Oregon Convention Center 

In survey methodology, general compliance with protocols and individual interviewer performance has been analyzed with audio recordings. This is a resource intensive task since audios listening must be performed. On the other hand, little work has been done in analyzing subjective probabilistic expectations questions. In economics, agents form expectations for unknown quantities to take decisions, and very often the research problem is to infer the subjective probability distributions that express such expectations. In this paper, we develop a state-of-the-art audio transcription and speaker diarization machine learning pipeline and apply it to audio recordings of a subjective probabilistic expectations question from the Spanish Survey of Household Finances. We first compare the variables from the pipeline with a question evaluation sheet completed by the survey team. Then, we evaluate the interviewer question reading behavior using novel natural language processing techniques. We find that the extracted audio features are useful for assessing compliance, interviewer performance and for detecting biased responses from interviewer-induced household probabilistic expectations.

Machine Learning

Audio Transcription

Survey Methodology

Household Expectations 

Survey Research Methods Section