012: Quantifying Polarization in Newspaper Media.

According to recent research, Americans are more divided and polarized in recent years. In this project, we aim to characterize and quantify polarization trends throughout a historical record of US-based, primarily regional, newspapers. Newspapers were selected from a variety of US markets in an attempt to capture any regional differences that might exist in how issues/topics are discussed. Our modeling approach is based on a Structural Topic Model (STM) that identifies topics within a given corpus and then measures the tonal differences of articles discussing the same topic. Specifically, we utilize the STM for inferring potentially correlated topics and a sentiment analyzer called VADER to identify topics that exhibit a high level of semantic disparity. Using this technique, we measure the polarization of developing and evolving topics, such as sports, politics, and entertainment, and compare how polarization between and within these topics has varied through time. Through this, we develop topic-specific distributions of sentiments that we refer to as polarization distributions. We conclude by demonstrating the utility of these distributions in both identifying polarization and show how instances of high polarization coincide with significant social events.

Polarization

Topic Modeling

Sentiment

Unsupervised Clustering

Machine Learning 

Implementation and Analysis

Conference on Statistical Practice (CSP) 2023