Causal inference methods for time series data in a digital media service

Randomized controlled experiments (or A/B testing) remain the gold standard for measuring causal effects. However, we are sometimes faced with challenges, like limited resources or technical capabilities, that prevent us from A/B testing. When A/B testing is not feasible, data scientists often look to causal inference techniques to help draw conclusions about causal effects. User data from web and mobile applications pose challenges to traditional causal inference methods, like Difference-in-Differences and Regression Discontinuity. For applications with a regularly returning user base, pre v. post-intervention comparisons become difficult as the assumption of independent observations is often violated and the impact of confounders varies over time. Instead, the Data Science team at the Los Angeles Times has adopted causal inference techniques for time series data, such as Causal Impact analysis and Regression Discontinuity in Time, which allow control over autocorrelation in the data. In this session, we'll discuss the specific challenges we faced in designing traditional causal inference studies for product changes, and the alternative approaches we took for working with time series data. We'll share findings from projects that utilized Causal Impact analysis and Regression Discontinuity in Time, and limitations that persist even with these approaches.

causal inference

timeseries analysis

causal impact analysis

regression discontinuity in time 

Mid-Level

Knowledge

Women in Statistics and Data Science 2024