Drop if 2020? Reconsidering difference-in-differences in the post-COVID era

COVID-19 has induced global historic disruptions in health, politics, and the economy. This has resulted in large-scale shocks, often several standard deviations in magnitude, in common indicators used for observational research, including unemployment, mortality, and educational outcomes. In this paper, we explore the implications of these shocks for evaluating three types of non-experimental interventions: 1) those that began prior to 2020 with major shocks in the post-intervention period; 2) those implemented alongside or as part of pandemic response; and 3) those that will be implemented in the coming years for which 2020 would normally be included in the pre-intervention period. While the traditional "common shocks" assumption requires that comparison units on average move in parallel prior to and during a major shock, we present statistical and practical reasons why alternative approaches may be more appropriate. We characterize a flexible set of assumptions related to shock processes, including "removable shocks", "common after-shocks" and "common recovery", and propose a method for selecting an optimal estimator based on analysis of untreated units. We present case studies, retrospectively exploring unemployment and uninsurance trajectories following the 2008 financial crisis.