Hybrid Frequentist-Bayesian Estimation of Causal Effects in the Primary Care First Evaluation

Music City Center 

Multiplicity is a key problem in applied research. Estimating causal effects across many time periods, subgroups, or outcomes increases the risk of spurious findings. Bayesian methods address this issue by borrowing information across estimates from the same study or past studies to rein in extreme estimates. However, these methods are computationally intensive and can be difficult to align with frequentist approaches. As a solution, we pioneered a hybrid frequentist-Bayesian approach in the evaluation of Primary Care First (PCF), a primary care model from the Center for Medicare & Medicaid Innovation. In this approach, we fit a Bayesian meta-regression (Lipman et al. 2022) to frequentist difference-in-differences effect estimates from PCF's first three performance years. We found similar probabilities that PCF increased or reduced acute hospitalizations (51 and 49 percent, respectively), and a 72 percent probability that PCF increased Medicare expenditures by at least 1 percent. For subgroups, hybrid frequentist-Bayesian results were more moderate than frequentist estimates. These results supplement frequentist estimates to clarify model impacts across groups and over time.

Causal inference

Bayesian statistics

Difference-in-differences

Health policy evaluation

Multiplicity correction

Heterogeneous treatment effects 

Health Policy Statistics Section