A simulation study to assess performance of three study designs in vaccine safety studies

Music City Center 

Using real-world data to simulate outcomes under hypothetical scenarios enable comparisons of vaccine safety methods. We compared Concurrent Comparison Analyses (CCA), Self-Controlled Case Series (SCCS) adjusting for seasonality, and Self-Controlled Risk Interval (SCRI) without adjusting seasonality using a cohort of Kaiser Permanente Southern California during 2023–2024 respiratory season. Outcomes were simulated based on demographics, vaccination, confounder X, and seasonality, assuming no impact of prior outcomes on vaccination. We evaluated: SCCS, SCRI with follow-up of 60-day, 90-day and full follow-up, and four CCA approaches (stratified/unstratified by X and adjusting X as a covariate). In 500 simulations across 27 scenarios, rare outcomes caused convergence issues in SCRI and biases in SCCS and SCRI estimates. CCA adjusting for X showed less bias, while unadjusted CCA estimates had biases up to 56.7%. When outcomes were more common, SCCS and CCA (adjusted for X) performed comparably, but CCA was more efficient. Extending SCRI follow-up improved estimates. Proper adjustment for confounders and tailored follow-up are essential for minimizing bias in vaccine safety studies.

Vaccine safety

Concurrent Comparison Analyses (CCA)

Self-Controlled Case Series (SCCS)

Self-Controlled Risk Interval (SCRI)

Confounder adjustment 

Section on Statistics in Epidemiology