Monday, Aug 4: 2:00 PM - 3:50 PM
4081
Contributed Papers
Music City Center
Room: CC-207C
Main Sponsor
Section on Statistics in Epidemiology
Presentations
Using real-world data to simulate outcomes under hypothetical scenarios enable comparisons of vaccine safety methods. We compared Concurrent Comparison Analyses (CCA), Self-Controlled Case Series (SCCS) adjusting for seasonality, and Self-Controlled Risk Interval (SCRI) without adjusting seasonality using a cohort of Kaiser Permanente Southern California during 2023–2024 respiratory season. Outcomes were simulated based on demographics, vaccination, confounder X, and seasonality, assuming no impact of prior outcomes on vaccination. We evaluated: SCCS, SCRI with follow-up of 60-day, 90-day and full follow-up, and four CCA approaches (stratified/unstratified by X and adjusting X as a covariate). In 500 simulations across 27 scenarios, rare outcomes caused convergence issues in SCRI and biases in SCCS and SCRI estimates. CCA adjusting for X showed less bias, while unadjusted CCA estimates had biases up to 56.7%. When outcomes were more common, SCCS and CCA (adjusted for X) performed comparably, but CCA was more efficient. Extending SCRI follow-up improved estimates. Proper adjustment for confounders and tailored follow-up are essential for minimizing bias in vaccine safety studies.
Keywords
Vaccine safety
Concurrent Comparison Analyses (CCA)
Self-Controlled Case Series (SCCS)
Self-Controlled Risk Interval (SCRI)
Confounder adjustment
Co-Author(s)
Lina Sy, Kaiser Permanente Southern California
Xuan Huang, Kaiser Permanente Southern California
Vennis Hong, Kaiser Permanente Southern California
Bing Han, Kaiser Permanente Southern California
Katia Bruxvoort, University of Alabama at Birmingham
Bruno Lewin, Kaiser Permanente Southern California
Kimberly Holmquist, Kaiser Permanente Southern California
Lei Qian, Kaiser Permanente Southern California
First Author
Stanley Xu, Kaiser Permanente Southern California
Presenting Author
Stanley Xu, Kaiser Permanente Southern California
Unobserved confounders have long posed a major challenge in causal inference. Traditional methods that adjust for these confounders use auxiliary variables. In this paper, we propose a new framework that does not model the unobserved confounders directly but rather assumes that their average effects on multiple negative control outcomes follow some unknown prior distribution. Based on this assumption, we achieve identification of the target causal effect distribution. Further, we propose two methods for constructing confidence intervals of the target parameter. We applied our new method to a study on real-world effectiveness of GLP1-RA on mental health conditions using electronic health record data from Penn Medicine Health System.
Keywords
Bias correction
Causal inference
Distributional identification
Negative control outcomes
Co-Author(s)
Yumou Qiu, Peking University
Yong Chen, University of Pennsylvania, Perelman School of Medicine
First Author
Huiyuan Wang, University of Pennsylvania
Presenting Author
Huiyuan Wang, University of Pennsylvania
Randomized controlled trials (RCTs) are the gold standard for assessing new treatments, but they are often infeasible due to ethical or practical challenges. In these cases, single-arm trials, which lack a control arm, are useful, and external control data from previous studies can be leveraged to estimate treatment effects. This paper introduces a method for integrating published data summaries from external control groups into the analysis of single-arm trials. While individual patient-level data is preferable, it is often inaccessible due to privacy and economic constraints. As a result, investigators often have to rely on aggregated summaries, leading to challenges in estimating treatment effects accurately. To overcome these challenges, we propose a method that estimates an interval of potential effects (IPE), offering more reliable and interpretable results than single-point estimates. Our method provides a practical framework for using aggregated external data to inform treatment effect estimates and support decision-making in drug development. To assess the effectiveness of the proposed strategy, we conduct extensive simulation studies and provide theoretical guarantees.
Keywords
causal Inference
external data
real world data
clinical trials
Randomized experiments are considered the gold standard for estimating causal effects. However, out of the set of possible randomized assignments, some may be likely to produce poor effect estimates and misleading conclusions. Restricted randomization is an experimental design strategy that filters out undesirable treatment assignments, but its application has primarily been limited to ensuring covariate balance in two-arm studies where the target estimand is the average treatment effect. We introduce Inspection-Guided Randomization (IGR), a transparent and flexible framework for restricted randomization that filters out undesirable treatment assignments by inspecting assignments against analyst-specified, domain-informed design desiderata. In IGR, the acceptable treatment assignments are locked in ex ante and pre-registered in the trial protocol, thus safeguarding against p-hacking and promoting reproducibility. Through illustrative simulation studies motivated by behavioral health and education interventions, we demonstrate how IGR can be used to improve effect estimates compared to benchmark designs in experiments with interference and in group formation experiments.
Keywords
Causal inference
Experimental design
Reproducibility
Interference
Social context
Statistical blocking is a technique for organizing experimental units into homogeneous groups to reduce the impact of confounding variables when estimating treatment effects.
While there has been considerable work on methods for finding optimal blockings, or more generally, optimal experimental designs, these methods are fairly rigid and do not allow for flexibility in the optimization criterion.
In this talk, we develop a novel optimal blocking technique where we view statistical blocking as a graph partitioning problem, where experimental units are vertices and edges restrict considered blockings.
We then express all partitions as a zero-suppressed binary decision diagram (ZDD), which is a directed acyclic graph in which each path corresponds to a different statistical blocking.
For small experiments, we enumerate all ZDD paths for optimal blocking, while for larger ones, we can find an approximately optimal blocking by sampling paths in the ZDD.
Our approach can accommodate any objective function, and can take into account restrictions on the number of blocks in total and the size of the blocks.
We validate our method through a small simulation study.
Keywords
Statistical blocking
zero-suppressed binary decision diagram (ZDD)
graph partitioning
optimal blocking
vertices and edges
The use of negative control outcomes (NCOs) has gained much attention in recent years. However, many existing methods rely on the validity of the negative control, which may break down when invalid NCOs exist. We propose a new method to estimate average causal effects that allows the existence of invalid negative outcomes. The key idea is to utilize the heterogeneity among multiple datasets to distinguish the valid and invalid NCOs. First, we give the identification of the causal effect under the multi-site NCO framework. With the usage of multi-site data, we avoid the majority rule that is commonly assumed in related literature. Then, we provide an estimation method based on the identification condition, and we establish the asymptotic properties of the proposed estimator. We conduct simulations to illustrate the good performance of the proposed estimator. We applied our method to GLP1 data to investigate the effectiveness of the drug.
Keywords
Causal inference
Multi-site
Negative control outcomes
Co-Author(s)
Yumou Qiu, Peking University
Yong Chen, University of Pennsylvania, Perelman School of Medicine
First Author
Wenjie Hu, University of Pennsylvania\ School of Medicine - Philadelphia, PA
Presenting Author
Wenjie Hu, University of Pennsylvania\ School of Medicine - Philadelphia, PA
Platform trials efficiently evaluate treatment effects by permitting the addition of multiple treatment groups using a common control group within a trial. While non-concurrent control data can improve statistical power, it can also introduce serious bias in the estimated treatment effect if heterogeneity occurs between the non-concurrent and concurrent control data. Existing methods fully borrow non-concurrent control data, ignoring the possibility of reduced heterogeneity among data closer in time to the concurrent controls.
We developed a novel method for evaluating treatment effects, in that, based on the likelihood ratio, the non-concurrent control data regarded as the low possibility heterogeneity for the concurrent control data back in time is determined.
A simulation study comparing the performance of the proposed method with five others revealed two distinct types; one is unbiased estimates, non-inflated α error rates and typical power, and the other is biased estimates, inflated α error rates and higher power. The proposed method belonged to the latter type and, in this type, had a smaller bias and less inflated α error rates than the other methods.
Keywords
Platform Trial
Non-concurrent control
Time-trend heterogeneity