Robusify p-value Calibration in Observational Studies with Partially Valid Negative Control Outcomes

Music City Center 

In observational studies, empirical calibration of p-values using negative control outcomes (NCOs) has emerged as a powerful tool for detecting and adjusting for systematic bias in treatment effect estimation. However, existing methods assume that all NCOs are valid-i.e., they have a true null effect-an assumption often violated in real-world settings. This study introduces a mixture model-based approach to account for the presence of invalid NCOs. Our method estimates the null distribution of effect estimates while accommodating heterogeneous NCO validity, enhancing robustness against bias. Through simulation studies, we demonstrate that our approach improves bias correction and controls false discoveries. We apply this methodology to real-world healthcare datasets, showcasing its practical benefits in ensuring reliable causal inference. Our findings underscore the importance of flexible p-value calibration strategies in observational research, particularly when some NCOs may deviate from the true null hypothesis. By tolerating partial misclassification of NCOs, our approach advances empirical calibration toward greater robustness and generalizability.

Hypothesis Testing

Mixture Models

Negative Control Outcomes

Observational Studies

p-value Calibration 

ENAR