Causal Inference and Adaptive Design for Evaluating Effectiveness of Medical Tests and Devices

Music City Center 

Diagnostic tests play a critical role in detecting and monitoring diseases. However, the effects of test results on health outcomes are indirect through their downstream influence on treatment decisions, posing a challenge in evaluating their true effectiveness. In this work, we propose causal estimands and targeted maximum likelihood estimation (TMLE) to evaluate the effectiveness of medical tests from their explanatory and/or pragmatic utility in medical care. We further propose an adaptive experimental design to better evaluate medical tests and devices. This framework is generally applicable to evaluate the effectiveness of any device output - e.g., artificial intelligence (Al) enabled prediction score - whose effect on the primary outcome of interest is mediated by the consequent treatment decision. The performance of our estimators and designs is demonstrated through simulation studies.

targeted maximum likelihood estimation (TMLE)

adaptive experimental designs

causal inference

real-world evidence (RWE)

medical test effectiveness

machine learning 

Section on Medical Devices and Diagnostics