13: An Estimand-Focused Approach for AUC Generalization and Cross-Study Benchmarking

Thomas M. Menino Convention & Exhibition Center 

The area under the ROC curve (AUC) is the standard measure of a biomarker's discriminatory accuracy; however, AUC is rarely treated as a population-specific estimand. When validation cohorts differ from the intended target population in case mix, na\"ive AUC estimates can mislead both generalization and cross-study comparison. We develop an estimand-focused framework that anchors biomarker AUC inference to a prespecified target population, aligning with the ICH E9(R1) estimand perspective adapted to discrimination rather than treatment effect. The framework supports two scientific goals: generalizing a study-specific AUC to a clinically relevant target population, and benchmarking AUCs across studies on a common population footing. Methodologically, we extend calibration weighting to the U-statistic formulation of AUC, proposing a family of estimators that accommodate either patient-level or summary-level target covariate information; augmented variants attain double robustness. We establish asymptotic properties and study their performances through comprehensive simulations. Furthermore, we demonstrate the proposed framework on the POWER trials, evaluating baseline stair-climb power (SCP) as a prognostic marker for 6-month survival in advanced non-small-cell lung cancer (NSCLC). Unlike prior work on transporting model-based predictive accuracy, our framework targets the biomarker-level estimand directly and addresses cross-study comparability — an issue not resolved by current methods.

Biomarker evaluation

Calibration weighting

Covariate shift

Diagnosis accuracy

Prediction medicine

U-statistics 

Biopharmaceutical Section