01: A Simulation Method for Sample Size Determination in Calibration Studies of Prognostic Biomarkers

Music City Center 

Evaluating the calibration of prognostic biomarkers is crucial for assessing how well predicted risks match observed event rates. A common approach bins predicted risks and compares the observed event rates in each bin against the predicted rates. However, small sample sizes can yield wide confidence intervals (CIs), which can obscure whether observed deviations reflect actual miscalibration or random variation.

A simulation framework was developed to determine the necessary sample size for reliable estimation of bin-specific event rates. We leverage assumptions on the marginal distribution of biomarker risk groups (RGs), the conditional bin distribution per RG, and event rates within RGs. Using a Dirichlet‐multinomial process, individuals are assigned to a RG, then randomly allocated to bins conditional on the RG assignment. Using an exponential survival model, event times are generated to calculate the bin‐level survival estimates and CIs at a fixed time point.

The method provides practical guidance for choosing sample size to ensure robust calibration assessments using an adaptable approach towards different bin schemes and biomarker assumptions.

Calibration

Sample Size

Biomarker

Survival Analysis

Simulation

Confidence Interval 

Biometrics Section