Enhanced clinical risk prediction accuracy provided by real and artificial intelligence

We compared methods to project absolute risk, the probability of experiencing the outcome of interest in a given projection interval accommodating competing risks, for a person from the target population with missing predictors. Without missing data, a perfectly calibrated model gives unbiased absolute risk estimates in a new target population, even if the predictor distribution differs from the training data. However, if predictors are missing in target population members, a reference dataset with complete data is needed to impute them and to estimate absolute risk, conditional only on the observed predictors. If the predictor distributions of the reference data and the target population differ, this approach yields biased estimates. We compared the bias and mean squared error of absolute risk predictions for seven methods that assume predictors are missing at random (MAR), including imputing individual missing predictors and imputing linear predictor combinations (risk scores). Simulations were based on real breast cancer predictor distributions and outcome data. We also analyzed a real breast cancer dataset. The largest bias for all methods resulted from different predictor distributions of the reference and target populations. No method was unbiased in this situation. Surprisingly, violating the MAR assumption did not induce severe biases. Most multiple imputation methods performed similarly and were less biased (but more variable) than a method that used a single expected risk score. Our work shows the importance of selecting predictor reference datasets similar to the target population to reduce bias of absolute risk predictions with missing risk factors.  

Moderate to severe traumatic brain injury carries a poor prognosis, with mortality in 20% of patients within 6 months after injury. The IMPACT and CRASH investigators have developed prognostic calculators for the probability of poor 6 month outcome in adult patients with Glasgow Coma Scale <=12 on admission (https://crash.lshtm.ac.uk/Risk%20calculator/index.html; http://www.tbi-impact.org/?p=impact/calc). By entering clinical characteristics into the calculators, absolute risk predictions are provided for individual patients. We aim to discuss the validity and applicability of these predictions at the population and individual level.
A literature review identified over 50 external validation studies for these models. Standard performance measures such as the area under the ROC curve showed satisfactory results (typical range: 0.7 – 0.85 at external validation). Calibration was more variable, with systematic overestimation of risks.
We conclude that the adequate discrimination supports the application of the calculators and underlying prediction for purposes of classification and characterization of large cohorts of patients. Extreme caution is required when applying the estimated prognosis to individual patients. Other lessons from the field of traumatic brain injury will also be discussed, relating to the importance of study design (sample size, setting); the irrelevance of specific methods for model development (regression versus machine learning); and implementation aspects (presentation of risk, uncertainty, and guidance on medical decision making). 

When patients present with an ovarian tumor, it is important to provide optimal management. Benign tumors can often be managed conservatively, whereas malignant tumors require surgery that depends on the type of malignancy. A 2014 systematic review identified 116 models, based on based on score systems or binary risk models. The International Ovarian Tumor Analysis consortium developed the first multinomial model (ADNEX) that differentiates between 4 types of malignancy based on clinical and ultrasound data. A 2024 systematic review of ADNEX identified 47 validation studies reporting on 17,000 tumors. The meta-analysis resulted in an AUROC of 0.93 (95% prediction interval 0.85-0.98). ADNEX has been incorporated in ultrasound machines from several manufacturers, implemented in iOS and Android apps and as an online calculator. Currently, the apps and calculator are down due to absence of a CE label, which is near impossible to obtain from a purely academic point of view. Most studies only include women that were operated, such that histology can be used as a reference standard. This may have resulted in selection bias. A next objective is therefore to update ADNEX to the full population of women with ovarian tumors. Finally, to avoid subjectivity in the measurement of ultrasound predictors, we have developed a prototype ADNEX-AI model where ultrasound features are measured using deep learning.