Evaluation of Health Algorithms in the Era of Artificial Intelligence

Music City Center 

Evaluation of algorithmic performance is critical to ascertain that only the best technologies move forward to clinical applications. This is particularly true with the ubiquity of algorithms developed using modern artificial intelligence methods.
In this talk we will review the principles for development and evaluation of risk prediction algorithms, paying special attention to the different stages of the algorithmic lifecycle articulated in the recent FDA guidance, emphasizing the interdisciplinary nature of the process. We will also contrast the predictive and generative applications of mathematical algorithms and point out specific challenges in the evaluation of technologies used in the generative setting.
These issues will be illustrated using practical examples of evaluation of predictive and generative technologies developed using traditional regression methods as well as novel large language models. We will also emphasize the need and present a model of a well-defined governance process for managing risk algorithms in clinical care.