#1 Old Principles, New Models: Revisiting the Foundations of Good Prediction

Music City Center 

An important question for any model is ``how useful will it be when applied to other data sets?'' Before diving into any particular metric for this, it is important to step back and ask exactly what we mean by "useful". We will find that the answer to that question is very often problem dependent. For example assume a model M is focused on survival time of patients with a particular cancer. One possible use of M would be to predict which subjects have an expected survival of 6 months or less, for recommendation to supportive care; in this case "good" is a simple yes/no decision. A separate use might be assign subjects to a trial of a new adjuvant therapy, one which is not expected to have any influence on deaths in the first year. In this case the ability of M to discriminate within the first year is immaterial, but dividing 1-2, 2-4 and 4+ might be quite important.
The main message of this talk is thus "first stop and think". We will also discuss some variations on the concordance that are more applicable to different cases.

prediction

concordance

survival