Diversifying conformal selections

Music City Center 

When selecting from a list of potential candidates, it is important to not only ensure that those selected are of high quality, but also that the selected candidates are diverse. For instance, in drug discovery, scientists aim to select potent drugs from a library of unsynthesized candidates, but recognize that it is wasteful to repeatedly synthesize highly similar compounds. In contrast to prior works, which study the problem of making many selections subject to a false discovery rate (FDR) constraint, this paper considers the problem of making a diverse set of selections subject to the same constraint. Our method diversity-aware conformal selection (DACS) works with a
user-specified notion of diversity and runs an optimization procedure to construct a maximally diverse selection set subject to a simple constraint involving certain stopping-time-based conformal e-values. The practitioner has flexibility in the choice of e-values, and DACS's key insight is to use optimal stopping theory to make this choice in a way which (approximately) maximizes diversity. We demonstrate the empirical performance of our method both in simulation and on real datasets.

Conformal prediction

E-values

Optimal stopping 

IMS