Improving the Aggregation and Evaluation of NBA Mock Drafts

Oregon Convention Center 

If professional teams can accurately predict the order of their league's draft, they would have a competitive advantage when using or trading their draft picks. Many experts and enthusiasts publish forecasts of the order players are drafted into professional sports leagues, known as mock drafts. Using a novel dataset of mock drafts for the National Basketball Association (NBA), we explore mock drafts' ability to forecast the actual draft. We analyze authors' mock draft accuracy over time and ask how we can reasonably aggregate information from multiple authors. For both of these tasks, mock drafts are usually analyzed as ranked lists, and in this paper we propose ways to improve on these methods. We propose that rank-biased distance is the appropriate error metric for measuring accuracy of mock drafts as ranked lists. To best combine information from multiple mock drafts into a single consensus mock draft, we also propose a combination method based on the ideas of ranked-choice voting. We show that this method provides improved forecasts over the standard Borda count combination method used for most similar analyses in sports, and that either combination method provides a more accurate forecast across seasons than any single author.

Expert elicitation

Rank-biased overlap

Borda count

Ranked-choice voting

Instant-runoff voting

Rank aggregation 

Section on Statistics in Sports