When to Consult Precision-Recall Curves

Abstract

ROC curves are commonly used to evaluate predictions of binary outcomes. When there are a small percentage of items of interest (as would be the case with fraud detection, for example), ROC curves can provide an inflated view of performance. This can cause challenges in trying to determine which set of predictions is better. This article discusses the conditions under which precision-recall curves may be preferable to ROC curves. As an illustrative example, we compare two commonly used fraud predictors (Beneish's (1999) M-score and Dechow et al.'s (2011) F-score) using both ROC and precision-recall curves. To aid the reader with using precision-recall curves, this article also introduces a Stata module to plot them. This module is now available on the Boston College Statistical Software Components (SSC) archive and can be installed by typing ssc install prtab in Stata.