Two optimal threshold criteria for ROC analysis

Abstract

Abstract Among many optimal threshold criteria from ROC curve, the closest-to-(0,1) andamended closest-to-(0,1) criteria are considered. An ROC curve that passes close to the(0,1) point indicates that two models are well classi ed. In this case, the ROC curve islocated far from the (1,0) point. Hence we propose two criteria: the farthest-to-(1,0) andamended farthest-to-(1,0) criteria. These criteria are found to have a relationship withthe KolmogorovSmirnov statistic as well as some optimal threshold criteria. Moreover,we derive that a de nition for the proposed criteria with more than two dimensionsand with relations to multi-dimensional optimal threshold criteria.Keywords: ROC, threshold, true negative rate, true positive rate. 1. Introduction There are many criteria for determining the optimal threshold for two classi ed models.Some of them can be explained using the receiver operating characteristic (ROC) curve(Provost and Fawcett, 2001; Sobehart and Keenan, 2001; Engelmann et al:, 2003; Fawcett,2003; Zho et al:, 2007; Hong, 2009; Hong et al:, 2013). Among them, there are the closest-to-(0,1) and amended closest-to-(0,1) criteria of Perkins and Schisterman (2006). These twocriteria are based on the idea that models are well classi ed when the ROC curve is closer tothe (0,1) point. Since the ROC curve that plots close to the (0,1) point is far from the (1,0)point, we propose two criteria in this paper: the farthest-to-(1,0) and amended farthest-to-(1,0) criteria.De nitions of the farthest-to-(1,0) and amended farthest-to-(1,0) criteria, denoted as Fand AF, respectively, are presented and explained in Section 2. The F and AF criteria couldbe extended to more than two dimensions, as explained in Section 3. Some relationships havebeen found between these criteria and others such as MVD (maximum vertical distance;Krzanowski and Hand, 2009), J (Youden index; Youden, 1950), SSS (sum of sensitivity andspeci city; Connell and Koepsell, 1985), TR (true rate: Velez et al:, 2007; Hong and Joo,2010), AA (accuracy area; Brasil, 2010) and CCSR (correct classi cation simple rate; Hongand Wu, 2014).