Business measures of scorecard benefit

Abstract

In many applications of scoring, analysts measure the performance of a proposed scorecard statistically. Some common measures are divergence, Kolmogorov–Smirnov (KS), and the receiver operating characteristic (ROC) curve (the mirror image of the Lorentz diagram). Statistical measures can be useful, but they do not answer the fundamental question: how much more money can we expect to make if we use the proposed score? We start with the traditional binary classification model, where an accepted ‘good’ provides a positive profit and an accepted ‘bad’ provides a negative profit. We then define a relative measure called holistic profit (HP), which is profit expressed as a percentage of profit under perfect discrimination. From this, two new measures are defined. The first is the HP scan, which is optimal HP as a function of the important economic ratio. The second is the HP curve, which measures sensitivity to the use of suboptimal cutoff scores. We also show how these two measures generalize the KS statistic and the ROC curve.