Underperforming performance measures? A review of measures for loss given default models

Abstract

As far as Probability of Default (PD) prediction is concerned, the model performance is typically measured with the Gini coefficient and/or the Kolmogorov-Smirnov (KS) statistic. For Loss Given Default (LGD) models, there are no standard performance measures, though, and more than 15 different measures are used, including Mean Square Error (MSE), Mean Absolute Error (MAE), coefficient of determination (R-squared) as well as correlation coefficients between the observed and predicted LGD. However, some measures cannot be readily recommended for LGD models, even though they have been used for this purpose. It is argued that there are measures that should only be employed for specific types of models. It is also pointed out that some measures can be applied interchangeably to avoid information redundancy. Moreover, the Area Under the Receiver Operating Characteristic Curve (AUC) is critically discussed in the LGD context. Four new measures are then proposed: Mean Area Under the Receiver Operating Characteristic Curve (MAUROC), Mean Accuracy Ratio (MAR), Mean Enhanced Lin-Lin Error (MELLE) and a generalized lift. The review is illustrated using an empirical example.