Calming Jittery Models, with a Credit Application

Abstract

Decision makers, such as loan officers, risk managers, traders, and portfolio managers, who operate in the presence of transaction costs of one sort or another and take actions (e.g., trade, allocate economic capital, etc.) that are informed by probabilistic models, are sensitive to the “volatility” of the probabilistic models on which they rely. We explore various natural new definitions of probabilistic model volatility and introduce a new, highly practical, and robust method to control the volatility of an important class of conditional probability models. This method allows us to develop an efficient frontier of models that describes the tradeoff between model accuracy, measured in terms of likelihood, and model volatility so that a model user can select the model that best reflects his preferences. We illustrate our method by generating a collection of (physical) probability of default (PD) models. We find that, if we are willing to accept a very modest decrease in model performance (measured by likelihood or ROC), we can substantially reduce the volatility of our PD models, with respect to various measures of volatility and substantially increase the signal/noise ratio for defaulted obligors prior to default.