On a Quest for Robustness: About Model Risk, Randomness and Discretion in Credit Risk Stress Tests

Abstract

In this paper we study the impact of model uncertainty, which occurs when linking a scenario to default probabilities, on reduced-form credit risk testing. This type of uncertainty is omnipresent in most macroeconomic testing applications due to short time series for banks' portfolio risk parameters and highly collinear macroeconomic covariates. We quantify the effect of model uncertainty on supervisory and bank tests in terms of predicted portfolio loss distributions and implied capital shortfalls by conducting a full-edged top-down credit risk test for over 1,500 German banks. Our results suggest that the impact of model uncertainty on predicted capital shortfalls can be huge, even among models with similar predictive power. This leaves both banks and supervisors with uncertainty when calculating impacts and implied capital requirements. To mitigate the impact of uncertainty, we suggest a modeling approach which filters the model space by combining the standard Bayesian model averaging (BMA) paradigm with a structural filter derived from the Merton/Vasicek credit risk model. Applying our testing framework, the dispersion decreases and the median effect is reduced from -5.0pp of CET1 ratio under the BMA model to -2.5pp under the structurally augmented BMA model, while the predicted capital shortfall is reduced by 70 %. The structural filter eliminates extreme outcomes on both sides of the forecast distribution, leading in our application to the German banking sector to a reduction in impact compared to the model without the stress testing plausibility filter.