Quantifying Model Risk Research Articles

Model risk quantification based on relative entropy

Model risk can be broadly defined as the possibility of suffering losses due to errors either in the development or use of models. Financial pricing and risk measurement are subject to model risk, and its quantification is a hot topic in both academia and the financial industry. This paper presents a technique for challenging derivative pricing models that will also help to quantify model risk. The proposed approach is based on the fundamental theorems of asset pricing, which allow a model to be interpreted as a pricing measure, and on the use of the minimum relative entropy technique as a way of changing between measures. This paper makes three contributions to the literature. First, it overcomes many of the limitations of previous approaches that identified models and probability measures. Second, a statistical divergence is defined for both quantifying the divergence between measures, and therefore between models, and determining the set of optimal market instruments in the calibration of models. Third, the proposed methodology is able to assess the model risk of a target portfolio. Further, it is theoretically possible to determine a model without model risk for such a portfolio. This last framework has been explicitly designed to be easily applied by financial industry practitioners.

Deviation-Based Model Risk Measures

In practice, risk forecasts are obtained by risk measures based on a given probability measure on a measurable space. In our study, we consider the probability measures as alternative scenarios, which refer to, for instance, different distribution assumptions, models, or economic situations. Using an improper probability measure can affect risk forecasting and lead to wrong financial decisions. In this context, we propose a Deviation-based approach for quantifying model risk associated with choosing an inappropriate probability measure for risk forecasting. This measuring approach provides us with information about how far our risk measurement process could be affected by model risk. We provide examples of Deviation-based model risk measures defined in the literature. Moreover, we are proposing new alternatives to quantify model risk, for example, Gini and Extended Gini-type model risk measures. We provide a practical example using Value-at-risk (VaR) and Expected Shortfall forecasting to illustrate our approach. Our results indicate that using an inadequate probability measure (distribution assumptions) can largely affect risk forecasting. We verify that model risk estimates present skewness and heavy tail, have significant auto-correlation and do increase in periods that coincide with the highest variability of returns.

Quantification of model risk that is caused by model misspecification

In this paper, we suggest a technique to quantify model risk, particularly model misspecification for binary response regression problems found in financial risk management, such as in credit risk modelling. We choose the probability of default model as one instance of many other credit risk models that may be misspecified in a financial institution. By way of illustrating the model misspecification for probability of default, we carry out quantification of two specific statistical predictive response techniques, namely the binary logistic regression and complementary log–log. The maximum likelihood estimation technique is employed for parameter estimation. The statistical inference, precisely the goodness of fit and model performance measurements, are assessed. Using the simulation dataset and Taiwan credit card default dataset, our finding reveals that with the same sample size and very small simulation iterations, the two techniques produce similar goodness-of-fit results but completely different performance measures. However, when the iterations increase, the binary logistic regression technique for balanced dataset reveals prominent goodness of fit and performance measures as opposed to the complementary log–log technique for both simulated and real datasets.

MEASURING MODEL RISK IN FINANCIAL RISK MANAGEMENT AND PRICING

Risk measurement and pricing of financial positions are based on modeling assumptions, which are common assumptions on the probability distribution of the position’s outcomes. We associate a model with a probability measure and investigate model risk by considering a model space. First, we incorporate model risk into market risk measures by introducing model weighted and superposed market risk measures. Second, we quantify model risk itself and propose axioms for model risk measures. We introduce superposed model risk measures that quantify model risk relative to a reference model, which is the financial institution’s model of choice. Several risk measures that we propose require a probability distribution on the model space, which can be obtained from data by applying Bayesian analysis. Examples and a case study illustrate our approaches.

Model Risk Measures: A Review and New Proposals on Risk Forecasting

In financial decisions, model risk has been recognized as an important source of uncertainty. The revision of the Basel II suggests that financial institutions quantify and manage their model risk. Focusing on risk forecasting literature, we identify two main approaches to quantify model risk: the worst-case and loss function. The first approach includes measures with a similar structure to deviation measures, which are applied to a set of forecasts obtained by different models. For the second approach, monetary risk measures are employed under a loss or error function, from some forecasting procedure. Based on the untapped features of model risk for both approaches we suggest new proposals, which include measures to quantify upside and downside model risk, and average costs associated with risk overestimation and underestimation. We also conduct an empirical assessment of model risk measures using Value at Risk (VaR) and Expected Shortfall (ES) forecasting. We verify, using worst-case measures, that model risk tends to increase before crisis period and, according to loss function measures, model risk values increase in the crisis period. We also conclude that a model with good performance to risk forecasting (lower realized loss) does not indicate this model has lower model risk. Furthermore, we highlight insights into future research directions regarding this topic.

The impact of parameter uncertainty in insurance pricing and reserve with the temperature‐related mortality model

AbstractChanges in mortality rates have an impact on the life insurance industry, the financial sector (as a significant proportion of the financial markets is driven by pension funds), governmental agencies, and decision makers and policymakers. Thus the pricing of financial, pension and insurance products that are contingent upon survival or death and which is related to the accuracy of central mortality rates is of key importance. Recently, a temperature‐related mortality (TRM) model was proposed by Seklecka et al. (Journal of Forecasting, 2017, 36(7), 824–841), and it has shown evidence of outperformance compared with the Lee and Carter (Journal of the American Statistical Association, 1992, 87, 659–671) model and several others of its extensions, when mortality‐experience data from the UK are used. There is a need for awareness, when fitting the TRM model, of model risk when assessing longevity‐related liabilities, especially when pricing long‐term annuities and pensions. In this paper, the impact of uncertainty on the various parameters involved in the model is examined. We demonstrate a number of ways to quantify model risk in the estimation of the temperature‐related parameters, the choice of the forecasting methodology, the structures of actuarial products chosen (e.g., annuity, endowment and life insurance), and the actuarial reserve. Finally, several tables and figures illustrate the main findings of this paper.

The impact of model risk on dynamic portfolio selection under multi-period mean-standard-deviation criterion

We quantify model risk of a financial portfolio whereby a multi-period mean-standard-deviation criterion is used as a selection criterion. In this work, model risk is defined as the loss due to uncertainty of the underlying distribution of the returns of the assets in the portfolio. The uncertainty is measured by the Kullback–Leibler divergence, i.e., the relative entropy. In the worst case scenario, the optimal robust strategy can be obtained in a semi-analytical form as a solution of a system of nonlinear equations. Several numerical results are presented which allow us to compare the performance of this robust strategy with the optimal non-robust strategy. For illustration, we also quantify the model risk associated with an empirical dataset.

Insurance Pricing and Reserving with the Temperature-Related Mortality Model

Changes in mortality rates have an impact on the life insurance industry, the financial sector (as a significant proportion of the financial markets is driven by pension funds), the governmental agencies, and the decision and policy makers. Thus, the pricing of financial, pension and insurance products that are contingent upon survival or death and which is related to the accuracy of central mortality rates is of key importance. Recently, a temperature-related mortality (TRM) model was proposed by Seklecka et al. (2017), and it has show evidence of outperformance compared with the Lee and Carter (1992) model and several other of its extensions, when mortality-experience data from the United Kingdom is used. There is a need for awareness, when fitting the TRM model, of model risk when assessing longevity-related liabilities, especially when pricing long term annuities and pensions. In this paper, the impact of uncertainty on the various parameters involved in the model is examined. We demonstrate a number of ways to quantify model risk in the estimation of the temperature-related parameters, the choice of the forecasting methodology, the structures of actuarial products chosen (e.g., annuity, endowment and life insurance), and the actuarial reserve. Finally, several tables and figures illustrate the main findings of this paper.

Measuring Model Risk in Financial Risk Management and Pricing

Risk measurement and pricing of financial positions are based on modeling assumptions, which are common assumptions on the probability distribution of the position's outcomes. We associate a model with a probability measure and investigate model risk by considering a model space. First, we incorporate model risk into market risk measures by introducing model weighted and superposed market risk measures. Second, we quantify model risk itself and propose axioms for model risk measures. We introduce superposed model risk measures that quantify model risk relative to a reference model, which is the financial institution's model of choice. Several risk measures that we propose require a probability distribution on the model space, which can be obtained from data by applying Bayesian analysis. Examples and a case study illustrate our approaches.

Model risk – daring to open up the black box

AbstractWith the increasing use of complex quantitative models in applications throughout the financial world, model risk has become a major concern. Such risk is generated by the potential inaccuracy and inappropriate use of models in business applications, which can lead to substantial financial losses and reputational damage. In this paper, we deal with the management and measurement of model risk. First, a model risk framework is developed, adapting concepts such as risk appetite, monitoring, and mitigation to the particular case of model risk. The usefulness of such a framework for preventing losses associated with model risk is demonstrated through case studies. Second, we investigate the ways in which different ways of using and perceiving models within an organisation both lead to different model risks. We identify four distinct model cultures and argue that in conditions of deep model uncertainty, each of those cultures makes a valuable contribution to model risk governance. Thus, the space of legitimate challenges to models is expanded, such that, in addition to a technical critique, operational and commercial concerns are also addressed. Third, we discuss through the examples of proxy modelling, longevity risk, and investment advice, common methods and challenges for quantifying model risk. Difficulties arise in mapping model errors to actual financial impact. In the case of irreducible model uncertainty, it is necessary to employ a variety of measurement approaches, based on statistical inference, fitting multiple models, and stress and scenario analysis.

Robust measurement of (heavy-tailed) risks: Theory and implementation

Every model presents an approximation of reality and thus modeling inevitably implies model risk. We quantify model risk in a non-parametric way, i.e., in terms of the divergence from a so-called nominal model. Worst-case risk is defined as the maximal risk among all models within a given divergence ball. We derive several new results on how different divergence measures affect the worst case. Moreover, we present a novel, empirical way built on model confidence sets (MCS) for choosing the radius of the divergence ball around the nominal model, i.e., for calibrating the amount of model risk. We demonstrate the implications of heavy-tailed risks for the choice of the divergence measure and the empirical divergence estimation. For heavy-tailed risks, the simulation of the worst-case distribution is numerically intricate. We present a Sequential Monte Carlo algorithm which is suitable for this task. An extended practical example, assessing the robustness of a hedging strategy, illustrates our approach.

Robust Measurement of (Heavy-Tailed) Risks: Theory and Implementation

Every model presents an approximation of reality and thus modeling inevitably implies model risk. We quantify model risk in a non-parametric way, i.e., in terms of the divergence from a so-called nominal model. Worst-case risk is defined as the maximal risk among all models within a given divergence ball. We derive several new results on how different divergence measures affect the worst case. Moreover, we present a novel, empirical way built on model confidence sets (MCS) for choosing the radius of the divergence ball around the nominal model, i.e., for calibrating the amount of model risk. We demonstrate the implications of heavy-tailed risks for the choice of the divergence measure and the empirical divergence estimation. For heavy-tailed risks, the simulation of the worst-case distribution is numerically intricate. We present an SMC (Sequential Monte Carlo) algorithm which is suitable for this task. An extended practical example, assessing the robustness of a hedging strategy, illustrates our approach.

Quantifying Model Risk

WilmottVolume 2014, Issue 69 p. 56-59 Columnist Quantifying Model Risk Sri Krishnamurthy, Sri KrishnamurthySearch for more papers by this author Sri Krishnamurthy, Sri KrishnamurthySearch for more papers by this author First published: 29 January 2014 https://doi.org/10.1002/wilm.10289Citations: 2AboutPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinked InRedditWechat Citing Literature Volume2014, Issue69January 2014Pages 56-59 RelatedInformation

Quantile Uncertainty and Value‐at‐Risk Model Risk

This article develops a methodology for quantifying model risk in quantile risk estimates. The application of quantile estimates to risk assessment has become common practice in many disciplines, including hydrology, climate change, statistical process control, insurance and actuarial science, and the uncertainty surrounding these estimates has long been recognized. Our work is particularly important in finance, where quantile estimates (called Value-at-Risk) have been the cornerstone of banking risk management since the mid 1980s. A recent amendment to the Basel II Accord recommends additional market risk capital to cover all sources of "model risk" in the estimation of these quantiles. We provide a novel and elegant framework whereby quantile estimates are adjusted for model risk, relative to a benchmark which represents the state of knowledge of the authority that is responsible for model risk. A simulation experiment in which the degree of model risk is controlled illustrates how to quantify Value-at-Risk model risk and compute the required regulatory capital add-on for banks. An empirical example based on real data shows how the methodology can be put into practice, using only two time series (daily Value-at-Risk and daily profit and loss) from a large bank. We conclude with a discussion of potential applications to nonfinancial risks.

Quantifying model risk within a CreditRisk+ framework

In order to quantify an aspect of model risk in CreditRisk+, the assumption of gamma-distributed sectors is omitted by introducing a more flexible family of distributions: tempered stable distributions. The theoretical results are applied to the original Credit Suisse Financial Products framework and to the common background vector model recently launched by Fischer and Dietz. In order to calculate the loss distribution, a short review of the Panjer recursion is given and, in accordance with Gerhold et al, an extension that guarantees numerical stability in cases where the recursion might fail is presented. In addition to the distribution of the portfolio loss, contributions to value-at-risk are calculated as well, using the nested evaluation algorithm of Haaf et al. Finally, some empirical results on risk figures and risk contributions for a benchmark portfolio are given, emphasizing the effects of heavy-tailedness of the sector distributions.