Adjusted Evaluation Measures for Asymmetrically Important Data

Despoina Skilogianni,George-Jason Siouris,Alex Karagrigoriou

doi:10.33119/erfin.2019.4.1.3

Abstract

In this paper we introduce adjustments for standard evaluation measures appropriate for the analysis of data with asymmetrical importance. In risk analysis, it is understood that the returns of an asset do not all provide the same amount of information. This asymmetry of information is crucial for choosing the most appropriate model and evaluating its forecasting ability. In risk analysis, measures like value at risk (VaR) and expected shortfall (ES) concentrate on the left tail of the distribution of returns so that failures in fitting a model on the right tail are not important. Therefore, when we estimate the VaR of an asset, the days of violations are more important than the days of non-violations. The proposed adjustments take into consideration the asymmetry in importance and are filling the gap in the theory of evaluation of percentiles measures. The measures are divided into fixed partition, based on prior information or the goal of forecasting, and non fixed partition, based on the time proximity of the model failure. The performance of the proposed measures is illustrated with the use of a stock from the industrial metals and minerals index of the American Stock Exchange (NYSE MKT), as well as a warrant, from the Athens Exchange (ATHEX).

Highlights

Risk measures have been proposed and used, over the years, to quantify overall risk exposure for the purpose of financial supervision, including internal control and banking supervision
We provide the respective formulas for the adjusted mean absolute error (MAE), the adjusted mean absolute error (AMAE), the adjusted mean absolute percent error (AMAPE), and the adjusted heteroskedasticity mean square error (AHMSE): 1n
We have presented adjusted evaluation measures, applicable to situations in which not all observations are important

Summary

Introduction

Risk measures have been proposed and used, over the years, to quantify overall risk exposure for the purpose of financial supervision, including internal control and banking supervision. In order to judge the forecasting quality of typical methodologies such as the above, one may rely on a number of popular evaluation measures, such as the mean square error (MSE), the mean absolute error (MAE), and the mean absolute percent error (MAPE) The problem with these measures is that they fail to evaluate the risk measure estimators such as VaR, because these are percentiles. The proposed measures will be divided into two general categories based on the method of partitioning the dataset; fixed partition, based on prior information or the goal of the forecasting, and non fixed partition, based on the time proximity of the model failure.

Methodology

Type 1 Evaluation Measures

Type 2 Evaluation Measures

Applications

The NBG warrant

The NAK stock

Findings

Conclusions