Abstract
Many accuracy measures have been proposed in the past for time series forecasting comparisons. However, many of these measures suffer from one or more issues such as poor resistance to outliers and scale dependence. In this paper, while summarising commonly used accuracy measures, a special review is made on the symmetric mean absolute percentage error. Moreover, a new accuracy measure called the Unscaled Mean Bounded Relative Absolute Error (UMBRAE), which combines the best features of various alternative measures, is proposed to address the common issues of existing measures. A comparative evaluation on the proposed and related measures has been made with both synthetic and real-world data. The results indicate that the proposed measure, with user selectable benchmark, performs as well as or better than other measures on selected criteria. Though it has been commonly accepted that there is no single best accuracy measure, we suggest that UMBRAE could be a good choice to evaluate forecasting methods, especially for cases where measures based on geometric mean of relative errors, such as the geometric mean relative absolute error, are preferred.
Highlights
Forecasting has always been an attractive research area since it plays an important role in daily life
SMAPE, geometric mean relative absolute error (GMRAE) and Unscaled Mean Bounded Relative Absolute Error (UMBRAE) are less sensitive to this single forecasting outlier
We have proposed a new accuracy measure UMBRAE based on bounded relative errors
Summary
Forecasting has always been an attractive research area since it plays an important role in daily life. Many comparative studies have been conducted with the aim of identifying the most accurate methods for time series forecasting [6]. Research findings indicate that the performance of forecasting methods varies according to the accuracy measure being used [7]. Various accuracy measures have been proposed as the best to use in the past decades. Many of these measures are not generally applicable due to issues such as being infinite or undefined under certain circumstances, which may produce misleading results. The criteria required for accuracy measures have been explicitly addressed by Armstrong and Collopy [6] and further discussed by Fildes [8] and Clements and Hendry [9].
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
