Evaluation of Error Metrics for Meta-learning Label Definition in the Forecasting Task

Abstract

AbstractMeta-learning has been successfully applied to time series forecasting. For such, it uses meta-datasets created by previous machine learning applications. Each row in a meta-dataset represents a time series dataset. Each row, apart from the last, is meta-feature describing aspects of the related dataset. The last column is a target value, a meta-label. Here, the meta-label is the forecasting model with the best predictive performance for a specific error metric. In the previous studies applying meta-learning to time series forecasting, error metrics have been arbitrarily chosen. We believe that the error metric used can affect the results obtained by meta-learning. This study presents an experimental analysis of the predictive performance obtained by using different error metrics for the definition of the meta-label value. The experiments performed used 100 time series collected from the ICMC time series prediction open access repository, which has time series from a large variety of application domains. A traditional meta-learning framework for time series forecasting was used in this work. According to the experimental results, the mean absolute error can be the best metric for meta-label definition.KeywordsMeta-learningMeta-labelError metricsTime series