Deep Echo State Networks with Multi-Span Features for Nonlinear Time Series Prediction

Abstract

Nonlinear time-series prediction is one of the challenging topics in machine learning due to complex non-stationarity in the temporal dynamics. Many recurrent neural network models have been proposed for enhancing the prediction accuracy in time-series prediction tasks. Echo state networks (ESNs) are a variant of recurrent neural networks, which have great potential for addressing machine learning tasks with a very low learning cost. However, the existing ESN-based models have used only single-span features to our best knowledge. In this study, we propose two deep ESN models incorporating multi-span features to improve the prediction performance. We show that the two deep ESN models yield better prediction performance compared to the other state-of-the-art ESN-based methods in benchmark time-series prediction tasks with three models: the Lorenz system, the Mackey-Glass system, and the NARMA-10 system. Our analyses illustrate that deeper structures decrease the multicollinearity of the extracted features and thus contribute to improved performance. The presented results suggest that the proposed models contribute to the development of artificial intelligence for temporal information processing.