Multi-Reservoir Echo State Networks with Hodrick–Prescott Filter for nonlinear time-series prediction

Abstract

The Echo State Network (ESN) is a representative model for reservoir computing, which is capable of high-speed model training for machine learning tasks with time series data. Extended models of the ESN, such as Multi-Reservoir ESNs (MRESNs), have been intensively studied for performance improvement in recent years. In this study, we propose a new model called an HP-MRESN by combining an MRESN with the Hodrick–Prescott (HP) filter for nonlinear time series prediction. The proposed HP-MRESN comprises three basic components: a time series decomposer, a reservoir state extractor, and an ensemble decoder. In the time series decomposer, we recursively leverage the HP filter to decompose original time-series data into multiple trend and cycle components. In the reservoir state extractor, each time series component is fed into a corresponding reservoir-state encoder for generating a reservoir state which is extracted as it is or through the principal component analysis. In the ensemble decoder, the states of multiple reservoirs are collected and processed to produce model outputs. Moreover, we propose a greedy algorithm to automatically find the best model architectures under designated hyperparameters for different prediction tasks. Experimental results on a total of 24 nonlinear time-series prediction tasks with 6 real-world datasets demonstrate that our proposed HP-MRESN not only can outperform some existing representative MRESN models and fully-trained RNN models but also can have relatively low training time. In addition, performance comparisons between the HP-MRESN and related MRESN models with other prepossessing methods show the benefit of time series decompositions using the HP filter. The codes of the proposed method are publicly available on https://github.com/Ziqiang-IRCN/HP-MRESN.