Abstract
As a recurrent neural network that requires no training, the reservoir computing (RC) model has attracted widespread attention in the last decade, especially in the context of time series prediction. However, most time series have a multiscale structure, which a single-hidden-layer RC model may have difficulty capturing. In this paper, we propose a novel multiple projection-encoding hierarchical reservoir computing framework called Deep Projection-encoding Echo State Network (DeePr-ESN). The most distinctive feature of our model is its ability to learn multiscale dynamics through stacked ESNs, connected via subspace projections. Specifically, when an input time series is projected into the high-dimensional echo-state space of a reservoir, a subsequent encoding layer (e.g., an autoencoder or PCA) projects the echo-state representations into a lower-dimensional feature space. These representations are the principal components of the echo-state representations, which removes the high frequency components of the representations. These can then be processed by another ESN through random connections. By using projection layers and encoding layers alternately, our DeePr-ESN can provide much more robust generalization performance than previous methods, and also fully takes advantage of the temporal kernel property of ESNs to encode the multiscale dynamics of time series. In our experiments, the DeePr-ESNs outperform both standard ESNs and existing hierarchical reservoir computing models on some artificial and real-world time series prediction tasks.
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.