T-distributed stochastic neighbor embedding echo state network with state matrix dimensionality reduction for time series prediction

Abstract

Echo state network (ESN), a novel type of recurrent neural network, possesses high nonlinear mapping capability, which is particularly appropriate for time series prediction. However, the huge reservoir may lead to ill-conditioned solutions in the output weight matrix, reducing the generalization ability and prediction performance of the network. To address this issue, a t-distributed stochastic neighbor embedding ESN (TESN) is proposed in this paper to replace the initial large-scale reservoir state matrix with a low-dimensional manifold. By maintaining the local neighbor relationship of the data in the original high-dimensional space, the ill-conditioned dilemma of the output weight matrix is successfully solved. Moreover, the proposed TESN has a strong ability to preserve the global features of the data, which effectively improves the prediction performance of the network. The superiority of the TESN model is demonstrated through two benchmark prediction tasks and a practical application.