Robust echo state network with Cauchy loss function and hybrid regularization for noisy time series prediction

Abstract

Noisy time series prediction is a hot research topic in practical applications. Echo state networks (ESNs) have superior performance on time series prediction. However, the ill-posed problem may weaken the generalization performance of ESNs when dealing with real data, because of the noise and the overlarge reservoir. To solve this problem, this paper proposes a robust ESN with a Cauchy loss function and hybrid regularizations for noisy time series prediction. First, the Cauchy loss function is used to suppress the large noise mixed in the real data. Then, an improved L1 regularization and the L2 regularization are combined to shrink the larger output weights and produce the sparse solution. Particularly, a simple iterative algorithm is proposed to calculate the output weights of the network. Finally, simulation results on both synthetic and real-world data sets show the super performance of the proposed model.