A novel echo state network for multivariate and nonlinear time series prediction

Abstract

A robust and adaptive multivariate nonlinear time series prediction model is proposed based on echo state network and variational inference and we call it robust variational echo state network (RVESN). RVESN uses a heavy-tailed and more robust Gaussian mixture distribution as the likelihood function of the model output. The variational inference procedure which has an advantage over the Laplace approximation is utilized to handle the marginal likelihood function of the model output which is analytically intractable for the mixture distribution. RVESN not only has strong capability of nonlinear approximation as echo state network and avoids the cross-validation process of estimating model parameters but also is more robust to outliers compared to the traditional Bayesian learning with a single Gaussian distribution as the likelihood function of the model output. And furthermore, the Gaussian mixture distribution can describe the underlying dynamic characteristics of the multivariate time series more comprehensively and practically than a single Laplace or Gaussian one by an adaptive parameter. The experimental results of artificial and real-world multivariate nonlinear time series with and without outliers demonstrate that RVESN has better prediction performance.