Nonlinear process modelling using echo state networks optimised by covariance matrix adaption evolutionary strategy

Abstract

Echo state networks (ESN) have been shown to be an effective alternative to conventional recurrent neural networks (RNNs) due to its fast training process and good performance in dynamic system modelling. However, the performance of ESN can be affected by the randomly generated reservoir. This paper presents nonlinear process modelling using ESN optimized by covariance matrix adaption evolutionary strategy (CMA-ES). CMA-ES is used to optimize the structural parameters of ESN: reservoir size, spectral radius, and leak rate. The proposed method is applied to three case studies: modelling a time series, modelling a conic tank, and modelling a fed-batch penicillin fermentation process. The results are compared with those from the original ESN, long short-term memory network, GA-ESN (ESN optimized by genetic algorithm), and feedforward neural networks. It is shown that the proposed method gives much better performance than the original ESN and other networks on all the three case studies.