Echo state network structure optimization algorithm based on correlation analysis

Abstract

Echo State Network (ESN) is an effective variant of Recurrent Neural Network (RNN). However, it is difficult for traditional ESN to determine the reservoir size that matches a given task. In this paper, an ESN structure optimization pruning algorithm based on correlation analysis, called PCESN, is proposed to design the size of the reservoir automatically. First, a characteristic matrix is constructed utilizing probability theory and information theory to measure the correlation between each neuron in the reservoir and the output neuron. On this basis, a pruning criterion is proposed to achieve a sparse reservoir structure by dynamically removing neurons with low correlation. Second, in order to retain the sample information of the neurons in the removed reservoir during the network reduction, the input weights of the retained reservoir are updated by means of the average transverse propagation of the weights. Finally, the performance of PCESN is tested on multiple time series. Simulation results show that the proposed PCESN outperforms some fixed size ESN and other dynamic ESN in terms of prediction accuracy, generalization performance and model complexity.