Reservoir structure optimization of echo state networks: A detrended multiple cross-correlation pruning perspective

Abstract

Reservoir structure optimization of echo state networks (ESN) is an important enabler for improving network performance. In this regard, pruning provides an effective means to optimize reservoir structure by removing redundant components in the network. Existing studies achieve reservoir pruning by removing insignificant neuronal connections. However, such processing causes the optimized neurons to still remain in the reservoir and thus hinder network inference by participating in computations, leading to suboptimal utilization of pruning benefits by the network. To solve this problem, this paper proposes an adaptive pruning algorithm for ESN within the detrended multiple cross-correlation (DMC2) framework, i.e., DMAP. On the whole, it contains two main functional parts: DMC2 measure of reservoir neurons and reservoir pruning. Specifically, the former is used to quantify the correlation among neurons. Based on this, the latter can remove neurons with high correlation from the reservoir completely, and finally obtain the optimal network structure by retraining the output weights. Experiment results show that DMAP-ESN outperforms its competitors in nonlinear approximation capability and reservoir stability.