The architecture of dynamic reservoir in the echo state network

Abstract

Echo state network (ESN) has recently attracted increasing interests because of its superior capability in modeling nonlinear dynamic systems. In the conventional echo state network model, its dynamic reservoir (DR) has a random and sparse topology, which is far from the real biological neural networks from both structural and functional perspectives. We hereby propose three novel types of echo state networks with new dynamic reservoir topologies based on complex network theory, i.e., with a small-world topology, a scale-free topology, and a mixture of small-world and scale-free topologies, respectively. We then analyze the relationship between the dynamic reservoir structure and its prediction capability. We utilize two commonly used time series to evaluate the prediction performance of the three proposed echo state networks and compare them to the conventional model. We also use independent and identically distributed time series to analyze the short-term memory and prediction precision of these echo state networks. Furthermore, we study the ratio of scale-free topology and the small-world topology in the mixed-topology network, and examine its influence on the performance of the echo state networks. Our simulation results show that the proposed echo state network models have better prediction capabilities, a wider spectral radius, but retain almost the same short-term memory capacity as compared to the conventional echo state network model. We also find that the smaller the ratio of the scale-free topology over the small-world topology, the better the memory capacities.