Demystifying echo state network with deterministic simple topologies

Abstract

Echo state networks (ESN) are a special type of recurrent neural networks (RNN) with distinct performance in the field of reservoir computing (RC). The state space of the ESN is initially randomised and the reservoir weights are fixed with training done only on the state readout. Beside the advantages of ESN, there remains some opacity in the dynamic properties of the reservoir due to the presence of randomisation. Our aim in this paper is to demystify the model of ESN in a complete deterministic structure with the use of different proposed reservoir structures (topologies) and compare their performance with the random ESN on different benchmark datasets. All applied topologies maintain the simplicity of random ESN computation complexity. Most of the topologies showed comparable or even better performance.