Asymptotic Fisher memory of randomized linear symmetric Echo State Networks

Abstract

Abstract We study asymptotic properties of Fisher memory of linear Echo State Networks with randomized symmetric state space coupling. In particular, two reservoir constructions are considered: (1) More direct dynamic coupling construction using a class of Wigner matrices and (2) positive semi-definite dynamic coupling obtained as a product of unconstrained stochastic matrices. We show that the maximal Fisher memory is achieved when the input-to-state coupling is collinear with the dominant eigenvector of the reservoir coupling matrix. In the case of Wigner reservoirs we show that as the system size grows, the contribution to the Fisher memory of self-coupling of reservoir units is negligible. We also prove that when the input-to-state coupling is collinear with the sum of eigenvectors of the state space coupling, the expected normalized memory is four and eight time smaller than the maximal memory value for the Wigner and product constructions, respectively.