Memory in Echo State Networks and the Controllability Matrix Rank

Abstract

Echo State Networks (ESNs) are a variant of recurrent neural networks (RNNs). ESNs perform as nonlinear fading memory filters and excel in prediction of "chaotic" signals. Predictions are made using a forced nonlinear dynamical system called the "reservoir" which incorporates past information into new states. The length of memory is critical to a task effective ESN. We examine the rank behavior of minimal task-effective ESNs predicting the chaotic Lorenz 1963 system for single and multi-variable input/output. Relationships are observed between the rank of the controllability matrix, memory length, and attractor features. We find that reservoir memory varies dependent on input forcing and location in state space. Knowledge of controllability matrix rank indicates a signal specific range for memory length. This variability corresponds to the reservoir varying between stable and unstable. The controllability matrix rank can facilitate efficient use of data and ESN construction.