Effects of singular value spectrum on the performance of echo state network

Abstract

Reservoir computing (RC), as an efficient and simple paradigm of training recurrent networks, has attracted considerable attention for its superior performance on many tasks. Echo state network (ESN) is one of typical representatives of RC, whose good performance mainly depends on the selection of right parameters. Hence, it is necessary for users to understand the inter-relationship between the performance and the parameters of ESN. Using computational experiments and statistical analysis, this paper systematically analyzes the effects of singular value spectrum on the performance of ESN, such as memory capacity, robustness, generalization performance, dynamics and so on. To simultaneously control the singular value spectrum and the sparsity when designing an ESN, two algorithms are introduced to construct the reservoir with the predefined singular value spectrum and sparsity, and their benefits are assessed on some benchmark problems.