Weak Sensitivity to Initial Conditions for Generating Temporal Patterns in Recurrent Neural Networks: A Reservoir Computing Approach

Abstract

A function for generating temporal patterns such as melody of the music and motor commands for body movements is one of major roles in the brain. In this paper, we study how such temporal patterns can be generated from nonlinear dynamics of recurrent neural networks (RNNs) and clarify the hidden mechanism that supports the functional ability of RNNs from reservoir computing (RC) approach. We show that when the reservoir (random recurrent neural network) shows weak instability to initial conditions, the error of the output from the reservoir and the target pattern is sufficiently small and robust to noise. It is also shown that the output from the spontaneous activity of the trained system intermittently exhibits response-like activity to the trigger input, which may be related to recent experimental findings in the neuroscience.