Beyond the edge: Amplification and temporal integration by recurrent networks in the chaotic regime.

Abstract

Event Abstract Back to Event Beyond the edge: Amplification and temporal integration by recurrent networks in the chaotic regime. Taro Toyoizumi1* and L. F. Abbott1 1 Columbia University, Department of Neuroscience, United States Randomly connected networks exhibit a transition from fixed-point to chaotic activity as the variance of their synaptic connection strengths is increased. At the transition point, known as the edge of chaos, networks display a number of desirable features (Bertchinger and Natschläger, 2004), including large gains and integration times for responses to external inputs. Away from this edge, in the fixed-point regime that has been the focus of most models and studies, gains and integration times fall off dramatically, which implies that parameters must be fine tuned with considerable precision if high performance is required. Here we show that the fall-off in gains and integration times is slower on the chaotic side of the transition, meaning that good performance can, under appropriate conditions, be achieved with less fine tuning. Our study is based on a dynamical mean-field calculation of the Fisher information provided by a large recurrently connected network about its external input. The Fisher information bounds decoding accuracy for both static and dynamical stimuli, and can also be used to quantify the integration time or memory lifetime of a network (Ganguli et al., 2008). To quantify the behavior of the Fisher information near the transition point, we evaluate the critical exponents of the Fisher information under a simple case. For any antisymmetric smoothly saturating response nonlinearity, the Fisher information diverges more rapidly on the chaotic side than the non-chaotic side of the edge of chaos. We show that, with observation noise, the Fisher information is maximized at the edge of chaos, where the network time-constant shows a critical slowdown and small inputs are highly amplified. Furthermore, at a given distance away from the transition point, the chaotic state is often more informative than the non-chaotic state, and it provides a longer lasting memory. The analytical expression for the Fisher information provides an intuitive picture of the trade-off between increasing the signal due to a larger gain and increasing chaotic "noise". The presence of observation noise emphasizes the importance of increasing the signal over decreasing the internally generated noise, providing an advantage to the chaotic state.