Temporal memory and network dynamics

Abstract

Event Abstract Back to Event Temporal memory and network dynamics The brain is easily able to process and categorize complex time-varying signals. For example, the two sentences "it is cold in Utah this time of year" and "it is hot in Utah this time of year" have different meanings, even though the words "hot" and "cold" appear about 3000 ms before the ends of the two sentences. A network that can perform this kind of processing must, therefore, have a long memory. In other words, the current state of the network must depend on events that happened many seconds ago, as well as events in the last few milliseconds. This is particularly difficult because neurons are dominated by relatively short time constants -- tens to hundreds of milliseconds. Recently Jaeger and Maass et al. (2002) proposed that randomly connected networks could exhibit the long memories necessary for complex temporal processing. This is an attractive idea, both for its simplicity and because little fine tuning is required. However, a necessary condition is that the underlying network dynamics must be neither chaotic nor must it be an attractor network; that is, it must exhibit Lyapunov exponents very close to zero (White et al., 2004; Bertschinger and Natschlager, 2004). Biologically plausible model networks, though, tend to be chaotic (van Vreeswijk and Sompolinsky, 1998), an observation that we have corroborated based on an extension of the analysis used by Bertschinger and Natschlager. Real networks also tend to be very noisy -- synaptic failures occur about 50% of the time. The question we address here, then, is: given the chaotic dynamics and high noise intrinsic to biologically realistic networks, can randomly connected networks exhibit memories that are significantly longer than the time constants of their constituent neurons? The answer, not surprisingly, is "no". This answer is consistent with recent work by Ganguli et al. (2008), in which they analyzed temporal memory in a very general setting.