Abstract
Delays are ubiquitous in biological systems, ranging from genetic regulatory networks and synaptic conductances, to predator/pray population interactions. The evidence is mounting, not only to the presence of delays as physical constraints in signal propagation speed, but also to their functional role in providing dynamical diversity to the systems that comprise them. The latter observation in biological systems inspired the recent development of a computational architecture that harnesses this dynamical diversity, by delay-coupling a single nonlinear element to itself. This architecture is a particular realization of Reservoir Computing, where stimuli are injected into the system in time rather than in space as is the case with classical recurrent neural network realizations. This architecture also exhibits an internal memory which fades in time, an important prerequisite to the functioning of any reservoir computing device. However, fading memory is also a limitation to any computation that requires persistent storage. In order to overcome this limitation, the current work introduces an extended version to the single node Delay-Coupled Reservoir, that is based on trained linear feedback. We show by numerical simulations that adding task-specific linear feedback to the single node Delay-Coupled Reservoir extends the class of solvable tasks to those that require nonfading memory. We demonstrate, through several case studies, the ability of the extended system to carry out complex nonlinear computations that depend on past information, whereas the computational power of the system with fading memory alone quickly deteriorates. Our findings provide the theoretical basis for future physical realizations of a biologically-inspired ultrafast computing device with extended functionality.
Highlights
Some neuron types are endowed with extensive dendritic trees
The goal of the current paper is to extend the Delay-Coupled Reservoir (DCR)’s applicability to cases that require stable memories, while enhancing its ability to encode and predict dynamical systems by teacher forcing or FORCE is beyond its scope
A closer look at the output signal immediately following cue u− onset in the top panel of Fig 3B demonstrates that the standard DCR produces output that is fairly close to the desired constant value
Summary
Each dendrite is characterized by its spatial location within the tree, and the delay required for a postsynaptic action potential to propagate to the soma. While several studies investigate the computational role of the dendrites’ spatial distribution [1,2,3], the functionality of dendritic propagation delays is scarcely. One suggestion is that propagation delays enrich the dynamics of recurrent neural networks by turning them into infinite-dimensional dynamical systems [4]. The latter observation was the basis of a neurally-inspired computational paradigm, the single node Delay-Coupled Reservoir (DCR), where a single nonlinear neuron is delay-coupled to itself [5, 6]. Some of the authors have shown that applying homeostatic plasticity [7] directly to the delays dramatically improved the computational capabilities of the DCR [8]
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.