Path integration based on a recurrent spiking neural network

Feedback-driven recurrent spiking neural networks (RSNNs) are powerful computational models that can mimic dynamic systems. However, the presence of a feedback loop from the readout to the recurrent layer de-stabilizes the learning mechanism and prevents it from converging. Here, we propose a supervised training procedure for RSNNs, where a second network is introduced only during the training, to provide hint for the target dynamics. The proposed training procedure consists of generating targets for both recurrent and readout layers (i.e., for a full RSNN system). It uses the recursive least square-based First-Order and Reduced Control Error (FORCE) algorithm to fit the activity of each layer to its target. The proposed full-FORCE training procedure reduces the amount of modifications needed to keep the error between the output and target close to zero. These modifications control the feedback loop, which causes the training to converge. We demonstrate the improved performance and noise robustness of the proposed full-FORCE training procedure to model 8 dynamical systems using RSNNs with leaky integrate and fire (LIF) neurons and rate coding. For energy-efficient hardware implementation, an alternative time-to-first-spike (TTFS) coding is implemented for the full-FORCE training procedure. Compared to rate coding, full-FO RCE with TTFS coding generates fewer spikes and facilitates faster convergence to the target dynamics.

Active Tuning is an optimization paradigm specifically designed to increase the robustness and generalization ability of temporal forward models like recurrent neural networks (RNNs). This work explores how the Active Tuning method can be used to optimize the internal dynamics of recurrent spiking neural networks (RSNNs). Active Tuning decouples the network from direct influence of the data stream and instead tunes its internal dynamics. This is based on the temporal gradient signals from propagating the error between outputs and observations backwards through time. Meanwhile, the network is running in a closed-loop prediction cycle, where the own output is used as the next input. As modern ANNs often demand excessive amounts of computational resources, spiking neural networks (SNNs) aim for the energy efficiency demonstrated by the human brain. This is accomplished by using an event-driven model inspired by the spiking behavior of biological neurons. Target of the Active Tuning optimization in RSNNs is the membrane potential of the neurons in the hidden layer. We show in two scenarios how RSNNs handle noisy inputs and that Active Tuning is a reliable method to increase their robustness as well as general prediction performance.

In neural circuits, recurrent connectivity plays a crucial role in network function and stability. However, existing recurrent spiking neural networks (RSNNs) are often constructed by random connections without optimization. While RSNNs can produce rich dynamics that are critical for memory formation and learning, systemic architectural optimization of RSNNs is still an open challenge. We aim to enable systematic design of large RSNNs via a new scalable RSNN architecture and automated architectural optimization. We compose RSNNs based on a layer architecture called Sparsely-Connected Recurrent Motif Layer (SC-ML) that consists of multiple small recurrent motifs wired together by sparse lateral connections. The small size of the motifs and sparse inter-motif connectivity leads to an RSNN architecture scalable to large network sizes. We further propose a method called Hybrid Risk-Mitigating Architectural Search (HRMAS) to systematically optimize the topology of the proposed recurrent motifs and SC-ML layer architecture. HRMAS is an alternating two-step optimization process by which we mitigate the risk of network instability and performance degradation caused by architectural change by introducing a novel biologically-inspired "self-repairing" mechanism through intrinsic plasticity. The intrinsic plasticity is introduced to the second step of each HRMAS iteration and acts as unsupervised fast self-adaptation to structural and synaptic weight modifications introduced by the first step during the RSNN architectural "evolution." We demonstrate that the proposed automatic architecture optimization leads to significant performance gains over existing manually designed RSNNs: we achieve 96.44% on TI46-Alpha, 94.66% on N-TIDIGITS, 90.28% on DVS-Gesture, and 98.72% on N-MNIST. To the best of the authors' knowledge, this is the first work to perform systematic architecture optimization on RSNNs.

A key issue is how networks in the brain learn to perform path integration, that is update a represented position using a velocity signal. Using head direction cells as an example, we show that a competitive network could self-organize to learn to respond to combinations of head direction and angular head rotation velocity. These combination cells can then be used to drive a continuous attractor network to the next head direction based on the incoming rotation signal. An associative synaptic modification rule with a short term memory trace enables preceding combination cell activity during training to be associated with the next position in the continuous attractor network. The network accounts for the presence of neurons found in the brain that respond to combinations of head direction and angular head rotation velocity. Analogous networks in the hippocampal system could self-organize to perform path integration of place and spatial view representations.

Temporal Encoding of Place Sequences by Hippocampal Cell Assemblies

Vision and Locomotion Combine to Drive Path Integration Sequences in Mouse Retrosplenial Cortex.

Some neurons encode information about the orientation or position of an animal, and can maintain their response properties in the absence of visual input. Examples include head direction cells in rats and primates, place cells in rats and spatial view cells in primates. `Continuous attractor' neural networks model these continuous physical spaces by using recurrent collateral connections between the neurons which reflect the distance between the neurons in the state space (e.g. head direction space) of the animal. These networks maintain a localized packet of neuronal activity representing the current state of the animal. We show how the synaptic connections in a one-dimensional continuous attractor network (of for example head direction cells) could be self-organized by associative learning. We also show how the activity packet could be moved from one location to another by idiothetic (self-motion) inputs, for example vestibular or proprioceptive, and how the synaptic connections could self-organize to implement this. The models described use `trace' associative synaptic learning rules that utilize a form of temporal average of recent cell activity to associate the firing of rotation cells with the recent change in the representation of the head direction in the continuous attractor. We also show how a nonlinear neuronal activation function that could be implemented by NMDA receptors could contribute to the stability of the activity packet that represents the current state of the animal.

Some neurons encode information about the orientation or position of an animal, and can maintain their response properties in the absence of visual input. Examples include head direction cells in rats and primates, place cells in rats and spatial view cells in primates. 'Continuous attractor' neural networks model these continuous physical spaces by using recurrent collateral connections between the neurons which reflect the distance between the neurons in the state space (e.g. head direction space) of the animal. These networks maintain a localized packet of neuronal activity representing the current state of the animal. We show how the synaptic connections in a one-dimensional continuous attractor network (of for example head direction cells) could be self-organized by associative learning. We also show how the activity packet could be moved from one location to another by idiothetic (self-motion) inputs, for example vestibular or proprioceptive, and how the synaptic connections could self-organize to implement this. The models described use 'trace' associative synaptic learning rules that utilize a form of temporal average of recent cell activity to associate the firing of rotation cells with the recent change in the representation of the head direction in the continuous attractor. We also show how a nonlinear neuronal activation function that could be implemented by NMDA receptors could contribute to the stability of the activity packet that represents the current state of the animal.

Some neurons encode information about the orientation or position of an animal, and can maintain their response properties in the absence of visual input. Examples include head direction cells in rats and primates, place cells in rats and spatial view cells in primates. `Continuous attractor' neural networks model these continuous physical spaces by using recurrent collateral connections between the neurons which reflect the distance between the neurons in the state space (e.g. head direction space) of the animal. These networks maintain a localized packet of neuronal activity representing the current state of the animal. We show how the synaptic connections in a one-dimensional continuous attractor network (of for example head direction cells) could be self-organized by associative learning. We also show how the activity packet could be moved from one location to another by idiothetic (self-motion) inputs, for example vestibular or proprioceptive, and how the synaptic connections could self-organize to implement this. The models described use `trace' associative synaptic learning rules that utilize a form of temporal average of recent cell activity to associate the firing of rotation cells with the recent change in the representation of the head direction in the continuous attractor. We also show how a nonlinear neuronal activation function that could be implemented by NMDA receptors could contribute to the stability of the activity packet that represents the current state of the animal.

The problem of designing recurrent continuous-time and spiking neural networks is NP-Hard. A common practice is to utilize stochastic searches, such as evolutionary algorithms, to automatically construct acceptable networks. The outcome of the stochastic search is related to its ability to navigate the search space of neural networks and discover those of high quality. In this paper we investigate the search space associated with designing the above recurrent neural networks in order to differentiate which network should be easier to automatically design via a stochastic search. Our investigation utilizes two popular dynamic systems problems; (1) the Henon map and (2) the inverted pendulum as a benchmark.

Interaction between the body and the brain network is important for cognitive and behavioral development. Sensory feedback to the brain, originating from the body, accelerates self-organization of the brain network. This self-organization may lead to the acquisition of new behaviors. However, how self-organization promotes behavior acquisition and how this brain-behavior interaction develops remain unclear. We propose a recurrent spiking neural network (RSN) model of the acquisition of canonical babbling, and show that self-organization of the RSN based on auditory feedback can promote such acquisition. In this model, the output of the RSN is converted to vocalization, and its sound spectrum is fed back to the RSN. Synaptic weights in the RSN are updated via spike-timing-dependent plasticity (STDP). The output weights of the RSN are modulated by the dopamine STDP, i.e., reward learning to acquire the babbling. The study demonstrated that in the model incorporating STDP under auditory feedback, babbling was acquired faster than it was in the model without STDP. Our analysis indicated that self-organization enhanced the complexity of dynamics of the RSN, resulting in faster reward learning. We also found that there was an optimal balance between STDP and dopamine STDP, which implies that self-organization that is too fast or too slow may be disadvantageous with regard to behavior acquisition.

Learning the underlying dynamics of periodic and aperiodic systems evolving over space and time provides substantial challenges across scientific fields. Despite recent advances in machine learning (ML), our understanding of its limits to learning, identifying, and emulating real physical systems remains incomplete. With the increasing complexity of dynamical systems and higher power consumption, ML systems take inspiration from the complex learning mechanisms of the mammalian brain, for biologically plausible learning. As we delve into the designing of brain- inspired ML systems, we deal with the challenges of the learning procedures, network sizes, energy efficiency and training complexities involved with it. It's crucial to characterize their performance in learning and controlling unknown dynamical systems to address them. We address these issues by designing sparsity aware learning mechanisms in feedback driven recurrent neural networks (RNNs), incorporate biologically plausible neuron models, and integrate computational neuroscience to build data driven AI systems. We explore brain-inspired neuronal dynamics to build AI systems that can mimic and classify dynamical systems. We propose full-FORCE (FF) SNN, Reservoir-in-reservoir (R-i-R) architecture, and SNN based applications learning, We pro- pose sparsity aware learning in RNNs, full-FORCE based sparsity regularization in recurrent spiking neural networks (RSNN) and an RSNN based BLE application for dynamical systems learning. Our proposed learning methods and architectures exhibit strong performance, noise robustness and energy efficiency in simulated and real-world adaptive system emulation. We evaluate bio signals, motion trajectories, Lorenz systems, real-world dynamic variable datasets, and other dynamical systems in this dissertation.

A recurrent network using a simple, biologically plausible learning rule can learn the successor representation, suggesting that long-horizon predictions are computations that are easily accessible in neural circuits.

A recurrent network using a simple, biologically plausible learning rule can learn the successor representation, suggesting that long-horizon predictions are computations that are easily accessible in neural circuits.

A recurrent network using a simple, biologically plausible learning rule can learn the successor representation, suggesting that long-horizon predictions are computations that are easily accessible in neural circuits.