STDP Rule Research Articles

Compact Model for Describing the Plasticity of Memristors Based on Nanolayers of LiNbO3 and (Co–Fe–B)х(LiNbO3)100–х Composite According to the Biosimilar STDP Rule

Compact Model for Describing the Plasticity of Memristors Based on Nanolayers of LiNbO3 and (Co–Fe–B)х(LiNbO3)100–х Composite According to the Biosimilar STDP Rule

ReplaceNet: real-time replacement of a biological neural circuit with a hardware-assisted spiking neural network.

Recent developments in artificial neural networks and their learning algorithms have enabled new research directions in computer vision, language modeling, and neuroscience. Among various neural network algorithms, spiking neural networks (SNNs) are well-suited for understanding the behavior of biological neural circuits. In this work, we propose to guide the training of a sparse SNN in order to replace a sub-region of a cultured hippocampal network with limited hardware resources. To verify our approach with a realistic experimental setup, we record spikes of cultured hippocampal neurons with a microelectrode array (in vitro). The main focus of this work is to dynamically cut unimportant synapses during SNN training on the fly so that the model can be realized on resource-constrained hardware, e.g., implantable devices. To do so, we adopt a simple STDP learning rule to easily select important synapses that impact the quality of spike timing learning. By combining the STDP rule with online supervised learning, we can precisely predict the spike pattern of the cultured network in real-time. The reduction in the model complexity, i.e., the reduced number of connections, significantly reduces the required hardware resources, which is crucial in developing an implantable chip for the treatment of neurological disorders. In addition to the new learning algorithm, we prototype a sparse SNN hardware on a small FPGA with pipelined execution and parallel computing to verify the possibility of real-time replacement. As a result, we can replace a sub-region of the biological neural circuit within 22 μs using 2.5 × fewer hardware resources, i.e., by allowing 80% sparsity in the SNN model, compared to the fully-connected SNN model. With energy-efficient algorithms and hardware, this work presents an essential step toward real-time neuroprosthetic computation.

Information theoretical properties of a spiking neuron trained with Hebbian and STDP learning rules

AbstractUsing formal methods complemented by large-scale simulations we investigate information theoretical properties of spiking neurons trained using Hebbian and STDP learning rules. It is shown that weight space contains meta-stable states, which are points where the average weight change under the learning rule vanishes. These points may capture the random walker transiently. The dwell time in the vicinity of the meta-stable state is either quasi-infinite or very short and depends on the level of noise in the system. Moreover, important information theoretic quantities, such as the amount of information the neuron transmits are determined by the meta-stable state. While the Hebbian learning rule reliably leads to meta-stable states, the STDP rule tends to be unstable in the sense that for most choices of hyper-parameters the weights are not captured by meta-stable states, except for a restricted set of choices. It emerges that stochastic fluctuations play an important role in determining which meta-stable state the neuron takes. To understand this, we model the trajectory of the neuron through weight space as an inhomogeneous Markovian random walk, where the transition probabilities between states are determined by the statistics of the input signal.

Motion detection and direction recognition in a photonic spiking neural network consisting of VCSELs-SA.

Motion detection and direction recognition are two important fundamental visual functions among the many cognitive functions performed by the human visual system. The retina and visual cortex are indispensable for composing the visual nervous system. The retina is responsible for transmitting electrical signals converted from light signals to the visual cortex of the brain. We propose a photonic spiking neural network (SNN) based on vertical-cavity surface-emitting lasers with an embedding saturable absorber (VCSELs-SA) with temporal integration effects, and demonstrate that the motion detection and direction recognition tasks can be solved by mimicking the visual nervous system. Simulation results reveal that the proposed photonic SNN with a modified supervised algorithm combining the tempotron and the STDP rule can correctly detect the motion and recognize the direction angles, and is robust to time jitter and the current difference between VCSEL-SAs. The proposed approach adopts a low-power photonic neuromorphic system for real-time information processing, which provides theoretical support for the large-scale application of hardware photonic SNN in the future.

Spontaneous dynamics of synaptic weights in stochastic models with pair-based spike-timing-dependent plasticity.

We investigate spike-timing dependent plasticity (STPD) in the case of a synapse connecting two neuronal cells. We develop a theoretical analysis of several STDP rules using Markovian theory. In this context there are two different timescales, fast neuronal activity and slower synaptic weight updates. Exploiting this timescale separation, we derive the long-time limits of a single synaptic weight subject to STDP. We show that the pairing model of presynaptic and postsynaptic spikes controls the synaptic weight dynamics for small external input on an excitatory synapse. This result implies in particular that mean-field analysis of plasticity may miss some important properties of STDP. Anti-Hebbian STDP favors the emergence of a stable synaptic weight. In the case of an inhibitory synapse the pairing schemes matter less, and we observe convergence of the synaptic weight to a nonnull value only for Hebbian STDP. We extensively study different asymptotic regimes for STDP rules, raising interesting questions for future work on adaptative neuronal networks and, more generally, on adaptative systems.

Repetitive magnetic stimulation induces synaptic plasticity through cooperative pre- and postsynaptic activation

Abstract Repetitive TMS (rTMS) induces changes in cortical excitability. However, the cellular and molecular mechanisms which underlie rTMS-induced plasticity remain incompletely understood. Using a combination of functional optical imaging, electrophysiological recordings, and multi-scale computational modeling we study how the physical input parameters of rTMS affect the ability of neurons to express synaptic plasticity. Our experiments in pyramidal neurons of mouse organotypic tissue cultures show that a cooperative pre- and postsynaptic activation during stimulation asserts metaplastic effects, which result in robust LTP-induction. At the molecular level we demonstrate that these effects depend on the activation of NMDA receptors, L-type voltage gated calcium channels and functional intracellular calcium stores. Finally, a computational model that is based on a voltage-dependent STDP rule with fast BCM-like metaplasticity reproduces our findings, and predicts biological differences between 10 Hz and iTBS protocols. These results confirm and expand upon our previous work on rTMS-induced synaptic plasticity in animal models. They provide a framework towards the development of multi-scale computational models that can predict biological effects of rTMS. Keywords: LTP, STDP, Metaplasticity, Multi-scale computational modeling

Perception Understanding Action: Adding Understanding to the Perception Action Cycle With Spiking Segmentation.

Traditionally the Perception Action cycle is the first stage of building an autonomous robotic system and a practical way to implement a low latency reactive system within a low Size, Weight and Power (SWaP) package. However, within complex scenarios, this method can lack contextual understanding about the scene, such as object recognition-based tracking or system attention. Object detection, identification and tracking along with semantic segmentation and attention are all modern computer vision tasks in which Convolutional Neural Networks (CNN) have shown significant success, although such networks often have a large computational overhead and power requirements, which are not ideal in smaller robotics tasks. Furthermore, cloud computing and massively parallel processing like in Graphic Processing Units (GPUs) are outside the specification of many tasks due to their respective latency and SWaP constraints. In response to this, Spiking Convolutional Neural Networks (SCNNs) look to provide the feature extraction benefits of CNNs, while maintaining low latency and power overhead thanks to their asynchronous spiking event-based processing. A novel Neuromorphic Perception Understanding Action (PUA) system is presented, that aims to combine the feature extraction benefits of CNNs with low latency processing of SCNNs. The PUA utilizes a Neuromorphic Vision Sensor for Perception that facilitates asynchronous processing within a Spiking fully Convolutional Neural Network (SpikeCNN) to provide semantic segmentation and Understanding of the scene. The output is fed to a spiking control system providing Actions. With this approach, the aim is to bring features of deep learning into the lower levels of autonomous robotics, while maintaining a biologically plausible STDP rule throughout the learned encoding part of the network. The network will be shown to provide a more robust and predictable management of spiking activity with an improved thresholding response. The reported experiments show that this system can deliver robust results of over 96 and 81% for accuracy and Intersection over Union, ensuring such a system can be successfully used within object recognition, classification and tracking problem. This demonstrates that the attention of the system can be tracked accurately, while the asynchronous processing means the controller can give precise track updates with minimal latency.

A biologically plausible supervised learning method for spiking neural networks using the symmetric STDP rule

Spiking neural networks (SNNs) possess energy-efficient potential due to event-based computation. However, supervised training of SNNs remains a challenge as spike activities are non-differentiable. Previous SNNs training methods can be generally categorized into two basic classes, i.e., backpropagation-like training methods and plasticity-based learning methods. The former methods are dependent on energy-inefficient real-valued computation and non-local transmission, as also required in artificial neural networks (ANNs), whereas the latter are either considered to be biologically implausible or exhibit poor performance. Hence, biologically plausible (bio-plausible) high-performance supervised learning (SL) methods for SNNs remain deficient. In this paper, we proposed a novel bio-plausible SNN model for SL based on the symmetric spike-timing dependent plasticity (sym-STDP) rule found in neuroscience. By combining the sym-STDP rule with bio-plausible synaptic scaling and intrinsic plasticity of the dynamic threshold, our SNN model implemented SL well and achieved good performance in the benchmark recognition task (MNIST dataset). To reveal the underlying mechanism of our SL model, we visualized both layer-based activities and synaptic weights using the t-distributed stochastic neighbor embedding (t-SNE) method after training and found that they were well clustered, thereby demonstrating excellent classification ability. Furthermore, to verify the robustness of our model, we trained it on another more realistic dataset (Fashion-MNIST), which also showed good performance. As the learning rules were bio-plausible and based purely on local spike events, our model could be easily applied to neuromorphic hardware for online training and may be helpful for understanding SL information processing at the synaptic level in biological neural systems.

SYNAPTIC PLASTICITY OF MEMRISTIVE STRUCTURES BASED ON POLY-P-XYLYLENE

Abstract—Neuromorphic computer networks (NCNs) with synaptic connections based on memristors can provide much greater efficiency in the hardware implementation of bio-inspired spiking neural networks than digital synaptic elements based on complementary technology. To achieve energy-efficient and, in the long-term, self-learning NCNs, the resistance of a memristor connecting pre- and postsynaptic neurons needs to be changeable according to local rules, e.g., according to the rules of spike-timing-dependent plasticity—STDP. The possibility of memristor training according to STDP rules was demonstrated by the example of Cu/poly-p-xylylene (PPX)/indium tin oxide (ITO) memristive structures, in which the top electrode (copper) acted as the presynaptic input, and the bottom (ITO), as the postsynaptic input. The optimal pulse amplitude and duration values are found for rectangular and triangular training pulses. The results open up prospects for creating autonomous NCNs capable of supervised and unsupervised learning to solve complex cognitive problems.

Fusion of Image Storage and Operation Based on Ag-Chalcogenide Memristor with Synaptic Plasticity

With the discovery of the memristor devices, most researchers have focused on the simulation and application of memristive characteristics, especially the nonvolatile storage potential of the memristor. However, the fusion of image storage and operation is seldom discussed. Therefore, this paper focuses on the electrical characterization of silver chalcogenide based on memristor devices, which can be used as a synapse to mimic the STDP and SRDP learning rules in neuromorphic systems. Then on this basis, a improved memristive cross array structure is designed, whose storage and operation are set in a single unit. Furthermore, this improved memristive cross array and image processing are combined with adjusting the memristor resistance by synaptic plasticity to achieve the integration of storage and processing. Finally, the effectiveness of the design is verified by the LTSPICE experiments of the nonlinear grayscale transformation and logic operation of the image. The design breaks the gap between the processing and storage in the Von Neumann architecture, which will be helpful to the development of the parallel computing in the future.

On the applicability of STDP-based learning mechanisms to spiking neuron network models

The ways to creating practically effective method for spiking neuron networks learning, that would be appropriate for implementing in neuromorphic hardware and at the same time based on the biologically plausible plasticity rules, namely, on STDP, are discussed. The influence of the amount of correlation between input and output spike trains on the learnability by different STDP rules is evaluated. A usability of alternative combined learning schemes, involving artificial and spiking neuron models is demonstrated on the iris benchmark task and on the practical task of gender recognition.

A comparison of learning abilities of spiking networks with different spike timing-dependent plasticity forms

A study of possibility to model the learning process on base of different forms of timing-dependent plasticity (STDP) was performed. It is shown that the learning ability depends on the choice of spike pairing scheme and the type of input signal used for learning. The comparison of performance of several STDP rules along with several neuron models (leaky integrate-and-fire, static, Izhikevich and Hodgkin-Huxley) was carried out using the NEST simulator. The combinations of input signal and STDP spike pairing scheme, which demonstrate the best learning abilities, were extracted.

Balancing the critical period of spiking neurons with attractor-less STDP

Attractor-based (multiplicative) STDP has been called a biologically more realistic form of spike timing dependent plasticity than attractor-less (additive) variants [1-4], as it produces unimodal distributions of synaptic weights [5,6] when given poisson-distributed random input spike trains [1,2,4,7-9]. While unimodal weight distributions have been observed as an outcome in some in vitro experiments [5], the actual biochemical process that changes synaptic connection strengths has yet to be fully understood. Unfortunately, attractor-based STDP has been repeatedly shown to be computationally less powerful than attractor-less STDP [9] and successful implementations of attractor-based STDP used attractors that were either very weak [8] or very close to some minimum weight [2,3], causing a more additive-like [2,8] behaviour of the plasticity rule. We therefore examined possible biological interpretations of attractor-less STDP rules while keeping in mind that any STDP rule is just an abstraction from the hidden biophysical reality. We show how unimodal weight distributions can reliably result from attractor-less STDP when negative synaptic drift is combined with activity-independent synaptic growth. A bimodal distribution is then only formed when non-random (polychromous [10]) poisson-distributed inputs are presented to a neuron [11]. In practice, this produces a plasticity rule that keeps the postsynaptic neuron unselective and responsive to a broad range of inputs while receiving only random spikes, but quickly allows the neuron's receptive field to become highly selective as soon as some inputs start repeating a non-random ordering of spikes. This stabilization procedure preserves STDP's sensitivity to temporally shifted correlations in input spike data, which in turn gives us several beneficial features for biologically more realistic and computationally more powerful [12] implementations of plasticity in spiking neural networks.

Исследование спонтанной активности в модели нейрональной культуры с долговременной пластичностью

Existence of spontaneous population bursts is a widely studied phenomenon observed in neuronal cultures in vitro. Recent models of neuronal cultures network activity consist of a number of burst generating mechanisms such as synaptic noise and presence of pacemaker neurons in the network. In the previous simulations of bursting in neuronal cultures synaptic weights change in accordance with the rule of short-term plasticity whereas the long-term values of them, and hence the network structure, remain unchanged. In this paper we reproduce neuronal network models with static synapses, and then investigate spontaneous activity changes in neuronal networks with long-term plasticity defined by STDP rule. Our results demonstrate that introduction of long-term plasticity in the model leads to discrepancy with the experimental data.

Two-trace model for spike-timing-dependent synaptic plasticity.

We present an effective model for timing-dependent synaptic plasticity (STDP) in terms of two interacting traces, corresponding to the fraction of activated NMDA receptors and the [Formula: see text] concentration in the dendritic spine of the postsynaptic neuron. This model intends to bridge the worlds of existing simplistic phenomenological rules and highly detailed models, thus constituting a practical tool for the study of the interplay of neural activity and synaptic plasticity in extended spiking neural networks. For isolated pairs of pre- and postsynaptic spikes, the standard pairwise STDP rule is reproduced, with appropriate parameters determining the respective weights and timescales for the causal and the anticausal contributions. The model contains otherwise only three free parameters, which can be adjusted to reproduce triplet nonlinearities in hippocampal culture and cortical slices. We also investigate the transition from time-dependent to rate-dependent plasticity occurring for both correlated and uncorrelated spike patterns.

Adrenergic Gating of Hebbian Spike-Timing-Dependent Plasticity in Cortical Interneurons

In pyramidal cells, the induction of spike-dependent plasticity (STDP) follows a simple Hebbian rule in which the order of presynaptic and postsynaptic firing dictates the induction of LTP or LTD. In contrast, cortical fast spiking (FS) interneurons, which control the rate and timing of pyramidal cell firing, reportedly express timing-dependent LTD, but not timing-dependent LTP. Because a mismatch in STDP rules could impact the maintenance of the excitation/inhibition balance, we examined the neuromodulation of STDP in FS cells of mouse visual cortex. We found that stimulation of adrenergic receptors enables the induction of Hebbian bidirectional STDP in FS cells in a manner consistent with a pull-push mechanism previously characterized in pyramidal cells. However, in pyramidal cells, STDP induction depends on NMDA receptors, whereas in FS cells it depends on mGluR5 receptors. We propose that neuromodulators control the polarity of STDP in different synapses in the same manner, and independently of the induction mechanism, by acting downstream in the plasticity cascade. By doing so, neuromodulators may allow coordinated plastic changes in FS and pyramidal cells.

A unifying perspective on neuromodulatory effects on signal transmission and plasticity in D1-dominant MSN neurons

How could phasic variations in dopamine level affect the learning outcome of a spiking neural network? How may neuromodulation affect the network's instantaneous response to simultaneously arriving glutamatergic inputs? How may this depend on the brain regions involved? In our spiking phenomenological model for signal transmission across the synapse and along the dendritic tree, we propose a new approach for the influence of dopamine-like neuromodulators on the ascribed aspects, which unifies diverging views on its role in (reinforcement) learning and (attentional) contrast. We call into question the common practice of simulating dopaminergic influence on an STDP rule as a third factor, and instead show how an instantaneous effect of a dopamine-like neuromodulator on postsynaptic activity can also lead to reinforced learning outcomes. As the phasic change of neuromodulator needs to be present during glutamatergic transmission in our model, we do not account for delayed reward as stated in the distal reward problem. Instead, we assume an involvement of hippocampus and cortical working memory for long delays of reward. However, as our transmission-based model does not interfere with the standard two-factor STDP rule, it may be freely combined with existing extensions to STDP if needed.

Multi-modal novelty and familiarity detection

Presented is further development of the architecture presented in [1] where a top-down feature-based and spatial attention have been incorporated in a large scale visual module and novelty and familiarity detectors based on the model presented in [2]. These have been developed in the perceptual (visual and auditory) and motor modalities. In addition to the novelty/familiarity detection shown in [2,3], the architecture is able to partially recognise familiar features in each perceptual modality, and furthermore in a distributed fashion activate associated familiar features from one perceptual modality to another and/or to the motor programmes and affordances. The architecture is implemented on a mobile robot operating in a dynamic environment. The proposed distributed multi-modal familiarity detection integrated in the architecture improves the recognition and action performance in a noisy environment, as well as contributing to the multi-modal association and learning of novel objects and actions. The neural network is built on integrate-and-fire spiking neurons with active dendrites (ADDS) and trained with STDP rule presented in [4]. The overall architecture is shown in Figure ​Figure11 (left). It integrates representation of concepts in different modalities. The distributed representation in the working memory area achieve temporal binding via phase-locking, whereas different concepts are separated in time. The robot implementation provides interpretation and execution of simple instructions given via the auditory area and interpreted in the working memory which is integrating sequential language input into instruction representation. The separate entities of the instruction and the instruction as a whole run in nested gamma-theta oscillations. The novelty and familiarity detectors network is shown on Figure ​Figure11 (right). The familiarity detector (FD) and novelty detector (ND) areas are 25x25 ADDS neurons trained with STDP. After training approximately 68% (± 15 in the different trials / training sessions) of the FD neurons become unimodal action/object specific. About 22% (±15) of the ND neurons were also observed to become unimodal action/object specific, responding to when a particular object/action are not present in the verbal instruction or the visual field. Figure 1 Multi-modal Robot Control Architecture (left). Novelty and familiarity Detectors (right).

Emergence of Optimal Decoding of Population Codes Through STDP

The brain faces the problem of inferring reliable hidden causes from large populations of noisy neurons, for example, the direction of a moving object from spikes in area MT. It is known that a theoretically optimal likelihood decoding could be carried out by simple linear readout neurons if weights of synaptic connections were set to certain values that depend on the tuning functions of sensory neurons. We show here that such theoretically optimal readout weights emerge autonomously through STDP in conjunction with lateral inhibition between readout neurons. In particular, we identify a class of optimal STDP learning rules with homeostatic plasticity, for which the autonomous emergence of optimal readouts can be explained on the basis of a rigorous learning theory. This theory shows that the network motif we consider approximates expectation-maximization for creating internal generative models for hidden causes of high-dimensional spike inputs. Notably, we find that this optimal functionality can be well approximated by a variety of STDP rules beyond those predicted by theory. Furthermore, we show that this learning process is very stable and automatically adjusts weights to changes in the number of readout neurons, the tuning functions of sensory neurons, and the statistics of external stimuli.

Reward-based learning under hardware constraints—using a RISC processor embedded in a neuromorphic substrate

In this study, we propose and analyze in simulations a new, highly flexible method of implementing synaptic plasticity in a wafer-scale, accelerated neuromorphic hardware system. The study focuses on globally modulated STDP, as a special use-case of this method. Flexibility is achieved by embedding a general-purpose processor dedicated to plasticity into the wafer. To evaluate the suitability of the proposed system, we use a reward modulated STDP rule in a spike train learning task. A single layer of neurons is trained to fire at specific points in time with only the reward as feedback. This model is simulated to measure its performance, i.e., the increase in received reward after learning. Using this performance as baseline, we then simulate the model with various constraints imposed by the proposed implementation and compare the performance. The simulated constraints include discretized synaptic weights, a restricted interface between analog synapses and embedded processor, and mismatch of analog circuits. We find that probabilistic updates can increase the performance of low-resolution weights, a simple interface between analog synapses and processor is sufficient for learning, and performance is insensitive to mismatch. Further, we consider communication latency between wafer and the conventional control computer system that is simulating the environment. This latency increases the delay, with which the reward is sent to the embedded processor. Because of the time continuous operation of the analog synapses, delay can cause a deviation of the updates as compared to the not delayed situation. We find that for highly accelerated systems latency has to be kept to a minimum. This study demonstrates the suitability of the proposed implementation to emulate the selected reward modulated STDP learning rule. It is therefore an ideal candidate for implementation in an upgraded version of the wafer-scale system developed within the BrainScaleS project.