Plasticity Learning Rule Research Articles

(Invited) Analog HfO2-RRAM Switches for Neural Networks

Resistive random access memories (RRAM) are one of the main constituents of the class of memristive technologies that are today considered very promising in semiconductor industry because of their high potential for several applications ranging from non-volatile memories to neuromorphic hardware. The latter application is particularly interesting, since bio-inspired electronic systems have the ability to treat ill-posed problems with higher efficiency than conventional computing paradigms. In this work, we focus on HfO2-based RRAM devices and we analyse their switching dynamics in order to reach neuromorphic requirements. We present analogue memristive behaviour in HfO2 RRAM, which allows realizing a simple version of spike timing dependent plasticity learning rule. Finally, the experimental data are used to simulate an unsupervised spiking neuromorphic network for pattern recognition suitable for real-time applications.

Analog Memristive Synapse in Spiking Networks Implementing Unsupervised Learning.

Emerging brain-inspired architectures call for devices that can emulate the functionality of biological synapses in order to implement new efficient computational schemes able to solve ill-posed problems. Various devices and solutions are still under investigation and, in this respect, a challenge is opened to the researchers in the field. Indeed, the optimal candidate is a device able to reproduce the complete functionality of a synapse, i.e., the typical synaptic process underlying learning in biological systems (activity-dependent synaptic plasticity). This implies a device able to change its resistance (synaptic strength, or weight) upon proper electrical stimuli (synaptic activity) and showing several stable resistive states throughout its dynamic range (analog behavior). Moreover, it should be able to perform spike timing dependent plasticity (STDP), an associative homosynaptic plasticity learning rule based on the delay time between the two firing neurons the synapse is connected to. This rule is a fundamental learning protocol in state-of-art networks, because it allows unsupervised learning. Notwithstanding this fact, STDP-based unsupervised learning has been proposed several times mainly for binary synapses rather than multilevel synapses composed of many binary memristors. This paper proposes an HfO2-based analog memristor as a synaptic element which performs STDP within a small spiking neuromorphic network operating unsupervised learning for character recognition. The trained network is able to recognize five characters even in case incomplete or noisy images are displayed and it is robust to a device-to-device variability of up to ±30%.

Training a Hidden Markov Model with a Bayesian Spiking Neural Network

It is of some interest to understand how statistically based mechanisms for signal processing might be integrated with biologically motivated mechanisms such as neural networks. This paper explores a novel hybrid approach for classifying segments of sequential data, such as individual spoken works. The approach combines a hidden Markov model (HMM) with a spiking neural network (SNN). The HMM, consisting of states and transitions, forms a fixed backbone with nonadaptive transition probabilities. The SNN, however, implements a biologically based Bayesian computation that derives from the spike timing-dependent plasticity (STDP) learning rule. The emission (observation) probabilities of the HMM are represented in the SNN and trained with the STDP rule. A separate SNN, each with the same architecture, is associated with each of the states of the HMM. Because of the STDP training, each SNN implements an expectation maximization algorithm to learn the emission probabilities for one HMM state. The model was studied on synthesized spike-train data and also on spoken word data. Preliminary results suggest its performance compares favorably with other biologically motivated approaches. Because of the model’s uniqueness and initial promise, it warrants further study. It provides some new ideas on how the brain might implement the equivalent of an HMM in a neural circuit.

Slow feature analysis with spiking neurons and its application to audio stimuli.

Extracting invariant features in an unsupervised manner is crucial to perform complex computation such as object recognition, analyzing music or understanding speech. While various algorithms have been proposed to perform such a task, Slow Feature Analysis (SFA) uses time as a means of detecting those invariants, extracting the slowly time-varying components in the input signals. In this work, we address the question of how such an algorithm can be implemented by neurons, and apply it in the context of audio stimuli. We propose a projected gradient implementation of SFA that can be adapted to a Hebbian like learning rule dealing with biologically plausible neuron models. Furthermore, we show that a Spike-Timing Dependent Plasticity learning rule, shaped as a smoothed second derivative, implements SFA for spiking neurons. The theory is supported by numerical simulations, and to illustrate a simple use of SFA, we have applied it to auditory signals. We show that a single SFA neuron can learn to extract the tempo in sound recordings.

Effect of spike-timing-dependent plasticity on neural assembly computing

Spiking neural network (SNNs) are practical and realistic neural models, which have attracted considerable attention and many valuable physical realizations have been implemented. Neural assembly computing (NAC) is a new approach to SNNs, which is known to be a promising mechanism for explaining large-scale neural behavior, and it has been used to examine and explore the computational activities of neural cell assemblies. To obtain algorithms based on NAC, neural coalitions are considered to be responsible for patterns, memorizing them, and controlling their hierarchical relatives. In addition, spike timing-dependent plasticity (STDP) can be employed as a synaptic plasticity learning rule to modify the synaptic weights in neural networks, thereby allowing the convergence of neural activities to a spatiotemporal neuron-network pattern. Thus, applying STDP to NAC can be a powerful tool in neuroscience computing. Investigations in this area are also useful for understanding biological systems. In this study, we investigated the effect of applying STDP to NAC. Our simulation results showed that applying STDP rule increased the average number of firings for each event in neural assemblies by adjusting the weights of the connections. Moreover, the firing of neural assemblies resulted in a sequence of events in a closed loop. We determined correlations to measure the similarity between the patterns in each assembly, which showed that STDP made the network fire with a more distinctive and similar pattern. In addition, after memorizing a pattern, the frequency of events increased and the firing patterns became faster. Therefore, STDP can improve and accelerate the overall NAC process. Thus, our simulation results demonstrate that STDP can help NAC to obtain a more distinctive pattern in the network output.

Effects of bursting dynamic features on the generation of multi-clustered structure of neural network with symmetric spike-timing-dependent plasticity learning rule.

In this paper, the generation of multi-clustered structure of self-organized neural network with different neuronal firing patterns, i.e., bursting or spiking, has been investigated. The initially all-to-all-connected spiking neural network or bursting neural network can be self-organized into clustered structure through the symmetric spike-timing-dependent plasticity learning for both bursting and spiking neurons. However, the time consumption of this clustering procedure of the burst-based self-organized neural network (BSON) is much shorter than the spike-based self-organized neural network (SSON). Our results show that the BSON network has more obvious small-world properties, i.e., higher clustering coefficient and smaller shortest path length than the SSON network. Also, the results of larger structure entropy and activity entropy of the BSON network demonstrate that this network has higher topological complexity and dynamical diversity, which benefits for enhancing information transmission of neural circuits. Hence, we conclude that the burst firing can significantly enhance the efficiency of clustering procedure and the emergent clustered structure renders the whole network more synchronous and therefore more sensitive to weak input. This result is further confirmed from its improved performance on stochastic resonance. Therefore, we believe that the multi-clustered neural network which self-organized from the bursting dynamics has high efficiency in information processing.

Programmable Spike-Timing-Dependent Plasticity Learning Circuits in Neuromorphic VLSI Architectures

Hardware implementations of spiking neural networks offer promising solutions for computational tasks that require compact and low-power computing technologies. As these solutions depend on both the specific network architecture and the type of learning algorithm used, it is important to develop spiking neural network devices that offer the possibility to reconfigure their network topology and to implement different types of learning mechanisms. Here we present a neuromorphic multi-neuron VLSI device with on-chip programmable event-based hybrid analog/digital circuits; the event-based nature of the input/output signals allows the use of address-event representation infrastructures for configuring arbitrary network architectures, while the programmable synaptic efficacy circuits allow the implementation of different types of spike-based learning mechanisms. The main contributions of this article are to demonstrate how the programmable neuromorphic system proposed can be configured to implement specific spike-based synaptic plasticity rules and to depict how it can be utilised in a cognitive task. Specifically, we explore the implementation of different spike-timing plasticity learning rules online in a hybrid system comprising a workstation and when the neuromorphic VLSI device is interfaced to it, and we demonstrate how, after training, the VLSI device can perform as a standalone component (i.e., without requiring a computer), binary classification of correlated patterns.

A neuromorphic implementation of multiple spike-timing synaptic plasticity rules for large-scale neural networks.

We present a neuromorphic implementation of multiple synaptic plasticity learning rules, which include both Spike Timing Dependent Plasticity (STDP) and Spike Timing Dependent Delay Plasticity (STDDP). We present a fully digital implementation as well as a mixed-signal implementation, both of which use a novel dynamic-assignment time-multiplexing approach and support up to 226 (64M) synaptic plasticity elements. Rather than implementing dedicated synapses for particular types of synaptic plasticity, we implemented a more generic synaptic plasticity adaptor array that is separate from the neurons in the neural network. Each adaptor performs synaptic plasticity according to the arrival times of the pre- and post-synaptic spikes assigned to it, and sends out a weighted or delayed pre-synaptic spike to the post-synaptic neuron in the neural network. This strategy provides great flexibility for building complex large-scale neural networks, as a neural network can be configured for multiple synaptic plasticity rules without changing its structure. We validate the proposed neuromorphic implementations with measurement results and illustrate that the circuits are capable of performing both STDP and STDDP. We argue that it is practical to scale the work presented here up to 236 (64G) synaptic adaptors on a current high-end FPGA platform.

A Reconfigurable Digital Neuromorphic Processor with Memristive Synaptic Crossbar for Cognitive Computing

This article presents a brain-inspired reconfigurable digital neuromorphic processor (DNP) architecture for large-scale spiking neural networks. The proposed architecture integrates an arbitrary number of N digital leaky integrate-and-fire (LIF) silicon neurons to mimic their biological counterparts and on-chip learning circuits to realize spike-timing-dependent plasticity (STDP) learning rules. We leverage memristor nanodevices to build an N × N crossbar array to store not only multibit synaptic weight values but also network configuration data with significantly reduced area overhead. Additionally, the crossbar array is designed to be accessible both column- and row-wise to expedite the synaptic weight update process for learning. The proposed digital pulse width modulator (PWM) produces binary pulses with various durations for reading and writing the multilevel memristive crossbar. The proposed column based analog-to-digital conversion (ADC) scheme efficiently accumulates the presynaptic weights of each neuron and reduces silicon area overhead by using a shared arithmetic unit to process the LIF operations of all N neurons. With 256 silicon neurons, learning circuits and 64K synapses, the power dissipation and area of our DNP are 6.45 mW and 1.86 mm 2 , respectively, when implemented in a 90-nm CMOS technology. The functionality of the proposed DNP architecture is demonstrated by realizing an unsupervised-learning based character recognition system.

A Minimal Spiking Neural Network to Rapidly Train and Classify Handwritten Digits in Binary and 10-Digit Tasks

This paper reports the results of experiments to develop a minimal neural network for pattern classification. The network uses biologically plausible neural and learning mechanisms and is applied to a subset of the MNIST dataset of handwritten digits. The research goal is to assess the classification power of a very simple biologically motivated mechanism. The network architecture is primarily a feedforward spiking neural network (SNN) composed of Izhikevich regular spiking (RS) neurons and conductance-based synapses. The weights are trained with the spike timing-dependent plasticity (STDP) learning rule. The proposed SNN architecture contains three neuron layers which are connected by both static and adaptive synapses. Visual input signals are processed by the first layer to generate input spike trains. The second and third layers contribute to spike train segmentation and STDP learning, respectively. The network is evaluated by classification accuracy on the handwritten digit images from the MNIST dataset. The simulation results show that although the proposed SNN is trained quickly without error-feedbacks in a few number of iterations, it results in desirable performance (97.6%) in the binary classification (0 and 1). In addition, the proposed SNN gives acceptable recognition accuracy in 10-digit (0-9) classification in comparison with statistical methods such as support vector machine (SVM) and multi-perceptron neural network.

Ferroelectric Artificial Synapses for Recognition of a Multishaded Image

We demonstrate, for the first time, the on-chip pattern recognition of a multishaded grayscale image in a neural network circuit with multiple neurons. This pattern recognition is based on a spiking neural network model that uses multiple three-terminal ferroelectric memristors (3T-FeMEMs) as synapses. The synapse chip of the neural network is formed by stacking CMOS circuits and 3T-FeMEMs. The conductance of the 3T-FeMEM is gradually changed in the linear range by varying the amplitude of the applied voltage pulse. Using the analog and nonvolatile conductance change of the 3T-FeMEM as synaptic weight, the matrix patterns are learned after the spike timing-dependent plasticity learning rule. Even when an incomplete multishaded pattern is input to the neural network circuit, it automatically completes and recalls a previously learned pattern.

Bio-Inspired Stochastic Computing Using Binary CBRAM Synapses

In this paper, we present an alternative approach to neuromorphic systems based on multilevel resistive memory synapses and deterministic learning rules. We demonstrate an original methodology to use conductive-bridge RAM (CBRAM) devices as, easy to program and low-power, binary synapses with stochastic learning rules. New circuit architecture, programming strategy, and probabilistic spike-timing dependent plasticity (STDP) learning rule for two different CBRAM configurations with-selector (1T-1R) and without-selector (1R) are proposed. We show two methods (intrinsic and extrinsic) for implementing probabilistic STDP rules. Fully unsupervised learning with binary synapses is illustrated through two example applications: 1) real-time auditory pattern extraction (inspired from a 64-channel silicon cochlea emulator); and 2) visual pattern extraction (inspired from the processing inside visual cortex). High accuracy (audio pattern sensitivity > 2, video detection rate > 95%) and low synaptic-power dissipation (audio 0.55 μW, video 74.2 μW) are shown. The robustness and impact of synaptic parameter variability on system performance are also analyzed.

Self-organizing spiking neural model for learning fault-tolerant spatio-motor transformations.

In this paper, we present a spiking neural model that learns spatio-motor transformations. The model is in the form of a multilayered architecture consisting of integrate and fire neurons and synapses that employ spike-timing-dependent plasticity learning rule to enable the learning of such transformations. We developed a simple 2-degree-of-freedom robot-based reaching task which involves the learning of a nonlinear function. Computer simulations demonstrate the capability of such a model for learning the forward and inverse kinematics for such a task and hence to learn spatio-motor transformations. The interesting aspect of the model is its capacity to be tolerant to partial absence of sensory or motor inputs at various stages of learning. We believe that such a model lays the foundation for learning other complex functions and transformations in real-world scenarios.

Visual Pattern Extraction Using Energy-Efficient “2-PCM Synapse” Neuromorphic Architecture

We introduce a novel energy-efficient methodology “2-PCM Synapse” to use phase-change memory (PCM) as synapses in large-scale neuromorphic systems. Our spiking neural network architecture exploits the gradual crystallization behavior of PCM devices for emulating both synaptic potentiation and synaptic depression. Unlike earlier attempts to implement a biological-like spike-timing-dependent plasticity learning rule with PCM, we use a simplified rule where long-term potentiation and long-term depression can both be produced with a single invariant crystallizing pulse. Our architecture is simulated on a special purpose event-based simulator, using a behavioral model for the PCM devices validated with electrical characterization. The system, comprising about 2 million synapses, directly learns from event-based dynamic vision sensors. When tested with real-life data, it is able to extract complex and overlapping temporally correlated features such as car trajectories on a freeway. Complete trajectories can be learned with a detection rate above 90 %. The synaptic programming power consumption of the system during learning is estimated and could be as low as 100 nW for scaled down PCM technology. Robustness to device variability is also evidenced.

VLSI implementation of a bio-inspired olfactory spiking neural network.

This paper presents a low-power, neuromorphic spiking neural network (SNN) chip that can be integrated in an electronic nose system to classify odor. The proposed SNN takes advantage of sub-threshold oscillation and onset-latency representation to reduce power consumption and chip area, providing a more distinct output for each odor input. The synaptic weights between the mitral and cortical cells are modified according to an spike-timing-dependent plasticity learning rule. During the experiment, the odor data are sampled by a commercial electronic nose (Cyranose 320) and are normalized before training and testing to ensure that the classification result is only caused by learning. Measurement results show that the circuit only consumed an average power of approximately 3.6 μW with a 1-V power supply to discriminate odor data. The SNN has either a high or low output response for a given input odor, making it easy to determine whether the circuit has made the correct decision. The measurement result of the SNN chip and some well-known algorithms (support vector machine and the K-nearest neighbor program) is compared to demonstrate the classification performance of the proposed SNN chip.The mean testing accuracy is 87.59% for the data used in this paper.

Energy-Efficient Neuron, Synapse and STDP Integrated Circuits

Ultra-low energy biologically-inspired neuron and synapse integrated circuits are presented. The synapse includes a spike timing dependent plasticity (STDP) learning rule circuit. These circuits have been designed, fabricated and tested using a 90 nm CMOS process. Experimental measurements demonstrate proper operation. The neuron and the synapse with STDP circuits have an energy consumption of around 0.4 pJ per spike and synaptic operation respectively.

Approximating distributions in stochastic learning

On-line machine learning algorithms, many biological spike-timing-dependent plasticity (STDP) learning rules, and stochastic neural dynamics evolve by Markov processes. A complete description of such systems gives the probability densities for the variables. The evolution and equilibrium state of these densities are given by a Chapman-Kolmogorov equation in discrete time, or a master equation in continuous time. These formulations are analytically intractable for most cases of interest, and to make progress a nonlinear Fokker-Planck equation (FPE) is often used in their place. The FPE is limited, and some argue that its application to describe jump processes (such as in these problems) is fundamentally flawed. We develop a well-grounded perturbation expansion that provides approximations for both the density and its moments. The approach is based on the system size expansion in statistical physics (which does not give approximations for the density), but our simple development makes the methods accessible and invites application to diverse problems. We apply the method to calculate the equilibrium distributions for two biologically-observed STDP learning rules and for a simple nonlinear machine-learning problem. In all three examples, we show that our perturbation series provides good agreement with Monte-Carlo simulations in regimes where the FPE breaks down.

Stochastic Perturbation Methods for Spike-Timing-Dependent Plasticity

Online machine learning rules and many biological spike-timing-dependent plasticity (STDP) learning rules generate jump process Markov chains for the synaptic weights. We give a perturbation expansion for the dynamics that, unlike the usual approximation by a Fokker-Planck equation (FPE), is well justified. Our approach extends the related system size expansion by giving an expansion for the probability density as well as its moments. We apply the approach to two observed STDP learning rules and show that in regimes where the FPE breaks down, the new perturbation expansion agrees well with Monte Carlo simulations. The methods are also applicable to the dynamics of stochastic neural activity. Like previous ensemble analyses of STDP, we focus on equilibrium solutions, although the methods can in principle be applied to transients as well.

Stable Learning in Stochastic Network States

The mammalian cerebral cortex is characterized in vivo by irregular spontaneous activity, but how this ongoing dynamics affects signal processing and learning remains unknown. The associative plasticity rules demonstrated in vitro, mostly in silent networks, are based on the detection of correlations between presynaptic and postsynaptic activity and hence are sensitive to spontaneous activity and spurious correlations. Therefore, they cannot operate in realistic network states. Here, we present a new class of spike-timing-dependent plasticity learning rules with local floating plasticity thresholds, the slow dynamics of which account for metaplasticity. This novel algorithm is shown to both correctly predict homeostasis in synaptic weights and solve the problem of asymptotic stable learning in noisy states. It is shown to naturally encompass many other known types of learning rule, unifying them into a single coherent framework. The mixed presynaptic and postsynaptic dependency of the floating plasticity threshold is justified by a cascade of known molecular pathways, which leads to experimentally testable predictions.

An Electronic Synapse Device Based on Metal Oxide Resistive Switching Memory for Neuromorphic Computation

The multilevel capability of metal oxide resistive switching memory was explored for the potential use as a single-element electronic synapse device. TiN/HfOx/AlOx/ Pt resistive switching cells were fabricated. Multilevel resistance states were obtained by varying the programming voltage amplitudes during the pulse cycling. The cell conductance could be continuously increased or decreased from cycle to cycle, and about 105 endurance cycles were obtained. Nominal energy consumption per operation is in the subpicojoule range with a maximum measured value of 6 pJ. This low energy consumption is attractive for the large-scale hardware implementation of neuromorphic computing and brain simulation. The property of gradual resistance change by pulse amplitudes was exploited to demonstrate the spike-timing-dependent plasticity learning rule, suggesting that metal oxide memory can potentially be used as an electronic synapse device for the emerging neuromorphic computation system.