Unsupervised Spike Timing Dependent Plasticity Research Articles

Paired competing neurons improving STDP supervised local learning in Spiking Neural Networks

Direct training of Spiking Neural Networks (SNNs) on neuromorphic hardware has the potential to significantly reduce the energy consumption of artificial neural network training. SNNs trained with Spike Timing-Dependent Plasticity (STDP) benefit from gradient-free and unsupervised local learning, which can be easily implemented on ultra-low-power neuromorphic hardware. However, classification tasks cannot be performed solely with unsupervised STDP. In this paper, we propose Stabilized Supervised STDP (S2-STDP), a supervised STDP learning rule to train the classification layer of an SNN equipped with unsupervised STDP for feature extraction. S2-STDP integrates error-modulated weight updates that align neuron spikes with desired timestamps derived from the average firing time within the layer. Then, we introduce a training architecture called Paired Competing Neurons (PCN) to further enhance the learning capabilities of our classification layer trained with S2-STDP. PCN associates each class with paired neurons and encourages neuron specialization toward target or non-target samples through intra-class competition. We evaluate our methods on image recognition datasets, including MNIST, Fashion-MNIST, and CIFAR-10. Results show that our methods outperform state-of-the-art supervised STDP learning rules, for comparable architectures and numbers of neurons. Further analysis demonstrates that the use of PCN enhances the performance of S2-STDP, regardless of the hyperparameter set and without introducing any additional hyperparameters.

A Compact Model for Stochastic Spike-Timing-Dependent Plasticity (STDP) Based on Resistive Switching Memory (RRAM) Synapses

Resistive switching memory (RRAM) devices have been proposed to boost the density and the bio-realistic plasticity in neural networks. One of the main limitations to the development of neuromorphic systems with RRAM devices is the lack of compact models for the simulation of spiking neural networks, including neuron spike processing, synaptic plasticity, and stochastic learning. Here, we present a predictive model for neuromorphic networks with unsupervised spike timing-dependent plasticity (STDP) in HfO2 RRAM devices. Our compact model can predict the learning behavior of experimental networks and can speed up the simulation of unsupervised learning compared to Monte Carlo (MC) approaches. The model can be used to optimize the classification accuracy of data sets, such as MNIST, and to estimate the time of learning and the energy consumption.

Controlled Forgetting: Targeted Stimulation and Dopaminergic Plasticity Modulation for Unsupervised Lifelong Learning in Spiking Neural Networks.

Stochastic gradient descent requires that training samples be drawn from a uniformly random distribution of the data. For a deployed system that must learn online from an uncontrolled and unknown environment, the ordering of input samples often fails to meet this criterion, making lifelong learning a difficult challenge. We exploit the locality of the unsupervised Spike Timing Dependent Plasticity (STDP) learning rule to target local representations in a Spiking Neural Network (SNN) to adapt to novel information while protecting essential information in the remainder of the SNN from catastrophic forgetting. In our Controlled Forgetting Networks (CFNs), novel information triggers stimulated firing and heterogeneously modulated plasticity, inspired by biological dopamine signals, to cause rapid and isolated adaptation in the synapses of neurons associated with outlier information. This targeting controls the forgetting process in a way that reduces the degradation of accuracy for older tasks while learning new tasks. Our experimental results on the MNIST dataset validate the capability of CFNs to learn successfully over time from an unknown, changing environment, achieving 95.24% accuracy, which we believe is the best unsupervised accuracy ever achieved by a fixed-size, single-layer SNN on a completely disjoint MNIST dataset.

Classifying Melanoma Skin Lesions Using Convolutional Spiking Neural Networks With Unsupervised STDP Learning Rule

Deep learning methods have made some achievements in the automatic skin lesion recognition, but there are still some problems such as limited training samples, too complicated network structure, and expensive computational costs. Considering the inherent power-efficiency, biological plausibility and good image recognition performance of spiking neural networks (SNNs), in this paper we make malignant melanoma and benign melanocytic nevi skin lesions classification using convolutional SNNs with unsupervised spike-timing-dependent plasticity (STDP) learning rule. Efficient temporal coding, event driven learning rule and winner-take-all (WTA) mechanism together ensure sparse spike coding and efficient learning of our networks which achieve an average accuracy of 83.8%. We further propose to use feature selection to select more diagnostic features to improve the classification performance of our networks. Our SNNs with feature selection reach an average accuracy of 87.7%. Experimental results show that comparing to CNNs that need to be trained from scratch, our SNNs (with and without feature selection) not only achieve much better classification accuracies but also have much better runtime efficiency. Moreover, although the pretrained CNNs models can achieve similar running time, our proposed SNNs are more stable and easier to use than the pretrained CNNs because we do not need to try many pretrained models any more, and our SNNs also have much better classification accuracies than the pretrained CNNs. In addition, our networks have only three convolutional layers, and the complexity of the model and the parameters that need to be trained in the networks are greatly reduced. Our works show that STDP-based SNNs are very beneficial for the implementation of automated skin lesion classifiers on small portable devices.

Unsupervised online learning of temporal information in spiking neural network using thin-film transistor-type NOR flash memory devices

Brain-inspired analog neuromorphic systems based on the synaptic arrays have attracted large attention due to low-power computing. Spike-timing-dependent plasticity (STDP) algorithm is considered as one of the appropriate neuro-inspired techniques to be applied for on-chip learning. The aim of this study is to investigate the methodology of unsupervised STDP based learning in temporal encoding systems. The system-level simulation was performed based on the measurement results of thin-film transistor-type asymmetric floating-gate NOR flash memory. With proposed learning methods, 91.53% of recognition accuracy is obtained in inferencing MNIST standard dataset with 200 output neurons. Moreover, temporal encoding rules showed that the number of input pulses and the computing power can be compressed without significant loss of recognition accuracy compared to the conventional rate encoding scheme. In addition, temporal computing in a multi-layer network is suitable for learning data sequences, suggesting the possibility of applying to real-world tasks such as classifying direction of moving objects.

Energy-efficient FPGA Spiking Neural Accelerators with Supervised and Unsupervised Spike-timing-dependent-Plasticity

The liquid state machine (LSM) is a model of recurrent spiking neural networks (SNNs) and provides an appealing brain-inspired computing paradigm for machine-learning applications. Moreover, operated by processing information directly on spiking events, the LSM is amenable to efficient event-driven hardware implementation. However, training SNNs is, in general, a difficult task as synaptic weights shall be updated based on neural firing activities while achieving a learning objective. In this article, we explore bio-plausible spike-timing-dependent-plasticity (STDP) mechanisms to train liquid state machine models with and without supervision. First, we employ a supervised STDP rule to train the output layer of the LSM while delivering good classification performance. Furthermore, a hardware-friendly unsupervised STDP rule is leveraged to train the recurrent reservoir to further boost the performance. We pursue efficient hardware implementation of FPGA LSM accelerators by performing algorithm-level optimization of the two proposed training rules and exploiting the self-organizing behaviors naturally induced by STDP. Several recurrent spiking neural accelerators are built on a Xilinx Zync ZC-706 platform and trained for speech recognition with the TI46 speech corpus as the benchmark. Adopting the two proposed unsupervised and supervised STDP rules outperforms the recognition accuracy of a competitive non-STDP baseline training algorithm by up to 3.47%.

A 4096-Neuron 1M-Synapse 3.8-pJ/SOP Spiking Neural Network With On-Chip STDP Learning and Sparse Weights in 10-nm FinFET CMOS

A reconfigurable 4096-neuron, 1M-synapse chip in 10-nm FinFET CMOS is developed to accelerate inference and learning for many classes of spiking neural networks (SNNs). The SNN features digital circuits for leaky integrate and fire neuron models, on-chip spike-timing-dependent plasticity (STDP) learning, and high-fan-out multicast spike communication. Structured fine-grained weight sparsity reduces synapse memory by up to 16 $\times $ with less than 2% overhead for storing connections. Approximate computing co-optimizes the dropping flow control and benefits from algorithmic noise to process spatiotemporal spike patterns with up to 9.4 $\times $ lower energy. The SNN achieves a peak throughput of 25.2 GSOP/s at 0.9 V, peak energy efficiency of 3.8 pJ/SOP at 525 mV, and 2.3- $\mu \text{W}$ /neuron operation at 450 mV. On-chip unsupervised STDP trains a spiking restricted Boltzmann machine to de-noise Modified National Institute of Standards and Technology (MNIST) digits and to reconstruct natural scene images with RMSE of 0.036. Near-threshold operation, in conjunction with temporal and spatial sparsity, reduces energy by $17.4\times $ to 1.0- $\mu \text{J}$ /classification in a $236 \times 20$ feed-forward network that is trained to classify MNIST digits using supervised STDP. A binary-activation multilayer perceptron with 50% sparse weights is trained offline with error backpropagation to classify MNIST digits with 97.9% accuracy at 1.7- $\mu \text{J}$ /classification.

Memristive Spiking Neural Networks Trained with Unsupervised STDP

Neuromorphic computing systems are promising alternatives in the fields of pattern recognition, image processing, etc. especially when conventional von Neumann architectures face several bottlenecks. Memristors play vital roles in neuromorphic computing systems and are usually used as synaptic devices. Memristive spiking neural networks (MSNNs) are considered to be more efficient and biologically plausible than other systems due to their spike-based working mechanism. In contrast to previous SNNs with complex architectures, we propose a hardware-friendly architecture and an unsupervised spike-timing dependent plasticity (STDP) learning method for MSNNs in this paper. The architecture, which is friendly to hardware implementation, includes an input layer, a feature learning layer and a voting circuit. To reduce hardware complexity, some constraints are enforced: the proposed architecture has no lateral inhibition and is purely feedforward; it uses the voting circuit as a classifier and does not use additional classifiers; all neurons can generate at most one spike and do not need to consider firing rates and refractory periods; all neurons have the same fixed threshold voltage for classification. The presented unsupervised STDP learning method is time-dependent and uses no homeostatic mechanism. The MNIST dataset is used to demonstrate our proposed architecture and learning method. Simulation results show that our proposed architecture with the learning method achieves a classification accuracy of 94.6%, which outperforms other unsupervised SNNs that use time-based encoding schemes.

First-Spike-Based Visual Categorization Using Reward-Modulated STDP.

Reinforcement learning (RL) has recently regained popularity with major achievements such as beating the European game of Go champion. Here, for the first time, we show that RL can be used efficiently to train a spiking neural network (SNN) to perform object recognition in natural images without using an external classifier. We used a feedforward convolutional SNN and a temporal coding scheme where the most strongly activated neurons fire first, while less activated ones fire later, or not at all. In the highest layers, each neuron was assigned to an object category, and it was assumed that the stimulus category was the category of the first neuron to fire. If this assumption was correct, the neuron was rewarded, i.e., spike-timing-dependent plasticity (STDP) was applied, which reinforced the neuron's selectivity. Otherwise, anti-STDP was applied, which encouraged the neuron to learn something else. As demonstrated on various image data sets (Caltech, ETH-80, and NORB), this reward-modulated STDP (R-STDP) approach has extracted particularly discriminative visual features, whereas classic unsupervised STDP extracts any feature that consistently repeats. As a result, R-STDP has outperformed STDP on these data sets. Furthermore, R-STDP is suitable for online learning and can adapt to drastic changes such as label permutations. Finally, it is worth mentioning that both feature extraction and classification were done with spikes, using at most one spike per neuron. Thus, the network is hardware friendly and energy efficient.

Fast unsupervised learning for visual pattern recognition using spike timing dependent plasticity

Real-time learning needs algorithms operating in a fast speed comparable to human or animal, however this is a huge challenge in processing visual inputs. Research shows a biological brain can process complicated real-life recognition scenarios at milliseconds scale. Inspired by biological system, in this paper, we proposed a novel real-time learning method by combing the spike timing-based feed-forward spiking neural network (SNN) and the fast unsupervised spike timing dependent plasticity learning method with dynamic post-synaptic thresholds. Fast cross-validated experiments using MNIST database showed the high efficiency of the proposed method at an acceptable accuracy.

Spiking neural network model of reinforcement learning in the honeybee implemented on the GPU

Honeybees can learn and perform complex behavioral tasks despite their small brains that contain less than a million neurons. At the same time they are accessible to physiological experiments and the relatively small number of neurons in their brain lends itself to quite detailed numerical simulations. Bees therefore are a good model system for studying sensory cognition and reinforcement learning. We have shown in earlier work [1] that the anatomy and known electrophysiological properties of the olfactory pathway of insects in combination with spike-timing dependent plasticity (STDP) and lateral inhibition lend themselves to an unsupervised self-organization of synaptic connections for the recognition of odors. Here we extend this model by adding mechanisms of reinforcement learning, as suggested by [2] (see Figure ​Figure1).1). We employ a three factor learning rule where plasticity is governed by pre-synaptic and post-synaptic activity and a global octopaminergic/dopaminergic reinforcement signal, triggered by a reward. We investigated the role of feed-forward and feedback mechanisms, as well as the role of the connectivity initially achieved by unsupervised STDP. Figure 1 Network diagram for the hypothesized model of reinforcement learning in the honeybee olfactory system. Excitatory connections are shown in black, inhibitory connections in blue and learning synapses in red. Grey arrows represent the abstractions modeled ... Our model is implemented in the GeNN [3] framework, which facilitates the use of GPUs for spiking neural network simulations using a code generation framework. Because of the massive parallelism provided by GPUs, we can simulate tens of thousands of neurons in real time in the sparse firing regime relevant here. We investigated optimization strategies and neuron and synapse model choices for a better performance on the GPU. The model presented here is a stepping-stone to more sophisticated learning models and multi-sensory integration in the Green Brain Project [4], in which we aim to control a flying robot with a simulation of learning and decision making mechanisms in the honeybee related both to the olfactory and visual pathways.

A Spiking Network Model of Decision Making Employing Rewarded STDP

Reward-modulated spike timing dependent plasticity (STDP) combines unsupervised STDP with a reinforcement signal that modulates synaptic changes. It was proposed as a learning rule capable of solving the distal reward problem in reinforcement learning. Nonetheless, performance and limitations of this learning mechanism have yet to be tested for its ability to solve biological problems. In our work, rewarded STDP was implemented to model foraging behavior in a simulated environment. Over the course of training the network of spiking neurons developed the capability of producing highly successful decision-making. The network performance remained stable even after significant perturbations of synaptic structure. Rewarded STDP alone was insufficient to learn effective decision making due to the difficulty maintaining homeostatic equilibrium of synaptic weights and the development of local performance maxima. Our study predicts that successful learning requires stabilizing mechanisms that allow neurons to balance their input and output synapses as well as synaptic noise.

A Neuromorphic Architecture for Object Recognition and Motion Anticipation Using Burst-STDP

In this work we investigate the possibilities offered by a minimal framework of artificial spiking neurons to be deployed in silico. Here we introduce a hierarchical network architecture of spiking neurons which learns to recognize moving objects in a visual environment and determine the correct motor output for each object. These tasks are learned through both supervised and unsupervised spike timing dependent plasticity (STDP). STDP is responsible for the strengthening (or weakening) of synapses in relation to pre- and post-synaptic spike times and has been described as a Hebbian paradigm taking place both in vitro and in vivo. We utilize a variation of STDP learning, called burst-STDP, which is based on the notion that, since spikes are expensive in terms of energy consumption, then strong bursting activity carries more information than single (sparse) spikes. Furthermore, this learning algorithm takes advantage of homeostatic renormalization, which has been hypothesized to promote memory consolidation during NREM sleep. Using this learning rule, we design a spiking neural network architecture capable of object recognition, motion detection, attention towards important objects, and motor control outputs. We demonstrate the abilities of our design in a simple environment with distractor objects, multiple objects moving concurrently, and in the presence of noise. Most importantly, we show how this neural network is capable of performing these tasks using a simple leaky-integrate-and-fire (LIF) neuron model with binary synapses, making it fully compatible with state-of-the-art digital neuromorphic hardware designs. As such, the building blocks and learning rules presented in this paper appear promising for scalable fully neuromorphic systems to be implemented in hardware chips.