Input Spike Research Articles

A Supervised Learning Algorithm for Spiking Neurons Using Spike Train Kernel Based on a Unit of Pair-Spike

In recent years, neuroscientists have discovered that the neural information is encoded by spike trains with precise times. Supervised learning algorithm based on the precise times for spiking neurons becomes an important research field. Although many existing algorithms have the excellent learning ability, most of their mechanisms still have some complex computations and certain limitations. Moreover, the discontinuity of spiking process also makes it very difficult to build an efficient algorithm. This paper proposes a supervised learning algorithm for spiking neurons using the kernel function of spike trains based on a unit of pair-spike. Firstly, we comprehensively divide the intervals of spike trains. Then, we construct an optimal selection and computation method of spikes based on the unit of pair-spike. This method avoids some wrong computations and reduces the computational cost by using each effective input spike only once in every epoch. Finally, we use the kernel function defined by an inner product operator to solve the computing problem of discontinue spike process and multiple output spikes. The proposed algorithm is successfully applied to many learning tasks of spike trains, where the effect of our optimal selection and computation method is verified and the influence of learning factors such as learning kernel, learning rate, and learning epoch is analyzed. Moreover, compared with other algorithms, all experimental results show that our proposed algorithm has the higher learning accuracy and good learning efficiency.

Spiking time-dependent plasticity leads to efficient coding of predictions

Latency reduction in postsynaptic spikes is a well-known effect of spiking time-dependent plasticity. We expand this notion for long postsynaptic spike trains on single neurons, showing that, for a fixed input spike train, STDP reduces the number of postsynaptic spikes and concentrates the remaining ones. Then, we study the consequences of this phenomena in terms of coding, finding that this mechanism improves the neural code by increasing the signal-to-noise ratio and lowering the metabolic costs of frequent stimuli. Finally, we illustrate that the reduction in postsynaptic latencies can lead to the emergence of predictions.

A Model for the Diffusive Filling-In Algorithm Operating in Spike Mode

To run cortical circuit simulations in spike mode, i.e. taking into account the neural representation of information in terms of sequences of electrical pulses (also known as spikes), the use of customized hardware, which is specific for this purpose, is recommended. Simulations using more traditional hardware can be prohibitive. In this context, theoretical predictions are important for customized hardware design. For example, theoretical predictions lead to an adequate neuron model choice. To make such theoretical predictions, the cortical circuit simulations are carried out in amplitude mode. Differently from the spike mode, in amplitude mode information is represented by sequences of scalar values that describe neural input and output spike rates. In this paper, it was proposed amplitude and spike mode simulations of a cortical algorithm, namely the diffusive filling-in algorithm, to investigate whether predictions based on the amplitude-mode results approximate well the behavior of the customized hardware (spike mode results). The diffusive filling-in algorithm was chosen because it is simple enough for spike-mode simulation in a conventional computer, but the proposed amplitude-mode prediction method is the same for more complex algorithms or circuits. We provide a highly realistic comparison between amplitude-mode and spike-mode in the diffusive filling-in case, which suggests that the amplitude mode is reliable for theoretical predictions useful for customized hardware design for cortical circuit simulation. The goal of this paper is not to bring closure to these discussions but to suggest a way of avoiding possible issues that could compromise the success of the customized device design.

STDP-Based Unsupervised Spike Pattern Learning in a Photonic Spiking Neural Network With VCSELs and VCSOAs

We propose a photonic spiking neural network (SNN) consisting of photonic spiking neurons based on vertical-cavity surface-emitting lasers (VCSELs). The photonic spike timing dependent plasticity (STDP) is implemented in a vertical-cavity semiconductor optical amplifier (VCSOA). A versatile computational model of the photonic SNN is presented based on the rate equation models. Through numerical simulation, a spike pattern learning and recognition task is performed based on the photonic STDP. The results show that the post-synaptic spike timing (PST) is eventually converged iteratively to the first spike timing of the input spike pattern via unsupervised learning. Additionally, the convergence rate of the PST can be accelerated for a photonic SNN with more pre-synaptic neurons. The effects of VCSOA parameters on the convergence performance of the unsupervised spike learning are also considered. To the best of our knowledge, such a versatile computational model of photonic SNN for unsupervised learning and recognition of arbitrary spike pattern has not yet been reported, which would contribute one step forward toward numerical implementation of a large-scale energy-efficient photonic SNN, and hence is interesting for neuromorphic photonic systems and spiking information processing.

An Introduction to Probabilistic Spiking Neural Networks: Probabilistic Models, Learning Rules, and Applications

Spiking neural networks (SNNs) are distributed trainable systems whose computing elements, or neurons, are characterized by internal analog dynamics and by digital and sparse synaptic communications. The sparsity of the synaptic spiking inputs and the corresponding event-driven nature of neural processing can be leveraged by energy-efficient hardware implementations, which can offer significant energy reductions as compared to conventional artificial neural networks (ANNs). The design of training algorithms for SNNs, however, lags behind hardware implementations: most existing training algorithms for SNNs have been designed either for biological plausibility or through conversion from pretrained ANNs via rate encoding.

Spike-Based Winner-Take-All Computation: Fundamental Limits and Order-Optimal Circuits.

Winner-take-all (WTA) refers to the neural operation that selects a (typically small) group of neurons from a large neuron pool. It is conjectured to underlie many of the brain's fundamental computational abilities. However, not much is known about the robustness of a spike-based WTA network to the inherent randomness of the input spike trains. In this work, we consider a spike-based -WTA model wherein randomly generated input spike trains compete with each other based on their underlying firing rates and winners are supposed to be selected. We slot the time evenly with each time slot of length 1 ms and model the input spike trains as independent Bernoulli processes. We analytically characterize the minimum waiting time needed so that a target minimax decision accuracy (success probability) can be reached. We first derive an information-theoretic lower bound on the waiting time. We show that to guarantee a (minimax) decision error (where ), the waiting time of any WTA circuit is at least [Formula: see text]where is a finite set of rates and is a difficulty parameter of a WTA task with respect to set for independent input spike trains. Additionally, is independent of , , and . We then design a simple WTA circuit whose waiting time is [Formula: see text]provided that the local memory of each output neuron is sufficiently long. It turns out that for any fixed , this decision time is order-optimal (i.e., it matches the above lower bound up to a multiplicative constant factor) in terms of its scaling in , , and .

Spike Counts Based Low Complexity SNN Architecture With Binary Synapse.

In this paper, we present an energy and area efficient spike neural network (SNN) processor based on novel spike counts based methods. For the low cost SNN design, we propose hardware-friendly complexity reduction techniques for both of learning and inferencing modes of operations. First, for the unsupervised learning process, we propose a spike counts based learning method. The novel learning approach utilizes pre- and post-synaptic spike counts to reduce the bit-width of synaptic weights as well as the number of weight updates. For the energy efficient inferencing operations, we propose an accumulation based computing scheme, where the number of input spikes for each input axon is accumulated without instant membrane updates until the pre-defined number of spikes are reached. In addition, the computation skip schemes identify meaningless computations and skip them to improve energy efficiency. Based on the proposed low complexity design techniques, we design and implement the SNN processor using 65 nm CMOS process. According to the implementation results, the SNN processor achieves 87.4% of recognition accuracy in MNIST dataset using only 1-bit 230 k synaptic weights with 400 excitatory neurons. The energy consumptions are 0.26 pJ/SOP and 0.31 μJ/inference in inferencing mode, and 1.42pJ/SOP and 2.63 μJ/learning in learning mode of operations.

Pattern Classification by Spiking Neural Networks Combining Self-Organized and Reward-Related Spike-Timing-Dependent Plasticity

Abstract Many recent studies have applied to spike neural networks with spike-timing-dependent plasticity (STDP) to machine learning problems. The learning abilities of dopamine-modulated STDP (DA-STDP) for reward-related synaptic plasticity have also been gathering attention. Following these studies, we hypothesize that a network structure combining self-organized STDP and reward-related DA-STDP can solve the machine learning problem of pattern classification. Therefore, we studied the ability of a network in which recurrent spiking neural networks are combined with STDP for non-supervised learning, with an output layer joined by DA-STDP for supervised learning, to perform pattern classification. We confirmed that this network could perform pattern classification using the STDP effect for emphasizing features of the input spike pattern and DA-STDP supervised learning. Therefore, our proposed spiking neural network may prove to be a useful approach for machine learning problems.

A CMOS Compatible Bulk FinFET-Based Ultra Low Energy Leaky Integrate and Fire Neuron for Spiking Neural Networks

The fundamental building block of an artificial spiking neural network (SNN) is an element which can effectively mimic a biological neuron. There are several electronic and spintronic devices which have been demonstrated as a neuron. But the main concern here is the energy consumption and large area of those artificial neurons. In this letter, we propose and demonstrate a highly scalable and CMOS compatible bulk FinFET with an ${n}^{+}$ buried layer for ultra low energy artificial neuron using well calibrated TCAD simulations. The proposed device shows the signature spiking frequency versus input voltage curve of a biological neuron. The energy per spike of the integrate block of the proposed leaky integrate and fire (LIF) neuron is ${6.3}~\textit {fJ}$ /spike which is the minimum reported till date.

Constructing an Associative Memory System Using Spiking Neural Network.

Development of computer science has led to the blooming of artificial intelligence (AI), and neural networks are the core of AI research. Although mainstream neural networks have done well in the fields of image processing and speech recognition, they do not perform well in models aimed at understanding contextual information. In our opinion, the reason for this is that the essence of building a neural network through parameter training is to fit the data to the statistical law through parameter training. Since the neural network built using this approach does not possess memory ability, it cannot reflect the relationship between data with respect to the causality. Biological memory is fundamentally different from the current mainstream digital memory in terms of the storage method. The information stored in digital memory is converted to binary code and written in separate storage units. This physical isolation destroys the correlation of information. Therefore, the information stored in digital memory does not have the recall or association functions of biological memory which can present causality. In this paper, we present the results of our preliminary effort at constructing an associative memory system based on a spiking neural network. We broke the neural network building process into two phases: the Structure Formation Phase and the Parameter Training Phase. The Structure Formation Phase applies a learning method based on Hebb's rule to provoke neurons in the memory layer growing new synapses to connect to neighbor neurons as a response to the specific input spiking sequences fed to the neural network. The aim of this phase is to train the neural network to memorize the specific input spiking sequences. During the Parameter Training Phase, STDP and reinforcement learning are employed to optimize the weight of synapses and thus to find a way to let the neural network recall the memorized specific input spiking sequences. The results show that our memory neural network could memorize different targets and could recall the images it had memorized.

Synaptic Strengths Dominate Phasing of Motor Circuit: Intrinsic Conductances of Neuron Types Need Not Vary across Animals.

Identified neurons and the networks they compose produce stereotypical, albeit individually unique, activity across members of a species. We propose, for a motor circuit driven by a central pattern generator (CPG), that the uniqueness derives mainly from differences in synaptic strength rather than from differences in intrinsic membrane conductances. We studied a dataset of recordings from six leech (Hirudo sp.) heartbeat control networks, containing complete spiking activity patterns from inhibitory premotor interneurons, motor output spike patterns, and synaptic strength patterns to investigate the source of uniqueness. We used a conductance-based multicompartmental motor neuron model to construct a bilateral motor circuit model, and controlled it by playing recorded input spike trains from premotor interneurons to generate output inhibitory synaptic patterns similar to experimental measurements. By generating different synaptic conductance parameter sets of this circuit model, we found that relative premotor synaptic strengths impinging onto motor neurons must be different across individuals to produce animal-specific output burst phasing. Obtaining unique outputs from each individual's circuit model did not require different intrinsic ionic conductance parameters. Furthermore, changing intrinsic conductances failed to compensate for modified synaptic strength patterns. Thus, the pattern of synaptic strengths of motor neuron inputs is critical for the phasing of this motor circuit and can explain individual differences. When intrinsic conductances were allowed to vary, they exhibited the same conductance correlations across individuals, suggesting a motor neuron "type" required for proper network function. Our results are general and may translate to other systems and neuronal networks that control output phasing.

Serotonergic modulation of slow inward rectification in mesencephalic trigeminal neurons

Inward rectification in response to membrane hyperpolarization is a prominent feature of mesencephalic trigeminal (Mes V) neurons and the hyperpolarization-activated inward current (Ih), as the basis of this property, regulates the spike discharge characteristics and input frequency preference (resonance) in these neurons, suggesting that Ih modulation is an important regulator of oral motor activity. To examine a possible contribution of serotonin (5-HT) to the modulation of Ih activation characteristics, in the present study, we investigated the modulatory effects of 5-HT receptor activation on Ih in postnatal day (P) 2–12 rat Mes V neurons by whole-cell patch-clamp recording. Bath application of 5-HT suppressed the Ih-dependent voltage sag and Ih conductance, but induced only a modest shift in the voltage dependence of Ih activation. This 5-HT-induced suppression of Ih was greater in P10–12 than P2–4 neurons, and involved the cAMP/protein kinase A (PKA) signaling pathway but not the PKC pathway. Pharmacological activation of the 5-HT1A receptor mimicked the effect of 5-HT, while modulation of other receptor subtypes, including 5-HT1B,1D, 5-HT2, and 5-HT3, had little or no effect on Ih. Low-frequency (<10 Hz) resonance at membrane potentials below the resting potential were reduced by 5-HT, suggesting that serotonergic Ih modulation can substantially alter the frequency preference to synaptic inputs. These results suggest that changes in resonance properties through serotonergic modulation of Ih may tune the firing of Mes V neurons to different afferent input frequencies and alter motor outputs to the jaw, thereby regulating oral motor activity.

Temporal pattern separation in hippocampal neurons through multiplexed neural codes.

Pattern separation is a central concept in current theories of episodic memory: this computation is thought to support our ability to avoid confusion between similar memories by transforming similar cortical input patterns of neural activity into dissimilar output patterns before their long-term storage in the hippocampus. Because there are many ways one can define patterns of neuronal activity and the similarity between them, pattern separation could in theory be achieved through multiple coding strategies. Using our recently developed assay that evaluates pattern separation in isolated tissue by controlling and recording the input and output spike trains of single hippocampal neurons, we explored neural codes through which pattern separation is performed by systematic testing of different similarity metrics and various time resolutions. We discovered that granule cells, the projection neurons of the dentate gyrus, can exhibit both pattern separation and its opposite computation, pattern convergence, depending on the neural code considered and the statistical structure of the input patterns. Pattern separation is favored when inputs are highly similar, and is achieved through spike time reorganization at short time scales (< 100 ms) as well as through variations in firing rate and burstiness at longer time scales. These multiplexed forms of pattern separation are network phenomena, notably controlled by GABAergic inhibition, that involve many celltypes with input-output transformations that participate in pattern separation to different extents and with complementary neural codes: a rate code for dentate fast-spiking interneurons, a burstiness code for hilar mossy cells and a synchrony code at long time scales for CA3 pyramidal cells. Therefore, the isolated hippocampal circuit itself is capable of performing temporal pattern separation using multiplexed coding strategies that might be essential to optimally disambiguate multimodal mnemonic representations.

An optimal time interval of input spikes involved in synaptic adjustment of spike sequence learning

The supervised learning methods for spiking neurons based on temporal encoding are important foundation for the development of spiking neural networks. During the learning process, the synaptic weights of a spiking neuron are adjusted to make the neuron emit a specific spike train. Because various learning methods use the information of input spikes to calculate the adjustment of synaptic weights, how many input spikes participated in the calculation is a critical factor that can influence learning performance. This paper chooses an important category of learning methods as the research object to study the factor. The input spikes participated in weight adjustment are contained in a time interval. An optimal time interval that contains the most appropriate number of input spikes is proposed based on the characteristic of the category of learning methods. The length of the optimal time interval is determined by comprehensive consideration of desired and actual output spikes. The results of a lot of experiments show that the optimal time interval can obtain the highest learning performance under various experimental settings. If other time intervals are longer than the optimal time interval, an overlapping problem of input spikes will occur and the learning performance will decline a lot. The learning accuracy of the optimal time interval can be about 55% higher than the learning accuracy of an other longer time interval. If other time intervals are shorter than the optimal time interval, the input spikes contained in them will be insufficient to adjust synaptic weights and the learning performance will also decline. The learning accuracy of the optimal time interval can be about 8% higher than the learning accuracy of an other shorter time interval. In addition, the optimal time interval can also improve the generalization ability and pattern storage capability of the category of learning methods.

Short-term synaptic depression can increase the rate of information transfer at a release site.

The release of neurotransmitters from synapses obeys complex and stochastic dynamics. Depending on the recent history of synaptic activation, many synapses depress the probability of releasing more neurotransmitter, which is known as synaptic depression. Our understanding of how synaptic depression affects the information efficacy, however, is limited. Here we propose a mathematically tractable model of both synchronous spike-evoked release and asynchronous release that permits us to quantify the information conveyed by a synapse. The model transits between discrete states of a communication channel, with the present state depending on many past time steps, emulating the gradual depression and exponential recovery of the synapse. Asynchronous and spontaneous releases play a critical role in shaping the information efficacy of the synapse. We prove that depression can enhance both the information rate and the information rate per unit energy expended, provided that synchronous spike-evoked release depresses less (or recovers faster) than asynchronous release. Furthermore, we explore the theoretical implications of short-term synaptic depression adapting on longer time scales, as part of the phenomenon of metaplasticity. In particular, we show that a synapse can adjust its energy expenditure by changing the dynamics of short-term synaptic depression without affecting the net information conveyed by each successful release. Moreover, the optimal input spike rate is independent of the amplitude or time constant of synaptic depression. We analyze the information efficacy of three types of synapses for which the short-term dynamics of both synchronous and asynchronous release have been experimentally measured. In hippocampal autaptic synapses, the persistence of asynchronous release during depression cannot compensate for the reduction of synchronous release, so that the rate of information transmission declines with synaptic depression. In the calyx of Held, the information rate per release remains constant despite large variations in the measured asynchronous release rate. Lastly, we show that dopamine, by controlling asynchronous release in corticostriatal synapses, increases the synaptic information efficacy in nucleus accumbens.

Self-Biased Ultralow Power Current-Reused Neural Amplifier With On-Chip Analog Spike Detections

An ultralow power 0.6 V neural amplifier with on-chip analog spike detection is presented. A capacitively-coupled instrumentation amplifier (CCIA) with the current-reused and self-biased scheme is proposed to reduce the overall power consumption and to enhance the noise efficiency. The transistors in the amplifier are operated in the subthreshold region to enhance noise performance. The analog-domain spike detection based on a low power peak detector can reduce the overall power consumption. The circuit is fabricated using the standard 0.18 μm CMOS process. The passband of the circuit is from 6.4 Hz to 4.46 kHz. Input-referred noise is 10.68 μVrms. The supply voltage is 0.6 V, and the power consumption of the singlestage CCIA is 50.6 nW. The CCIA achieves a good noise efficiency factor and power efficiency factor of 1.79 and 1.93, respectively. The overall power consumption including two CCIAs, the programmable gain amplifier, and the analog spike detector is 269.8 nW. Input spikes with an amplitude of 50 μV at 100-Hz intervals are accurately detected.

Joint Optimization of Input Spike Rate and Receiver Decision Threshold to Maximize Achievable Bit Rate of Neuro-Spike Communication Channel

Nano-networks employ novel nano-scale communication techniques. These techniques are inspired by biological systems. Neuro-spike communication is an example of this communication paradigm. A new example of nano-networks is the artificial neural system where nano-machines are linked to neurons to treat the neurodegenerative diseases. In these networks, nano-machines are used to replace the damaged segments of the nervous system and they behave exactly like biological entities. In this paper, by considering a point-to-point model for the neuro-spike communication which contains several sources of randomness such as the axonal noise, random amplitude, and the synaptic noise, we analyze the achievable bit rate of the neuro-spike communication channel. This model can be divided into two main parts. The first part is the axonal pathway and the second one encompasses the synaptic transmission and the spike generation. First, we focus on the axonal pathway and we model this part as a binary channel through defining the axonal shot noise probability. Next, we investigate the synaptic transmission part to model this part as a binary channel. Then, we consider the neuro-spike communication channel as two cascaded binary channels to obtain its error probability and achievable bit rate. To enhance the achievable bit rate of the neuro-spike communication, we design an optimum spike detection receiver and also we propose an optimum input spike rate. We derive closed-form descriptions for the decision threshold of the optimum receiver and the optimum input spike rate. Since they are coupled, we propose a recursive scheme to derive their optimum values. Finally, simulation results are used to investigate the impact of the axonal noise, the synaptic noise, and modifications of the spikes' shape due to propagation along the axon on the achievable bit rate of the neuro-spike communication channel.

Ultrathin Conformable Organic Artificial Synapse for Wearable Intelligent Device Applications.

Ultrathin conformable artificial synapse platforms that can be used as on-body or wearable chips suggest a path to build next-generation, wearable, intelligent electronic systems that can mimic the synaptic operations of the human brain. So far, an artificial synapse architecture with ultimate mechanical flexibility in a freestanding form while maintaining its functionalities with high stability and accuracy on any conformable substrate has not been demonstrated yet. Here, we demonstrate the first ultrathin artificial synapse (∼500 nm total thickness) that features freestanding ferroelectric organic neuromorphic transistors (FONTs), which can stand alone without a substrate or an encapsulation layer. Our simple dry peel-off process allows integration of the freestanding FONTs with an extremely thin film that is transferable to various conformable substrates. The FONTs exhibit excellent and reliable synaptic functions, which can be modulated by diverse electrical stimuli and relative timing (or temporal order) between the pre- and postsynaptic spikes. Furthermore, the FONTs show sustainable synaptic plasticity even under folded condition ( R = 50 μm, ε = 0.48%) for more than 6000 input spikes. Our study suggests that the ultrathin conformable organic artificial synapse platforms are considered as one of key technologies for realization of wearable intelligent electronics in the future.

The Relationship Between Ictal Multi-Unit Activity and the Electrocorticogram.

During neocortical seizures in patients with epilepsy, microelectrode array recordings from the ictal core show a strong correlation between the fast, cellular spiking activities and the low-frequency component of the potential field, reflected in the electrocorticogram (ECoG). Here, we model the relationship between the cellular spike activity and this low-frequency component as the input and output signals of a linear time invariant system. Our approach is based on the observation that this relationship can be characterized by a so-called sinc function, the unit impulse response of an ideal (brick-wall) filter. Accordingly, using a brick-wall filter, we are able to convert ictal cellular spike inputs into an output that significantly correlates with the observed seizure activity in the ECoG , while ECoG recordings of subsequent seizures within patients also show significant, but lower, correlations . Furthermore, we can produce seizure-like output signals using synthetic spike trains with ictal properties. We propose a possible physiological mechanism to explain the observed properties associated with an ideal filter, and discuss the potential use of our approach for the evaluation of anticonvulsant strategies.

Resonate and fire neuron with fixed magnetic skyrmions

In the brain, the membrane potential of many neurons oscillates in a subthreshold damped fashion and fire when excited by an input frequency that nearly equals their eigen frequency. In this work, we investigate theoretically the artificial implementation of such “resonate-and-fire” neurons by utilizing the magnetization dynamics of a fixed magnetic skyrmion in the free layer of a magnetic tunnel junction (MTJ). To realize firing of this nanomagnetic implementation of an artificial neuron, we propose to employ voltage control of magnetic anisotropy or voltage generated strain as an input (spike or sinusoidal) signal, which modulates the perpendicular magnetic anisotropy. This results in continual expansion and shrinking (i.e., breathing) of a skyrmion core that mimics the subthreshold oscillation. Any subsequent input pulse having an interval close to the breathing period or a sinusoidal input close to the eigen frequency drives the magnetization dynamics of the fixed skyrmion in a resonant manner. The time varying electrical resistance of the MTJ layer due to this resonant oscillation of the skyrmion core is used to drive a Complementary Metal Oxide Semiconductor buffer circuit, which produces spike outputs. By rigorous micromagnetic simulation, we investigate the interspike timing dependence and response to different excitatory and inhibitory incoming input pulses. Finally, we show that such resonate and fire neurons have potential application in coupled nanomagnetic oscillator based associative memory arrays.