Spike Encoding Research Articles

A Low-Cost Hardware-Friendly Spiking Neural Network Based on Binary MRAM Synapses, Accelerated Using In-Memory Computing

In recent years, the scaling down that Moore’s Law relies on has been gradually slowing down, and the traditional von Neumann architecture has been limiting the improvement of computing power. Thus, neuromorphic in-memory computing hardware has been proposed and is becoming a promising alternative. However, there is still a long way to make it possible, and one of the problems is to provide an efficient, reliable, and achievable neural network for hardware implementation. In this paper, we proposed a two-layer fully connected spiking neural network based on binary MRAM (Magneto-resistive Random Access Memory) synapses with low hardware cost. First, the network used an array of multiple binary MRAM cells to store multi-bit fixed-point weight values. This helps to simplify the read/write circuit. Second, we used different kinds of spike encoders that ensure the sparsity of input spikes, to reduce the complexity of peripheral circuits, such as sense amplifiers. Third, we designed a single-step learning rule, which fit well with the fixed-point binary weights. Fourth, we replaced the traditional exponential Leak-Integrate-Fire (LIF) neuron model to avoid the massive cost of exponential circuits. The simulation results showed that, compared to other similar works, our SNN with 1184 neurons and 313,600 synapses achieved an accuracy of up to 90.6% in the MNIST recognition task with full-resolution (28 × 28) and full-bit-depth (8-bit) images. In the case of low-resolution (16 × 16) and black-white (1-bit) images, the smaller version of our network with 384 neurons and 32,768 synapses still maintained an accuracy of about 77%, extending its application to ultra-low-cost situations. Both versions need less than 30,000 samples to reach convergence, which is a >50% reduction compared to other similar networks. As for robustness, it is immune to the fluctuation of MRAM cell resistance.

Balancing the Cost and Performance Trade-Offs in SNN Processors

Spiking neural network (SNN), known as the third generation of neural networks, is attracting more and more researchers' attention because of its high energy efficiency. However, due to the spatiotemporal characteristics, SNN involved a few complicated computations such as exponential function and logarithmic function, making it hard to implement on hardware. In this brief, we propose a SNN processor, which balances the trade-offs between cost and performance. In terms of cost, through software and hardware co-design, a high-robustness SNN model with 2-bit weights and a novel synapse delay management mechanism are adopted to reduce memory utilization. In terms of performance, a spike encoder and a VFA (Vote-For-All) decoder are used to reduce latency and improve inference accuracy respectively. We implement the design on the Xilinx ZCU102 FPGA board and apply it to the MNIST handwritten digit classification, which achieves 90.53% classification accuracy. Compared to a previous proposed SNN processor of a similar neuron scale, our design achieves a 156 × inference speed-up and consumes 0,32 × hardware resources.

CVnCoV and CV2CoV protect human ACE2 transgenic mice from ancestral B BavPat1 and emerging B.1.351 SARS-CoV-2

The ongoing SARS-CoV-2 pandemic necessitates the fast development of vaccines. Recently, viral mutants termed variants of concern (VOC) which may escape host immunity have emerged. The efficacy of spike encoding mRNA vaccines (CVnCoV and CV2CoV) against the ancestral strain and the VOC B.1.351 was tested in a K18-hACE2 transgenic mouse model. Naive mice and mice immunized with a formalin-inactivated SARS-CoV-2 preparation were used as controls. mRNA-immunized mice develop elevated SARS-CoV-2 RBD-specific antibody and neutralization titers which are readily detectable, but significantly reduced against VOC B.1.351. The mRNA vaccines fully protect from disease and mortality caused by either viral strain. SARS-CoV-2 remains undetected in swabs, lung, or brain in these groups. Despite lower neutralizing antibody titers compared to the ancestral strain BavPat1, CVnCoV and CV2CoV show complete disease protection against the novel VOC B.1.351 in our studies.

Large-Scale Convergence of Receptor Cell Arrays Onto Afferent Terminal Arbors in the Lorenzinian Electroreceptors of Polyodon.

Certain sensory receptors contain many transducers, converging onto few afferents. Convergence creates star-topology neural networks, of iterative parallel organization, that may yield special functional properties. We quantitated large-scale convergence in electroreceptors on the rostrum of preadult paddlefish, Polyodon spathula (Acipenseriforme vertebrates), and analyzed the afferent terminal branching underlying the convergence. From neurophysiological mapping, a recorded afferent innervated 23.3 ± 9.1 (range 6–45) ampullary organs, and innervated every ampullary organ within the receptive field’s sharp boundary. Ampullary organs each contained ∼665 Lorenzinian receptor cells, from imaging and modeling. We imaged three serial types of afferent branching at electroreceptors, after immunofluorescent labeling for neurite filaments, glial sheaths, or nodal ion channels, or by DiI tracing. (i) Myelinated tree: Each of 3.08 ± 0.51 (2–4) parallel afferents from a cranial nerve (ALLn) entered a receptive field from deeper tissue, then branched into a laminar tree of large myelinated dendrites, parallel to the skin, that branched radially until ∼9 extremities with heminodes, which were candidate sites of spike encoders. (ii) Inline transition: Each myelinated extremity led distally into local unmyelinated arbors originating at inline branching structures covered by terminal (satellite) glia. The unmyelinated transition zones included globular afferent modules, 4–6 microns wide, from which erupted fine fascicles of parallel submicron neurites, a possibly novel type of neuronal branching. The neurite fascicles formed loose bundles projecting ∼105 microns distally to innervate local groups of ∼3 adjacent ampullary organs. (iii) Radial arbors: Receptor cells in an electrosensory neuroepithelium covering the basal pole of each ampullary organ were innervated by bouton endings of radial neurites, unmyelinated and submicron, forming a thin curviplanar lamina distal to the lectin+ basal lamina. The profuse radial neurites diverged from thicker (∼2 micron) basolateral trunks. Overall, an average Polyodon electroreceptor formed a star topology array of ∼9 sensor groups. Total convergence ratios were 15,495 ± 6,052 parallel receptor cells per afferent per mean receptive field, assuming 100% innervation. Large-scale convergence likely increases the signal-to-noise ratio (SNR) of stimulus encoding into spiking afferent output, increasing receiver sensitivity. Unmyelinated arbors may also regenerate and repair the afferent innervation of ampullary organs.LSID: urn:lsid:zoobank.org:act:09BCF04C-3C3C-4B6C-9DC9-A2BF43087369

Spike Encoding with Optic Sensory Neurons Enable a Pulse Coupled Neural Network for Ultraviolet Image Segmentation.

Drawing inspiration from biology, neuromorphic systems are of great interest in direct interaction and efficient processing of analogue signals in the real world and could be promising for the development of smart sensors. Here, we demonstrate an artificial sensory neuron consisting of an InGaZnO4 (IGZO4)-based optical sensor and NbOx-based oscillation neuron in series, which can simultaneously sense the optical information even beyond the visible light region and encode them into electrical impulses. Such artificial vision sensory neurons can convey visual information in a parallel manner analogous to biological vision systems, and the output spikes can be effectively processed by a pulse coupled neural network, demonstrating the capability of image segmentation out of a complex background. This study could facilitate the construction of artificial visual systems and pave the way for the development of light-driven neurorobotics, bioinspired optoelectronics, and neuromorphic computing.

Pulsewidth Modulation-Based Algorithm for Spike Phase Encoding and Decoding of Time-Dependent Analog Data.

This article proposes a new spike encoding and decoding algorithm for analog data. The algorithm uses the pulsewidth modulation principles to achieve a high reconstruction accuracy of the signal, along with a high level of data compression. Two benchmark data sets are used to illustrate the method: stock index time series and human voice data. Applications of the method for spiking neural network (SNN) modeling and neuromorphic implementations are discussed. The proposed method would allow the development of new applications of SNNs as regression techniques for predictive time-series modeling.

Chronic defects in intraspinal mechanisms of spike encoding by spinal motoneurons following chemotherapy

Chemotherapy-induced sensorimotor disabilities, including gait and balance disorders, as well as physical fatigue often persist for months and sometimes years into disease free survival from cancer. While associated with impaired sensory function, chronic sensorimotor disorders might also depend on chemotherapy-induced defects in other neuron types. In this report, we extend consideration to motoneurons, which, if chronically impaired, would necessarily degrade movement behavior. The present study was undertaken to determine whether motoneurons qualify as candidate contributors to chronic sensorimotor disability independently from sensory impairment. We tested this possibility in vivo from rats 5 weeks following human-scaled treatment with one of the platinum-based compounds, oxaliplatin, widely used in chemotherapy for a variety of cancers. Action potential firing of spinal motoneurons responding to different fixed levels of electrode-current injection was measured in order to assess the neurons' intrinsic capacity for stimulus encoding. The encoding of stimulus duration and intensity corroborated in untreated control rats was severely degraded in oxaliplatin treated rats, in which motoneurons invariably exhibited erratic firing that was unsustained, unpredictable from one stimulus trial to the next, and unresponsive to changes in current strength. Direct measurements of interspike oscillations in membrane voltage combined with computer modeling pointed to aberrations in subthreshold conductances as a plausible contributor to impaired firing behavior. These findings authenticate impaired spike encoding as a candidate contributor to, in the case of motoneurons, deficits in mobility and fatigue. Aberrant firing also becomes a deficit worthy of testing in other CNS neurons as a potential contributor to perceptual and cognitive disorders induced by chemotherapy in patients.

Rank order coding based spiking convolutional neural network architecture with energy-efficient membrane voltage updates

Spiking neural network (SNN) system that uses rank order coding (ROC) as input spike encoding, generally suffers from low recognition accuracy and unnecessary computations that increase complexities. In this paper, we present a Spiking convolutional neural network (Spiking CNN) architecture that significantly improves recognition accuracy as well as computation efficiencies based on a novel ROC and modified kernel sizes. The proposed ROC generates spike trains based on maximum input value without sorting operations. In addition, as the recognition accuracy is affected by the reduced number of spikes as layers become deeper, the proposed ROC is inserted just before the final layer to increase the number of input spikes. The 2 × 2 pooling kernels are also replaced with 4 × 4 to reduce the network size. The hardware architecture of the proposed Spiking CNN has been implemented using 65 nm CMOS process. Neuron-centric membrane voltage update approach is also efficiently exploited in convolutional and fully connected layers to improve the hardware energy efficiencies. The Spiking CNN processor is seamlessly processing 2.85 K classifications per second with 6.79 uJ/classification. It also achieves 90.2% of recognition accuracy for MNIST dataset using unsupervised learning with STDP.

An end-to-end functional spiking model for sequential feature learning

Spiking Neural Network (SNN) has recently gained significant momentum in the neuromorphic low-power systems. However, the existing SNN models have limited use in time-sequential feature learning, and the exhausting spike encoding and decoding make the SNNs not straightforward to use. Inspired by the functional organization in the primate visual system, we propose an end-to-end functional spiking model in this paper to address these issues. Specifically, we propose the functional spike response to make each neuron special, and the dynamic synaptic efficiency to make the transmission of each input signal controllable. We represent inputs by a simple two-tuple set instead of conventional complex encoding, which achieves end-to-end learning. Experiments on synthetic datasets demonstrate that employing the two-tuple encoding strategy, our method improves the accuracy of the traditional SNN model significantly. In addition, we apply our method to seven real-world datasets and one human motion prediction dataset to investigate its performance. Experimental results show that the proposed functional spike response organization saves the running time of our model compared with the LSTM, GRU and one of the state-of-the-art time series processing methods.

Reconstruction of natural visual scenes from neural spikes with deep neural networks

Neural coding is one of the central questions in systems neuroscience for understanding how the brain processes stimulus from the environment, moreover, it is also a cornerstone for designing algorithms of brain–machine interface, where decoding incoming stimulus is highly demanded for better performance of physical devices. Traditionally researchers have focused on functional magnetic resonance imaging (fMRI) data as the neural signals of interest for decoding visual scenes. However, our visual perception operates in a fast time scale of millisecond in terms of an event termed neural spike. There are few studies of decoding by using spikes. Here we fulfill this aim by developing a novel decoding framework based on deep neural networks, named spike-image decoder (SID), for reconstructing natural visual scenes, including static images and dynamic videos, from experimentally recorded spikes of a population of retinal ganglion cells. The SID is an end-to-end decoder with one end as neural spikes and the other end as images, which can be trained directly such that visual scenes are reconstructed from spikes in a highly accurate fashion. Our SID also outperforms on the reconstruction of visual stimulus compared to existing fMRI decoding models. In addition, with the aid of a spike encoder, we show that SID can be generalized to arbitrary visual scenes by using the image datasets of MNIST, CIFAR10, and CIFAR100. Furthermore, with a pre-trained SID, one can decode any dynamic videos to achieve real-time encoding and decoding of visual scenes by spikes. Altogether, our results shed new light on neuromorphic computing for artificial visual systems, such as event-based visual cameras and visual neuroprostheses.

Selection and Optimization of Temporal Spike Encoding Methods for Spiking Neural Networks.

Spiking neural networks (SNNs) receive trains of spiking events as inputs. In order to design efficient SNN systems, real-valued signals must be optimally encoded into spike trains so that the task-relevant information is retained. This paper provides a systematic quantitative and qualitative analysis and guidelines for optimal temporal encoding. It proposes a methodology of a three-step encoding workflow: method selection by signal characteristics, parameter optimization by error metrics between original and reconstructed signals, and validation by comparison of the original signal and the encoded spike train. Four encoding methods are analyzed: one stimulus estimation [Ben's Spiker algorithm (BSA)] and three temporal contrast [threshold-based, step-forward (SW), and moving-window (MW)] encodings. A short theoretical analysis is provided, and the extended quantitative analysis is carried out applying four types of test signals: step-wise signal, smooth (sinusoid) signal with added noise, trended smooth signal, and event-like smooth signal. Various time-domain and frequency spectrum properties are explored, and a comparison is provided. BSA, the only method providing unipolar spikes, was shown to be ineffective for step-wise signals, but it can follow smoothly changing signals if filter coefficients are scaled appropriately. Producing bipolar (positive and negative) spike trains, SW encoding was most effective for all types of signals as it proved to be robust and easy to optimize. Signal-to-noise ratio (SNR) can be recommended as the error metric for parameter optimization. Currently, only a visual check is available for final validation.

Photonic spiking neural network based on excitable VCSELs-SA for sound azimuth detection.

We propose a photonic spiking neural network (SNN) based on excitable vertical-cavity surface-emitting lasers with an embedded saturable absorber (VCSELs-SA) for emulating the sound azimuth detection function of the brain for the first time. Here, the spike encoding and response properties based on the excitability of VCSELs-SA are employed, and the difference between spike timings of two postsynaptic neurons serves as an indication of sound azimuth. Furthermore, the weight matrix contributing to the successful sound azimuth detection is carefully identified, and the effect of the time interval between two presynaptic spikes is considered. It is found that the weight range that can achieve sound azimuth detection decreases gradually with the increase of the time interval between the sound arriving at the left and right ears. Besides, the effective detection range of the time interval between two presynaptic spikes is also identified, which is similar to that of the biological auditory system, but with a much higher resolution which is at the nanosecond time scale. We further discuss the effect of device variations on the photonic sound azimuth detection. Hence, this photonic SNN is biologically plausible, which has comparable low energy consumption and higher resolution compared with the biological system. This work is valuable for brain-inspired information processing and a promising foundation for more complex spiking information processing implemented by photonic neuromorphic computing systems.

A Low-Power Actor-Critic Framework Based on Memristive Spiking Neural Network

Traditional deep reinforcement learning (DRL) algorithms consume much energy. Energy-efficient spiking neural networks (SNNs) are promising technologies to bulid a low-power reinforcement learning architecture. In this paper, an actor-critic framework based on memrisitive SNN is proposed. To convey and process information in SNN, spike encoding and decoding systems are created. Then, an improved learning algorithm based on spike-timing-dependent plasticity (STDP) learning rule is designed to combine actor-critic method with SNN. Moreover, this learning algorithm is also hardware-friendly. Besides, memristive synapse is designed to accelerate this learning algorithm. Finally, a continuous control problem is applied to illustrate the effectiveness of the proposed framework. The results show the proposed framework is prior to traditional methods.

Polarization-resolved and polarization- multiplexed spike encoding properties in photonic neuron based on VCSEL-SA

The spike encoding properties of two polarization-resolved modes in vertical-cavity surface-emitting laser with an embedded saturable absorber (VCSEL-SA) are investigated numerically, based on the spin-flip model combined with the Yamada model. The results show that the external input optical pulse (EIOP) can be encoded into spikes in X-polarization (XP) mode, Y-polarization (YP) mode, or both XP and YP modes. Furthermore, the numerical bifurcation diagrams show that a lower (higher) strength of EIOP is beneficial for generating tonic (phasic) spikes; a small amplitude anisotropy contributes to wide (narrow) tonic spiking range in XP (YP) mode; a large current leads to low thresholds of EIOP strength for both XP and YP modes. However, the spike encoding properties are hardly affected by the phase anisotropy. The encoding rate is shown to be improved by increasing EIOP strength. Moreover, dual-channel polarization-multiplexed spike encoding can also be achieved in a single VCSEL-SA. To the best of our knowledge, such single channel polarization-resolved and dual-channel polarization-multiplexed spike encoding schemes have not yet been reported. Hence, this work is valuable for ultrafast photonic neuromorphic systems and brain-inspired information processing.

Spike encoding and storage properties in mutually coupled vertical-cavity surface-emitting lasers subject to optical pulse injection.

The generation and storage properties of different spike codes in vertical-cavity surface-emitting lasers with an embedded saturable absorber (VCSELs-SA) are investigated numerically. The results show that different spike codes are generated by injecting an optical pulse into one single VCSEL-SA and can be stored in two mutually coupled VCSELs-SA. In particular, in the case of the generation of spike codes, the effects of the input strength and the temporal duration of the input optical pulse are studied; in the case of the storage of spike codes, the roles of the coupling weight and the coupling delay between the two mutually coupled VCSELs-SA are examined. Simulations reveal that spikes can be triggered if the input strength and the temporal duration exceed the threshold values, and higher values of the input strength and the temporal duration are beneficial for generating more spikes. Moreover, successful storage of a perfectly formed train of spike codes in two mutually coupled VCSELs-SA can also be realized provided that the coupling weight and the coupling delay are larger than the corresponding threshold values.

Using Spike-Based Bio-Inspired Olfactory Model for Data Processing in Electronic Noses

Conventional electronic noses need complicated data preprocessing and tedious feature reduction steps for different sensors and applications. To overcome the drawbacks, a bio-inspired data processing method using a spike-based olfactory model is proposed in this paper, which consists of spike encoding by virtual olfactory receptor neurons (VORNs) and subsequent processing in a bionic olfactory bulb (BOB) model. Each VORN transduces the continuous sensor responses into spike time points, which are relayed to the BOB to enhance the operation efficiency. It is easy to extract useful features from BOB’s outputs due to their specific oscillation patterns, which simplifies the subsequent steps of feature generation. Three classification methods are used to identify seven Chinese liquors. The experimental results show that the proposed method achieves a better classification performance than the conventional methods.

Thermodynamic-RAM technology stack

We introduce a technology stack or specification describing the multiple levels of abstraction and specialisation needed to implement a neuromorphic processor (NPU) based on the previously-described concept of AHaH Computing and integrate it into today’s digital computing systems. The general purpose NPU implementation described here is called Thermodynamic-RAM (kT-RAM) and is just one of many possible architectures, each with varying advantages and trade offs. Bringing us closer to brain-like neural computation, kT-RAM will provide a general-purpose adaptive hardware resource to existing computing platforms enabling fast and low-power machine learning capabilities that are currently hampered by the separation of memory and processing, a.k.a the von Neumann bottleneck. Because understanding such a processor based on non-traditional principles can be difficult, by presenting the various levels of the stack from the bottom up, layer by layer, explaining kT-RAM becomes a much easier task. The levels of the Thermodynamic-RAM technology stack include the memristor, synapse, AHaH node, kT-RAM, instruction set, sparse spike encoding, kT-RAM emulator, and SENSE server.

Ischemia deteriorates spike encoding at cortical GABAergic neurons and cerebellar Purkinje cells by increasing the intracellular Ca2.

GABAergic neurons play a critical role in the central nervous system, such as well-organized behaviors. The ischemic cell death is presumably initiated by neuronal excitotoxicity resulted from the dysfunction of GABAergic neurons. It is not clear how ischemia influences different types of GABAergic neurons and whether intracellular Ca2+ plays a key role in the ischemic excitotoxicity. We have investigated this issue at cortical GABAergic neurons and cerebellar Purkinje cells by whole-cell recording in mouse brain slices, and the roles of intracellular Ca2+ are examined by BABTA infusion. Compare with the data from a group of control, ischemia causes by lowering purfusion rate lowers spike encoding at cortical GABAergic neurons and enhances encoding ability at cerebellar Purkinje cells. These differential effects of ischemia on spike encoding are mechanistically associated with the changes in the refractory periods and threshold potentials of sequential spikes. These ischemia-induced dysfunction of spike encoding at two types of GABAergic cells are prevented by BABTA infusion. Therefore, the ischemia destabilizes the spike encoding of GABAergic cells via raising intracellular Ca2+. Our findings indicate that ischemia preferentially causes the dysfunction of spike encoding at GABAergic neurons by the up-regulation of intracellular Ca2+ level, which leads to neuronal excitotoxicity.

Hypocretin/Orexin Peptides Alter Spike Encoding by Serotonergic Dorsal Raphe Neurons through Two Distinct Mechanisms That Increase the Late Afterhyperpolarization

Orexins (hypocretins) are neuropeptides that regulate multiple homeostatic processes, including reward and arousal, in part by exciting serotonergic dorsal raphe neurons, the major source of forebrain serotonin. Here, using mouse brain slices, we found that, instead of simply depolarizing these neurons, orexin-A altered the spike encoding process by increasing the postspike afterhyperpolarization (AHP) via two distinct mechanisms. This orexin-enhanced AHP (oeAHP) was mediated by both OX1 and OX2 receptors, required Ca(2+) influx, reversed near EK, and decayed with two components, the faster of which resulted from enhanced SK channel activation, whereas the slower component decayed like a slow AHP (sAHP), but was not blocked by UCL2077, an antagonist of sAHPs in some neurons. Intracellular phospholipase C inhibition (U73122) blocked the entire oeAHP, but neither component was sensitive to PKC inhibition or altered PKA signaling, unlike classical sAHPs. The enhanced SK current did not depend on IP3-mediated Ca(2+) release but resulted from A-current inhibition and the resultant spike broadening, which increased Ca(2+) influx and Ca(2+)-induced-Ca(2+) release, whereas the slower component was insensitive to these factors. Functionally, the oeAHP slowed and stabilized orexin-induced firing compared with firing produced by a virtual orexin conductance lacking the oeAHP. The oeAHP also reduced steady-state firing rate and firing fidelity in response to stimulation, without affecting the initial rate or fidelity. Collectively, these findings reveal a new orexin action in serotonergic raphe neurons and suggest that, when orexin is released during arousal and reward, it enhances the spike encoding of phasic over tonic inputs, such as those related to sensory, motor, and reward events. Orexin peptides are known to excite neurons via slow postsynaptic depolarizations. Here we elucidate a significant new orexin action that increases and prolongs the postspike afterhyperpolarization (AHP) in 5-HT dorsal raphe neurons and other arousal-system neurons. Our mechanistic studies establish involvement of two distinct Ca(2+)-dependent AHP currents dependent on phospholipase C signaling but independent of IP3 or PKC. Our functional studies establish that this action preserves responsiveness to phasic inputs while attenuating responsiveness to tonic inputs. Thus, our findings bring new insight into the actions of an important neuropeptide and indicate that, in addition to producing excitation, orexins can tune postsynaptic excitability to better encode the phasic sensory, motor, and reward signals expected during aroused states.

Real-time encoding and compression of neuronal spikes by metal-oxide memristors.

Advanced brain-chip interfaces with numerous recording sites bear great potential for investigation of neuroprosthetic applications. The bottleneck towards achieving an efficient bio-electronic link is the real-time processing of neuronal signals, which imposes excessive requirements on bandwidth, energy and computation capacity. Here we present a unique concept where the intrinsic properties of memristive devices are exploited to compress information on neural spikes in real-time. We demonstrate that the inherent voltage thresholds of metal-oxide memristors can be used for discriminating recorded spiking events from background activity and without resorting to computationally heavy off-line processing. We prove that information on spike amplitude and frequency can be transduced and stored in single devices as non-volatile resistive state transitions. Finally, we show that a memristive device array allows for efficient data compression of signals recorded by a multi-electrode array, demonstrating the technology's potential for building scalable, yet energy-efficient on-node processors for brain-chip interfaces.