Neuromorphic Hardware Research Articles

Co-learning synaptic delays, weights and adaptation in spiking neural networks.

Spiking neural network (SNN) distinguish themselves from artificial neural network (ANN) because of their inherent temporal processing and spike-based computations, enabling a power-efficient implementation in neuromorphic hardware. In this study, we demonstrate that data processing with spiking neurons can be enhanced by co-learning the synaptic weights with two other biologically inspired neuronal features: (1) a set of parameters describing neuronal adaptation processes and (2) synaptic propagation delays. The former allows a spiking neuron to learn how to specifically react to incoming spikes based on its past. The trained adaptation parameters result in neuronal heterogeneity, which leads to a greater variety in available spike patterns and is also found in the brain. The latter enables to learn to explicitly correlate spike trains that are temporally distanced. Synaptic delays reflect the time an action potential requires to travel from one neuron to another. We show that each of the co-learned features separately leads to an improvement over the baseline SNN and that the combination of both leads to state-of-the-art SNN results on all speech recognition datasets investigated with a simple 2-hidden layer feed-forward network. Our SNN outperforms the benchmark ANN on the neuromorphic datasets (Spiking Heidelberg Digits and Spiking Speech Commands), even with fewer trainable parameters. On the 35-class Google Speech Commands dataset, our SNN also outperforms a GRU of similar size. Our study presents brain-inspired improvements in SNN that enable them to excel over an equivalent ANN of similar size on tasks with rich temporal dynamics.

Fast learning without synaptic plasticity in spiking neural networks

Spiking neural networks are of high current interest, both from the perspective of modelling neural networks of the brain and for porting their fast learning capability and energy efficiency into neuromorphic hardware. But so far we have not been able to reproduce fast learning capabilities of the brain in spiking neural networks. Biological data suggest that a synergy of synaptic plasticity on a slow time scale with network dynamics on a faster time scale is responsible for fast learning capabilities of the brain. We show here that a suitable orchestration of this synergy between synaptic plasticity and network dynamics does in fact reproduce fast learning capabilities of generic recurrent networks of spiking neurons. This points to the important role of recurrent connections in spiking networks, since these are necessary for enabling salient network dynamics. We show more specifically that the proposed synergy enables synaptic weights to encode more general information such as priors and task structures, since moment-to-moment processing of new information can be delegated to the network dynamics.

Brain-inspired spiking neural networks in Engineering Mechanics: a new physics-based self-learning framework for sustainable Finite Element analysis

AbstractThe present study aims to develop a sustainable framework employing brain-inspired neural networks for solving boundary value problems in Engineering Mechanics. Spiking neural networks, known as the third generation of artificial neural networks, are proposed for physics-based artificial intelligence. Accompanied by a new pseudo-explicit integration scheme based on spiking recurrent neural networks leading to a spike-based pseudo explicit integration scheme, the underlying differential equations are solved with a physics-informed strategy. We propose additionally a third-generation spike-based Legendre Memory Unit that handles large sequences. These third-generation networks can be implemented on the coming-of-age neuromorphic hardware resulting in less energy and memory consumption. The proposed framework, although implicit, is viewed as a pseudo-explicit scheme since it requires almost no or fewer online training steps to achieve a converged solution even for unseen loading sequences. The proposed framework is deployed in a Finite Element solver for plate structures undergoing cyclic loading and a Xylo-Av2 SynSense neuromorphic chip is used to assess its energy performance. An acceleration of more than 40% when compared to classical Finite Element Method simulations and the capability of online training is observed. We also see a reduction in energy consumption down to the thousandth order.

Hardware in Loop Learning with Spin Stochastic Neurons

Despite the promise of superior efficiency and scalability, real‐world deployment of emerging nanoelectronic platforms for brain‐inspired computing have been limited thus far, primarily because of inter‐device variations and intrinsic non‐idealities. In this work, mitigation of these issues is demonstrated by performing learning directly on practical devices through a hardware‐in‐loop approach, utilizing stochastic neurons based on heavy metal/ferromagnetic spin–orbit torque heterostructures. The probabilistic switching and device‐to‐device variability of the fabricated devices of various sizes is characterized to showcase the effect of device dimension on the neuronal dynamics and its consequent impact on network‐level performance. The efficacy of the hardware‐in‐loop scheme is illustrated in a deep learning scenario achieving equivalent software performance. This work paves the way for future large‐scale implementations of neuromorphic hardware and realization of truly autonomous edge‐intelligent devices.

Vertically Integrated Monolithic Neuromorphic Nanowire Device for Physiological Information Processing.

This study demonstrates the conceptual design and fabrication of a vertically integrated monolithic (VIM) neuromorphic device. The device comprises an n-type SnO2 nanowire bottom channel connected by a shared gate to a p-type P3HT nanowire top channel. This architecture establishes two distinct neural pathways with different response behaviors. The device generates excitatory and inhibitory postsynaptic currents, mimicking the corelease mechanism of bilingual synapses. To enhance the signal processing efficiency, we employed a bipolar spike encoding strategy to convert fluctuating sensory signals to spike trains containing positive and negative pulses. Utilizing the neuromorphic platform for synaptic processing, physiological signals featuring bidirectional fluctuations, including electrocardiogram and breathing signals, can be classified with an accuracy of over 90%. The VIM device holds considerable promise as a solution for developing highly integrated neuromorphic hardware for healthcare and edge intelligence applications.

1-Selector 1-Memristor Configuration with Multifunctional a-IGZO Memristive Devices Fabricated at Room Temperature.

Serving as neuromorphic hardware accelerators, memristors play a crucial role in large-scale neuromorphic computing. Herein, two-terminal memristors utilizing amorphous indium-gallium-zinc oxide (a-IGZO) are fabricated through room-temperature sputtering. The electrical characteristics of these memristors are effectively modulated by varying the oxygen flow during the deposition process. The optimized a-IGZO memristor, fabricated under 3 sccm oxygen flow, presents a 5 × 103 ratio between its high- and low-resistance states, which can be maintained over 1 × 104 s with minimal degradation. Meanwhile, desirable properties such as electroforming-free and self-compliance, crucial for low-energy consumption, are also obtained in the a-IGZO memristor. Moreover, analog conductance switching is observed, demonstrating an interface-type behavior, as evidenced by its device-size-dependent performance. The coexistence of negative differential resistance with analog switching is attributed to the migration of oxygen vacancies and the trapping/detrapping of charges. Furthermore, the device demonstrates optical storage capabilities by exploiting the optical properties of a-IGZO, which can stably operate for up to 50 sweep cycles. Various synaptic functions have been demonstrated, including paired-pulse facilitation and spike-timing-dependent plasticity. These functionalities contribute to a simulated recognition accuracy of 90% for handwritten digits. Importantly, a one-selector one-memristor (1S1M) architecture is successfully constructed at room temperature by integrating a-IGZO memristor on a TaOx-based selector. This architecture exhibits a 107 on/off ratio, demonstrating its potential to suppress sneak currents among adjacent units in a memristor crossbar.

MoOx Synaptic Memristor with Programmable Multilevel Conductance for Reliable Neuromorphic Hardware.

Memristor holds great potential for enabling next-generation neuromorphic computing hardware. Controlling the interfacial characteristics of the device is critical for seamlessly integrating and replicating the synaptic dynamic behaviors; however, it is commonly overlooked. Herein, we report the straightforward oxidation of a Mo electrode in air to design MoOx memristors that exhibit nonvolatile ultrafast switching (0.6-0.8 mV/decade, <1 mV/decade) with a high on/off ratio (>104), a long durability (>104 s), a low power consumption (17.9 μW), excellent device-to-device uniformity, ingeniously synaptic behavior, and finely programmable multilevel analog switching. The analyzed physical mechanism of the observed resistive switching behavior might be the conductive filaments formed by the oxygen vacancies. Intriguingly, upon organization into memristor-based crossbar arrays, in addition to simulated multipattern memorization, edge detection on random images can be implemented well by parallel processing of pixels using a 3 × 3 × 2 array of Prewitt filter groups. These are vital functions for neural system hardware in efficient in-memory computing neural systems with massive parallelism beyond a von Neumann architecture.

Closing the loop: High-speed robotics with accelerated neuromorphic hardware.

The BrainScaleS-2 system is an established analog neuromorphic platform with versatile applications in the diverse fields of computational neuroscience and spike-based machine learning. In this work, we extend the system with a configurable realtime event interface that enables a tight coupling of its distinct analog network core to external sensors and actuators. The 1,000-fold acceleration of the emulated nerve cells allows us to target high-speed robotic applications that require precise timing on a microsecond scale. As a showcase, we present a closed-loop setup for commuting brushless DC motors: we utilize PyTorch to train a spiking neural network emulated on the analog substrate to control an electric motor from a sensory event stream. The presented system enables research in the area of event-driven controllers for high-speed robotics, including self-supervised and biologically inspired online learning for such applications.

Memory-Efficient Reversible Spiking Neural Networks

Spiking neural networks (SNNs) are potential competitors to artificial neural networks (ANNs) due to their high energy-efficiency on neuromorphic hardware. However, SNNs are unfolded over simulation time steps during the training process. Thus, SNNs require much more memory than ANNs, which impedes the training of deeper SNN models. In this paper, we propose the reversible spiking neural network to reduce the memory cost of intermediate activations and membrane potentials during training. Firstly, we extend the reversible architecture along temporal dimension and propose the reversible spiking block, which can reconstruct the computational graph and recompute all intermediate variables in forward pass with a reverse process. On this basis, we adopt the state-of-the-art SNN models to the reversible variants, namely reversible spiking ResNet (RevSResNet) and reversible spiking transformer (RevSFormer). Through experiments on static and neuromorphic datasets, we demonstrate that the memory cost per image of our reversible SNNs does not increase with the network depth. On CIFAR10 and CIFAR100 datasets, our RevSResNet37 and RevSFormer-4-384 achieve comparable accuracies and consume 3.79x and 3.00x lower GPU memory per image than their counterparts with roughly identical model complexity and parameters. We believe that this work can unleash the memory constraints in SNN training and pave the way for training extremely large and deep SNNs.

Enhancing the Robustness of Spiking Neural Networks with Stochastic Gating Mechanisms

Spiking neural networks (SNNs) exploit neural spikes to provide solutions for low-power intelligent applications on neuromorphic hardware. Although SNNs have high computational efficiency due to spiking communication, they still lack resistance to adversarial attacks and noise perturbations. In the brain, neuronal responses generally possess stochasticity induced by ion channels and synapses, while the role of stochasticity in computing tasks is poorly understood. Inspired by this, we elaborate a stochastic gating spiking neural model for layer-by-layer spike communication, introducing stochasticity to SNNs. Through theoretical analysis, our gating model can be viewed as a regularizer that prevents error amplification under attacks. Meanwhile, our work can explain the robustness of Poisson coding. Experimental results prove that our method can be used alone or with existing robust enhancement algorithms to improve SNN robustness and reduce SNN energy consumption. We hope our work will shed new light on the role of stochasticity in the computation of SNNs. Our code is available at https://github.com/DingJianhao/StoG-meets-SNN/.

Enhancing Training of Spiking Neural Network with Stochastic Latency

Spiking neural networks (SNNs) have garnered significant attention for their low power consumption when deployed on neuromorphic hardware that operates in orders of magnitude lower power than general-purpose hardware. Direct training methods for SNNs come with an inherent latency for which the SNNs are optimized, and in general, the higher the latency, the better the predictive powers of the models, but at the same time, the higher the energy consumption during training and inference. Furthermore, an SNN model optimized for one particular latency does not necessarily perform well in lower latencies, which becomes relevant in scenarios where it is necessary to switch to a lower latency because of the depletion of onboard energy or other operational requirements. In this work, we propose Stochastic Latency Training (SLT), a direct training method for SNNs that optimizes the model for the given latency but simultaneously offers a minimum reduction of predictive accuracy when shifted to lower inference latencies. We provide heuristics for our approach with partial theoretical justification and experimental evidence showing the state-of-the-art performance of our models on datasets such as CIFAR-10, DVS-CIFAR-10, CIFAR-100, and DVS-Gesture. Our code is available at https://github.com/srinuvaasu/SLT

Gated Attention Coding for Training High-Performance and Efficient Spiking Neural Networks

Spiking neural networks (SNNs) are emerging as an energy-efficient alternative to traditional artificial neural networks (ANNs) due to their unique spike-based event-driven nature. Coding is crucial in SNNs as it converts external input stimuli into spatio-temporal feature sequences. However, most existing deep SNNs rely on direct coding that generates powerless spike representation and lacks the temporal dynamics inherent in human vision. Hence, we introduce Gated Attention Coding (GAC), a plug-and-play module that leverages the multi-dimensional gated attention unit to efficiently encode inputs into powerful representations before feeding them into the SNN architecture. GAC functions as a preprocessing layer that does not disrupt the spike-driven nature of the SNN, making it amenable to efficient neuromorphic hardware implementation with minimal modifications. Through an observer model theoretical analysis, we demonstrate GAC's attention mechanism improves temporal dynamics and coding efficiency. Experiments on CIFAR10/100 and ImageNet datasets demonstrate that GAC achieves state-of-the-art accuracy with remarkable efficiency. Notably, we improve top-1 accuracy by 3.10% on CIFAR100 with only 6-time steps and 1.07% on ImageNet while reducing energy usage to 66.9% of the previous works. To our best knowledge, it is the first time to explore the attention-based dynamic coding scheme in deep SNNs, with exceptional effectiveness and efficiency on large-scale datasets. Code is available at https://github.com/bollossom/GAC.

Study on the Sodium-Doped Titania Interface-Type Memristor.

Memristors integrated into a crossbar-array architecture (CAA) are promising candidates for analog in-memory computing accelerators. However, the relatively low reliability of the memristor device and sneak current issues in CAA remain the main obstacles. Alkali ion-based interface-type memristors are promising solutions for implementing highly reliable memristor devices and neuromorphic hardware. This interface-type device benefits from self-rectifying and forming-free resistive switching (RS), and exhibits relatively low variation from device to device and cycle to cycle. In a previous report, we introduced an in situ grown Na/TiO2 memristor using atomic layer deposition (ALD) and proposed a RS mechanism from experimentally measured Schottky barrier modulation. Self-rectifying RS characteristics were observed by the asymmetric distribution of Na dopants and oxygen vacancies as the Ti metal used as the adhesion layer for the bottom electrode diffuses over the Pt electrode at 250 °C during the ALD process and is doped into the TiO2 layer. Here, we theoretically verify the modulation of the Schottky barrier at the TiO2/Pt electrode interface by Na ions. This study fabricated a Pt/Na/TiO2/Pt memristor device and confirmed its self-rectifying RS characteristics and stable retention characteristics for 24 h at 85 °C. Additionally, this device exhibited relative standard deviations of 27 and 7% in the high and low resistance states, respectively, in terms of cycle-to-cycle variation. To verify the RS mechanism, we conducted density functional theory simulations to analyze the impact of Na cations at interstitial sites on the Schottky barrier. Our findings can contribute to both fundamental understanding and the design of high-performance memristor devices for neuromorphic computing.

In situ edge extraction enabled by reconfigurable van der Waals infrared photodetectors

At present, the widely used artificial intelligence image perception technology is composed of discrete detection and processing components, which is mismatching the demand for increasingly sizeable redundant image information processing. The transmission of non-critical information between components limits the efficiency of image perception systems. Inspired by the human vision system, which can extract the key features and reduce the transmission burden at the low-level detection end, we propose a symmetric structured mixed-dimensional n/p/n [n-molybdenum disulfide (MoS2)/p-germanium (Ge)/n-MoS2] interdigital van der Waals heterojunction infrared photodetector that can be tuned by bias voltage in response polarity and magnitude. The MoS2/Ge/MoS2 infrared photodetector has a bias-symmetric optoelectronic response and covers the detection band from visible to short-wave infrared. Through the joint detection of a pair of MoS2/Ge/MoS2 infrared photodetectors, we demonstrate the image in situ edge extraction at the detection end, which provides key features for high-level processing. This work has discovered the potential of retina-inspired infrared photodetectors on a 2D/3D integration platform, providing distinct opportunities for a neuromorphic visual perception hardware.

Vertically Stackable Ovonic Threshold Switch Oscillator Using Atomic Layer Deposited Ge0.6Se0.4 Film for High-Density Artificial Neural Networks.

Nanodevice oscillators (nano-oscillators) have received considerable attention to implement in neuromorphic computing as hardware because they can significantly improve the device integration density and energy efficiency compared to complementary metal oxide semiconductor circuit-based oscillators. This work demonstrates vertically stackable nano-oscillators using an ovonic threshold switch (OTS) for high-density neuromorphic hardware. A vertically stackable Ge0.6Se0.4 OTS-oscillator (VOTS-OSC) is fabricated with a vertical crossbar array structure by growing Ge0.6Se0.4 film conformally on a contact hole structure using atomic layer deposition. The VOTS-OSC can be vertically integrated onto peripheral circuits without causing thermal damage because the fabrication temperature is <400 °C. The fabricated device exhibits oscillation characteristics, which can serve as leaky integrate-and-fire neurons in spiking neural networks (SNNs) and coupled oscillators in oscillatory neural networks (ONNs). For practical applications, pattern recognition and vertex coloring are demonstrated with SNNs and ONNs, respectively, using semiempirical simulations. This structure increases the oscillator integration density significantly, enabling complex tasks with a large number of oscillators. Moreover, it can enhance the computational speed of neural networks due to its rapid switching speed.

Local prediction-learning in high-dimensional spaces enables neural networks to plan

Planning and problem solving are cornerstones of higher brain function. But we do not know how the brain does that. We show that learning of a suitable cognitive map of the problem space suffices. Furthermore, this can be reduced to learning to predict the next observation through local synaptic plasticity. Importantly, the resulting cognitive map encodes relations between actions and observations, and its emergent high-dimensional geometry provides a sense of direction for reaching distant goals. This quasi-Euclidean sense of direction provides a simple heuristic for online planning that works almost as well as the best offline planning algorithms from AI. If the problem space is a physical space, this method automatically extracts structural regularities from the sequence of observations that it receives so that it can generalize to unseen parts. This speeds up learning of navigation in 2D mazes and the locomotion with complex actuator systems, such as legged bodies. The cognitive map learner that we propose does not require a teacher, similar to self-attention networks (Transformers). But in contrast to Transformers, it does not require backpropagation of errors or very large datasets for learning. Hence it provides a blue-print for future energy-efficient neuromorphic hardware that acquires advanced cognitive capabilities through autonomous on-chip learning.

A universal ANN-to-SNN framework for achieving high accuracy and low latency deep Spiking Neural Networks

Spiking Neural Networks (SNNs) have become one of the most prominent next-generation computational models owing to their biological plausibility, low power consumption, and the potential for neuromorphic hardware implementation. Among the various methods for obtaining available SNNs, converting Artificial Neural Networks (ANNs) into SNNs is the most cost-effective approach. The early challenges in ANN-to-SNN conversion work revolved around the susceptibility of converted SNNs to conversion errors. Some recent endeavors have attempted to mitigate these conversion errors by altering the original ANNs. Despite their ability to enhance the accuracy of SNNs, these methods lack generality and cannot be directly applied to convert the majority of existing ANNs. In this paper, we present a framework named DNISNM for converting ANN to SNN, with the aim of addressing conversion errors arising from differences in the discreteness and asynchrony of network transmission between ANN and SNN. The DNISNM consists of two mechanisms, Data-based Neuronal Initialization (DNI) and Signed Neuron with Memory (SNM), designed to respectively address errors stemming from discreteness and asynchrony disparities. This framework requires no additional modifications to the original ANN and can result in SNNs with improved accuracy performance, simultaneously ensuring universality, high precision, and low inference latency. We verify it experimentally on challenging object recognition datasets, including CIFAR10, CIFAR100, and ImageNet-1k. Experimental results show that the SNN converted by our framework has very high accuracy even at extremely low latency.

Shift-Equivariant Similarity-Preserving Hypervector Representations of Sequences

Hyperdimensional Computing (HDC), also known as Vector-Symbolic Architectures (VSA), is a promising framework for the development of cognitive architectures and artificial intelligence systems, as well as for technical applications and emerging neuromorphic and nanoscale hardware. HDC/VSA operate with hypervectors, i.e., neural-like distributed vector representations of large fixed dimension (usually > 1000). One of the key ingredients of HDC/VSA are the methods for encoding various data types (from numeric scalars and vectors to graphs) by hypervectors. In this paper, we propose an approach for the formation of hypervectors of sequences that provides both an equivariance with respect to the shift of sequences and preserves the similarity of sequences with identical elements at nearby positions. Our methods represent the sequence elements by compositional hypervectors and exploit permutations of hypervectors for representing the order of sequence elements. We experimentally explored the proposed representations using a diverse set of tasks with data in the form of symbolic strings. Although we did not use any features here (hypervector of a sequence was formed just from the hypervectors of its symbols at their positions), the proposed approach demonstrated the performance on a par with the methods that exploit various features, such as subsequences. The proposed techniques were designed for the HDC/VSA model known as Sparse Binary Distributed Representations. However, they can be adapted to hypervectors in formats of other HDC/VSA models, as well as for representing sequences of types other than symbolic strings. Directions for further research are discussed.

VCSEL-based photonic spiking neural networks for ultrafast detection and tracking

Inspired by efficient biological spike-based neural networks, we demonstrate for the first time the detection and tracking of target patterns in image and video inputs at high-speed rates with networks of multiple artificial spiking optical neurons. Using photonic systems of in-parallel spiking vertical cavity surface emitting lasers (VCSELs), we demonstrate the implementation of multiple convolutional kernel operators which, in combination with optical spike signalling, enable the detection and tracking of target features in images/video feeds at an ultrafast photonic operation speed of 1 ns per pixel. Alongside a single layer optical spiking neural network (SNN) demonstration, a multi-layer network of photonic (GHz-rate) spike-firing neurons is reported where the photonic system successfully tracks a large complex feature (Handwritten Digit 3). The consecutive photonic layers perform spike-enabled image reduction and convolution operations, and interact with a software-implemented SNN, that learns the feature patterns that best identify the target to provide a high detection efficiency even in the presence of a distractor feature. This work therefore highlights the effectiveness of combining neuromorphic photonic hardware and software SNNs, for efficient learning and ultrafast operation, thanks to the use of spiking light signals, towards tackling complex AI and computer vision problems.

Neuromorphic hardware for real-time intraoperative HFO detection

Neuromorphic hardware for real-time intraoperative HFO detection