Neuromorphic Hardware Research Articles

Bimodal alteration of cognitive accuracy for spintronic artificial neural networks.

Spintronics-based artificial neural networks (ANNs) exhibiting nonvolatile, fast, and energy-efficient computing capabilities are promising neuromorphic hardware for performing complex cognitive tasks of artificial intelligence and machine learning. Early experimental efforts focused on multistate device concepts to enhance synaptic weight precisions, albeit compromising on cognitive accuracy due to their low magnetoresistance. Here, we propose a hybrid approach based on the tuning of tunnel magnetoresistance (TMR) and the number of states in the compound magnetic tunnel junctions (MTJs) to improve the cognitive performance of an all-spin ANN. A TMR variation of 33-78% is controlled by the free layer (FL) thickness wedge (1.6-2.6 nm) across the wafer. Meanwhile, the number of resistance states in the compound MTJ is manipulated by varying the number of constituent MTJ cells (n = 1-3), generating n + 1 states with a TMR difference between consecutive states of at least 21%. Using MNIST handwritten digit and fashion object databases, the test accuracy of the compound MTJ ANN is observed to increase with the number of intermediate states for a fixed FL thickness or TMR. Meanwhile, the test accuracy for a 1-cell MTJ increases linearly by 8.3% and 7.4% for handwritten digits and fashion objects, respectively, with increasing TMR. Interestingly, a multifarious TMR dependence of test accuracy is observed with the increasing synaptic complexity in the 2- and 3-cell MTJs. By leveraging on the bimodal tuning of multilevel and TMR, we establish viable paths for enhancing the cognitive performance of spintronic ANN for in-memory and neuromorphic computing.

Ferroelectric Tunnel Junction Memristors for In‐Memory Computing Accelerators

Neuromorphic computing has seen great interest as leaps in artificial intelligence (AI) applications have exposed limitations due to heavy memory access, with the von Neumann computing architecture. The parallel in‐memory computing provided by neuromorphic computing has the potential to significantly improve latency and power consumption. Key to analog neuromorphic computing hardware are memristors, providing non‐volatile multistate conductance levels, high switching speed, and energy efficiency. Ferroelectric tunnel junction (FTJ) memristors are prime candidates for this purpose, but the impact of the particular characteristics for their performance upon integration into large crossbar arrays, the core compute element for both inference and training in deep neural networks, requires close investigation. In this work, a W/Hf x Zr1−x O2/TiN FTJ with 60 programmable conductance states, a dynamic range (DR) up to 10, current density >3 A m−2 at V read = 0.3 V and highly nonlinear current–voltage (I–V) characteristics (>1100) is experimentally demonstrated. Using a circuit macro‐model, the system level performance of a true crossbar array is evaluated and a 92% classification accuracy of the modified nation institute of science and technology (MNIST) dataset is achieved. Finally, the low on conductance in combination with the highly nonlinear I–V characteristics enable the realization of large selector‐free crossbar arrays for neuromorphic hardware accelerators.

Versatile Neuromorphic Modulation and Biosensing based on N‐type Small‐molecule Organic Mixed Ionic‐Electronic Conductors

AbstractThe ion/chemical‐based modulation feature of organic mixed ionic‐electronic conductors (OMIECs) are critical to advancing next generation bio‐integrated neuromorphic hardware. Despite achievements with polymeric OMIECs in organic electrochemical neuronal synapse (OENS). However, small molecule OMIECs based OENS has not yet been realized. Here, for the first time, we demonstrate an effective materials design concept of combining n‐type fused all‐acceptor small molecule OMIECs with subtle side chain optimization that enables robustly and flexibly modulating versatile synaptic behavior and sensing neurotransmitter in solid or aqueous electrolyte, operating in accumulation modes. By judicious tuning the ending side chains, the linear oligoether and butyl chain derivative gNR‐Bu exhibits higher recognition accuracy for a model artificial neural network (ANN) simulation, higher steady conductance states and more outstanding ambient stability, which is superior to the state‐of‐art n‐type OMIECs based OENS. These superior artificial synapse characteristics of gNR‐Bu can be attributed to its higher crystallinity with stronger ion bonding capacities. More impressively, we unprecedentedly realized n‐type small‐molecule OMIECs based OENS as a neuromorphic biosensor enabling to respond synaptic communication signals of dopamine even at sub‐μM level in aqueous electrolyte. This work may open a new path of small‐molecule ion‐electron conductors for next‐generation ANN and bioelectronics.

Versatile Neuromorphic Modulation and Biosensing based on N-type Small-molecule Organic Mixed Ionic-Electronic Conductors.

The ion/chemical-based modulation feature of organic mixed ionic-electronic conductors (OMIECs) are critical to advancing next generation bio-integrated neuromorphic hardware. Despite achievements with polymeric OMIECs in organic electrochemical neuronal synapse (OENS). However, small molecule OMIECs based OENS has not yet been realized. Here, for the first time, we demonstrate an effective materials design concept of combining n-type fused all-acceptor small molecule OMIECs with subtle side chain optimization that enables robustly and flexibly modulating versatile synaptic behavior and sensing neurotransmitter in solid or aqueous electrolyte, operating in accumulation modes. By judicious tuning the ending side chains, the linear oligoether and butyl chain derivative gNR-Bu exhibits higher recognition accuracy for a model artificial neural network (ANN) simulation, higher steady conductance states and more outstanding ambient stability, which is superior to the state-of-art n-type OMIECs based OENS. These superior artificial synapse characteristics of gNR-Bu can be attributed to its higher crystallinity with stronger ion bonding capacities. More impressively, we unprecedentedly realized n-type small-molecule OMIECs based OENS as a neuromorphic biosensor enabling to respond synaptic communication signals of dopamine even at sub-μM level in aqueous electrolyte. This work may open a new path of small-molecule ion-electron conductors for next-generation ANN and bioelectronics.

Biomimetic oculomotor control with spiking neural networks

Spiking neural networks (SNNs) are comprised of artificial neurons that, like their biological counterparts, communicate via electrical spikes. SNNs have been hailed as the next wave of deep learning as they promise low latency and low-power consumption when run on neuromorphic hardware. Current deep neural network models for computer vision often require power-hungry GPUs to train and run, making them great candidates to replace with SNNs. We develop and train a biomimetic, SNN-driven, neuromuscular oculomotor controller for a realistic biomechanical model of the human eye. Inspired by the ON and OFF bipolar cells of the retina, we use event-based data flow in the SNN to direct the necessary extraocular muscle-driven eye movements. We train our SNN models from scratch, using modified deep learning techniques. Classification tasks are straightforward to implement with SNNs and have received the most research attention, but visual tracking is a regression task. We use surrogate gradients and introduce a linear layer to convert membrane voltages from the final spiking layer into the desired outputs. Our SNN foveation network enhances the biomimetic properties of the virtual eye model and enables it to perform reliable visual tracking. Overall, with event-based data processed by an SNN, our oculomotor controller successfully tracks a visual target while activating 87.3% fewer neurons than a conventional neural network.

In operando cryo-STEM of pulse-induced charge density wave switching in TaS2

The charge density wave material 1T-TaS2 exhibits a pulse-induced insulator-to-metal transition, which shows promise for next-generation electronics such as memristive memory and neuromorphic hardware. However, the rational design of TaS2 devices is hindered by a poor understanding of the switching mechanism, the pulse-induced phase, and the influence of material defects. Here, we operate a 2-terminal TaS2 device within a scanning transmission electron microscope at cryogenic temperature, and directly visualize the changing charge density wave structure with nanoscale spatial resolution and down to 300 μs temporal resolution. We show that the pulse-induced transition is driven by Joule heating, and that the pulse-induced state corresponds to the nearly commensurate and incommensurate charge density wave phases, depending on the applied voltage amplitude. With our in operando cryogenic electron microscopy experiments, we directly correlate the charge density wave structure with the device resistance, and show that dislocations significantly impact device performance. This work resolves fundamental questions of resistive switching in TaS2 devices, critical for engineering reliable and scalable TaS2 electronics.

A Design Flow for Scheduling Spiking Deep Convolutional Neural Networks on Heterogeneous Neuromorphic System-on-Chip

Neuromorphic systems-on-chip (NSoCs) integrate CPU cores and neuromorphic hardware accelerators on the same chip. These platforms can execute spiking deep convolutional neural networks (SDCNNs) with a low energy footprint. Modern NSoCs are heterogeneous in terms of their computing, communication, and storage resources. This makes scheduling SDCNN operations a combinatorial problem of exploring an exponentially-large state space in determining mapping, ordering, and timing of operations to achieve a target hardware performance, e.g., throughput. We propose a systematic design flow to schedule SDCNNs on an NSoC. Our scheduler, called SMART ( S DCNN MA pping, Orde R ing, and T iming), branches the combinatorial optimization problem into computationally-relaxed sub-problems that generate fast solutions without significantly compromising the solution quality. SMART improves performance by efficiently incorporating the heterogeneity in computing, communication, and storage resources. SMART operates in four steps. First, it creates a self-timed execution schedule to map operations to compute resources, maximizing throughput. Second, it uses an optimization strategy to distribute activation and synaptic weights to storage resources, minimizing data communication-related overhead. Third, it constructs an inter-processor communication (IPC) graph with a transaction order for its communication actors. This transaction order is created using a transaction partial order algorithm, which minimizes contention on the shared communication resources. Finally, it schedules this IPC graph to hardware by overlapping communication with the computation, and leveraging operation, pipeline, and batch parallelism. We evaluate SMART using 10 representative image, object, and language-based SDCNNs. Results show that SMART increases throughput by an average 23%, compared to a state-of-the-art scheduler. SMART is implemented entirely in software as a compiler extension. It doesn’t require any change in a neuromorphic hardware or its interface to CPUs. It improves throughput with only a marginal increase in the compilation time. SMART is released under the open-source MIT licensing at https://github.com/drexel-DISCO/SMART to foster future research.

How can neuromorphic hardware attain brain-like functional capabilities?

The author provides 4 design principles of how to make cortical microcircuits into neuromorphic hardwares, shedding light for the next generation neuromorphic hardware design.

2D Materials-based Neuromorphic Computing Electronic Device

Nowadays, with the rapid information explosion connected to all devices, there is a huge demand for effectively processing big data. In particular, conventional von Neumann computing system with physically separated processing and memory units face significant problems in dealing with massive unstructured data such as sound, images, and video because of a von Neumann bottleneck. As a key feature of parallel operations, neuromorphic computing systems can analyze massive unstructured data in a time and energy efficient manner. However, critical issues related to reliability and variability of nonlinearity and asymmetric weight update, have been great challenges in the implementation of artificial synaptic device in practical neuromorphic hardware system. Also, hardware systems enabling artificial neural networks in-situ personal data are essential for adaptive wearable neuromorphic edge computing.

A Visually Inspired Computational Model for Recognition of Optic Flow

Foundation models trained on vast quantities of data have demonstrated impressive performance in capturing complex nonlinear relationships and accurately predicting neuronal responses. Due to the fact that deep learning neural networks depend on massive amounts of data samples and high energy consumption, foundation models based on spiking neural networks (SNNs) have the potential to significantly reduce calculation costs by training on neuromorphic hardware. In this paper, a visually inspired computational model composed of an SNN and echo state network (ESN) is proposed for the recognition of optic flow. The visually inspired SNN model serves as a foundation model that is trained using spike-timing-dependent plasticity (STDP) for extracting core features. The ESN model makes readout decisions for recognition tasks using the linear regression method. The results show that STDP can perform similar functions as non-negative matrix decomposition (NMF), i.e., generating sparse and linear superimposed readouts based on basis flow fields. Once the foundation model is fully trained from enough input samples, it can considerably reduce the training samples required for ESN readout learning. Our proposed SNN-based foundation model facilitates efficient and cost-effective task learning and could also be adapted to new stimuli that are not included in the training of the foundation model. Moreover, compared with the NMF algorithm, the foundation model trained using STDP does not need to be retrained during the testing procedure, contributing to a more efficient computational performance.

Impact of spiking neurons leakages and network recurrences on event-based spatio-temporal pattern recognition

Spiking neural networks coupled with neuromorphic hardware and event-based sensors are getting increased interest for low-latency and low-power inference at the edge. However, multiple spiking neuron models have been proposed in the literature with different levels of biological plausibility and different computational features and complexities. Consequently, there is a need to define the right level of abstraction from biology in order to get the best performance in accurate, efficient and fast inference in neuromorphic hardware. In this context, we explore the impact of synaptic and membrane leakages in spiking neurons. We confront three neural models with different computational complexities using feedforward and recurrent topologies for event-based visual and auditory pattern recognition. Our results showed that, in terms of accuracy, leakages are important when there are both temporal information in the data and explicit recurrence in the network. Additionally, leakages do not necessarily increase the sparsity of spikes flowing in the network. We also investigated the impact of heterogeneity in the time constant of leakages. The results showed a slight improvement in accuracy when using data with a rich temporal structure, thereby validating similar findings obtained in previous studies. These results advance our understanding of the computational role of the neural leakages and network recurrences, and provide valuable insights for the design of compact and energy-efficient neuromorphic hardware for embedded systems.

An ultrasmall organic synapse for neuromorphic computing

High‐performance organic neuromorphic devices with miniaturized device size and computing capability are essential elements for developing brain‐inspired humanoid intelligence technique. However, due to the structural inhomogeneity of most organic materials, downscaling of such devices to nanoscale and their high‐density integration into compact matrices with reliable device performance remain challenging at the moment. Herein, based on the design of a semicrystalline polymer PBFCL10 with ordered structure to regulate dense and uniform formation of conductive nanofilaments, we realize an organic synapse with the smallest device dimension of 50 nm and highest integration size of 1 Kb reported thus far. The as‐fabricated PBFCL10 synapses can switch between 32 conductance states linearly with a high cycle‐to‐cycle uniformity of 98.89% and device‐to‐device uniformity of 99.71%, which are the best results of organic devices. A mixed-signal neuromorphic hardware system based on the organic neuromatrix and FPGA controller is implemented to execute spiking‐plasticity‐related algorithm for decision-making tasks.

Lightweight spiking neural network training based on spike timing dependent backpropagation

Spiking neural networks are energy efficient and biological interpretability, communicating through sparse, asynchronous spikes, which makes them suitable for neuromorphic hardware. However, due to the nature of binary weights and spike trains in time-coded binarized spiking neural networks, their forward propagation may cause neurons to not fire spikes, and their backward propagation has non-differentiable problems. Moreover, the current use of deep and complex network structures generates a large number of redundant parameters. Therefore, we need effective methods to improve energy efficiency without reducing accuracy. We propose a dynamic threshold model that can reduce the number of dead neurons. We combine the backpropagation algorithm and the spike timing dependent plasticity algorithm to avoid the non-differentiable problem. We propose a neuron pruning strategy based on adaptive firing time threshold. This pruning strategy prunes 267 neurons in a network of 600 neurons, reducing the network size and obtaining a more compact network structure. The energy efficiency is improved by 0.55×, while the classification accuracy is lost by 1.1%.

Recurrent neural network implementation of digital integrated circuits to mitigate challenges in design verification

Design verification is the dominant stage that consumes the most resources in the digital integrated circuit (IC) design process. Design verification is important because human designers imperfectly convert high-level specifications to low-level circuit implementations using standard cell logic, which is nonlinear and complex to predict and characterize. The widening process variations in shrinking process technologies while digital designs grow in scale and complexity to the extent of being impossible to fully or intuitively identify all temporal interactions of a specific design. Deep neural networks (DNN) are being progressively integrated into the sophisticated software tool chain and design process flow of digital IC as artificial intelligence can learn relationships and correlations in complex, high-dimensional, and multi-factorial problems. In this work, we propose to apply DNN to implement digital IC to minimize the complexity of digital design verification. We posit that DNN can learn to implement circuit functions directly from high-level specifications without requiring detailed specifications from the designer. Trained neural networks can be implemented on neuromorphic hardware to achieve greater power and compute efficiencies than the conventional standard cell implementation. We demonstrate that over 150 randomly generated finite state machines (FSM) can be learned effectively with Recurrent Neural Network (RNN) comprising Gated Recurrent Units (GRU) with different complexity as indicated by the number of states and inputs to the FSM. Our proposed methodology of learning RNN GRU implementations of FSM demonstrates a way forward to reduce the cost and effort of design verification, ultimately leading towards faster digital IC design cycles.

Simulation-based inference for model parameterization on analog neuromorphic hardware

The BrainScaleS-2 (BSS-2) system implements physical models of neurons as well as synapses and aims for an energy-efficient and fast emulation of biological neurons. When replicating neuroscientific experiments on BSS-2, a major challenge is finding suitable model parameters. This study investigates the suitability of the sequential neural posterior estimation (SNPE) algorithm for parameterizing a multi-compartmental neuron model emulated on the BSS-2 analog neuromorphic system. The SNPE algorithm belongs to the class of simulation-based inference methods and estimates the posterior distribution of the model parameters; access to the posterior allows quantifying the confidence in parameter estimations and unveiling correlation between model parameters. For our multi-compartmental model, we show that the approximated posterior agrees with experimental observations and that the identified correlation between parameters fits theoretical expectations. Furthermore, as already shown for software simulations, the algorithm can deal with high-dimensional observations and parameter spaces when the data is generated by emulations on BSS-2. These results suggest that the SNPE algorithm is a promising approach for automating the parameterization and the analyzation of complex models, especially when dealing with characteristic properties of analog neuromorphic substrates, such as trial-to-trial variations or limited parameter ranges.

Low-variance Forward Gradients using Direct Feedback Alignment and momentum

Supervised learning in deep neural networks is commonly performed using error backpropagation. However, the sequential propagation of errors during the backward pass limits its scalability and applicability to low-powered neuromorphic hardware. Therefore, there is growing interest in finding local alternatives to backpropagation. Recently proposed methods based on forward-mode automatic differentiation suffer from high variance in large deep neural networks, which affects convergence. In this paper, we propose the Forward Direct Feedback Alignment algorithm that combines Activity-Perturbed Forward Gradients with Direct Feedback Alignment and momentum. We provide both theoretical proofs and empirical evidence that our proposed method achieves lower variance than forward gradient techniques. In this way, our approach enables faster convergence and better performance when compared to other local alternatives to backpropagation and opens a new perspective for the development of online learning algorithms compatible with neuromorphic systems.

Human-Level Control Through Directly Trained Deep Spiking Q -Networks.

As the third-generation neural networks, spiking neural networks (SNNs) have great potential on neuromorphic hardware because of their high energy efficiency. However, deep spiking reinforcement learning (DSRL), that is, the reinforcement learning (RL) based on SNNs, is still in its preliminary stage due to the binary output and the nondifferentiable property of the spiking function. To address these issues, we propose a deep spiking Q -network (DSQN) in this article. Specifically, we propose a directly trained DSRL architecture based on the leaky integrate-and-fire (LIF) neurons and deep Q -network (DQN). Then, we adapt a direct spiking learning algorithm for the DSQN. We further demonstrate the advantages of using LIF neurons in DSQN theoretically. Comprehensive experiments have been conducted on 17 top-performing Atari games to compare our method with the state-of-the-art conversion method. The experimental results demonstrate the superiority of our method in terms of performance, stability, generalization and energy efficiency. To the best of our knowledge, our work is the first one to achieve state-of-the-art performance on multiple Atari games with the directly trained SNN.

Analysis of Integration Technologies for High-Speed Analog Neuromorphic Photonics

While the use of graphic processing units fueled the success of artificial intelligence models, their future evolution will likely require overcoming the speed and energy efficiency limitations of current implementations with the use of specialized neuromorphic hardware. In this scenario, neuromorphic photonic processors have recently proved to be a feasible solution. In this paper, we first discuss basic analog photonic processing elements based on Mach-Zehnder modulators and assess their effective bit resolution. Then we evaluate how different photonic integration technologies affect the performance and the scalability of analog optical processors, in order to provide a clearer path toward real-world implementations of such engines. To this aim, we focus our analysis on the silicon on insulator (SOI), lithium niobate on insulator (LNOI), and indium phosphide (InP) platforms. In particular, we have numerically evaluated the performance of the Photonic Electronic Multiply- Accumulate Neuron (PEMAN) and its tensorial version, both based on Mach-Zehnder modulators, with the three technologies in terms of resolution, energy efficiency, and footprint efficiency. LNOI modulators achieve the best resolution at high speed, with 4.3 bits at 56 GMAC/s for the single PEMAN and 3.6 bits at 224 GMAC/S for the tensorial version. The energy consumption in InP and LNOI platforms is the lowest, accounting for just 13.2 pJ/MAC and 4.6 pJ/MAC for the single and tensorial PEMAN, respectively. Nonetheless, SOI devices outperform the others in terms of footprint efficiency, reaching 18.6 GMAC/s/mm <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^{2}$</tex-math></inline-formula> in the single-neuron version and 29.6 GMAC/s/mm <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^{2}$</tex-math></inline-formula> in the tensorial version.

Learning inverse kinematics using neural computational primitives on neuromorphic hardware

Current low-latency neuromorphic processing systems hold great potential for developing autonomous artificial agents. However, the variable nature and low precision of the underlying hardware substrate pose severe challenges for robust and reliable performance. To address these challenges, we adopt hardware-friendly processing strategies based on brain-inspired computational primitives, such as triplet spike-timing dependent plasticity, basal ganglia-inspired disinhibition, and cooperative-competitive networks and apply them to motor control. We demonstrate this approach by presenting an example of robust online motor control using a hardware spiking neural network implemented on a mixed-signal neuromorphic processor, trained to learn the inverse kinematics of a two-joint robotic arm. The final system is able to perform low-latency control robustly and reliably using noisy silicon neurons. The spiking neural network, trained to control two joints of the iCub robot arm simulator, performs a continuous target-reaching task with 97.93% accuracy, 33.96 ms network latency, 102.1 ms system latency, and with an estimated power consumption of 26.92 μW during inference (control). This work provides insights into how specific computational primitives used by real neural systems can be applied to neuromorphic computing for solving real-world engineering tasks. It represents a milestone in the design of end-to-end spiking robotic control systems, relying on event-driven sensory encoding, neuromorphic processing, and spiking motor control.

Spiking neural networks for predictive and explainable modelling of multimodal streaming data with a case study on financial time series and online news

In a first study, this paper argues and demonstrates that spiking neural networks (SNN) can be successfully used for predictive and explainable modelling of multimodal streaming data. The paper proposes a new method, where both time series and on-line news are integrated as numerical streaming data in the same time domain and then used to train incrementally a SNN model. The connectivity and the spiking activity of the SNN are then analyzed through clustering and dynamic graph extraction to reveal on-line interaction between all input variables in regard to the predicted one. The paper answers the main research question of how to understand the dynamic interaction of time series and on-line news through their integrative modelling. It offers a new method to evaluate the efficiency of using on-line news on the predictive modelling of time series. Results on financial stock time series and online news are presented. In contrast to traditional machine learning techniques, the method reveals the dynamic interaction between stock variables and news and their dynamic impact on model accuracy when compared to models that do not use news information. Along with the used financial data, the method is applicable to a wide range of other multimodal time series and news data, such as economic, medical, environmental and social. The proposed method, being based on SNN, promotes the use of massively parallel and low energy neuromorphic hardware for multivariate on-line data modelling.