Memristor-based Neuromorphic Computing Research Articles

Enhanced read resolution in reconfigurable memristive synapses for Spiking Neural Networks

The synapse is a key element circuit in any memristor-based neuromorphic computing system. A memristor is a two-terminal analog memory device. Memristive synapses suffer from various challenges including high voltage, SET or RESET failure, and READ margin issues that can degrade the distinguishability of stored weights. Enhancing READ resolution is very important to improving the reliability of memristive synapses. Usually, the READ resolution is very small for a memristive synapse with a 4-bit data precision. This work considers a step-by-step analysis to enhance the READ current resolution or the read current difference between two resistance levels for a current-controlled memristor-based synapse. An empirical model is used to characterize the HfO2\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$${\\hbox {HfO}}_{2}$$\\end{document} based memristive device. 1st\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$1\ extrm{st}$$\\end{document} and 2nd\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$2\ extrm{nd}$$\\end{document} stage device of our proposed synapse design can be scaled to enhance the READ current margin up to ∼\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\sim$$\\end{document} 4.3×\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\ imes$$\\end{document} and ∼\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\sim$$\\end{document} 21%, respectively. Moreover, READ current resolution can be enhanced with run-time adaptation techniques such as READ voltage scaling and body biasing. The READ voltage scaling and body biasing can improve the READ current resolution by about 46% and 15%, respectively. TENNLab’s neuromorphic computing framework is leveraged to evaluate the effect of READ current resolution on classification, control, and reservoir computing applications. Higher READ current resolution shows better accuracy than lower resolution even when facing different levels of read noise.

Reconfigurable multifunctional neuromorphic memristor fabricated from two-dimensional ReSe2 ferroelectric nanosheet films

Memristor-based neuromorphic computing is beneficial for artificial intelligence to process external information autonomously with high speed and high efficiency. Two-dimensional (2D) layered van der Waals rhenium selenide (ReSe2) has optoelectronic and semiconductor properties, but its ferroelectricity has not been confirmed fully experimentally and the application exploration is currently limited. Here, we experimentally confirmed the room-temperature ferroelectricity of 2D ReSe2 and proposed a reconfigurable ReSe2 memristor that can realize multiple functions. The device can realize the conductance bidirectional regulation, and under the action of electrical signals, it exhibits the high 0.99 and 0.98 linearity and accurate bidirectional update of weights. Under the complementary effect of ReSe2 ferroelectric polarization flipping and interface defects, the device exhibits the memcapacitor and memristor reconfigurable behavior and multiple functions such as visible light perception, logical “OR” calculation, and long/short-term synaptic plasticity. In addition, the six-layer convolutional neural network built based on ReSe2 memristors can perform feature extraction and classification recognition of handwritten digital pictures, and its recognition accuracy can reach 97.04%. In addition to obtain substantial experimental evidence for the ferroelectricity of 2D ReSe2, this work also provides a new avenue for the implementation of ReSe2 ferroelectric memristors in the neuromorphic computing system with the front-end sensing and back-end processing.

A carbon conductive filament-induced robust resistance switching behavior for brain-inspired computing.

Memristors have revolutionized the path forward for brain-inspired computing. However, the instability of the nucleation process of conductive filaments based on active metal electrodes leads to the discrete distribution of switching parameters, which hinders the realization of high-performance and low-power devices for neuromorphic computing. In response, a carbon conductive filament-induced robust memristor is demonstrated with variation coefficients as low as 3.9%/-1.18%, a threshold power as low as 10-9 W, and 3 × 106 s retention and 107 cycle endurance behaviors can be maintained. The recognition accuracy for Modified National Institute of Standards and Technology (MNIST) handwriting is as high as 96.87%, attributed to the high linearity of the iterative updating of synaptic weights. The demodulation and storage functions of the American Standard Code for Information Interchange (ASCII) are demonstrated by programmable pulse modulation. Notably, the transmission electron microscopy (TEM) images allow the observation of carbon conductive filament paths formed in the low resistance state. First-principles calculations analyze the energetics of defects involved in the diffusion of carbon atoms into MoTe2 films. This work presents a novel guideline for studying memristor-based neuromorphic computing.

Diverse long-term potentiation and depression based on multilevel LiSiO x memristor for neuromorphic computing

Memristor-based neuromorphic computing is expected to overcome the bottleneck of von Neumann architecture. An artificial synaptic device with continuous conductance variation is essential for implementing bioinspired neuromorphic systems. In this work, a memristor based on Pt/LiSiO x /TiN structure is developed to emulate an artificial synapse, which shows non-volatile multilevel resistance state memory behavior. Moreover, the high nonlinearity caused by abrupt changes in the set process is optimized by adjusting the initial resistance. 100 levels of continuously modulated conductance states are achieved and the nonlinearity factors are reduced to 1.31. The significant improvement is attributed to the decrease in the Schottky barrier height and the evolution of the conductive filaments. Finally, due to the improved linearity of the long-term potentiation/long-term depression behaviors in LiSiO x memristor, a robust recognition rate (∼94.58%) is achieved for pattern recognition with the modified National Institute of Standards and Technology handwriting database. The Pt/LiSiO x /TiN memristor shows significant potential in high-performance multilevel data storage and neuromorphic computing systems.

Internal ion transport in ionic 2D CuInP2S6 enabling multi-state neuromorphic computing with low operation current

Memristor-based neuromorphic computing is promising for artificial intelligence. However, most of the reported memristors have limited linear computing states and consume large operation energy which hinder their applications. Herein, we report a memristor based on ionic two-dimensional CuInP2S6 (2D CIPS), in which up to 1350 linear conductance states are achieved by controlling the migration of internal Cu ions in CIPS. In addition, the device shows a low operation current of ∼100 pA. Cu ions are proven to move along the electric field by in-situ scanning electron microscopy and energy dispersive spectroscopy measurements. Furthermore, complex signal transport among multiple neurons in the brain is imitated by 2D CIPS-based memristor arrays. Our results offer a new platform to fabricate high-performance memristors based on ion transport in 2D materials for neuromorphic computing.

PAWN: Programmed Analog Weights for Non-Linearity Optimization in Memristor-Based Neuromorphic Computing System

Memristors offer advantages as a hardware solution for neuromorphic systems. However, their nonlinear device property makes the weight update inaccurately and reduces the inference accuracy of a neural network. A Programmed Analog Weights for Nonlinearity (PAWN) method is proposed in this paper to update the conductance of a memristor by following the nonlinear curve during the training in a neuromorphic system. The experiment results indicates the PAWN method is effective to alleviate the nonlinearity influence to memristors in all different LTP/LTD conditions. Especially in extreme nonlinearity (LTP=6, LTD=−6), the memristor-based neuromorphic system has significantly low accuracy (51.77%), but the PAWN method enables large accuracy improvement (9.87%) without the inference energy and latency overhead. In addition, overall performance of the neuromorphic system is also evaluated for further verification. Finally, comprehensive experiments show that the PAWN method is still greatly valid even considering device-to-device variations, cycle-to-cycle variations, various technology nodes, and different architectures.

Neuromorphic behaviors of a symmetric LAM-based electronic neuron circuit: Numerical simulation and experimental measurement

Memristor-based neuromorphic computing can achieve the scalability and biological fidelity simultaneously by flexible applying the neuromorphic behaviors. The locally-active memristor (LAM) can work on the edge of chaos associating to stable yet excitable subdomain, which is regarded as a natural candidate in generating neuromorphic behaviors. This paper proposes a new LAM possessing origin symmetric DC characteristic and locally-active domains (LADs) is investigated in-depth and the edge of chaos in its LADs is explored. Afterwards, a second-order memristive circuit is built. Then, the Jacobi matrix-based stability and admittance function are theoretically and numerically analyzed, from which Hopf bifurcation is disclosed. The oscillating behaviors are revealed at neighborhood of the Hopf bifurcation. Then, a third-order electronic neuron circuit is built by parallel connecting a capacitor to the LAM, and a bipolar square-wave is applied to disclose the neuromorphic behaviors. Numerical results demonstrate that the electronic neuron circuit can generate neuromorphic behaviors of periodic/chaotic spiking. These spiking behaviors can be regulated by the period and duty ratio of the applied bipolar square-wave. Finally, FPGA-based experimental measurements validate the numerical simulations. What’s more, these numerical simulations and experimental measurements effectively show the feasibility of the LAM in constructing memristive circuit to generate neuromorphic behaviors.

QuantBayes: Weight Optimization for Memristive Neural Networks via Quantization-Aware Bayesian Inference

The memristor-based neuromorphic computing system (NCS) with emerging storage and computing integration architecture has drawn extensive attention. Because of the unique nonvolatility and programmability, the memristor is an ideal nano-device to realize neural synapses in VLSI circuit implementation of neural networks. However, in the hardware implementation, the performance of the memristive neural network is always affected by quantization error, writing error, and conductance drift, which seriously hinders its applications in practice. In this paper, a novel weight optimization scheme combining quantization and Bayesian inference is proposed to alleviate this problem. Specifically, the weight deviation in the memristive neural network is transformed into the weight uncertainty in the Bayesian neural network, which can make the network insensitive to unexpected weight changes. A quantization regularization term is designed and utilized during the training process of the Bayesian neural network, reducing the quantization error and improving the robustness of the network. Furthermore, a partial training method is raised to extend the applicability of the proposed scheme in large-scale neural networks. Finally, the experiments on a Multilayer Perceptron and LeNet demonstrate that the proposed weight optimization scheme can significantly enhance the robustness of memristive neural networks.

Competing memristors for brain-inspired computing.

SummaryThe expeditious development of information technology has led to the rise of artificial intelligence (AI). However, conventional computing systems are prone to volatility, high power consumption, and even delay between the processor and memory, which is referred to as the von Neumann bottleneck, in implementing AI. To address these issues, memristor-based neuromorphic computing systems inspired by the human brain have been proposed. A memristor can store numerous values by changing its resistance and emulate artificial synapses in brain-inspired computing. Here, we introduce six types of memristors classified according to their operation mechanisms: ionic migration, phase change, spin, ferroelectricity, intercalation, and ionic gating. We review how memristor-based neuromorphic computing can learn, infer, and even create, using various artificial neural networks. Finally, the challenges and perspectives in the competing memristor technology for neuromorphic computing systems are discussed.

Training memristor-based multilayer neuromorphic networks with SGD, momentum and adaptive learning rates

Neural networks implemented with traditional hardware face inherent limitation of memory latency. Specifically, the processing units like GPUs, FPGAs, and customized ASICs, must wait for inputs to read from memory and outputs to write back. This motivates memristor-based neuromorphic computing in which the memory units (i.e., memristors) have computing capabilities. However, training a memristor-based neural network is difficult since memristors work differently from CMOS hardware. This paper proposes a new training approach that enables prevailing neural network training techniques to be applied for memristor-based neuromorphic networks. Particularly, we introduce momentum and adaptive learning rate to the circuit training, both of which are proven methods that significantly accelerate the convergence of neural network parameters. Furthermore, we show that this circuit can be used for neural networks with arbitrary numbers of layers, neurons, and parameters. Simulation results on four classification tasks demonstrate that the proposed circuit achieves both high accuracy and fast speed. Compared with the SGD-based training circuit, on the WBC data set, the training speed of our circuit is increased by 37.2% while the accuracy is only reduced by 0.77%. On the MNIST data set, the new circuit even leads to improved accuracy.

Cascaded Architecture for Memristor Crossbar Array Based Larger-Scale Neuromorphic Computing

Multiply-accumulate calculations using a memristor crossbar array is an important method to realize neuromorphic computing. However, the memristor array fabrication technology is still immature, and it is difficult to fabricate large-scale arrays with high-yield, which restricts the development of memristor-based neuromorphic computing technology. Therefore, cascading small-scale arrays to achieve the neuromorphic computational ability that can be achieved by large-scale arrays, which is of great significance for promoting the application of memristor-based neuromorphic computing. To address this issue, we present a memristor-based cascaded framework with some basic computation units, several neural network processing units can be cascaded by this means to improve the processing capability of the dataset. Besides, we introduce a split method to reduce the pressure of the input terminal. Compared with VGGNet and GoogLeNet, the proposed cascaded framework can achieve 93.54% Fashion-MNIST accuracy and 86.64% CIFAR-10 accuracy under the 4.15M parameters. The extensive experiments with Ti/AlOx/TaOx/Pt we fabricated are conducted to show that the circuit simulation results can still provide a high recognition accuracy, and the recognition accuracy loss after circuit simulation can be controlled at around 0.39%.

Bipolar Analog Memristors as Artificial Synapses for Neuromorphic Computing

Synaptic devices with bipolar analog resistive switching behavior are the building blocks for memristor-based neuromorphic computing. In this work, a fully complementary metal-oxide semiconductor (CMOS)-compatible, forming-free, and non-filamentary memristive device (Pd/Al2O3/TaOx/Ta) with bipolar analog switching behavior is reported as an artificial synapse for neuromorphic computing. Synaptic functions, including long-term potentiation/depression, paired-pulse facilitation (PPF), and spike-timing-dependent plasticity (STDP), are implemented based on this device; the switching energy is around 50 pJ per spike. Furthermore, for applications in artificial neural networks (ANN), determined target conductance states with little deviation (<1%) can be obtained with random initial states. However, the device shows non-linear conductance change characteristics, and a nearly linear conductance change behavior is obtained by optimizing the training scheme. Based on these results, the device is a promising emulator for biology synapses, which could be of great benefit to memristor-based neuromorphic computing.

Three dimensional memristor-based neuromorphic computing system and its application to cloud robotics

Neuromorphic computing based on three-dimensional inetgraed circuits (3D-NCs) offers a novel hardware implementation of neuromorphic computing, and provides high device density, massively parallel signal processing capability, low power consumption, and direct analog signal processing capability. In this paper, by replacing conventional CPUs based on Von Neumann architecture with 3D-NCs, a novel neuromorphic computing based cloud robotics (NC-robotics) system is proposed, which is constructed by 1) cloud server center using 3D-NCs as computing units, 2) neuromorphic robotics based on neural network control technology. Besides the benefits of normal Cloud Robotics platform, this NC- Robotics system has more advantages on massive parallel-computing, analog signals processing, and lower power consumption. In order to implement this NC––Robotics system, a novel 3D-NCs architecture combining vertical RRAM structure is investigated and its concise equivalent circuit model is created, evaluated, and analyzed through SPICE simulations.

MNSIM: Simulation Platform for Memristor-based Neuromorphic Computing System

Memristor-based computation provides a promising solution to boost the power efficiency of the neuromorphic computing system. However, a behavior-level memristor-based neuromorphic computing simulator, which can model the performance and realize an early stage design space exploration, is still missing. In this paper, we propose a simulation platform for the memristor-based neuromorphic system, called MNSIM. A hierarchical structure for memristor-based neuromorphic computing accelerator is proposed to provides flexible interfaces for customization. A detailed reference design is provided for large-scale applications. A behavior-level computing accuracy model is incorporated to evaluate the computing error rate affected by interconnect lines and nonideal device factors. Experimental results show that MNSIM achieves over 7000 times speed-up than SPICE simulation. MNSIM can optimize the design and estimate the tradeoff relationships among different performance metrics for users.

Harmonica: A Framework of Heterogeneous Computing Systems With Memristor-Based Neuromorphic Computing Accelerators

Following technology scaling, on-chip heterogeneous architecture emerges as a promising solution to combat the power wall of microprocessors. This work presents <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><i>Harmonica</i></b> —aframework of heterogeneous computing system enhanced by memristor-based neuromorphic computing accelerators (NCAs). In Harmonica, a conventional pipeline is augmented with a NCA which is designed to speedup artificial neural network (ANN) relevant executions by leveraging the extremely efficient mixed-signal computation capability of nanoscale memristor-based crossbar (MBC) arrays. With the help of a mixed-signal interconnection network (M-Net), the hierarchically arranged MBC arrays can accelerate the computation of a variety of ANNs. Moreover, an inline calibration scheme is proposed to ensure the computation accuracy degradation incurred by the memristor resistance shifting within an acceptable range during NCA executions. Compared to general-purpose processor, Harmonica can achieve on average 27.06 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\times$</tex-math></inline-formula> performance speedup and 25.23 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\times$</tex-math></inline-formula> energy savings when the NCA is configured with auto-associative memory (AAM) implementation. If the NCA is configured with multilayer perception (MLP) implementation, the performance speedup and energy savings can be boosted to 178.41 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\times$</tex-math></inline-formula> and 184.24 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\times$</tex-math></inline-formula> , respectively, with slightly degraded computation accuracy. Moreover, the performance and power efficiency of Harmonica are superior to the designs with either digital neural processing units (D-NPUs) or MBC arrays cooperating with a digital interconnection network. Compared to the baseline of general-purpose processor, the classification rate degradation of Harmonica in MLP or AAM is less than 8% or 4%, respectively.