Memristor-based Neuromorphic System Research Articles

Noise-induced homeostasis in memristor-based neuromorphic systems

Noise-induced homeostasis in memristor-based neuromorphic systems

A Hybrid Weight Quantization Strategy for Memristive Neural Networks

Due to the ability to store data and process information, the memristor-based neuromorphic system has attracted extensive attention. Its efficient parallel computing approach allows it to implement neural networks in hardware. However, due to the limitation of the range of memristor conductance, it is difficult to represent high-precision weights in memristive neural network. During the off-chip learning, it is crucial to find an efficient weight quantization scheme and map it to the memristor array. Therefore, a hybrid weight quantization strategy that combines uniform and non-uniform quantization is proposed to overcome these problems. Specifically, the curve fitting of pulse modulation for tantalum oxide-based memristor is carried out, and the mapping rules of weights are proposed to simplify the process of reading verification. Furthermore, the hybrid quantization strategy is proposed and applied to a multilayer perceptron and a convolutional neural network, respectively. The effectiveness and robustness of the hybrid quantization scheme are verified in the MNIST dataset. Experiments show that the proposed hybrid quantization scheme can achieve 99.26% accuracy at 4 bits and tolerate 20% noise interference. The simulation results in this paper also provide an effective solution for the hardware implementation of memristive neural networks.

From memristive devices to neuromorphic systems

Progress in hardware and algorithms for artificial intelligence (AI) has ushered in large machine learning models and various applications impacting our everyday lives. However, today's AI, mainly artificial neural networks, still cannot compete with human brains because of two major issues: the high energy consumption of the hardware running AI models and the lack of ability to generalize knowledge and self-adapt to changes. Neuromorphic systems built upon emerging devices, for instance, memristors, provide a promising path to address these issues. Although innovative memristor devices and circuit designs have been proposed for neuromorphic computing and applied to different proof-of-concept applications, there is still a long way to go to build large-scale low-power memristor-based neuromorphic systems that can bridge the gap between AI and biological brains. This Perspective summarizes the progress and challenges from memristor devices to neuromorphic systems and proposes possible directions for neuromorphic system implementation based on memristive devices.

PAWN: Programmed Analog Weights for Non-Linearity Optimization in Memristor-Based Neuromorphic Computing System

Memristors offer advantages as a hardware solution for neuromorphic systems. However, their nonlinear device property makes the weight update inaccurately and reduces the inference accuracy of a neural network. A Programmed Analog Weights for Nonlinearity (PAWN) method is proposed in this paper to update the conductance of a memristor by following the nonlinear curve during the training in a neuromorphic system. The experiment results indicates the PAWN method is effective to alleviate the nonlinearity influence to memristors in all different LTP/LTD conditions. Especially in extreme nonlinearity (LTP=6, LTD=−6), the memristor-based neuromorphic system has significantly low accuracy (51.77%), but the PAWN method enables large accuracy improvement (9.87%) without the inference energy and latency overhead. In addition, overall performance of the neuromorphic system is also evaluated for further verification. Finally, comprehensive experiments show that the PAWN method is still greatly valid even considering device-to-device variations, cycle-to-cycle variations, various technology nodes, and different architectures.

Action potential and chaos near the edge of chaos in memristive circuits.

Memristor-based neuromorphic systems have a neuro-bionic function, which is critical for possibly overcoming Moore's law limitation and the von Neumann bottleneck problem. To explore neural behaviors and complexity mechanisms in memristive circuits, this paper proposes an N-type locally active memristor, based on which a third-order memristive circuit is constructed. Theoretical analysis shows that the memristive circuit can exhibit not only various action potentials but also self-sustained oscillation and chaos. Based on Chua's theory of local activity, this paper finds that the neural behaviors and chaos emerge near the edge of chaos through subcritical Hopf bifurcation, in which the small unstable limit cycle is depicted by the dividing line between the attraction basin of the large stable limit cycle and the attraction basin of the stable equilibrium point. Furthermore, an analog circuit is designed to imitate the action potentials and chaos, and the simulation results are in agreement with the theoretical analysis.

Improved analog switching characteristics of Ta2O5-based memristor using indium tin oxide buffer layer for neuromorphic computing

A memristor is defined as a non-volatile memory switching two-terminal resistor, and a memristor with digital switching characteristics is widely studied as a next-generation non-volatile memory because of its simple structure, high integration density, and low power consumption. Recently, analog memristors with gradual resistance switching (RS) characteristics have garnered great attention because of their potential to implement artificial synapses that can emulate the brain functions. Transition metal oxides are thought to be strong candidate materials for the RS. In particular, tantalum oxide (TaO x )-based memristive devices provide stable and durable switching characteristics. TaO x -based memristors utilize analog switching characteristics and have excellent durability and reliability, so they can be applied as artificial synaptic device. In this study, the characteristics of analog RS using Ta2O5-based memristive devices were investigated. The current level of the Pt/Ta2O5/Pt memristors was improved by adjusting the thickness of Ta2O5. In particular, when an indium-tin-oxide (ITO) buffer layer was added to Ta2O5 forming a Pt/ITO/Ta2O5/Pt heterostructured double-layer device, it showed more symmetrical potentiation and depression characteristics under both polarities than a single-layer device without ITO layer. The symmetrical and linear potentiation and depression characteristics are essential for the development of efficient memristor-based neuromorphic systems. Insertion of the ITO buffer layer improves linearity, symmetry, and stability of the analog RS properties of Ta2O5-based memristors to artificial synapses.

Multiscale Modeling for Application-Oriented Optimization of Resistive Random-Access Memory.

Memristor-based neuromorphic systems have been proposed as a promising alternative to von Neumann computing architectures, which are currently challenged by the ever-increasing computational power required by modern artificial intelligence (AI) algorithms. The design and optimization of memristive devices for specific AI applications is thus of paramount importance, but still extremely complex, as many different physical mechanisms and their interactions have to be accounted for, which are, in many cases, not fully understood. The high complexity of the physical mechanisms involved and their partial comprehension are currently hampering the development of memristive devices and preventing their optimization. In this work, we tackle the application-oriented optimization of Resistive Random-Access Memory (RRAM) devices using a multiscale modeling platform. The considered platform includes all the involved physical mechanisms (i.e., charge transport and trapping, and ion generation, diffusion, and recombination) and accounts for the 3D electric and temperature field in the device. Thanks to its multiscale nature, the modeling platform allows RRAM devices to be simulated and the microscopic physical mechanisms involved to be investigated, the device performance to be connected to the material’s microscopic properties and geometries, the device electrical characteristics to be predicted, the effect of the forming conditions (i.e., temperature, compliance current, and voltage stress) on the device’s performance and variability to be evaluated, the analog resistance switching to be optimized, and the device’s reliability and failure causes to be investigated. The discussion of the presented simulation results provides useful insights for supporting the application-oriented optimization of RRAM technology according to specific AI applications, for the implementation of either non-volatile memories, deep neural networks, or spiking neural networks.

Low-Voltage Oscillatory Neurons for Memristor-Based Neuromorphic Systems.

Neuromorphic systems consisting of artificial neurons and synapses can process complex information with high efficiency to overcome the bottleneck of von Neumann architecture. Artificial neurons are essentially required to possess functions such as leaky integrate‐and‐fire and output spike. However, previous reported artificial neurons typically have high operation voltage and large leakage current, leading to significant power consumption, which is contrary to the energy‐efficient biological model. Here, an oscillatory neuron based on Ag filamentary threshold switching memristor (TS) that has a low operation voltage (<0.6 V) with ultralow power consumption (<1.8 µW) is presented. It can trigger neuronal functions, including leaky integrate‐and‐fire and threshold‐driven spiking output, with high endurance (>108 cycles). Being connected to an external resistor or a resistive switching memristor (RS) as synaptic weight, the TS clearly demonstrates self‐oscillation behavior once the input pulse voltage exceeds the threshold voltage. Meanwhile, the oscillation frequency is proportional to the input pulse voltage and the conductance of RS synapse, which can be used to integrate the weighted sum current. As an energy‐efficient memristor‐based spiking neural network, this combination of TS oscillatory neuron with RS synapse is further evaluated for image recognition achieving an accuracy of 79.2 ± 2.4% for CIFAR‐10 subset.

An improved memristor model connecting plastic synapse and nonlinear spiking neuron

In this work, a flux-controlled memristor model with the state variable covering negative and positive states and having decay behavior is developed to link the plastic synapse and nonlinear spiking neuron. For the voltage dependence of the conductance, the trace-based voltage can be directly applied to the memristor without additional threshold to implement various synaptic plasticity such as STDP, anti-STDP, et al. At the same time, the memristor model can be introduced as adaption element in nonlinear models to reduce the model complexity while remaining the richness of firing patterns. A 2D nonlinear voltage-stimulated spiking neuron model with a memristor connecting to the input is developed. Stereotypically arranging the input voltages, various firing patterns observed in cortical neurons, such the tonic, adapting, burst, latency, and bistability, are successfully carried out in the neuron model. The flux-controlled memristor model can describe the electrical behavior of paired memristors with a power supply, and well link plastic synapses and nonlinear spiking neurons, which may be helpful for simulating and constructing a complex memristor-based neuromorphic system.

Optogenetics-Inspired Tunable Synaptic Functions in Memristors.

Two-terminal memristors with internal Ca2+-like dynamics can be used to faithfully emulate biological synaptic functions and have been intensively studied for neural network implementations. Inspired by the optogenetic technique that utilizes light to tune the Ca2+ dynamics and subsequently the synaptic plasticity, we develop a CH3NH3PbI3 (MAPbI3)-based memristor that exhibits light-tunable synaptic behaviors. Specifically, we show that by increasing the formation energy of iodine vacancy (VI·/VI×), light illumination can be used to control the VI·/VI× generation and annihilation dynamics, resembling light-controlled Ca2+ influx in biological synapses. We demonstrate that the memory formation and memory loss behaviors in the memristors can be modified by controlling the intensity and the wavelength of the illuminated light. Coincidence detection of electrical and light stimulations is also implemented in the memristive device with real-time (≤20 ms) response to light illumination. These results open options to modify the synaptic plasticity effects in memristor-based neuromorphic systems and can lead to the development of electronic systems that can faithfully emulate diverse biological processes.

Parasitic Effect Analysis in Memristor-Array-Based Neuromorphic Systems

Neuromorphic systems using memristors as artificial synapses have attracted broad interest for energy-efficient computing applications. However, networks based on these purely passive devices can be affected by parasitic effects such as series resistance and sneak path problems. Here, we analyze the effects of parasitic factors on the performance of memristor-based neuromorphic systems. During vector-array multiplication, the line resistance can cause significant distortion of the output current and the activity of the corresponding neurons. An approach to compensate the line resistance effects based on an approximate model consisting of only few known parameters is proposed and shows excellent ability to capture the complex network behavior. During training and feature detection, the series resistance can cause significant degradation of the learned dictionary, with only a few dominant neurons being trained. Using a scaling factor based on the proposed simple model, these effects can be successfully mitigated, and the correct network operations can be restored. These results provide insight and practical measures on the parasitic effects for implementation of the neuromorphic system using memristor arrays.

A Compact Memristor-Based Dynamic Synapse for Spiking Neural Networks

Recent advances in memristor technology lead to the feasibility of large-scale neuromorphic systems by leveraging the similarity between memristor devices and synapses. For instance, memristor cross-point arrays can realize dense synapse network among hundreds of neuron circuits, which is not affordable for traditional implementations. However, little progress was made in synapse designs that support both static and dynamic synaptic properties. In addition, many neuron circuits require signals in specific pulse shape, limiting the scale of system implementation. Last but not least, a bottom-up study starting from realistic memristor devices is still missing in the current research of memristor-based neuromorphic systems. Here, we propose a memristor-based dynamic (MD) synapse design with experiment-calibrated memristor models. The structure obtains both static and dynamic synaptic properties by using one memristor for weight storage and the other as a selector. We overcame the device nonlinearities and demonstrated spike-timing-based recall, weight tunability, and spike-timing-based learning functions on MD synapse. Furthermore, a temporal pattern learning application was investigated to evaluate the use of MD synapses in spiking neural networks, under both spike-timing-dependent plasticity and remote supervised method learning rules.

Flexible memristor based neuromorphic system for implementing multi-layer neural network algorithms

This paper describes a memristor-based neuromorphic system that can be used for ex situ training of various multi-layer neural network algorithms. This system is based on an analogue neuron circuit that is capable of performing an accurate dot product calculation. The presented ex situ programming technique can be used to map many key neural algorithms directly onto the grid of resistances in a memristor crossbar. Using this weight-to-crossbar mapping approach along with the memristor based circuit architecture, complex neural algorithms can be easily implemented using this system. Some existing memristor based circuits provide an approximated dot product based on conductance summation, but neuron outputs are not directly correlated to the numerical values obtained in a traditional software approach. To show the effectiveness and versatility of this circuit, two different powerful neural networks were simulated. These include a Restricted Boltzmann Machine for character recognition and a Multilayer Perceptron trained to perform Sobel edge detection. Following these simulations, an analysis was presented that shows how both memristor accuracy and neuron circuit gain relates to output error.

MNSIM: Simulation Platform for Memristor-based Neuromorphic Computing System

Memristor-based computation provides a promising solution to boost the power efficiency of the neuromorphic computing system. However, a behavior-level memristor-based neuromorphic computing simulator, which can model the performance and realize an early stage design space exploration, is still missing. In this paper, we propose a simulation platform for the memristor-based neuromorphic system, called MNSIM. A hierarchical structure for memristor-based neuromorphic computing accelerator is proposed to provides flexible interfaces for customization. A detailed reference design is provided for large-scale applications. A behavior-level computing accuracy model is incorporated to evaluate the computing error rate affected by interconnect lines and nonideal device factors. Experimental results show that MNSIM achieves over 7000 times speed-up than SPICE simulation. MNSIM can optimize the design and estimate the tradeoff relationships among different performance metrics for users.

A Memristor Device Model

This letter proposes a new mathematical model for memristor devices. It builds on existing models and is correlated against several published device characterizations. This letter identifies significant discrepancies between the existing models and published device characterization data. The proposed model addresses these discrepancies. In particular, it allows modeling of memristor-based neuromorphic systems.