Memristor-based Synapse Research Articles

Manufacturable Ti/ZrO2/Cu memristor-based synapses and biomimetic memory applications with circuit implementation

Manufacturable Ti/ZrO2/Cu memristor-based synapses and biomimetic memory applications with circuit implementation

Enhanced read resolution in reconfigurable memristive synapses for Spiking Neural Networks

The synapse is a key element circuit in any memristor-based neuromorphic computing system. A memristor is a two-terminal analog memory device. Memristive synapses suffer from various challenges including high voltage, SET or RESET failure, and READ margin issues that can degrade the distinguishability of stored weights. Enhancing READ resolution is very important to improving the reliability of memristive synapses. Usually, the READ resolution is very small for a memristive synapse with a 4-bit data precision. This work considers a step-by-step analysis to enhance the READ current resolution or the read current difference between two resistance levels for a current-controlled memristor-based synapse. An empirical model is used to characterize the HfO2\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$${\\hbox {HfO}}_{2}$$\\end{document} based memristive device. 1st\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$1\ extrm{st}$$\\end{document} and 2nd\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$2\ extrm{nd}$$\\end{document} stage device of our proposed synapse design can be scaled to enhance the READ current margin up to ∼\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\sim$$\\end{document} 4.3×\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\ imes$$\\end{document} and ∼\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\sim$$\\end{document} 21%, respectively. Moreover, READ current resolution can be enhanced with run-time adaptation techniques such as READ voltage scaling and body biasing. The READ voltage scaling and body biasing can improve the READ current resolution by about 46% and 15%, respectively. TENNLab’s neuromorphic computing framework is leveraged to evaluate the effect of READ current resolution on classification, control, and reservoir computing applications. Higher READ current resolution shows better accuracy than lower resolution even when facing different levels of read noise.

A Memristor Neural Network Based on Simple Logarithmic-Sigmoidal Transfer Function with MOS Transistors

Memristors are state-of-the-art, nano-sized, two-terminal, passive electronic elements with very good switching and memory characteristics. Owing to their very low power usage and a good compatibility to the existing CMOS ultra-high-density integrated circuits and chips, they are potentially applicable in artificial and spiking neural networks, memory arrays, and many other devices and circuits for artificial intelligence. In this paper, a complete electronic realization of an analog circuit model of the modified neural net with memristor-based synapses and transfer function with memristors and MOS transistors in LTSPICE is offered. Each synaptic weight is realized by only one memristor, providing enormously reduced circuit complexity. The summing and scaling implementation is founded on op-amps and memristors. The logarithmic-sigmoidal activation function is based on a simple scheme with MOS transistors and memristors. The functioning of the suggested memristor-based neural network for pulse input signals is evaluated both analytically in MATLAB-SIMULINK and in the LTSPICE environment. The obtained results are compared one to another and are successfully verified. The realized memristor-based neural network is an important step towards the forthcoming design of complex memristor-based neural networks for artificial intelligence, for implementation in very high-density integrated circuits and chips.

Dual functional states of working memory realized by memristor-based neural network.

Working memory refers to the brain's ability to store and manipulate information for a short period. It is disputably considered to rely on two mechanisms: sustained neuronal firing, and "activity-silent" working memory. To develop a highly biologically plausible neuromorphic computing system, it is anticipated to physically realize working memory that corresponds to both of these mechanisms. In this study, we propose a memristor-based neural network to realize the sustained neural firing and activity-silent working memory, which are reflected as dual functional states within memory. Memristor-based synapses and two types of artificial neurons are designed for the Winner-Takes-All learning rule. During the cognitive task, state transformation between the "focused" state and the "unfocused" state of working memory is demonstrated. This work paves the way for further emulating the complex working memory functions with distinct neural activities in our brains.

Topochemical Synthesis of Copper Phosphide Nanoribbons for Flexible Optoelectronic Memristors (Adv. Funct. Mater. 14/2022)

Flexible Optoelectronic Memristors In article number 2110900, Cong Ye, Jiahong Wang, and co-workers present a topochemical method to transform crystalline phosphorus nanoribbons into copper phosphide nanoribbons (Cu3P NRs) with well-maintained morphology. The flexible optoelectronic memristor constructed with Cu3P NRs exhibit excellent performance, and the memristor-based synapse arrays mimic the memory backtracking and pattern recognition.

SPICE Study of STDP Characteristics in a Drift and Diffusive Memristor-Based Synapse for Neuromorphic Computing

Neuromorphic hardware is a system with massive potential to enable efficient computing by mimicking the human brain. The novel system processes information using neuron spikes (Action Potentials) and the synaptic connections between neurons are trained using biologically plausible methods like spike-timing-dependent plasticity (STDP). Memristor is one of the promising candidates to implement such neuromorphic hardware. Two types of memristors, diffusive and drift, have been proposed to form a synapse showing faithful emulation of STDP, where the diffusion effect is used to trace the spike timing history crucial for STDP and the drift memristor keeps the weight information in a longer time scale. The purpose of this paper is to systematically investigate STDP characteristics in such a synapse with serially connected two memristors using SPICE models. The results show that STDP properties are strongly dependent on device parameters and even the shape of STDP curves is modified. Different shapes of the STDP curve were identified. The results and analysis could support the design of emerging device-based synapses, which can faithfully mimic biological STDP characteristics for future neuromorphic systems.

Metal-induced progressive alteration of conducting states in memristors for implementing an efficient analog memory: a DFT-supported experimental approach

Advancement of the memristor-based artificial synapse (AS) is urgently needed for rapid progress in neuromorphic devices.

An memristor-based synapse implementation using BCM learning rule

A novel memristive synapse model based on the HP memristor is proposed in this paper, which can address the problem of synaptic weight infinite modulations. The sliding threshold mechanism of the Bienenstock-Cooper-Munro rule (BCM) is used to redefine the memristance (i.e. synaptic weight) adjustment process of the memristive synapse model. Based on the proposed memristor-based synapse and Leaky Integrate-and-Fire neurons, a spiking neural network (SNN) hardware fragment is constructed, where spike trains with different frequencies are used to evaluate the stability performance of the proposed SNN hardware. Results show that compared to other approaches, the network is stable under different stimuli due to the characteristics of the memristor-based synapse model, and prove that the proposed synapse model is able to mimic biological synaptic behaviour and the problem of synaptic weight infinite modulations is addressed.

Analysis of the Memristor-Based Crossbar Synapse for Neuromorphic Systems.

In this study, we analyzed the memristor device typically used as a synapse in neuromorphic architecture and confirmed that the synaptic memristor device can be adopted to perform the machine learning algorithm. The nonlinear characteristics of the memristor complicates its use as the neuromorphic hardware in an artificial neural network (ANN) with a back-propagation algorithm. Using a memristor device with a nonlinear characteristic, we demonstrated that pattern classification can be implemented in ANNs using the Guide training algorithm without back-propagation. Furthermore, the memristor characteristics required to achieve accurate learning results are analyzed.

On the Application of a Diffusive Memristor Compact Model to Neuromorphic Circuits.

Memristive devices have found application in both random access memory and neuromorphic circuits. In particular, it is known that their behavior resembles that of neuronal synapses. However, it is not simple to come by samples of memristors and adjusting their parameters to change their response requires a laborious fabrication process. Moreover, sample to sample variability makes experimentation with memristor-based synapses even harder. The usual alternatives are to either simulate or emulate the memristive systems under study. Both methodologies require the use of accurate modeling equations. In this paper, we present a diffusive compact model of memristive behavior that has already been experimentally validated. Furthermore, we implement an emulation architecture that enables us to freely explore the synapse-like characteristics of memristors. The main advantage of emulation over simulation is that the former allows us to work with real-world circuits. Our results can give some insight into the desirable characteristics of the memristors for neuromorphic applications.

Supporting the Momentum Training Algorithm Using a Memristor-Based Synapse

Despite the increasing popularity of deep neural networks (DNNs), they cannot be trained efficiently on existing platforms, and efforts have thus been devoted to designing dedicated hardware for DNNs. In our recent work, we have provided direct support for the stochastic gradient descent (SGD) training algorithm by constructing the basic element of neural networks, the synapse, using emerging technologies, namely memristors. Due to the limited performance of SGD, optimization algorithms are commonly employed in DNN training. Therefore, DNN accelerators that only support SGD might not meet DNN training requirements. In this paper, we present a memristor-based synapse that supports the commonly used momentum algorithm. Momentum significantly improves the convergence of SGD and facilitates the DNN training stage. We propose two design approaches to support momentum: 1) a hardware friendly modification of the momentum algorithm using memory external to the synapse structure, and 2) updating each synapse with a built-in memory. Our simulations show that the proposed DNN training solutions are as accurate as training on a GPU platform while speeding up the performance by $886\times $ and decreasing energy consumption by $7\times $ , on average.

Operation Scheme of Multi-Layer Neural Networks Using NAND Flash Memory as High-Density Synaptic Devices

We propose a designing of multi-layer neural networks using 2D NAND flash memory cell as a high-density and reliable synaptic device. Our operation scheme eliminates the waste of NAND flash cells and allows analogue input values. A 3-layer perceptron network with 40,545 synapses is trained on a MNIST database set using an adaptive weight update method for hardware-based multi-layer neural networks. The conductance response of NAND flash cells is measured and it is shown that the unidirectional conductance response is suitable for implementing multi-layer neural networks using NAND flash memory cells as synaptic devices. Using an online-learning, we obtained higher learning accuracy with NAND synaptic devices compared to that with a memristor-based synapse regardless of weight update methods. Using an adaptive weight update method based on a unidirectional conductance response, we obtained a 94.19% learning accuracy with NAND synaptic devices. This accuracy is comparable to 94.69% obtained by synapses based on the ideal perfect linear device. Therefore, NAND flash memory which is mature technology and has great advantage in cell density can be a promising synaptic device for implementing high-density multi-layer neural networks.

Study of Recall Time of Associative Memory in a Memristive Hopfield Neural Network

By associative memory, people can remember a pattern in microseconds to seconds. In order to emulate human memory, an artificial neural network should also spend a reasonable time in recalling matters of different task difficulties or task familiarities. In this paper, we study the recall time in a memristive Hopfield network (MHN) implemented with memristor-based synapses. With the operating frequencies of 1-100 kHz, patterns can be stored into the network by altering the resistance of the memristors, and the pre-stored patterns can be successfully recalled, being similar to the associative memory behavior. For the same target pattern (the same familiarity), recall time of the MHN varies with the inputs, which is similar to the effect in the human brain that recall time depends on task difficulty. On the other hand, for the same input (i.e., the same initial state), the recall time may be different for different target patterns, which is similar to the effect in the brain that recall time depends on the familiarity. In addition, the effect of stimulation (updating frequency) on recall time may be complicated: a higher stimulation frequency may not always lead to a faster recall (it may even slow the recalling process in some circumstances). Our memristive Hopfield network shows good potential in mimicking the characteristics of human associative memory.

Bio-inspired fault detection circuits based on synapse and spiking neuron models

Recent studies have shown that the electronic hardware devices can be compromised by the faults and fault tolerance is a crucial capability. This paper addresses the challenge of fault detection in the CMOS circuits, using two bio-inspired structures based on the HP lab's memristor and the BSIM3v3.2.2 transistor models. The first fault detection circuit (FDC) includes the memristor-based synapses and a modified leaky integrate-and-fire (LIF)-based neuron. The memristor-based synapse circuits can be further optimized which is the proposed second fault detection method (O-FDC), and it has a lower hardware overhead and power consumption compared to the former. Experimental results demonstrate that the proposed structures can detect the circuit faults under the inputs of direct current (DC), alternating current (AC) voltage sources, and pulse signals. Under the input of DC, the fault detection times for the two proposed structures are about 0.16 ms and 1.2 ms, respectively; when the input source is AC, the corresponding fault detection times are about 0.206 ms and 0.758 ms; and it takes only 6.47us for fault detection under the input of pulse signals. This work provides an alternative solution to enhance the fault-tolerant capability of the hardware systems.

Monolithic 3D neuromorphic computing system with hybrid CMOS and memristor-based synapses and neurons

Because of fabrication compatibility to current semiconductor technology, three-dimensional integrated circuits (3D-ICs) offer promising near-term solutions for maintaining Moore’s Law. 3D-ICs proffer high system speeds, massively parallel processing, low power consumption, and their high densities result in small footprints. In this paper, a novel 3D neuromorphic IC architecture which combines monolithic 3D integration and a synaptic array based on vertical resistive random-access memory structure (V-RRAM) is proposed. To analyze the electrical characteristics of the proposed synaptic array, a concise equivalent circuit model of the system is developed, and analytical calculations for each parameter of the equivalent circuit are provided. Moreover, a novel signal intensity encoding neuron design that can directly convert analog signal into a spiking waveform sequence is proposed and analyzed. A feasible 3D neuromorphic computing architecture is demonstrated. Applying the monolithic 3D integration technology on neuromorphic computing system hardware implementation can reduce the power consumption by 50%, and shrink die areas by 35%.

Modular neuron comprises of memristor-based synapse

Design of analog modular neuron based on memristor is proposed here. Since neural networks are built by repetition of basic blocks that are called neurons, using modular neurons is essential for the neural network hardware. In this work modularity of the neuron is achieved through distributed neurons structure. Some major challenges in implementation of synaptic operation are weight programmability, weight multiplication by input signal and nonvolatile weight storage. Introduction of memristor bridge synapse addresses all of these challenges. The proposed neuron is a modular neuron based on distributed neuron structure which it uses the benefits of the memristor bridge synapse for synaptic operations. In order to test appropriate operation of the proposed neuron, it is used in a real-world application of neural network. Off-chip method is used to train the neural network. The results show 86.7 % correct classification and about 0.0695 mean square error for 4-5-3 neural network based on proposed modular neuron.

Reconfigurable Neuromorphic Computing System with Memristor-Based Synapse Design

Conventional CMOS technology is slowly approaching its physical limitations and researchers are increasingly utilizing nanotechnology to both extend CMOS capabilities and to explore potential replacements. Novel memristive systems continue to attract growing attention since their reported physical realization by HP in 2008. Unique characteristics like non-volatility, re-configurability, and analog storage properties make memristors a very promising candidate for the realization of artificial neural systems. In this work, we propose a memristor-based design of bidirectional transmission excitation/inhibition synapses and implement a neuromorphic computing system based on our proposed synapse designs. The robustness of our system is also evaluated by considering the actual manufacturing variability with emphasis on process variation.

Associative Learning of Integrate-and-Fire Neurons with Memristor-Based Synapses

A memrsitor is a two-terminal electronic device whose conductance can be precisely modulated by charge or flux through it. In this paper, we present a class of memristor-based neural circuits comprising leaky integrate-and-fire (I & F) neurons and memristor-based learning synapses. Employing these neuron circuits and corresponding SPICE models, the properties of a two neurons network are shown to be similar to biology. During correlated spiking of the pre- and post-synaptic neurons, the strength of the synaptic connection increases. Conversely, it is diminished when the spiking is uncorrelated. This synaptic plasticity and associative learning is essential for performing useful computation and adaptation in large scale artificial neural networks. Finally, future circuit design and consideration are discussed with the memristor-based neural networks.