Analog Resistive Random Access Memory Research Articles

SiNx‐Based Digital–Analog Hybrid Resistive Random Access Memory via Heterogeneous Integration

Herein, a digital–analog hybrid resistive random‐access memory (RRAM) is prepared by integrating the structurally similar SiNx‐based digital‐type RRAM with analog‐type RRAM by heterogeneous integration method. The Pt/SiNx/Ta/Ru is a digital RRAM due to the formation and breakage of the internal silicon dangling bonds conductive filaments, and the TiN/SiNx/Ta/Ru structure is an analog RRAM due to the traps‐filled limit region of the space‐charge limited current. The heterogeneously integrated digital RRAM has good cycling stability and endurance characteristics, and the heterogeneously integrated analog RRAM has high linearity and multistate counting characteristics. This heterogeneous integration approach is useful for the implementation of hybrid digital–analog RRAM.

Demonstration of transfer learning using 14 nm technology analog ReRAM array

Analog memory presents a promising solution in the face of the growing demand for energy-efficient artificial intelligence (AI) at the edge. In this study, we demonstrate efficient deep neural network transfer learning utilizing hardware and algorithm co-optimization in an analog resistive random-access memory (ReRAM) array. For the first time, we illustrate that in open-loop deep neural network (DNN) transfer learning for image classification tasks, convergence rates can be accelerated by approximately 3.5 times through the utilization of co-optimized analog ReRAM hardware and the hardware-aware Tiki-Taka v2 (TTv2) algorithm. A simulation based on statistical 14 nm CMOS ReRAM array data provides insights into the performance of transfer learning on larger network workloads, exhibiting notable improvement over conventional training with random initialization. This study shows that analog DNN transfer learning using an optimized ReRAM array can achieve faster convergence with a smaller dataset compared to training from scratch, thus augmenting AI capability at the edge.

Relaxation Signal Analysis and Optimization of Analog Resistive Random Access Memory for Neuromorphic Computing

Relaxation Signal Analysis and Optimization of Analog Resistive Random Access Memory for Neuromorphic Computing

A Low-Latency DNN Accelerator Enabled by DFT-Based Convolution Execution Within Crossbar Arrays.

Analog resistive random access memory (RRAM) devices enable parallelized nonvolatile in-memory vector-matrix multiplications for neural networks eliminating the bottlenecks posed by von Neumann architecture. While using RRAMs improves the accelerator performance and enables their deployment at the edge, the high tuning time needed to update the RRAM conductance states adds significant burden and latency to real-time system training. In this article, we develop an in-memory discrete Fourier transform (DFT)-based convolution methodology to reduce system latency and input regeneration. By storing the static DFT/inverse DFT (IDFT) coefficients within the analog arrays, we keep digital computational operations using digital circuits to a minimum. By performing the convolution in reciprocal Fourier space, our approach minimizes connection weight updates, which significantly accelerates both neural network training and interference. Moreover, by minimizing RRAM conductance update frequency, we mitigate the endurance limitations of resistive nonvolatile memories. We show that by leveraging the symmetry and linearity of DFT/IDFTs, we can reduce the power by 1.57 × for convolution over conventional execution. The designed hardware-aware deep neural network (DNN) inference accelerator enhances the peak power efficiency by 28.02 × and area efficiency by 8.7 × over state-of-the-art accelerators. This article paves the way for ultrafast, low-power, compact hardware accelerators.

Monolithic three-dimensional integration of RRAM-based hybrid memory architecture for one-shot learning

In this work, we report the monolithic three-dimensional integration (M3D) of hybrid memory architecture based on resistive random-access memory (RRAM), named M3D-LIME. The chip featured three key functional layers: the first was Si complementary metal-oxide-semiconductor (CMOS) for control logic; the second was computing-in-memory (CIM) layer with HfAlOx-based analog RRAM array to implement neural networks for feature extractions; the third was on-chip buffer and ternary content-addressable memory (TCAM) array for template storing and matching, based on Ta2O5-based binary RRAM and carbon nanotube field-effect transistor (CNTFET). Extensive structural analysis along with array-level electrical measurements and functional demonstrations on the CIM and TCAM arrays was performed. The M3D-LIME chip was further used to implement one-shot learning, where ~96% accuracy was achieved on the Omniglot dataset while exhibiting 18.3× higher energy efficiency than graphics processing unit (GPU). This work demonstrates the tremendous potential of M3D-LIME with RRAM-based hybrid memory architecture for future data-centric applications.

The Impact of Thermal Enhance Layers on the Relaxation Effect in Analog RRAM

Computing-in-memory (CIM) with analog resistive random access memory (RRAM) has recently shown great potential in building energy-efficient hardware for artificial intelligence (AI). However, the relaxation effect of analog RRAM featuring post-programming conductance drift has become a key performance-limiting factor. In this work, a comprehensive study of the relaxation effect is presented from the analysis of its causes to the strategy for device optimization as well as the impact on CIM applications. An application-oriented quantitative indicator (relative deviation [RD]) is proposed to fairly evaluate the relaxation effect of different devices. In particular, the influence of oxygen content in different thermal enhanced layers (TELs) on the relaxation and maximum conductance value <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${G}_{max}$ </tex-math></inline-formula> of analog RRAM is studied. A theory of ternary oxide TEL is proposed to mitigate relaxation while maintaining low <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${G}_{max}$ </tex-math></inline-formula> , which is experimentally validated by TaTiO <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><i>x</i></sub> as TEL. Furthermore, neural network simulation is carried out to analyze the requirement for RRAM relaxation for CIM applications. This work provides a useful strategy for device optimization to suppress the relaxation effect by engineering the TEL.

FangTianSim: High-Level Cycle-Accurate Resistive Random-Access Memory-Based Multi-Core Spiking Neural Network Processor Simulator.

Realization of spiking neural network (SNN) hardware with high energy efficiency and high integration may provide a promising solution to data processing challenges in future internet of things (IoT) and artificial intelligence (AI). Recently, design of multi-core reconfigurable SNN chip based on resistive random-access memory (RRAM) is drawing great attention, owing to the unique properties of RRAM, e.g., high integration density, low power consumption, and processing-in-memory (PIM). Therefore, RRAM-based SNN chip may have further improvements in integration and energy efficiency. The design of such a chip will face the following problems: significant delay in pulse transmission due to complex logic control and inter-core communication; high risk of digital, analog, and RRAM hybrid design; and non-ideal characteristics of analog circuit and RRAM. In order to effectively bridge the gap between device, circuit, algorithm, and architecture, this paper proposes a simulation model—FangTianSim, which covers analog neuron circuit, RRAM model and multi-core architecture and its accuracy is at the clock level. This model can be used to verify the functionalities, delay, and power consumption of SNN chip. This information cannot only be used to verify the rationality of the architecture but also guide the chip design. In order to map different network topologies on the chip, SNN representation format, interpreter, and instruction generator are designed. Finally, the function of FangTianSim is verified on liquid state machine (LSM), fully connected neural network (FCNN), and convolutional neural network (CNN).

Statistical temperature coefficient distribution in analog RRAM array: impact on neuromorphic system and mitigation method

Emerging analog resistive random access memory (RRAM) based on HfO x is an attractive device for non-von Neumann neuromorphic computing systems. The differences in temperature dependent conductance drift among cells hamper computing accuracy, characterized by the statistical distribution of temperature coefficient (Tα ). A compact model was presented in order to investigate the statistical distribution of Tα under different resistance states. Based on this model, the physical mechanism of thermal instability of cells with a positive Tα was elucidated. Furthermore, this model can also effectively evaluate the impact of conductance distribution of different levels under various temperatures in artificial neural networks. A current compensation scheme and hybird optimization method were proposed to reduce the impact of the distribution of Tα . The simulation results showed that recognition accuracy was improved from 79.8% to 91.3% for the application of Modified National Institute of Standards and Technology handwriting digits classification with a two-layer perceptron at 400 K after adopting the proposed optimization method.

Redundancy and Analog Slicing for Precise In-Memory Machine Learning—Part I: Programming Techniques

In-memory computing (IMC) is receiving considerable interest for accelerating artificial intelligence (AI) tasks, such as neural network training and inference. However, IMC can also accelerate other machine learning (ML) and scientific computing problems, such as recommendation systems, regression, and PageRank, which are ubiquitous in datacenters. These applications typically have higher precision requirements than neural networks, which can challenge analog-based IMC and sacrifice some of the expected energy efficiency benefits. In this article, we address these challenges experimentally, presenting new techniques improving the accuracy of the solution of linear algebra problems, such as eigenvector extraction for PageRank, in a fully integrated circuit (IC) with analog resistive random access memory (RRAM) devices. Our custom redundancy algorithm can improve the programming accuracy by using multiple memory devices for representing a single matrix entry. Accuracy is further improved by error compensation with analog slicing, which allows an ever more precise value representation.

Crossbar-Level Retention Characterization in Analog RRAM Array-Based Computation-in-Memory System

The reliability issues bring great challenges in the performance maintenance of computation-in-memory (CIM), especially based on large-scale resistive random access memory (RRAM) arrays. In this article, we directly characterize the retention of output differential/accumulation current for analog RRAM-based CIM applications. Different from the conventional concerns on the device-level conductance time-dependent fluctuation, this work focuses on the influence of crossbar-level weighted-sum currents on the accuracy loss over time in the general convolutional and fully connected (FC) networks. This is the first Mb-level long-term retention characterization and evaluation in analog RRAM arrays. Comparing with the simulation accuracy based on the short-term device-level test, the computing accuracy values based on crossbar-level characterizations are improved for about 16.8% and 31.3% at 500 and 1000 min at 125 °C and match well with the measured accuracy, indicating that the crossbar-level retention evaluation is more accurate. This work provides new insights for developing RRAM-based CIM systems with excellent reliability.

Compact Reliability Model of Analog RRAM for Computation-in-Memory Device-to-System Codesign and Benchmark

A physics-based compact model of reliability degradation in analog resistive random access memory (RRAM) is developed. The model captures the stochastic degradation behaviors of retention, bit yield, and endurance during analog resistive switching. The model is verified with statistical data measured from analog RRAM arrays. Based on this compact model, a device-to-system simulation framework for the computation-in-memory (CIM) system is developed. This simulation framework is a silicon-verified versatile simulator that supports both inference and training, and fully considers the device nonideal effects and circuit constraints. Based on the reliability evaluation results, optimization guidelines to suppress the impact of device reliability degradation are proposed. This work provides a useful device-system codesign tool for developing large-scale CIM systems with high performance.

Total Ionizing Dose Effects on Multistate HfOₓ-Based RRAM Synaptic Array

Emerging nonvolatile memories (eNVMs) have demonstrated satisfactory accuracy on various applications in deep learning. Characterized by high density and low leakage power consumption, resistive random access memory (RRAM) becomes very attractive in synaptic devices for deep neural networks (DNNs). RRAM-based synaptic devices include both analog and discrete versions. Unlike analog RRAM synapses which suffer from nonlinearity, discrete but multistate RRAM synapses are better suited for neural network hardware implementation. In this article, the multistate operation in RRAM arrays has been proposed as a synaptic device for DNN inference. Four-state conductance has been achieved in HfO <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">x</sub> -based RRAM synaptic arrays. The impact of total ionizing dose (TID) on the multistate behavior of HfO <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">x</sub> -based RRAM is investigated by irradiating a one-transistor-one-resistor (1T1R) 64-kb array with CMOS peripheral decoding circuitry fabricated at the 90-nm technology node with Co-60 gamma rays ( <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">60</sup> Co γ-ray irradiation).

Early-Stage Fluctuation in Low-Power Analog Resistive Memory: Impacts on Neural Network and Mitigation Approach

In this letter, a new reliability phenomenon, named early-stage resistance fluctuation (ERF), in analog resistive random-access memory (RRAM) devices and its impact on the neuromorphic computing are investigated. ERF is found to be a non-negligible random fluctuation behavior of RRAM resistance within ~10,000s after verification due to stochastic migration of vacancies. As a result, ERF can induce weight noise in RRAM-based neural network even with write-verify operation and cause a significant (~42%) drop of recognition accuracy on the MNIST dataset recognition tasks. An approach incorporating automatic early-stop and dropout during training is proposed to reduce the impacts of ERF. Results show that the recognition accuracy with ERF can be improved from 58% to 96% after adopting the proposed optimization method.

A Compact Model of Analog RRAM With Device and Array Nonideal Effects for Neuromorphic Systems

The parallelism and analog computing features of neuromorphic systems bring great challenges in developing a compact model of analog resistive random access memory (RRAM). In this article, we develop a physics-based compact model for analog RRAM devices and crossbar array. Nonideal effects of analog RRAM device, such as variability, ${I}$ – ${V}$ nonlinearity, programming nonlinearity and asymmetry, and tuning voltage sensitivity, are modeled and verified with the statistical data measured from RRAM array. Modeling of parallel-vector-matrix-multiplication and weight update process on RRAM crossbars with interconnect resistance enables fast and accurate estimation of the training accuracy. Benchmarks of neural networks under different hardware conditions validate the functionality of the proposed model. This model can provide valuable design guidelines for a practical neuromorphic system with high performance and computing accuracy.

Impacts of State Instability and Retention Failure of Filamentary Analog RRAM on the Performance of Deep Neural Network

In this work, an evaluation methodology is proposed to study the impacts of state instability and retention failure of filamentary analog resistive random access memory (FA-RRAM) on the performance of deep neural networks (DNNs). Based on the methodology, an analytic model for the statistical state instability and retention behaviors is applied to evaluate the impacts of the reliability of FA-RRAM on an 11-layer FA-RRAM-based DNN for CIFAR-10 recognition. Simulations indicate that the recognition accuracy of the 11-layer DNN decreases rapidly with the increase of the baking time (t = 104s, 16.3% accuracy loss at 125 °C) due to the overlapping among neighboring resistance levels. To mitigate the accuracy loss caused by state instability and retention failure, the optimization method including the optimized synapse cell and the refresh operation scheme is developed. With the optimization method, the robustness of the FA-RRAM-based DNN is enhanced significantly in which no accuracy loss is observed even after 107s (at 125 °C, ${5} \times {10}^{{3}}$ s/refresh).

Analog switching characteristics in TiW/Al2O3/Ta2O5/Ta RRAM devices

In this letter, we report analog switching characteristics in an analog resistive random access memory device based on a TiW/Al2O3/Ta2O5/Ta stack. For this device, both oxides were grown by using an atomic layer deposition system and the oxygen vacancies were found to exist at the interface of these oxides by using angle-resolved X-ray Photoelectron Spectroscopy. The device exhibits analog switching behaviors. Multiple states were achieved by applying 128 consecutive identical pulses of &amp;lt;20 μs duration and stable for at least 104 s. These characteristics show that the TiW/Al2O3/Ta2O5/Ta device is a promising candidate for synaptic applications.

Impact of Selector Devices in Analog RRAM-Based Crossbar Arrays for Inference and Training of Neuromorphic System

The impact of selector devices on the inference and training accuracy of a resistive random access memory (RRAM)-based neuromorphic computing system is rarely studied. In this paper, we analyze the weighted sum and weight update functions in a one-selector–one-RRAM (1S–1R)-based crossbar arrays. We first develop a Verilog-A model based on the lateral evolution of the filament to describe analog conductance tuning in the filamentary RRAM. We then perform an array-level SPICE simulation on the 1S–1R arrays, where the exponential and threshold selectors are employed. In the inference stage, the read-out current is vulnerable to the inevitable IR drop caused by the wire resistance. Our finding reveals that the use of a threshold selector allows the 1S–1R device to have a linear I–V relation, improving the immunity to the IR drop. On the other hand, the threshold selector distorts analog RRAM’s linear weight update during the training stage. Instead, an introduction of exponential selector enables the desirable properties of analog RRAM to be maintained even in the 1S–1R device. These results indicate that different selectors suitable for each operation mode (inference or training) are preferred in the neuromorphic computing system.

Unsupervised Learning on Resistive Memory Array Based Spiking Neural Networks.

Spiking Neural Networks (SNNs) offer great potential to promote both the performance and efficiency of real-world computing systems, considering the biological plausibility of SNNs. The emerging analog Resistive Random Access Memory (RRAM) devices have drawn increasing interest as potential neuromorphic hardware for implementing practical SNNs. In this article, we propose a novel training approach (called greedy training) for SNNs by diluting spike events on the temporal dimension with necessary controls on input encoding phase switching, endowing SNNs with the ability to cooperate with the inevitable conductance variations of RRAM devices. The SNNs could utilize Spike-Timing-Dependent Plasticity (STDP) as the unsupervised learning rule, and this plasticity has been observed on our one-transistor-one-resistor (1T1R) RRAM devices under voltage pulses with designed waveforms. We have also conducted handwritten digit recognition task simulations on MNIST dataset. The results show that the unsupervised SNNs trained by the proposed method could mitigate the requirement for the number of gradual levels of RRAM devices, and also have immunity to both cycle-to-cycle and device-to-device RRAM conductance variations. Unsupervised SNNs trained by the proposed methods could cooperate with real RRAM devices with non-ideal behaviors better, promising high feasibility of RRAM array based neuromorphic systems for online training.

Performance Impacts of Analog ReRAM Non-ideality on Neuromorphic Computing

Resistive random access memory (ReRAM) is often considered as a strong candidate for storing the weights in non-von Neumann neuromorphic computing systems. This paper studies how nonideal memory characteristics, which include programing error, read fluctuation, and retention, affect the inference accuracy of the analog ReRAM neural networks by incorporating memory characteristics extracted from 1-Mb ReRAM into a simulated inference-only neural network. This paper also shows that the different layer in the network can tolerate different amount of such imperfects. We learned four key points: 1) the conductance range of memory with less relative fluctuation is preferred for designing the weight-conductance mapping; 2) the control of programing error is essential for high inference accuracy; 3) retention-induced conductance drift can be fatal to the neuromorphic system. A compensation scheme is proposed in this paper which can effectively recover the inference accuracy; and 4) for multilayer networks, avoiding weight errors in the front layers can help to maintain the inference accuracy by reducing calculation error which may otherwise accumulate and pass down the networks. The concepts and approaches of this paper can also be applied to evaluate other types of nonvolatile memories for artificial neural networks.

Two-Step Read Scheme in One-Selector and One-RRAM Crossbar-Based Neural Network for Improved Inference Robustness

Introducing a threshold switching selector in a resistive random access memory (RRAM) is essential for implementing a crossbar array that accurately accelerates neuromorphic computations. But, at an expense, a read voltage ( ${V}_{\text {read}}$ ) to be used for inference tasks is inevitably boosted. Therefore, this brief shows the effect of the enlarged ${V}_{\text {read}}$ on the stability of conductance states of the RRAM relevant to the inference robustness. The multiple conductance states of the analog RRAM achieved by a SPICE simulation are stable under consecutive 106 cycles of nominal ${V}_{\text {read}}$ . However, each state of the one selector and one RRAM begins to be disturbed at ~104 cycles due to the boosted ${V}_{\text {read}}$ . More importantly, when a certain state exceeds to the next state due to the accumulated ${V}_{\text {read}}$ stress, a classification accuracy of the neural network is significantly degraded. We, thus, introduce a two-step read scheme that separates the roles of turning on the selector and reading the states. As the selector is turned on rapidly with an additional large pulse, the following ${V}_{\text {read}}$ can be lowered. As a result, the read disturbance is minimized, and the optimized two-step pulse scheme allows 106 MNIST images to be recognized with >95% accuracy in the neural network.