Resistive Random Access Memory Crossbar Array Research Articles

A Cascaded ReRAM-based Crossbar Architecture for Transformer Neural Network Acceleration

Emerging resistive random-access memory (ReRAM) based processing-in-memory (PIM) accelerators have been increasingly explored in recent years because they can efficiently perform in-situ matrix-vector multiplication (MVM) operations involved in a wide spectrum of artificial neural networks. However, there remain significant challenges to apply existing ReRAM-based PIM accelerators to the most popular Transformer neural networks. Since Transformers involve a series of matrix-matrix multiplication (MatMul) operations with data dependencies, they should write intermediate results of MatMuls to ReRAM crossbar arrays for further processing. Conventional ReRAM-based PIM accelerators often suffer from high latency of ReRAM writes and intra-layer pipeline stalls. In this paper, we propose ReCAT, a ReRAM-based PIM accelerator designed particularly for Transformers. ReCAT exploits transimpedance amplifiers (TIAs) to cascade a pair of crossbar arrays for MatMul operations involved in the self-attention mechanism. The intermediate result of a MatMul generated by one crossbar array can be directly mapped to another crossbar array, avoiding costly analog-to-digital conversions. In this way, ReCAT allows MVM operations to overlap with the corresponding data mapping, hiding the high latency of ReRAM writes. Furthermore, we propose an analog-to-digital converter (ADC) virtualization scheme to dynamically share scarce ADCs among a group of crossbar arrays, and thus significantly improve the utilization of ADCs to eliminate the performance bottleneck of MVM operations. Experimental results show that ReCAT achieves 207.3 ×, 2.11 ×, and 3.06 × performance improvement on average compared with other Transformer acceleration solutions—GPUs, ReBert, and ReTransformer, respectively.

Modeling of Self-Aligned Selector Based on Ultra-Thin Metal Oxide for Resistive Random-Access Memory (RRAM) Crossbar Arrays.

Resistive random-access memory (RRAM) is a crucial element for next-generation large-scale memory arrays, analogue neuromorphic computing and energy-efficient System-on-Chip applications. For these applications, RRAM elements are arranged into Crossbar arrays, where rectifying selector devices are required for correct read operation of the memory cells. One of the key advantages of RRAM is its high scalability due to the filamentary mechanism of resistive switching, as the cell conductivity is not dependent on the cell area. Thus, a selector device becomes a limiting factor in Crossbar arrays in terms of scalability, as its area exceeds the minimal possible area of an RRAM cell. We propose a tunnel diode selector, which is self-aligned with an RRAM cell and, thus, occupies the same area. In this study, we address the theoretical and modeling aspects of creating a self-aligned selector with optimal parameters to avoid any deterioration of RRAM cell performance. We investigate the possibilities of using a tunnel diode based on single- and double-layer dielectrics and determine their optimal physical properties to be used in an HfOx-based RRAM Crossbar array.

Fully Binarized Graph Convolutional Network Accelerator Based on In‐Memory Computing with Resistive Random‐Access Memory

Artificial intelligence for graph‐structured data has achieved remarkable success in applications such as recommendation systems, social networks, drug discovery, and circuit annotation. Graph convolutional networks (GCNs) are an effective way to learn representations of various graphs. The increasing size and complexity of graphs call for in‐memory computing (IMC) accelerators for GCN to alleviate massive data transmission between off‐chip memory and processing units. However, GCN implementation with IMC is challenging because of the large memory consumption, irregular memory access, and device nonidealities. Herein, a fully binarized GCN (BGCN) accelerator based on computational resistive random‐access memory (RRAM) through software–hardware codesign is presented. The essential operations including aggregation and combination in GCN are implemented on the RRAM crossbar arrays with cooperation between multiply‐and‐accumulation and content‐addressable memory operations. By leveraging the model quantization and IMC on the RRAM, the BGCN accelerator demonstrates less RRAM usage, high robustness to the device variations, high energy efficiency, and comparable classification accuracy compared to the current state‐of‐the‐art GCN accelerators on both graph classification task using the MUTAG and PTC datasets and node classification task using the Cora and CiteSeer datasets. These results provide a promising approach for edge intelligent systems to efficiently process graph‐structured data.

ReARTSim: an ReRAM ARray Transient Simulator with GPU optimized runtime acceleration

The demand for computation driven by machine learning and deep learning applications has experienced exponential growth over the past five years (Sevilla et al 2022 2022 International Joint Conference on Neural Networks (IJCNN) (IEEE) pp 1-8), leading to a significant surge in computing hardware products. Meanwhile, this rapid increase has exacerbated the memory wall bottleneck within mainstream Von Neumann architectures (Hennessy and Patterson et al 2011 Computer architecture: a quantitative approach (Elsevier)). For instance, NVIDIA graphical processing units (GPUs) have gained nearly a 200x increase in fp32 computing power, transitioning from P100 to H100 in the last five years (NVIDIA Tesla P100 2023 (www.nvidia.com/en-us/data-center/tesla-p100/); NVIDIA H100 Tensor Core GPU 2023 (www.nvidia.com/en-us/data-center/h100/)), accompanied by a mere 8x scaling in memory bandwidth. Addressing the need to mitigate data movement challenges, process-in-memory designs, especially resistive random-access memory (ReRAM)-based solutions, have emerged as compelling candidates (Verma et al 2019 IEEE Solid-State Circuits Mag. 11 43–55; Sze et al 2017 Proc. IEEE 105 2295–329). However, this shift in hardware design poses distinct challenges at the design phase, given the limitations of existing hardware design tools. Popular design tools today can be used to characterize analog behavior via SPICE tools (PrimeSim HSPICE 2023 (www.synopsys.com/implementation-and-signoff/ams-simulation/primesim-hspice.html)), system and logical behavior using Verilog tools (VCS 2023 (www.synopsys.com/verification/simulation/vcs.html)), and mixed signal behavior through toolbox like CPPSIM (Meninger 2023 (www.cppsim.org/Tutorials/wideband_fracn_tutorial.pdf)). Nonetheless, the design of in-memory computing systems, especially those involving non-CMOS devices, presents a unique need for characterizing mixed-signal computing behavior across a large number of cells within a memory bank. This requirement falls beyond the scope of conventional design tools. In this paper, we bridge this gap by introducing the ReARTSim framework—a GPU-accelerated mixed-signal transient simulator for analyzing ReRAM crossbar array. This tool facilitates the characterization of analog circuit and device behavior on a large scale, while also providing enhanced simulation performance for complex algorithm analysis, sign-off, and verification.

ReQUSA: a novel ReRAM-based hardware accelerator architecture for high-speed quantum computer simulation

Quantum computers are currently regarded as an emerging computing technology that can solve problems more quickly than classical computers. However, since constructing a general quantum computer is technically difficult, quantum computer simulation has been used instead of real quantum computers. Simulating quantum computers on classical computers is challenging because the time and resources required for the vectormatrix multiplication (VMM) increase exponentially with the number of qubits. This paper proposes a new accelerator architecture called ReQUSA that leverages resistive random access memory (ReRAM) to accelerate the quantum computer simulation. The ReQUSA employs a ReRAM crossbar array structure, which is specialized for implementing the VMM, and a realized state method for reduced VMM operation. To the best of our knowledge, ReRAM-based accelerators for quantum computer simulators has not been previously reported. Here, we describe the hardware design of the architecture and compare the performances (hardware resource, simulation time, and accuracy) of our accelerator with those of current quantum computer simulators (QuEST, QPlayer, and Qiskit). On average, our proposed architecture reduced the simulation times by factors of ×104 and ×103 (×102) on average from those of QuEST and QPlayer (also Qiskit), respectively. In addition, our architecture achieved 99% accuracy in 16-bit fixed-point data representation.

Insulator Metal Transition-Based Selector in Crossbar Memory Arrays

This article investigates resistive random access memory (ReRAM) crossbar memory arrays, which is a notable development in non-volatile memory technology. We highlight ReRAM’s competitive edge over NAND, NOR Flash, and phase-change memory (PCM), particularly in terms of endurance, speed, and energy efficiency. This paper focuses on the architecture of crossbar arrays, where memristive devices are positioned at intersecting metal wires. We emphasize the unique resistive switching mechanisms of memristors and the challenges of sneak path currents and delve into the roles and configurations of selectors, particularly focusing on the one-selector one-resistor (1S1R) architecture with an insulator–metal transition (IMT) based selector. We use SPICE simulations based on defined models to examine a 3 × 3 1S1R ReRAM array with vanadium dioxide selectors and titanium dioxide film memristors, assessing the impact of ambient temperature and critical IMT temperatures on array performance. We highlight the operational regions of low resistive state (LRS) and high resistive state (HRS), providing insights into the electrical behavior of these components under various conditions. Lastly, we demonstrate the impact of selector presence on sneak path currents. This research contributes to the overall understanding of ReRAM crossbar arrays integrated with IMT material-based selectors.

A physics-based compact model of thermal resistance in RRAMs

In this paper, we present a physics-based compact model of thermal resistance in Resistive Random-Access Memory (RRAM) devices considering (a) thermal properties of electrode materials, (b) temperature-induced variations in material thermal conductivity, and (c) parasitic heat losses through the oxide surrounding the filament. Fully coupled numerical simulation data obtained from the self-consistent solution of the drift-diffusion equation (for vacancy migration), carrier continuity equation (for electronic conduction), and the Fourier heat diffusion equation (accounting for electro-thermal heating) has been used for developing and validating the proposed model. Excellent agreement is observed between the model and the numerical simulation results. The proposed model is validated with experimentally measured conductive filament temperature obtained from two different temperature sensing methods. Finally, we present a modeling framework to estimate the potential thermal-cross talk in RRAM crossbar arrays. Our results illustrate that a power-dependent thermal resistance model significantly improves the accuracy in estimating the electro-thermal effects in RRAMs.

Vertically oriented 2D layered perovskite-based resistive random access memory (ReRAM) crossbar arrays

Resistive random-access memory (ReRAM) based on 2D layered perovskites have recently emerged as a new class of data storage devices, and the switching materials used in these devices have attracted extensive attention in recent years. There is a certain amount of variability in ReRAM owing to the stochastic nature of the formation and rupture of the conductive path, causing failure of the switching operation in the memory array. In this work, we fabricated both randomly and vertically oriented PEA2MA4Pb5I16-based ReRAM devices and conducted comparative studies of their switching variability. We demonstrated the formation of vertical alignment in PEA2MA4Pb5I16 films using methylammonium chloride (MACl) additives. Thereafter, the fabrication of 8×8 ReRAM crossbar arrays by patterning perovskites (∼600μm) and electrodes (line width ∼ 5 μm) using photolithography and e-beam lithography is demonstrated, and the resistive switching characteristics of both samples are discussed. Both cycle-to-cycle and device-to-device switching uniformity were improved in the vertically oriented film-based ReRAM devices, which can be explained by the narrow distribution of the energy barriers for VI migration owing to the vertically aligned charge transport path. These results suggest that vertically oriented 2D layered perovskites have great potential in ReRAM devices with improved switching uniformity.

Sneak Path Interference-Aware Adaptive Detection and Decoding for Resistive Memory Arrays

Resistive random-access memory (ReRAM) is an emerging non-volatile memory technology for high-density and high-speed data storage. However, the sneak path interference (SPI) occurred in the ReRAM crossbar array seriously affects its data recovery performance. In this letter, we first propose a quantized channel model of ReRAM, based on which we design both the one-bit and multi-bit channel quantizers by maximizing the mutual information of the channel. A key channel parameter that affects the quantizer design is the sneak path occurrence probability (SPOP) of the memory cell. We first use the average SPOP calculated statistically to design the quantizer, which leads to the same channel detector for different memory arrays. We then adopt the SPOP estimated separately for each memory array for the quantizer design, which is generated by an effective channel estimator and through an iterative detection and decoding scheme for the ReRAM channel. This results in an array-level SPI-aware adaptive detection and decoding approach. Moreover, since there is a strong correlation of the SPI that affects memory cells in the same rows/columns than that affecting cells in different rows/columns, we further derive a column-level scheme which outperforms the array-level scheme. We also propose a channel decomposition method that enables effective ways for theoretically analyzing the ReRAM channel. Simulation results show that the proposed SPI-aware adaptive detection and decoding schemes can approach the ideal performance with three quantization bits, with only one decoding iteration.

Conductance-Aware Quantization Based on Minimum Error Substitution for Non-Linear-Conductance-State Tolerance in Neural Computing Systems.

Emerging resistive random-access memory (ReRAM) has demonstrated great potential in the achievement of the in-memory computing paradigm to overcome the well-known “memory wall” in current von Neumann architecture. The ReRAM crossbar array (RCA) is a promising circuit structure to accelerate the vital multiplication-and-accumulation (MAC) operations in deep neural networks (DNN). However, due to the nonlinear distribution of conductance levels in ReRAM, a large deviation exists in the mapping process when the trained weights that are quantized by linear relationships are directly mapped to the nonlinear conductance values from the realistic ReRAM device. This deviation degrades the inference accuracy of the RCA-based DNN. In this paper, we propose a minimum error substitution based on a conductance-aware quantization method to eliminate the deviation in the mapping process from the weights to the actual conductance values. The method is suitable for multiple ReRAM devices with different non-linear conductance distribution and is also immune to the device variation. The simulation results on LeNet5, AlexNet and VGG16 demonstrate that this method can vastly rescue the accuracy degradation from the non-linear resistance distribution of ReRAM devices compared to the linear quantization method.

Long Short-Term Memory Implementation Exploiting Passive RRAM Crossbar Array

The ever-increasing demand to extract temporal correlations across sequential data and perform context-based learning in this era of big data has led to the development of long short-term memory (LSTM) networks. Furthermore, there is an urgent need to perform these time-series data-dependent applications including speech/video processing and recognition, language modelling and translation, etc. on compact internet-of-things (IoT) edge devices with limited energy. To this end, in this work, for the first time, we propose an extremely area- and energy-efficient LSTM network implementation exploiting the passive resistive random access memory (RRAM) crossbar array. We developed a hardware-aware LSTM network simulation framework and performed an extensive analysis of the proposed LSTM implementation considering the non-ideal hardware artifacts such as spatial (device-to-device) and temporal variations, non-linearity, noise, etc. utilizing an experimentally calibrated comprehensive phenomenological model for passive RRAM crossbar array. Our results indicate that the proposed passive RRAM crossbar-based LSTM network implementation not only outperforms the prior digital and active 1T-1R crossbar-based LSTM implementations by more than three orders of magnitude in terms of area and two orders of magnitude in terms of training energy for identical network accuracy, but also exhibits robustness against spatial and temporal variations and noise, and a faster convergence rate. Our work may provide the incentive for experimental realization of LSTM networks on passive RRAM crossbar arrays.

Oscillation neuron based on a low-variability threshold switching device for high-performance neuromorphic computing

Low-power and low-variability artificial neuronal devices are highly desired for high-performance neuromorphic computing. In this paper, an oscillation neuron based on a low-variability Ag nanodots (NDs) threshold switching (TS) device with low operation voltage, large on/off ratio and high uniformity is presented. Measurement results indicate that this neuron demonstrates self-oscillation behavior under applied voltages as low as 1 V. The oscillation frequency increases with the applied voltage pulse amplitude and decreases with the load resistance. It can then be used to evaluate the resistive random-access memory (RRAM) synaptic weights accurately when the oscillation neuron is connected to the output of the RRAM crossbar array for neuromorphic computing. Meanwhile, simulation results show that a large RRAM crossbar array (> 128 × 128) can be supported by our oscillation neuron owing to the high on/off ratio (> 108) of Ag NDs TS device. Moreover, the high uniformity of the Ag NDs TS device helps improve the distribution of the output frequency and suppress the degradation of neural network recognition accuracy (< 1%). Therefore, the developed oscillation neuron based on the Ag NDs TS device shows great potential for future neuromorphic computing applications.

Neural-PIM: Efficient Processing-In-Memory with Neural Approximation of Peripherals

Processing-in-memory (PIM) architectures have demonstrated great potential in accelerating numerous deep learning tasks. Particularly, resistive random-access memory (RRAM) devices provide a promising hardware substrate to build PIM accelerators due to their abilities to realize efficient in-situ vector-matrix multiplications (VMMs). However, existing PIM accelerators suffer from frequent and energy-intensive analog-to-digital (A/D) conversions, severely limiting their performance. This paper presents a new PIM architecture to efficiently accelerate deep learning tasks by minimizing the required A/D conversions with analog accumulation and neural approximated peripheral circuits. We first characterize the different dataflows employed by existing PIM accelerators, based on which a new dataflow is proposed to remarkably reduce the required A/D conversions for VMMs by extending shift and add (S+A) operations into the analog domain before the final quantizations. We then leverage a neural approximation method to design both analog accumulation circuits (S+A) and quantization circuits (ADCs) with RRAM crossbar arrays in a highly-efficient manner. Finally, we apply them to build an RRAM-based PIM accelerator (i.e., \textbf{Neural-PIM}) upon the proposed analog dataflow and evaluate its system-level performance. Evaluations on different benchmarks demonstrate that Neural-PIM can improve energy efficiency by 5.36x (1.73x) and speed up throughput by 3.43x (1.59x) without losing accuracy, compared to the state-of-the-art RRAM-based PIM accelerators, i.e., ISAAC (CASCADE).

A Crossbar-Based In-Memory Computing Architecture

To address the von Neumann bottleneck that leads to both energy and speed degradations, in-memory processing architectures have been proposed as a promising alternative for future computing applications. In this paper, we present an in-memory computing system based on resistive random-access memory (RRAM) crossbar arrays that is reconfigurable and can potentially perform parallel and general computing tasks. The system consists of small look-up tables (LUTs), a memory block, and two search auxiliary blocks, all implemented in the same RRAM crossbar array. External data access and data conversions are eliminated to allow operations fully in-memory. Details of addition, AND logic and multiplication operations are discussed on the basis of search and writeback steps. A compact instruction set consisting of 10 instructions is demonstrated on this architecture through circuit level simulations. Performance evaluations show that the proposed in-memory computing architecture is suitable for handling data-intensive problems. The average power consumption of the crossbar chip is estimated to be 45μW.

A Fully Integrated Reprogrammable CMOS-RRAM Compute-in-Memory Coprocessor for Neuromorphic Applications

Analog compute-in-memory with resistive random access memory (RRAM) devices promises to overcome the data movement bottleneck in data-intensive artificial intelligence (AI) and machine learning. RRAM crossbar arrays improve the efficiency of vector-matrix multiplications (VMMs), which is a vital operation in these applications. The prototype IC is the first complete, fully integrated analog-RRAM CMOS coprocessor. This article focuses on the digital and analog circuitry that supports efficient and flexible RRAM-based computation. A passive $54\times108$ RRAM crossbar array performs VMM in the analog domain. Specialized mixed-signal circuits stimulate and read the outputs of the RRAM crossbar. The single-chip CMOS prototype includes a reduced instruction set computer (RISC) processor interfaced to a memory-mapped mixed-signal core. In the mixed-signal core, ADCs and DACs interface with the passive RRAM crossbar. The RISC processor controls the mixed-signal circuits and the algorithm data path. The system is fully programmable and supports forward and backward propagation. As proof of concept, a fully integrated 0.18- $\mu \text{m}$ CMOS prototype with a postprocessed RRAM array demonstrates several key functions of machine learning, including online learning. The mixed-signal core consumes 64 mW at an operating frequency of 148 MHz. The total system power consumption considering the mixed-signal circuitry, the digital processor, and the passive RRAM array is 307 mW. The maximum theoretical throughput is 2.6 GOPS at an efficiency of 8.5 GOPS/W.

A Compact Model of Analog RRAM With Device and Array Nonideal Effects for Neuromorphic Systems

The parallelism and analog computing features of neuromorphic systems bring great challenges in developing a compact model of analog resistive random access memory (RRAM). In this article, we develop a physics-based compact model for analog RRAM devices and crossbar array. Nonideal effects of analog RRAM device, such as variability, ${I}$ – ${V}$ nonlinearity, programming nonlinearity and asymmetry, and tuning voltage sensitivity, are modeled and verified with the statistical data measured from RRAM array. Modeling of parallel-vector-matrix-multiplication and weight update process on RRAM crossbars with interconnect resistance enables fast and accurate estimation of the training accuracy. Benchmarks of neural networks under different hardware conditions validate the functionality of the proposed model. This model can provide valuable design guidelines for a practical neuromorphic system with high performance and computing accuracy.

Reduction of thermal disturbances in 3D 1S1R RRAM crossbar arrays for neuromorphic computing

Recently, analog behaviors in resistive random-access memory (RRAM) have been proposed as a promising circuit implementation for neuromorphic computing. However, to date, there have been relatively few studies on the issue of resistance degradation (computing weight) induced by thermal disturbances in high-density RRAM arrays. In this paper, thermal simulations of a three-dimensional (3D) one-selector, one-resistor (1S1R) RRAM crossbar array for neuromorphic computing were performed using the COMSOL Multiphysics software. Four factors influencing the thermal disturbance problem were considered: distances between adjacent devices, thermal conductivity of the insulating material, the resistance of the low-resistance state (LRS) of the RRAM, and the programming speed. Thermal disturbances were found to be more severe when the device spacing was less than 200 nm. Higher thermal conductivities of the insulating material, a larger LRS resistance, and a faster operational speed were found to effectively reduce the thermal disturbances in 3D 1S1R RRAM crossbar arrays. The use of sub-nanosecond pulses solved the thermal issues to the greatest extent.

Selectorless Oxide-Based Resistive Switching Memory with Nonuniform Dielectric for Low Power Crossbar Array Applications

Selectorless resistive random-access memory (RRAM) with self-rectifying behavior is presented as a way to suppress sneak path current without an additional transistor or switch device integration, which is beneficial for reducing the cost and complexity of crossbar array fabrication. In this work, selectorless 1R-only graphite-based memristor devices have been demonstrated by utilizing the nonlinear (NL) resistive switching (RS) characteristics with well-designed operation conditions. The early stage of cycling observation as seasoning effect is investigated, while the wear-out mechanism is also discussed for the selectorless 1R-only RRAM crossbar array for high storage class memory applications.

Near‐Infrared Annihilation of Conductive Filaments in Quasiplane MoSe2/Bi2Se3 Nanosheets for Mimicking Heterosynaptic Plasticity

It is desirable to imitate synaptic functionality to break through the memory wall in traditional von Neumann architecture. Modulating heterosynaptic plasticity between pre- and postneurons by another modulatory interneuron ensures the computing system to display more complicated functions. Optoelectronic devices facilitate the inspiration for high-performance artificial heterosynaptic systems. Nevertheless, the utilization of near-infrared (NIR) irradiation to act as a modulatory terminal for heterosynaptic plasticity emulation has not yet been realized. Here, an NIR resistive random access memory (RRAM) is reported, based on quasiplane MoSe2 /Bi2 Se3 heterostructure in which the anomalous NIR threshold switching and NIR reset operation are realized. Furthermore, it is shown that such an NIR irradiation can be employed as a modulatory terminal to emulate heterosynaptic plasticity. The reconfigurable 2D image recognition is also demonstrated by an RRAM crossbar array. NIR annihilation effect in quasiplane MoSe2 /Bi2 Se3 nanosheets may open a path toward optical-modulated in-memory computing and artificial retinal prostheses.

Bidirectional Database Storage and SQL Query Exploiting RRAM-Based Process-in-Memory Structure

With the coming of the “Big Data” era, a high-energy-efficiency database is demanded for the Internet of things (IoT) application scenarios. The emerging Resistive Random Access Memory (RRAM) has been considered as an energy-efficient replacement of DRAM for next-generation main memory. In this article, we propose an RRAM-based SQL query unit with process-in-memory (PIM) characteristics. A bidirectional storage structure for a database in RRAM crossbar array is proposed that avoids redundant data transfer to cache and reduces cache miss rate compared with the storage method in DRAM for an in-memory database. The proposed RRAM-based SQL query unit can support a representative subset of SQL queries in memory and thus can further reduce the data transfer cost. The corresponding query optimization method is proposed to fully utilize the PIM characteristics. Simulation results show that the energy efficiency of the proposed RRAM-based SQL query unit is increased by 4 to 6 orders of magnitudes compared with the traditional architecture.