Memristor Crossbar Research Articles

Enabling Energy-Efficient Deployment of Large Language Models on Memristor Crossbar: A Synergy of Large and Small.

Large language models (LLMs) have garnered substantial attention due to their promising applications in diverse domains. Nevertheless, the increasing size of LLMs comes with a significant surge in the computational requirements for training and deployment. Memristor crossbars have emerged as a promising solution, which demonstrated a small footprint and remarkably high energy efficiency in computer vision (CV) models. Memristors possess higher density compared to conventional memory technologies, making them highly suitable for effectively managing the extreme model size associated with LLMs. However, deploying LLMs on memristor crossbars faces three major challenges. Firstly, the size of LLMs increases rapidly, already surpassing the capabilities of state-of-the-art memristor chips. Secondly, LLMs often incorporate multi-head attention blocks, which involve non-weight stationary multiplications that traditional memristor crossbars cannot support. Third, while memristor crossbars excel at performing linear operations, they are not capable of executing complex nonlinear operations in LLM such as softmax and layer normalization. To address these challenges, we present a novel architecture for the memristor crossbar that enables the deployment of state-of-the-art LLM on a single chip or package, eliminating the energy and time inefficiencies associated with off-chip communication. Our testing on BERT Large showed negligible accuracy loss. Compared to traditional memristor crossbars, our architecture achieves enhancements of up to 39× in area overhead and 18× in energy consumption. Compared to modern TPU/GPU systems, our architecture demonstrates at least a 68× reduction in the area-delay product and a significant 69% energy consumption reduction.

AutoGMap: Learning to Map Large-Scale Sparse Graphs on Memristive Crossbars.

The sparse representation of graphs has shown great potential for accelerating the computation of graph applications (e.g., social networks and knowledge graphs) on traditional computing architectures (CPU, GPU, or TPU). But, the exploration of large-scale sparse graph computing on processing-in-memory (PIM) platforms (typically with memristive crossbars) is still in its infancy. To implement the computation or storage of large-scale or batch graphs on memristive crossbars, a natural assumption is that a large-scale crossbar is demanded, but with low utilization. Some recent works question this assumption; to avoid the waste of storage and computational resource, the fixed-size or progressively scheduled "block partition" schemes are proposed. However, these methods are coarse-grained or static and are not effectively sparsity-aware. This work proposes the dynamic sparsity-aware mapping scheme generating method that models the problem with a sequential decision-making model, and optimizes it by reinforcement learning (RL) algorithm (REINFORCE). Our generating model [long short-term memory (LSTM), combined with the dynamic-fill scheme] generates remarkable mapping performance on the small-scale graph/matrix data (complete mapping costs 43% area of the original matrix) and two large-scale matrix data (costing 22.5% area on qh882 and 17.1% area on qh1484). Our method may be extended to sparse graph computing on other PIM architectures, not limited to the memristive device-based platforms.

Memristive Crossbar Array-Based Probabilistic Graph Modeling.

Modern graph datasets with structural complexity and uncertainties due to incomplete information or data variability require advanced modeling techniques beyond conventional graph models. This study introduces a memristive crossbar array (CBA)-based probabilistic graph model (C-PGM) utilizing Cu0.3Te0.7/HfO2/Pt memristors, which exhibit probabilistic switching, self-rectifying, and memory characteristics. C-PGM addresses the complexities and uncertainties inherent in structural graph data across various domains, leveraging the probabilistic nature of memristors. C-PGM relies on the device-to-device variation across multiple memristive CBAs, overcoming the limitations of previous approaches that rely on sequential operations, which are slower and have a reliability concern due to repeated switching. This new approach enables the fast processing and massive implementation of probabilistic units at the expense of chip area. In this study, the hardware-based C-PGM feasibly expresses small-scale probabilistic graphs and shows minimal error in aggregate probability calculations. The probability calculation capabilities of C-PGM are applied to steady-state estimation and the PageRank algorithm, which is implemented on a simulated large-scale C-PGM. The C-PGM-based steady-state estimation and PageRank algorithm demonstrate comparable accuracy to conventional methods while significantly reducing computational costs.

Optimization of Memristor Crossbar’s Mapping Using Lagrange Multiplier Method and Genetic Algorithm for Reducing Crossbar’s Area and Delay Time

Memristor crossbars offer promising low-power and parallel processing capabilities, making them efficient for implementing convolutional neural networks (CNNs) in terms of delay time, area, etc. However, mapping large CNN models like ResNet-18, ResNet-34, VGG-Net, etc., onto memristor crossbars is challenging due to the line resistance problem limiting crossbar size. This necessitates partitioning full-image convolution into sub-image convolution. To do so, an optimized mapping of memristor crossbars should be considered to divide full-image convolution into multiple crossbars. With limited crossbar resources, especially in edge devices, it is crucial to optimize the crossbar allocation per layer to minimize the hardware resource in term of crossbar area, delay time, and area–delay product. This paper explores three optimization scenarios: (1) optimizing total delay time under a crossbar’s area constraint, (2) optimizing total crossbar area with a crossbar’s delay time constraint, and (3) optimizing a crossbar’s area–delay-time product without constraints. The Lagrange multiplier method is employed for the constrained cases 1 and 2. For the unconstrained case 3, a genetic algorithm (GA) is used to optimize the area–delay-time product. Simulation results demonstrate that the optimization can have significant improvements over the unoptimized results. When VGG-Net is simulated, the optimization can show about 20% reduction in delay time for case 1 and 22% area reduction for case 2. Case 3 highlights the benefits of optimizing the crossbar utilization ratio for minimizing the area–delay-time product. The proposed optimization strategies can substantially enhance the neural network’s performance of memristor crossbar-based processing-in-memory architectures, especially for resource-constrained edge computing platforms.

Random Projection‐Based Locality‐Sensitive Hashing in a Memristor Crossbar Array with Stochasticity for Sparse Self‐Attention‐Based Transformer

AbstractSelf‐attention mechanism is critically central to the state‐of‐the‐art transformer models. Because the standard full self‐attention has quadratic complexity with respect to the input's length L, resulting in prohibitively large memory for very long sequences, sparse self‐attention enabled by random projection (RP)‐based locality‐sensitive hashing (LSH) has recently been proposed to reduce the complexity to O(L log L). However, in current digital computing hardware with a von Neumann architecture, RP, which is essentially a matrix multiplication operation, incurs unavoidable time and energy‐consuming data shuttling between off‐chip memory and processing units. In addition, it is known that digital computers simply cannot generate provably random numbers. With the emerging analog memristive technology, it is shown that it is feasible to harness the intrinsic device‐to‐device variability in the memristor crossbar array for implementing the RP matrix and perform RP‐LSH computation in memory. On this basis, sequence prediction tasks are performed with a sparse self‐attention‐based Transformer in a hybrid software‐hardware approach, achieving a testing accuracy over 70% with much less computational complexity. By further harnessing the cycle‐to‐cycle variability for multi‐round hashing, 12% increase in the testing accuracy is demonstrated. This work extends the range of applications of memristor crossbar arrays to the state‐of‐the‐art large language models (LLMs).

An efficient full-size convolutional computing method based on memristor crossbar

Modern artificial intelligence systems based on neural networks need to perform a large number of repeated parallel operations quickly. Without hardware acceleration, they cannot achieve effectiveness and availability. Memristor-based neuromorphic computing systems are one of the promising hardware acceleration strategies. In this paper, we propose a full-size convolution algorithm (FSCA) for the memristor crossbar, which can store both the input matrix and the convolution kernel and map the convolution kernel to the entire input matrix in a full parallel method during the computation. This method dramatically increases the convolutional kernel computations in a single operation, and the number of operations no longer increases with the input matrix size. Then a bidirectional pulse control switch integrated with two extra memristors into CMOS devices is designed to effectively suppress the leakage current problem in the row and column directions of the existing memristor crossbar. The spice circuit simulation system is built to verify that the design convolutional computation algorithm can extract the feature map of the entire input matrix after only a few operations in the memristor crossbar-based computational circuit. System-level simulations based on the MNIST classification task verify that the designed algorithm and circuit can effectively implement Gabor filtering, allowing the multilayer neural network to improve the classification task recognition accuracy to 98.25% with a 26.2% reduction in network parameters. In comparison, the network can even effectively immunize various non-idealities of the memristive synaptic within 30%.

Overshoot‐Suppressed Memristor Crossbar Array with High Yield by AlOx Oxidation for Neuromorphic System

AbstractThere is a need to design a hardware synapse array appropriate for enhancing the efficiency of neuromorphic computing systems while minimizing energy consumption. This study introduces a memristor device with an AlOx overshoot suppression layer (A‐OSL) to achieve a self‐compliance effect. By optimizing each cell within the 16 × 16 crossbar array, synaptic devices are successfully fabricated with reliable characteristics and 3‐bit multilevel capabilities. In addition, the oxygen composition of TiOx and the annealing conditions are optimized to reduce the forming voltage and minimize the variation in the switching voltage. As a result, stable forming‐free characteristics are obtained through A‐OSL insertion, a reduction in forming voltage, and TiOx oxygen composition optimization. Also, target weights are accurately transferred to the A‐OSL memristor crossbar array and conducted the inference process by applying spike signals to the array following the designated time step. The spiking neural network (SNN) is demonstrated by measuring vector‐matrix multiplication (VMM) of the 16 × 16 crossbar array. The VMM results exhibit a classification accuracy of 90.80% for the MNIST dataset, which is close to the accuracy achieved by software‐based approaches, amounting to 91.85%.

Harnessing nonlinear conductive characteristic of TiO2/HfO2 memristor crossbar for implementing parallel vector–matrix multiplication

Memristor crossbar arrays are expected to achieve highly energy-efficient neuromorphic computing via implementing parallel vector–matrix multiplication (VMM) in situ. The similarities between memristors and neural synapses offer opportunities for realizing hardware-based brain-inspired computing, such as spike neural networks. However, the nonlinear I–V characteristics of the memristors limit the implementation of parallel VMM on passive memristor crossbar arrays. In our work, we propose to utilize differential conductance as a synaptic weight to implement linear VMM operations on a passive memristor array in parallel. We fabricated a TiO2/HfO2 memristor crossbar array, in which differential-conductance-based synaptic weight exhibits plasticity, nonvolatility, multi-states, and tunable ON/OFF ratio. The noise-dependent accuracy performance of VMM operations based on the proposed approach was evaluated, offering an optimization guideline. Furthermore, we demonstrated a spike neural network circuit capable of processing small spiking signals through the differential-conductance-based synapses. The experimental results showcase effective space-coded and time-coded spike pattern recognition. Importantly, our work opens up new possibilities for the development of passive memristor arrays, leading to increased energy and area efficiency in brain-inspired chips.

1-Selector 1-Memristor Configuration with Multifunctional a-IGZO Memristive Devices Fabricated at Room Temperature.

Serving as neuromorphic hardware accelerators, memristors play a crucial role in large-scale neuromorphic computing. Herein, two-terminal memristors utilizing amorphous indium-gallium-zinc oxide (a-IGZO) are fabricated through room-temperature sputtering. The electrical characteristics of these memristors are effectively modulated by varying the oxygen flow during the deposition process. The optimized a-IGZO memristor, fabricated under 3 sccm oxygen flow, presents a 5 × 103 ratio between its high- and low-resistance states, which can be maintained over 1 × 104 s with minimal degradation. Meanwhile, desirable properties such as electroforming-free and self-compliance, crucial for low-energy consumption, are also obtained in the a-IGZO memristor. Moreover, analog conductance switching is observed, demonstrating an interface-type behavior, as evidenced by its device-size-dependent performance. The coexistence of negative differential resistance with analog switching is attributed to the migration of oxygen vacancies and the trapping/detrapping of charges. Furthermore, the device demonstrates optical storage capabilities by exploiting the optical properties of a-IGZO, which can stably operate for up to 50 sweep cycles. Various synaptic functions have been demonstrated, including paired-pulse facilitation and spike-timing-dependent plasticity. These functionalities contribute to a simulated recognition accuracy of 90% for handwritten digits. Importantly, a one-selector one-memristor (1S1M) architecture is successfully constructed at room temperature by integrating a-IGZO memristor on a TaOx-based selector. This architecture exhibits a 107 on/off ratio, demonstrating its potential to suppress sneak currents among adjacent units in a memristor crossbar.

Memristor-CMOS Hybrid Circuits Implementing Event-Driven Neural Networks for Dynamic Vision Sensor Camera.

For processing streaming events from a Dynamic Vision Sensor camera, two types of neural networks can be considered. One are spiking neural networks, where simple spike-based computation is suitable for low-power consumption, but the discontinuity in spikes can make the training complicated in terms of hardware. The other one are digital Complementary Metal Oxide Semiconductor (CMOS)-based neural networks that can be trained directly using the normal backpropagation algorithm. However, the hardware and energy overhead can be significantly large, because all streaming events must be accumulated and converted into histogram data, which requires a large amount of memory such as SRAM. In this paper, to combine the spike-based operation with the normal backpropagation algorithm, memristor-CMOS hybrid circuits are proposed for implementing event-driven neural networks in hardware. The proposed hybrid circuits are composed of input neurons, synaptic crossbars, hidden/output neurons, and a neural network's controller. Firstly, the input neurons perform preprocessing for the DVS camera's events. The events are converted to histogram data using very simple memristor-based latches in the input neurons. After preprocessing the events, the converted histogram data are delivered to an ANN implemented using synaptic memristor crossbars. The memristor crossbars can perform low-power Multiply-Accumulate (MAC) calculations according to the memristor's current-voltage relationship. The hidden and output neurons can convert the crossbar's column currents to the output voltages according to the Rectified Linear Unit (ReLU) activation function. The neural network's controller adjusts the MAC calculation frequency according to the workload of the event computation. Moreover, the controller can disable the MAC calculation clock automatically to minimize unnecessary power consumption. The proposed hybrid circuits have been verified by circuit simulation for several event-based datasets such as POKER-DVS and MNIST-DVS. The circuit simulation results indicate that the neural network's performance proposed in this paper is degraded by as low as 0.5% while saving as much as 79% in power consumption for POKER-DVS. The recognition rate of the proposed scheme is lower by 0.75% compared to the conventional one, for the MNIST-DVS dataset. In spite of this little loss, the power consumption can be reduced by as much as 75% for the proposed scheme.

A memristor crossbar based on a novel ternary memristor model

A memristor crossbar based on a novel ternary memristor model

Self-Rectifying All-Optical Modulated Optoelectronic Multistates Memristor Crossbar Array for Neuromorphic Computing.

Researching optoelectronic memristors capable of integrating sensory and processing functions is essential for advancing the development of efficient neuromorphic vision. Here, we experimentally demonstrated an all-optical controlled and self-rectifying optoelectronic memristor (OEM) crossbar array with the function of multilevel storage under light stimuli. The NiO/TiO2 device exhibits an ultrahigh (>104) rectifying ratio (RR) thus overcoming the presence of sneak current. The reversible conductance modulation without electric signal involvement provides a novel way to realize ultrafast information processing. The proposed OEM array realized synaptic functions observed in the human brain, including long-term potentiation (LTP), long-term depression (LTD), paired-pulse facilitation (PPF), the transition from short-term memory (STM) to long-term memory (LTM), and learning experience behaviors successfully. The authors present a novel OEM crossbar that possesses complete light-modulation capabilities, potentially advancing the future development of efficient neuromorphic vision.

Implementation of Convolutional Neural Networks in Memristor Crossbar Arrays with Binary Activation and Weight Quantization.

We propose a hardware-friendly architecture of a convolutional neural network using a 32 × 32 memristor crossbar array having an overshoot suppression layer. The gradual switching characteristics in both set and reset operations enable the implementation of a 3-bit multilevel operation in a whole array that can be utilized as 16 kernels. Moreover, a binary activation function mapped to the read voltage and ground is introduced to evaluate the result of training with a boundary of 0.5 and its estimated gradient. Additionally, we adopt a fixed kernel method, where inputs are sequentially applied to a crossbar array with a differential memristor pair scheme, reducing unused cell waste. The binary activation has robust characteristics against device state variations, and a neuron circuit is experimentally demonstrated on a customized breadboard. Thanks to the analogue switching characteristics of the memristor device, the accurate vector-matrix multiplication (VMM) operations can be experimentally demonstrated by combining sequential inputs and the weights obtained through tuning operations in the crossbar array. In addition, the feature images extracted by VMM during the hardware inference operations on 100 test samples are classified, and the classification performance by off-chip training is compared with the software results. Finally, inference results depending on the tolerance are statistically verified through several tuning cycles.

A Genetic Algorithm Accelerator Based on Memristive Crossbar Array for Massively Parallel Computation

A Genetic Algorithm Accelerator Based on Memristive Crossbar Array for Massively Parallel Computation

ClipFormer: Key-Value Clipping of Transformers on Memristive Crossbars for Write Noise Mitigation

ClipFormer: Key-Value Clipping of Transformers on Memristive Crossbars for Write Noise Mitigation

Securing Binarized Neural Networks via PUF-Based Key Management in Memristive Crossbar Arrays

Securing Binarized Neural Networks via PUF-Based Key Management in Memristive Crossbar Arrays

Threshold Voltage based Dual Memristor Crossbar PUF

IoT security plays a crucial role where smart devices are interconnected and one of these can give an intrusion access to the network if they are not secured. Physical unclonable function (PUF) comes in handy as an emerging lightweight hardware security primitive for key management and device authentication. In this research work, we propose a new architecture of PUF hardware based on memristor threshold voltage variation. Memristor manufacturing variations are exploited for the threshold voltage variation which makes the device unique. The selection from the proposed crossbars forms two connected memristors. The selections connect the formed voltage divider network to the input and ground. The output is probed at the middle node, which is the selected rows of the two crossbars. Voltage (challenge) is applied at selected pair of memristors through the corresponding columns and rows, and output (response) is taken from the selected rows. We believe this work will set a new direction for the researchers to explore manufacturing errors for PUF applications.

Memristive patch attention neural network for facial expression recognition and edge computing.

Facial expression recognition has made a significant progress as a result of the advent of more and more convolutional neural networks (CNN). However, with the improvement of CNN, the models continues to get deeper and larger so as to a greater focus on the high-level features of the image and the low-level features tend to be lost. Because of the reason above, the dependence of low-level features between different areas of the face often cannot be summarized. In response to this problem, we propose a novel network based on the CNN model. To extract long-range dependencies of low-level features, multiple attention mechanisms has been introduced into the network. In this paper, the patch attention mechanism is designed to obtain the dependence between low-level features of facial expressions firstly. After fusion, the feature maps are input to the backbone network incorporating convolutional block attention module (CBAM) to enhance the feature extraction ability and improve the accuracy of facial expression recognition, and achieve competitive results on three datasets CK+(98.10%), JAFFE (95.12%) and FER2013 (73.50%). Further, according to the PA Net designed in this paper,a hardware friendly implementation scheme is designed based on memristor crossbars, which is expected to provide a software and hardware co-design scheme for edge computing of personal and wearable electronic products.

Parallel Density‐Based Spatial Clustering with Dual‐Functional Memristive Crossbar Array

AbstractAnalog and digital switching performances in a Ta/HfO2/RuO2 (THR) memristor are studied to implement a density‐based spatial clustering of applications with noise (DBSCAN) algorithm in a low‐power, parallel‐computing memristor crossbar structure. In the analog mode THR memristor, more than 256 states can be stored through a fine‐tuning process with a denoising scheme. The analog mode crossbar array facilitates Euclidean distance calculation between any points in the given graphic dataset. In the digital mode, the on/off ratio of more than three orders of magnitude between the binary states is achieved, providing functionality to cluster the data points with a reduced number of operations. The parallel computing capacity of the adopted crossbar decreases the time complexity of the original DBSCAN from O(n2) to O(n). Through array‐level simulations, the effectiveness of hardware functionality is validated using representative synthetic datasets and single‐cell RNA sequences datasets.

A Memristor Crossbar-Based Lyapunov Equation Solver

An analog memristor crossbar based Lyapunov equation solver is proposed in this paper. It is an extension of memristor crossbar linear equation solver, but taking into account of the regularity with Lyapunov equation in the organization of crossbar. Nonideal effects due to co-design with CMOS analog circuits are studied in detail. Verification using a Verilog-AMS tool is performed to justify that with proper scaling on the memristor conductance values and other circuit signals, the solution quality of Lyapunov matrix equation can be improved.