In-memory Computing Architecture Research Articles

Demonstration of In-Memory Biosignal Analysis: Novel High-Density and Low-Power 3D Flash Memory Array for Arrhythmia Detection.

Smart healthcare systems integrated with advanced deep neural networks enable real-time health monitoring, early disease detection, and personalized treatment. In this work, a novel 3D AND-type flash memory array with a rounded double channel for computing-in-memory (CIM) architecture to overcome the limitations of conventional smart healthcare systems: the necessity of high area and energy efficiency while maintaining high classification accuracy is proposed. The fabricated array, characterized by low-power operations and high scalability with double independent channels per floor, exhibits enhanced cell density and energy efficiency while effectively emulating the features of biological synapses. The CIM architecture leveraging the fabricated array achieves high classification accuracy (93.5%) for electrocardiogram signals, ensuring timely detection of potentially life-threatening arrhythmias. Incorporated with a simplified spike-timing-dependent plasticity learning rule, the CIM architecture is suitable for robust, area- and energy-efficient in-memory arrhythmia detection systems. This work effectively addresses the challenges of conventional smart healthcare systems, paving the way for a more refined healthcare paradigm.

Efficient and lightweight in-memory computing architecture for hardware security

This paper introduces an innovative solution for improving the efficiency and speed of the Advanced Encryption Standard (AES) based cryptographic algorithm. The approach leverages in-memory computing (IMC) and is versatile for application across a broad spectrum of IoT applications, including robotic autonomous vehicles and various other scenarios. To achieve this goal, memristor (MR) designs are proposed to emulate the arithmetic operations required for different phases of the AES algorithm, enabling efficient in-memory processing. The key contributions of this work include; 1) The development of a 4 bit-MR state element for implementing different arithmetic operations in an AES hardware prototype; 2) The creation of a pipeline AES design for massive parallelism and MR integration compatibility; and 3) The hardware implementation of the AES-IMC based architecture using the MR emulator. The results show that AES-IMC performs better than existing architectures in terms of higher throughput and energy efficiency. Compared to conventional AES hardware, AES-IMC achieves a 30% power enhancement with comparable throughput. Additionally, when compared to state-of-the-art AES-based NVM engines, AES-IMC demonstrates comparable power dissipation and a 62% increase in throughput. The IMC architecture enables cost-effective real-time deployment of AES, leading to high-performance computing. By leveraging the power of in-memory computing, this system is able to provide improved computational efficiency and faster processing speeds, making it a promising solution for a wide range of applications in the field of autonomous driving and robotics. The potential benefits of this system include improved safety and security of unmanned devices, as well as enhanced performance and cost-effectiveness in a variety of computing environments.

A reconfigurable in‐memory‐computation architecture with in‐situ update and shift capability

AbstractThis work proposes a storage element (SE) design for in‐memory computing (IMC). Using the proposed SE design, an IMC array has been constructed to enable in‐situ updates of stored weights. Compared with some existing related works which employ 2n bit cells for storing an n‐bit value, the proposed structure in this work exhibits a O(n) complexity of area overhead, which means that only n bit cells are needed to store n‐bit values, offering a significant improvement in silicon area usage. Compared to the purely digitally‐controlled weight update schemes, the approach proposed in this work demonstrates an approximate 1.47× increase in power consumption. Furthermore, it exhibits superior robustness compared to existing work. Additionally, the proposed SE design can achieve the necessary shift functionality in the cutting‐edge floating‐point (FP)‐IMC architecture through simple control mechanisms.

Configurable in-memory computing architecture based on dual-port SRAM

In the emerging field of in-memory computing (IMC), this study proposes a dual-port static random access memory (SRAM) IMC architecture with the distinct capability of realizing XOR encryption (XORE), thus serving as a potential solution for the Von Neumann bottleneck. Beyond providing traditional SRAM read and write operations, the proposed architecture carries out additional tasks such as multi-bit multiply and accumulate (MAC) and XOR accumulation (XORA). The architecture was simulated using a 28-nm Complementary Metal Oxide Semiconductor Process, demonstrating a minor standard deviation of 9.41 mV in bit line voltage at the SS process corner, as evidenced by Monte Carlo simulation. Energy expenditure for the MAC, XORA, and XORE, was found to be 1.65, 1.46, and 9.02 fJ/ops respectively at the TT process corner. Furthermore, the presented architecture showed considerable energy efficiency, with MAC, XORA, and XORE operations achieving energy efficiency values of 604.9, 682.7, and 110.8 TOPS/W respectively, at a supply voltage of 0.9 V at the TT process corner.

Side-Channel Attack Analysis on In-Memory Computing Architectures

In-memory computing (IMC) systems have great potential for accelerating data-intensive tasks such as deep neural networks (DNNs). As DNN models are generally highly proprietary, the neural network architectures become valuable targets for attacks. In IMC systems, since the whole model is mapped on chip and weight memory read can be restricted, the pre-mapped DNN model acts as a “black box” for users. However, the localized and stationary weight and data patterns may subject IMC systems to other attacks. In this paper, we propose a side-channel attack methodology on IMC architectures. We show that it is possible to extract model architectural information from power trace measurements without any prior knowledge of the neural network. We first developed a simulation framework that can emulate the dynamic power traces of the IMC macros. We then performed side-channel leakage analysis to reverse engineer model information such as the stored layer type, layer sequence, output channel/feature size and convolution kernel size from power traces of the IMC macros. Based on the extracted information, full networks can potentially be reconstructed without any knowledge of the neural network. Finally, we discuss potential countermeasures for building IMC systems that offer resistance to these model extraction attack.

Energy-Accuracy Trade-Offs for Resistive In-Memory Computing Architectures

Energy-Accuracy Trade-Offs for Resistive In-Memory Computing Architectures

A review of in-memory computing for machine learning: architectures, options

PurposeThis paper aims to review in-memory computing (IMC) for machine learning (ML) applications from history, architectures and options aspects. In this review, the authors investigate different architectural aspects and collect and provide our comparative evaluations.Design/methodology/approachCollecting over 40 IMC papers related to hardware design and optimization techniques of recent years, then classify them into three optimization option categories: optimization through graphic processing unit (GPU), optimization through reduced precision and optimization through hardware accelerator. Then, the authors brief those techniques in aspects such as what kind of data set it applied, how it is designed and what is the contribution of this design.FindingsML algorithms are potent tools accommodated on IMC architecture. Although general-purpose hardware (central processing units and GPUs) can supply explicit solutions, their energy efficiencies have limitations because of their excessive flexibility support. On the other hand, hardware accelerators (field programmable gate arrays and application-specific integrated circuits) win on the energy efficiency aspect, but individual accelerator often adapts exclusively to ax single ML approach (family). From a long hardware evolution perspective, hardware/software collaboration heterogeneity design from hybrid platforms is an option for the researcher.Originality/valueIMC’s optimization enables high-speed processing, increases performance and analyzes massive volumes of data in real-time. This work reviews IMC and its evolution. Then, the authors categorize three optimization paths for the IMC architecture to improve performance metrics.

Flash-Based Computing-in-Memory Architecture to Implement High-Precision Sparse Coding.

To address the concerns with power consumption and processing efficiency in big-size data processing, sparse coding in computing-in-memory (CIM) architectures is gaining much more attention. Here, a novel Flash-based CIM architecture is proposed to implement large-scale sparse coding, wherein various matrix weight training algorithms are verified. Then, with further optimizations of mapping methods and initialization conditions, the variation-sensitive training (VST) algorithm is designed to enhance the processing efficiency and accuracy of the applications of image reconstructions. Based on the comprehensive characterizations observed when considering the impacts of array variations, the experiment demonstrated that the trained dictionary could successfully reconstruct the images in a 55 nm flash memory array based on the proposed architecture, irrespective of current variations. The results indicate the feasibility of using Flash-based CIM architectures to implement high-precision sparse coding in a wide range of applications.

SpikeSim: An End-to-End Compute-in-Memory Hardware Evaluation Tool for Benchmarking Spiking Neural Networks

Spiking Neural Networks (SNNs) are an active research domain towards energy efficient machine intelligence. Compared to conventional artificial neural networks (ANNs), SNNs use temporal spike data and bio-plausible neuronal activation functions such as Leaky-Integrate Fire/Integrate Fire (LIF/IF) for data processing. However, SNNs incur significant dot-product operations causing high memory and computation overhead in standard von-Neumann computing platforms. To this end, In-Memory Computing (IMC) architectures have been proposed to alleviate the “memory-wall bottleneck" prevalent in von-Neumann architectures. Although recent works have proposed IMC-based SNN hardware accelerators, the following key implementation aspects have been overlooked 1) the adverse effects of crossbar non-ideality on SNN performance due to repeated analog dot-product operations over multiple time-steps 2) hardware overheads of essential SNN-specific components such as the LIF/IF and data communication modules. To this end, we propose SpikeSim, a tool that can perform realistic performance, energy, latency and area evaluation of IMC-mapped SNNs. SpikeSim consists of a practical monolithic IMC architecture called SpikeFlow for mapping SNNs. Additionally, the non-ideality computation engine (NICE) and energy-latency-area (ELA) engine performs hardware-realistic evaluation of SpikeFlow-mapped SNNs. Based on 65nm CMOS implementation and experiments on CIFAR10, CIFAR100 and TinyImagenet datasets, we find that the LIF/IF neuronal module has significant area contribution (>11% of the total hardware area). To this end, we propose SNN topological modifications that leads to 1.24× and 10× reduction in the neuronal module’s area and the overall energy-delay-product value, respectively. Furthermore, in this work, we perform a holistic comparison between IMC implemented ANN and SNNs and conclude that lower number of time-steps are the key to achieve higher throughput and energy-efficiency for SNNs compared to 4-bit ANNs. The code repository for the SpikeSim tool is available at https://github.com/Intelligent-Computing-Lab-Yale/Quanitzation-aware-SNN-training-and-hardware-evaluation-for-IMC-Architectures

FeCrypto: Instruction Set Architecture for Cryptographic Algorithms Based on FeFET-Based In-Memory Computing

Recently, computing-in-memory (CiM) becomes a promising technology for alleviating the memory wall bottleneck. CiM is suitable for data-intensive applications, especially cryptographic algorithms. Most current cryptographic accelerators are specific to a single function. It is expensive to accelerate different cryptographic algorithms with different accelerators. In this work, we first introduce a CiM architecture FeMIC that supports multi-operand CiM operations, by exploring advantages of state-of-the-art ferroelectric field-effect transistors. Based on that, we propose a novel instruction set together with an accelerator architecture named FeCrypto which supports the acceleration of various cryptographic algorithms. Evaluation results show that FeCrypto has better performance and energy efficiency than software implementations. The energy-delay product (EDP) of FeCrypto is 118.4× and 1.93× lower than that of the dedicated AES accelerator AIM that is built based on phase-change memories (PCMs) and magnetic random-access memories (MRAMs), respectively. EDP is reduced by 44.7× compared with PCM-based EIM, a recent AES accelerator. Compared with MRAM-based EIM, the EDP overhead of FeCrypto for supporting multiple functions is 23.2%.

An energy-efficient 10T SRAM in-memory computing macro for artificial intelligence edge processor

In-Memory Computing (IMC) is emerging as a new paradigm to address the von-Neumann bottleneck (VNB) in data-intensive applications. In this paper, an energy-efficient 10T SRAM-based IMC macro architecture is proposed to perform logic, arithmetic, and In-memory Dot Product (IMDP) operations. The average write margin and read margins of the proposed 10T SRAM are improved by 40% and 2.5%, respectively, compared to the 9T SRAM. The write energy and leakage power of the proposed 10T SRAM are reduced by 89% and 83.8%, respectively, with aproximatelly similar read energy compared to 9T SRAM. Additionally, a 4 Kb SRAM array based on 10T SRAM is implemented in 180-nm SCL technology to analyze the operation and performance of the proposed IMC macro architecture. The proposed IMC architecture achieves an energy efficiency of 5.3 TOPS/W for 1-bit logic, 4.1 TOPS/W for 1-bit addition, and 3.1 TOPS/W for IMDP operations at 1.8 V and 60 MHz. The area efficiency of 65.2% is achieved for a 136 × 32 array of proposed IMC macro architecture. Further, the proposed IMC macro is also tested for accelerating the IMDP operation of neural networks by importing linearity variation analysis in Tensorflow for image classification on MNIST and CIFAR datasets. According to Monte-Carlo simulations, the IMDP operation has a standard deviation of 0.07 percent in accumulation, equating to a classification accuracy of 97.02% on the MNIST dataset and 88.39% on the CIFAR dataset.

Ternary Output Binary Neural Network With Zero-Skipping for MRAM-Based Digital In-Memory Computing

This paper presents a novel ternary output binary neural network (BNN) and an MRAM-based digital in-memory computing (IMC) architecture. The proposed ternary output BNN and IMC architecture is capable of 1) improving array efficiency by using only one bit-cell for one synaptic weight, 2) no accuracy loss due to its digital nature, 3) high energy efficiency by employing a zero-skipping scheme, and 4) the use of normal memory and deep learning applications due to minimized array modification. System simulations with a two-layer perceptron show that the ternary output BNN achieves 92.12% inference accuracy measured against the MNIST dataset, while the conventional BNN shows 80.8% accuracy. In addition, when the zero-skipping scheme was employed, the energy efficiency of the proposed architecture improved from 8.13 to 58.69 TOPS/W.

A 6T-3M SOT-MRAM for in-memory computing with reconfigurable arithmetic operations

Traditional von Neumann architecture bottlenecks such as the “memory wall” limit artificial intelligence (AI) development, and in-memory computing (IMC) as a new computing architecture can solve the above problems. Spin-orbit-torque-magnetic random access memory (SOT-MRAM) has very good advantages in IMC architecture because of its good compatibility with CMOS, high tunneling magnetoresistance (TMR) ratio, and high energy efficiency. In this paper, we propose a 6T-3M-based MRAM-IMC architecture with reconfigurable memory mode and logical operation mode. The functionality of the proposed architecture is validated using the 28 nm process design kit and the SOT-MTJ model.

Analysis of resistive defects on a foundry 8T SRAM-based IMC architecture

A promising new alternative to efficiently solve the Von Neumann bottleneck problem is to adopt In-Memory Computing (IMC) architectures. Beyond the arithmetic operations, IMC architectures aim at integrating additional logic operators directly in the memory array or/and at the periphery in order to provide close computing abilities. However, they are subject to manufacturing defects in the same way as conventional memories. In this paper, a comprehensive model of a 128 × 128 bitcell array based on 28 nm FD-SOI process technology has been considered to analyze the behavior of IMC 8T SRAM bitcells in the presence of resistive defects (open and short) injected in the read port. A hierarchical analysis allowing a thorough study of each defect has been carried in order to identify their impact in both memory and computing modes, locally on the defective bitcell as well as globally on the array. Experimental results show that the IMC mode offers the most effective detectability of resistive-short and resistive-open defects.

A Fully Digital SRAM-Based Four-Layer In-Memory Computing Unit Achieving Multiplication Operations and Results Store

The separation of memory and arithmetic logic unit (ALU) in the von Neumann computing architecture hinders the development of big data and high-performance computing. In-memory computing (IMC) as a new computation method significantly reduces the latency and power consumption of data processing. In this study, we propose a fully digital static random access memory (SRAM)-based IMC architecture, which has the following advantages: 1) it simplifies multiplication to multicycle addition operations, reuses logic cells, and reduces hardware overhead; 2) by adding a pair of nMOS transistors to achieve internal write-back, the computational efficiency is improved, and at the same time, the final result of the multiplication can be stored locally, eliminating the need to read the computational result immediately; and 3) this scheme can be easily expanded to multiplication operations with different bit widths, which provides good scalability. A 4-kb SRAM-IMC macro chip is manufactured using the SMIC 55-nm technology to realize 4-bit multiplication, with an energy efficiency of 51.4 TOPS/W (0.9 V) and a throughput of 234.3 GOPS/mm <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$^{2}$</tex-math> </inline-formula> . The proposed multiplication–accumulation architecture is applied to a neural network, which achieves 98.7% accuracy with the Mixed National Institute of Standards and Technology database (MNIST) dataset.

Flash-based in-memory computing for stochastic computing in image edge detection

The “memory wall” of traditional von Neumann computing systems severely restricts the efficiency of data-intensive task execution, while in-memory computing (IMC) architecture is a promising approach to breaking the bottleneck. Although variations and instability in ultra-scaled memory cells seriously degrade the calculation accuracy in IMC architectures, stochastic computing (SC) can compensate for these shortcomings due to its low sensitivity to cell disturbances. Furthermore, massive parallel computing can be processed to improve the speed and efficiency of the system. In this paper, by designing logic functions in NOR flash arrays, SC in IMC for the image edge detection is realized, demonstrating ultra-low computational complexity and power consumption (25.5 fJ/pixel at 2-bit sequence length). More impressively, the noise immunity is 6 times higher than that of the traditional binary method, showing good tolerances to cell variation and reliability degradation when implementing massive parallel computation in the array.

Local bit line 8T SRAM based in-memory computing architecture for energy-efficient linear error correction codec implementation

Memory reliability is a critical issue in SRAM-based In-Memory Computing (IMC) architecture. The rapid advance in transistor technology makes SRAM more sensitive to soft errors. Various developments have recommended different methods to prevent store data corruption using Hamming Code. In order to encode or decode the information, a piece of Error Correction Code (ECC) hardware is necessary, which results in an area and energy overhead. In this paper, an IMC-based linear Error Correction Codec (IMC-ECC) hardware for parity check bit generation (encoding) and syndrome vector generation (decoding) is implemented by using the proposed Transmission Pass Gate (TPG) Local Bit-line (LB-8T) SRAM structure based IMC architecture. The local bit line structure resolves the half-select issues that occur for bit-interleave architecture. The TPG improves the write stability by delivering both strong VDD and GND. Furthermore, the proposed IMC-ECC scheme, along with bit-interleave architecture, makes the IMC architecture more resilient to multi-bit soft error. The proposed IMC-ECC scheme is validated by implementing a 4 Kb LB-8T SRAM array in 65 nm CMOS technology and Hamming code (7, 4) as an example. The proposed IMC-ECC scheme achieves the average energy consumption of 0.294 pJ/bit and 0.075 pJ/bit for the encoding and decoding process, respectively, at a supply voltage of 1 V.

SpinDrop: Dropout-Based Bayesian Binary Neural Networks With Spintronic Implementation

Neural Networks (NNs) provide an effective solution in numerous application domains, including autonomous driving and medical applications. Nevertheless, NN predictions can be incorrect if the input sample is outside of the training distribution or contaminated by noise. Consequently, quantifying the uncertainty of the NN prediction allows the system to make more insightful decisions by avoiding blind predictions. Therefore, uncertainty quantification is crucial for a variety of applications, including safety-critical applications. Bayesian NN (BayNN) using Dropout-based approximation provides a systematic approach for estimating the uncertainty of predictions. Despite such merit, BayNNs are not suitable for implementation in an embedded device or able to meet high-performance demands for certain applications. Computation in-memory (CiM) architecture with emerging non-volatile memories (NVMs) is a great candidate for high-performance and low-power acceleration BayNNs in hardware. Among NVMs, Magnetic Tunnel Junction (MTJ) offer many benefits, but they also suffer from various non-idealities and limited bit-level resolution. As a result, binarizing BayNNs is an attractive option that can directly implement BayNN into a CiM architecture and able to achieve benefits of both CiM architecture and BayNNs at the same time. Conventional in-memory hardware implementations emphasize conventional NNs, which can only make predictions, and do not account for both device and input uncertainty, thus, reducing both reliability and performance. In this paper, we propose for the first time Binary Bayesian NNs (BinBayNN) with an end-to-end approach (from algorithmic level to device level) for their implementation. Our approach takes the inherent stochastic properties of MTJs as a feature to implement Dropout-based Bayesian Neural Networks. We provide an extensive evaluation of our approach from the device level up to the algorithmic level on various benchmark datasets.

E-UPQ: Energy-Aware Unified Pruning-Quantization Framework for CIM Architecture

The wide adoption of convolutional neural networks (CNNs) in many applications has given rise to unrelenting computational demand and memory requirements. Computing-in-Memory (CIM) architecture has demonstrated great potential to break the memory wall and effectively execute CNN workloads. Ongoing research focuses on pruning or quantizing CNNs to achieve higher efficiency on CIM. However, prior works preclude the possibility of integrating both techniques in the same framework. On the other hand, directly incorporating energy estimation during the model compression process has not been well explored in the literature. In this paper, we present an Energy-aware Unified Pruning-Q uantization (E-UPQ) mechanism, a novel framework for automated compression (pruning + quantization) of CNNs while considering the energy-accuracy trade-off. Specifically, E-UPQ interweaves pruning and quantization seamlessly by viewing pruning as a special case of “0-bit” quantization during the mixed-precision search. In addition, E-UPQ introduces a set of trainable parameters to incorporate energy information during the compression process, closing the gap between compression policy and energy optimization. Experimental results evaluated on DNN+NeuroSim show that E-UPQ reduces energy consumption by up to 79.3% and 66.6% for VGG-16 and ResNet-18, respectively, compared with the state-of-the-art work, while achieving similar accuracy on CIFAR-100. Layer-wise analysis and ablation studies are provided to validate the effectiveness of the E-UPQ. We also present the corresponding CIM architecture to support the proposed E-UPQ framework.

The Opportunity of Negative Capacitance Behavior in Flash Memory for High‐Density and Energy‐Efficient In‐Memory Computing Applications

AbstractFlash memory is a promising candidate for use in in‐memory computing (IMC) owing to its multistate operations, high on/off ratio, non‐volatility, and the maturity of device technologies. However, its high operation voltage, slow operation speed, and string array structure severely degrade the energy efficiency of IMC. To address these challenges, a novel negative capacitance‐flash (NC‐flash) memory‐based IMC architecture is proposed. To stabilize and utilize the negative capacitance (NC) effect, a HfO2‐based reversible single‐domain ferroelectric (RSFE) layer is developed by coupling the flexoelectric and surface effects, which generates a large internal field and surface polarization pinning. Furthermore, NC‐flash memory is demonstrated for the first time by introducing a RSFE and dielectric heterostructure layer in which the NC effect is stabilized as a blocking layer. Consequently, an energy‐efficient and high‐throughput IMC is successfully demonstrated using an AND flash‐like cell arrangement and source‐follower/charge‐sharing vector‐matrix multiplication operation on a high‐performance NC‐flash memory.