Processor Memory Research Articles

A VOx-based optoelectronic memristor for application in visual perception

Abstract While biological vision systems excel at in-memory processing with low power consumption, traditional silicon-based vision chips struggle with high energy demands. This gap motivates the exploration of alternative materials for artificial intelligence applications. This paper presents a VOx-based optoelectronic synaptic memristive device. The proposed artificial synaptic device ITO/VO x /Pt mimics biological functions such as potentiation (P), depression (D), long-term memory, short-term memory (STM), and paired-pulse facilitation (PPF). The PPF index, standing at 105%, suggests a favorable pattern in STM function. The device served as synapses within a spiking neural network showing an achievable pattern classification accuracy of 88.68%, highlighting the potential of the VO x synaptic device for pattern classification tasks. The suggested VO x -based synaptic devices could represent an efficient pattern recognition and visual perception application.

BIMSA: Accelerating Long Sequence Alignment Using Processing-In-Memory.

Recent advances in sequencing technologies have stressed the critical role of sequence analysis algorithms and tools in genomics and healthcare research. In particular, sequence alignment is a fundamental building block in many sequence analysis pipelines and is frequently a performance bottleneck both in terms of execution time and memory usage. Classical sequence alignment algorithms are based on dynamic programming and often require quadratic time and memory with respect to the sequence length. As a result, classical sequence alignment algorithms fail to scale with increasing sequence lengths and quickly become memory-bound due to data-movement penalties. Processing-In-Memory (PIM) is an emerging architectural paradigm that seeks to accelerate memory-bound algorithms by bringing computation closer to the data to mitigate data-movement penalties. This work presents BIMSA (Bidirectional In-Memory Sequence Alignment), a PIM design and implementation for the state-of-the-art sequence alignment algorithm BiWFA (Bidirectional Wavefront Alignment), incorporating new hardware-aware optimizations for a production-ready PIM architecture (UPMEM). BIMSA supports aligning sequences up to 100K bases, exceeding the limitations of state-of-the-art PIM implementations. First, BIMSA achieves speedups up to 22.24 × (11.95× on average) compared to state-of-the-art PIM-enabled implementations of sequence alignment algorithms. Second, achieves speedups up to 5.84 × (2.83× on average) compared to the highest-performance multicore CPU implementation of BiWFA. Third, BIMSA exhibits linear scalability with the number of compute units in memory, enabling further performance improvements with upcoming PIM architectures equipped with more compute units and achieving speedups up to 9.56 × (4.7× on average). Code and documentation are publicly available at https://github.com/AlejandroAMarin/BIMSA.

A Cascaded ReRAM-based Crossbar Architecture for Transformer Neural Network Acceleration

Emerging resistive random-access memory (ReRAM) based processing-in-memory (PIM) accelerators have been increasingly explored in recent years because they can efficiently perform in-situ matrix-vector multiplication (MVM) operations involved in a wide spectrum of artificial neural networks. However, there remain significant challenges to apply existing ReRAM-based PIM accelerators to the most popular Transformer neural networks. Since Transformers involve a series of matrix-matrix multiplication (MatMul) operations with data dependencies, they should write intermediate results of MatMuls to ReRAM crossbar arrays for further processing. Conventional ReRAM-based PIM accelerators often suffer from high latency of ReRAM writes and intra-layer pipeline stalls. In this paper, we propose ReCAT, a ReRAM-based PIM accelerator designed particularly for Transformers. ReCAT exploits transimpedance amplifiers (TIAs) to cascade a pair of crossbar arrays for MatMul operations involved in the self-attention mechanism. The intermediate result of a MatMul generated by one crossbar array can be directly mapped to another crossbar array, avoiding costly analog-to-digital conversions. In this way, ReCAT allows MVM operations to overlap with the corresponding data mapping, hiding the high latency of ReRAM writes. Furthermore, we propose an analog-to-digital converter (ADC) virtualization scheme to dynamically share scarce ADCs among a group of crossbar arrays, and thus significantly improve the utilization of ADCs to eliminate the performance bottleneck of MVM operations. Experimental results show that ReCAT achieves 207.3 ×, 2.11 ×, and 3.06 × performance improvement on average compared with other Transformer acceleration solutions—GPUs, ReBert, and ReTransformer, respectively.

VSPIM: SRAM Processing-in-Memory DNN Acceleration via Vector-Scalar Operations

Processing-in-Memory (PIM) has been widely explored for accelerating data-intensive machine learning computation that mainly consists of general-matrix-multiplication (GEMM), by mitigating the burden of data movements and exploiting the ultra-high memory parallelism. The two mainstreams of PIM, the analog- and digital-type, have both been exploited in accelerating machine learning workloads by numerous outstanding prior works. Currently, the digital-PIM is increasingly favored due to the broader computing support and the avoidance of errors caused by intrinsic non-idealities, e.g., process variation. Nevertheless, it still lacks further optimization considering the characteristics of the GEMM computation, including better efficient data layout and scheduling, and the ability to handle the sparsity of activations at the bit-level. To boost the performance and efficiency of digital SRAM PIM, we propose the architecture called VSPIM that performs the computation in a bit-serial fashion, with unique support of vector-scalar computing pattern. The novelties of the VSPIM can be concluded as follows: 1) support bit-serial based scalar-vector computing via ingenious parallel bit-broadcasting; 2) refine the GEMM mapping strategy and computing pattern to enhance performance and efficiency; 3) powered by the introduced scalar-vector operation, the bit-sparsity of activation is leveraged to halt unnecessary computation to maximize efficiency and throughput. Our comprehensive evaluation shows that, compared to the state-of-the-art SRAM-based digital-PIM design (Neural Cache), VSPIM can significantly boost the performance and energy efficiency by up to <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$8.87\times$</tex-math></inline-formula> and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$4.81\times$</tex-math></inline-formula> respectively, with negligible area overhead, upon multiple representative neural networks.

MicroLEDs for optical neuromorphic computing—application potential and present challenges

Abstract The slow non-radiative surface recombination velocity of gallium nitride (GaN) in combination with its highly efficient radiative recombination makes this material ideally suited for microLEDs with dimensions as small as 1 µm and even below, serving as the fundamental building block of micro-displays. However, due to their superior miniaturization potential and energy efficiency, GaN-based microLEDs have applications that extend well beyond display technology. Their capability to produce optical patterns with high resolution, which can be modulated at extremely high frequencies, makes them suitable for numerous other applications. We suggest exploiting these exciting properties for a new and potentially equally significant application: utilizing microLEDs in optical processing units for artificial intelligence workloads. In neuromorphic computing, relevant aspects of biological neural networks are emulated directly with either electronic circuits or photonic devices, avoiding the shortcomings of conventional digital computer technology for AI workloads, which generally require massively parallel information processing. GaN microLEDs are discussed here as a promising enabling technology for optical neuromorphic processing units. We see great potential to substantially decrease power consumption through massively parallel in-memory processing combined with efficient photon production and detection. A theoretical analysis of scalability and energy efficiency is provided. A macroscopic bench-top optical microLED demonstrator is presented, which experimentally proves the feasibility of our approach. Future potential and challenges associated with miniaturizing and scaling microLED-based optical processing units are discussed. Finally, we summarize the open research questions that require attention before fully functional and miniaturized optical neuromorphic processing units based on GaN microLEDs can be realized.

Dynamic Performance and Power Optimization with Heterogeneous Processing-in-Memory for AI Applications on Edge Devices

The rapid advancement of artificial intelligence (AI) technology, combined with the widespread proliferation of Internet of Things (IoT) devices, has significantly expanded the scope of AI applications, from data centers to edge devices. Running AI applications on edge devices requires a careful balance between data processing performance and energy efficiency. This challenge becomes even more critical when the computational load of applications dynamically changes over time, making it difficult to maintain optimal performance and energy efficiency simultaneously. To address these challenges, we propose a novel processing-in-memory (PIM) technology that dynamically optimizes performance and power consumption in response to real-time workload variations in AI applications. Our proposed solution consists of a new PIM architecture and an operational algorithm designed to maximize its effectiveness. The PIM architecture follows a well-established structure known for effectively handling data-centric tasks in AI applications. However, unlike conventional designs, it features a heterogeneous configuration of high-performance PIM (HP-PIM) modules and low-power PIM (LP-PIM) modules. This enables the system to dynamically adjust data processing based on varying computational load, optimizing energy efficiency according to the application’s workload demands. In addition, we present a data placement optimization algorithm to fully leverage the potential of the heterogeneous PIM architecture. This algorithm predicts changes in application workloads and optimally allocates data to the HP-PIM and LP-PIM modules, improving energy efficiency. To validate and evaluate the proposed technology, we implemented the PIM architecture and developed an embedded processor that integrates this architecture. We performed FPGA prototyping of the processor, and functional verification was successfully completed. Experimental results from running applications with varying workload demands on the prototype PIM processor demonstrate that the proposed technology achieves up to 29.54% energy savings.

Long-term memory formation for voices during sleep in three-month-old infants

The ability to form long-term memories begins in early infancy. However, little is known about the specific mechanisms that guide memory formation during this developmental stage. We demonstrate the emergence of a long-term memory for a novel voice in three-month-old infants using the EEG mismatch response (MMR) to the word “baby”. In an oddball-paradigm, a frequent standard, and two rare deviant voices (novel and mother) were presented before (baseline), and after (test) familiarizing the infants with the novel voice and a subsequent nap. Only the mother deviant but not the novel deviant elicited a late frontal MMR (∼850 ms) at baseline, possibly reflecting a long-term memory representation for the mother’s voice. Yet, MMRs to the novel and mother deviant significantly increased in similarity after voice familiarization and sleep. Moreover, both MMRs showed an additional early (∼250 ms) frontal negative component that is potentially related to deviance processing in short-term memory. Enhanced spindle activity during the nap predicted an increase in late MMR amplitude to the novel deviant and increased MMR similarity between novel and mother deviant. Our findings indicate that the late positive MMR in infants might reflect emergent long-term memory that benefits from sleep spindles.

Static Scheduling of Weight Programming for DNN Acceleration with Resource Constrained PIM

Most existing architectural studies on ReRAM-based processing-in-memory (PIM) DNN accelerators assume that all weights of the DNN can be mapped to the crossbar at once. However, these studies are over-idealized. ReRAM crossbar resources for calculation are limited because of technological limitations, so multiple weight mapping procedures are required during the inference process. In this article, we propose a static scheduling framework which generates the mapping between DNN weights and ReRAM cells with minimum runtime weight programming cost. We first build a ReRAM crossbar programming latency model by simultaneously considering the DNN weight patterns, ReRAM programming operations, and PIM architecture characteristics. Then, the model is used in the searching process to obtain an optimized weight-to-OU mapping table with minimum online programming latency. Finally, an OU scheduler is used to coordinate the activation sequences of OUs in the crossbars to perform the inference computation correctly. Evaluation results show the proposed framework significantly reduces the weight programming overhead and the overall inference latency for various DNN models with different input datasets.

AutoGMap: Learning to Map Large-Scale Sparse Graphs on Memristive Crossbars.

The sparse representation of graphs has shown great potential for accelerating the computation of graph applications (e.g., social networks and knowledge graphs) on traditional computing architectures (CPU, GPU, or TPU). But, the exploration of large-scale sparse graph computing on processing-in-memory (PIM) platforms (typically with memristive crossbars) is still in its infancy. To implement the computation or storage of large-scale or batch graphs on memristive crossbars, a natural assumption is that a large-scale crossbar is demanded, but with low utilization. Some recent works question this assumption; to avoid the waste of storage and computational resource, the fixed-size or progressively scheduled "block partition" schemes are proposed. However, these methods are coarse-grained or static and are not effectively sparsity-aware. This work proposes the dynamic sparsity-aware mapping scheme generating method that models the problem with a sequential decision-making model, and optimizes it by reinforcement learning (RL) algorithm (REINFORCE). Our generating model [long short-term memory (LSTM), combined with the dynamic-fill scheme] generates remarkable mapping performance on the small-scale graph/matrix data (complete mapping costs 43% area of the original matrix) and two large-scale matrix data (costing 22.5% area on qh882 and 17.1% area on qh1484). Our method may be extended to sparse graph computing on other PIM architectures, not limited to the memristive device-based platforms.

Mixed-precision computing in the GRIST dynamical core for weather and climate modelling

Abstract. Atmosphere modelling applications are becoming increasingly memory-bound due to the inconsistent development rates between processor speeds and memory bandwidth. In this study, we mitigate memory bottlenecks and reduce the computational load of the Global–Regional Integrated Forecast System (GRIST) dynamical core by adopting a mixed-precision computing strategy. Guided by an application of the iterative development principle, we identify the coded equation terms that are precision insensitive and modify them from double to single precision. The results show that most precision-sensitive terms are predominantly linked to pressure gradient and gravity terms, while most precision-insensitive terms are advective terms. Without using more computing resources, computational time can be saved, and the physical performance of the model is largely kept. In the standard computational test, the reference runtime of the model's dry hydrostatic core, dry nonhydrostatic core, and the tracer transport module is reduced by 24 %, 27 %, and 44 %, respectively. A series of idealized tests, real-world weather and climate modelling tests, was performed to assess the optimized model performance qualitatively and quantitatively. In particular, in the long-term coarse-resolution climate simulation, the precision-induced sensitivity can manifest at the large scale, while in the kilometre-scale weather forecast simulation, the model's sensitivity to the precision level is mainly limited to small-scale features, and the wall-clock time is reduced by 25.5 % from the double- to mixed-precision full-model simulations.

Self and mother referential processing in phonological false memory

ABSTRACT The current study used phonological DRM and self-reference paradigms to examine whether phonological false memory was affected by self and mother referential processing. In Experiment 1 and 2, words in the Deese–Roediger–McDermott lists were presented to participants with their own name, their mother’s name or “Lu Xun”. Words were presented with their own name and “Lu Xun” in Experiment 3 to compare the effect of self and other referential processing specifically. The results revealed that true recognition rates were not different between self-reference and mother-reference conditions and both were higher than the other-reference condition; there were no significant differences in false recognition rates among the self-reference, mother-reference and other-reference conditions. The findings are discussed in terms of theories of the self and mother reference effects and phonological false memory.

SuperSIM: a comprehensive benchmarking framework for neural networks using superconductor Josephson devices

Abstract This paper introduces SuperSIM, a benchmarking framework tailored for neural networks using superconducting Josephson devices, specifically focusing on Adiabatic Quantum Flux Parametron (AQFP) based Processing-in-Memory (PIM) architectures. Our framework offers in-depth architecture-level simulations and performance assessments to enhance AQFP PIM chip development. It supports single and multi-bit PIM designs, various AQFP memory cell types, and diverse clocking methods. Additionally, it integrates circuit-level models for precise energy, delay, and area measurements, ensuring accurate performance evaluation. The framework includes application, device, and architectural layers for versatile configurations and cycle-accurate energy, latency, and area simulations. Experiments validate our framework, with case studies on algorithm and architecture-level features, examining data precision, crossbar size, operating frequency and clocking scheme impacts on computational accuracy, energy use, overall latency and hardware cost.

LUD-YOLO: A novel lightweight object detection network for unmanned aerial vehicle

Autonomous execution of tasks by unmanned aerial vehicles (UAVs) relies heavily on object detection. However, object detection in most images presents challenges such as complex backgrounds, small targets, and obstructions. Additionally, the limited computing speed and memory of the UAV processor affects the accuracy of conventional object detection algorithms. This paper proposes LUD-You Only Look Once (YOLO), a small and lightweight object detection algorithm for UAVs based on YOLOv8. The proposed algorithm introduces a new multiscale feature fusion mode that solves the degradation in feature propagation and interaction through the introduction of upsampling in the feature pyramid network and the progressive feature pyramid network. The application of the dynamic sparse attention mechanism in the Cf2 module achieves flexible computing allocation and content awareness. Furthermore, the proposed model is optimized to be sparse and lightweight, making it possible to deploy on UAV edge devices. Finally, the effectiveness and superiority of LUD-YOLO were verified on the VisDrone2019 and UAVDT datasets. The results of ablation and comparison experiments show that compared with the original algorithm, LUDY-N and LUDY-S have shown excellent performance in various evaluation indexes, indicating that the proposed improvement strategies make the model have better robustness and generalization. Moreover, compared with multiple other popular competitors, the proposed improvement strategies enable LUD-YOLO to have the best overall performance, providing an effective solution for UAVs object detection while balancing model size and detection accuracy.

TEFLON: Thermally Efficient Dataflow-aware 3D NoC for Accelerating CNN Inferencing on Manycore PIM Architectures

Resistive random-access memory (ReRAM)-based processing-in-memory (PIM) architectures are used extensively to accelerate inferencing/training with convolutional neural networks (CNNs). Three-dimensional (3D) integration is an enabling technology to integrate many PIM cores on a single chip. In this work, we propose the design of a t hermally e fficient data flo w-aware monolithic 3D (M3D) N oC architecture referred to as TEFLON to accelerate CNN inferencing without creating any thermal bottlenecks. TEFLON reduces the Energy-Delay-Product (EDP) by 42%, 46%, and 45% on an average compared to a conventional 3D mesh NoC for systems with 36-, 64-, and 100-PIM cores, respectively. TEFLON reduces the peak chip temperature by 25 K and improves the inference accuracy by up to 11% compared to sole performance-optimized SFC-based counterpart for inferencing with diverse deep CNN models using CIFAR-10/100 datasets on a 3D system with 100-PIM cores.

Data Pruning-enabled High Performance and Reliable Graph Neural Network Training on ReRAM-based Processing-in-Memory Accelerators

Graph Neural Networks (GNNs) have achieved remarkable accuracy in cognitive tasks such as predictive analytics on graph-structured data. Hence, they have become very popular in diverse real-world applications. However, GNN training with large real-world graph datasets in edge-computing scenarios is both memory- and compute-intensive. Traditional computing platforms such as CPUs and GPUs do not provide the energy efficiency and low latency required in edge intelligence applications due to their limited memory bandwidth. Resistive random-access memory (ReRAM)-based processing-in-memory (PIM) architectures have been proposed as suitable candidates for accelerating AI applications at the edge, including GNN training. However, ReRAM-based PIM architectures suffer from low reliability due to their limited endurance, and low performance when they are used for GNN training in real-world scenarios with large graphs. In this work, we propose a learning-for-data-pruning framework, which leverages a trained Binary Graph Classifier (BGC) to reduce the size of the input data graph by pruning subgraphs early in the training process to accelerate the GNN training process on ReRAM-based architectures. The proposed light-weight BGC model reduces the amount of redundant information in input graph(s) to speed up the overall training process, improves the reliability of the ReRAM-based PIM accelerator, and reduces the overall training cost. This enables fast, energy-efficient, and reliable GNN training on ReRAM-based architectures. Our experimental results demonstrate that using this learning for data pruning framework, we can accelerate GNN training and improve the reliability of ReRAM-based PIM architectures by up to 1.6×, and reduce the overall training cost by 100× compared to state-of-the-art data pruning techniques.

Efficient Processing-in-Memory System Based on RISC-V Instruction Set Architecture

A lot of research on deep learning and big data has led to efficient methods for processing large volumes of data and research on conserving computing resources. Particularly in domains like the IoT (Internet of Things), where the computing power is constrained, efficiently processing large volumes of data to conserve resources is crucial. The processing-in-memory (PIM) architecture was introduced as a method for efficient large-scale data processing. However, PIM focuses on changes within the memory itself rather than addressing the needs of low-cost solutions such as the IoT. This paper proposes a new approach using the PIM architecture to overcome memory bottlenecks effectively in domains with computing performance constraints. We adopt the RISC-V instruction set architecture for our proposed PIM system’s design, implementation, and comprehensive performance evaluation. Our proposal expects to efficiently utilize low-spec systems like the IoT by minimizing core modifications and introducing PIM instructions at the ISA level to enable solutions that leverage PIM capabilities. We evaluate the performance of our proposed architecture by comparing it with existing structures using convolution operations, the fundamental unit of deep-learning and big data computations. The experimental results show our proposed structure achieves a 34.4% improvement in processing speed and 18% improvement in power consumption compared to conventional von Neumann-based architectures. This substantiates its effectiveness at the application level, extending to fields such as deep learning and big data.

A Comprehensive Review of Processing-in-Memory Architectures for Deep Neural Networks

This comprehensive review explores the advancements in processing-in-memory (PIM) techniques and chiplet-based architectures for deep neural networks (DNNs). It addresses the challenges of monolithic chip architectures and highlights the benefits of chiplet-based designs in terms of scalability and flexibility. This review emphasizes dataflow-awareness, communication optimization, and thermal considerations in PIM-enabled manycore architectures. It discusses tailored dataflow requirements for different machine learning workloads and presents a heterogeneous PIM system for energy-efficient neural network training. Additionally, it explores thermally efficient dataflow-aware monolithic 3D (M3D) NoC architectures for accelerating CNN inferencing. Overall, this review provides valuable insights into the development and evaluation of chiplet and PIM architectures, emphasizing improved performance, energy efficiency, and inference accuracy in deep learning applications.

Representing a Model for the Anonymization of Big Data Stream Using In-Memory Processing

Representing a Model for the Anonymization of Big Data Stream Using In-Memory Processing

Load Balanced PIM-Based Graph Processing

Graph processing is widely used for many modern applications, such as social networks, recommendation systems, and knowledge graphs. However, processing large-scale graphs on traditional Von Neumann architectures is challenging due to the irregular graph data and memory-bound graph algorithms. Processing-in-memory (PIM) architecture has emerged as a promising approach for accelerating graph processing by enabling computation to be performed directly on memory. Despite having many processing units and high local memory bandwidth, PIM often suffers from insufficient global communication bandwidth and high synchronization overhead due to load imbalance. This article proposes GraphB, a novel PIM-based graph processing system, to address all these issues. From the algorithm perspective, we propose a degree-aware graph partitioning algorithm that can generate balanced partitioning at a low cost. From the architecture perspective, we introduce tile buffers incorporated with an on-chip 2D-Mesh, which provides high bandwidth for inter-node data transfer. Dataflow in GraphB is designed to enable computation–communication overlap and dynamic load balancing. In a PyMTL3-based cycle-accurate simulator with five real-world graphs and three common algorithms, GraphB achieves an average 2.2× and maximum 2.8× speedup compared to the SOTA PIM-based graph processing system GraphQ.

Scalability Limitations of Processing-in-Memory using Real System Evaluations

Processing-in-memory (PIM) has been widely explored in academia and industry to accelerate numerous workloads. By reducing the data movement and increasing parallelism, PIM offers great performance and energy efficiency. A large amount of cores or nodes present in PIM provide massive parallelism and compute throughput; however, this also proposes challenges and limitations for some workloads. In this work, we provide an extensive evaluation and analysis of a real PIM system from UPMEM. We specifically target emerging workloads featuring collective communication, demonstrating its role as the primary limitation within current PIM architecture.