Hybrid Memory Cube Research Articles

ALAMNI: Adaptive LookAside Memory based Near-Memory Inference Engine for Eliminating Multiplications in Real-Time

The CNN algorithm seeks high performance and energy efficiency in real-time inference. The costly off-chip memory accesses put additional burdens on CNN's execution. Towards avoiding off-chip accesses, we propose ALAMNI, a novel near-memory architecture that expedites the CNNs in the logic layer of the Hybrid Memory Cube. We exploit intra- and inter-vault parallelism to accelerate the highly parallel CNN operations. The proposed ALAMNI replaces costly multiplications of CNNs with lookaside memory (LAM) based searches. The proposed ALAMNI policy is effective on unseen data as it discards the data pre-profiling overhead by an adaptive LAM update policy. The ALAMNI controller keeps the most frequent triplets of weight (W), activation (A), and multiplication result (M), <W, A, M>, in the LAM to eliminate redundant computations. As an optimization, we incorporate a bitmasking concept to raise the hit rate of LAMs and further amortize computations. We also present a study on the relation between the amount of bitmasking and the loss of classification accuracy of the popular ConvNets. We keep the bitmasking as a reconfigurable feature of the ALAMNI units to achieve desired classification accuracy. Experimental results show substantial improvement in the system's performance and energy efficiency compared to the baseline and state-of-the-art.

High-throughput Near-Memory Processing on CNNs with 3D HBM-like Memory

This article discusses the high-performance near-memory neural network (NN) accelerator architecture utilizing the logic die in three-dimensional (3D) High Bandwidth Memory– (HBM) like memory. As most of the previously reported 3D memory-based near-memory NN accelerator designs used the Hybrid Memory Cube (HMC) memory, we first focus on identifying the key differences between HBM and HMC in terms of near-memory NN accelerator design. One of the major differences between the two 3D memories is that HBM has the centralized through- silicon-via (TSV) channels while HMC has distributed TSV channels for separate vaults. Based on the observation, we introduce the Round-Robin Data Fetching and Groupwise Broadcast schemes to exploit the centralized TSV channels for improvement of the data feeding rate for the processing elements. Using synthesized designs in a 28-nm CMOS technology, performance and energy consumption of the proposed architectures with various dataflow models are evaluated. Experimental results show that the proposed schemes reduce the runtime by 16.4–39.3% on average and the energy consumption by 2.1–5.1% on average compared to conventional data fetching schemes.

CLU

Convolutional/Deep Neural Networks (CNNs/DNNs) are rapidly growing workloads for the emerging AI-based systems. The gap between the processing speed and the memory-access latency in multi-core systems affects the performance and energy efficiency of the CNN/DNN tasks. This article aims to alleviate this gap by providing a simple and yet efficient near-memory accelerator-based system that expedites the CNN inference. Towards this goal, we first design an efficient parallel algorithm to accelerate CNN/DNN tasks. The data is partitioned across the multiple memory channels (vaults) to assist in the execution of the parallel algorithm. Second, we design a hardware unit, namely the convolutional logic unit (CLU), which implements the parallel algorithm. To optimize the inference, the CLU is designed, and it works in three phases for layer-wise processing of data. Last, to harness the benefits of near-memory processing (NMP), we integrate homogeneous CLUs on the logic layer of the 3D memory, specifically the Hybrid Memory Cube (HMC). The combined effect of these results in a high-performing and energy-efficient system for CNNs/DNNs. The proposed system achieves a substantial gain in the performance and energy reduction compared to multi-core CPU- and GPU-based systems with a minimal area overhead of 2.37%.

HAM: Hotspot-Aware Manager for Improving Communications With 3D-Stacked Memory

Emerging High-Performance Computing (HPC) workloads, such as graph analytics, machine learning, and big data science, are data-intensive. Data-intensive workloads usually present fine-grained memory accesses with limited or no data locality, and thus incur frequent cache misses and low utilization of memory bandwidth. 3D-stacked memory devices such as Hybrid Memory Cube (HMC) and High Bandwidth Memory (HBM) can provide significantly higher bandwidth than conventional memory modules. However, the traditional interfaces and optimization methods for JEDEC DDR devices do not allow to fully exploit the potential performance of 3D-stacked memory with the massive amount of irregular memory accesses of data-intensive applications. In this article, we propose a novel Hotspot-Aware Manager (HAM) infrastructure for 3D-stacked memory devices capable of optimizing memory access streams via request aggregation, hotspot detection, and in-memory prefetching. We present the HAM design and implementation, and simulate it on a system using RISC-V embedded cores with attached HMC devices. We extensively evaluate HAM with over 12 benchmarks and applications representing diverse irregular memory access patterns. The results show that, on average, HAM reduces redundant requests by 37.51 percent and increases the prefetch buffer hit rate by 4.2 times, compared to a baseline streaming prefetcher. On the selected benchmark set, HAM provides performance gains of 21.81 percent in average (up to 34.28 percent), and power savings of 35.07 percent over a standard 3D-stacked memory.

Enabling fast and energy-efficient FM-index exact matching using processing-near-memory

Memory bandwidth and latency constitutes a major performance bottleneck for many data-intensive applications. While high-locality access patterns take advantage of the deep cache hierarchies available in modern processors, unpredictable low-locality patterns cause a significant part of the execution time to be wasted waiting for data. An example of those memory bound applications is the exact matching algorithm based on FM-index, used in some well-known sequence alignment applications. Processing-Near-Memory (PNM) has been proposed as a strategy to overcome the memory wall problem, by placing computation close to data, speeding up memory bound workloads by reducing data movements. This paper presents a performance and energy evaluation of two classes of processor architectures when executing the FM-index exact matching algorithm, as a reference algorithm for exact sequence alignment. One architecture class is processor-centric, based on complex cores and DDR3/4 SDRAM memory technology. The other architecture class is memory-centric, based on simple cores and ultra-high-bandwidth hybrid memory cube (HMC) 3D-stacked memory technologies. The results show that the PNM solution improves performance between 1.26 $$\times$$ and 3.7 $$\times$$ and the energy consumption per operation is reduced between 21 $$\times$$ and 40 $$\times$$ . In addition, a synthetic benchmark RANDOM was developed that mimics the memory access pattern of the FM-index exact matching algorithm, but with a user configurable operational intensity. This benchmark allows us to extend the evaluation to the class of algorithms with similar memory behaviour but running over a range of operational intensity values.

Task Parallelism-Aware Deep Neural Network Scheduling on Multiple Hybrid Memory Cube-Based Processing-in-Memory

Processing-in-memory (PIM) comprises computational logic in the memory domain. It is the most promising solution to alleviate the memory bandwidth problem in deep neural network (DNN) processing. The hybrid memory cube (HMC), a 3D stacked memory structure, can efficiently implement the PIM architecture by maximizing the existing legacy hardware. To accelerate DNN inference, multiple HMCs can be connected, and data-independent tasks can be assigned to processing elements (PEs) within each HMC. However, owing to the packet-switched network structure, inter-HMC interconnects exhibit variable and unpredictable latencies depending on the data transmission path and link contention. A well-designed task schedule using context switching can effectively hide communication latency and improve PE utilization. Nevertheless, as the number of HMC increases, the variability of a wide range of inter-HMC communication latencies causes frequent context switching, degrading overall performance. This paper proposes a DNN task scheduling that can effectively utilize task parallelism by reducing the communication latency variance owing to HMC interconnect characteristics. Task partitions are generated to exploit parallelism while providing inter-HMC traffic within the sustainable link bandwidth. Task-to-HMC mapping is performed to hide the average communication latency of intermediate DNN processing results. A task schedule is generated using retiming to accelerate DNN inference while maximizing resource utilization. The effectiveness of the proposed method was verified through simulations using various realistic DNN applications performed on a ZSim x86-64 simulator. The simulations revealed that DNN processing with the proposed scheduling improved the DNN processing speed by reducing the processing time by 18.19% over conventional methods where each HMC operated independently.

Functionality-Based Processing-in-Memory Accelerator for Deep Convolutional Neural Networks

Processing-in-memory (PIM) architectures show the advantage of handling applications that generate complicated memory request patterns; usually, those kinds of memory streams degrade the application’s performance in conventional memory hierarchy systems. In particular, deep convolutional neural networks (DCNNs) processing that consists of several functionalities could be highly optimized if PIM cores can extend the processing capability and data accessibility. In this work, we propose a functionality-based PIM accelerator for DCNNs. We design several modules in addition to the conventional PIM system based on a hybrid memory cube (HMC). First, we compose a new buffer module, namely, a shared cache, in which PIM cores are provided DCNN functionalities and pre-trained weights. The PIM cores subsequently enhance computational utilization and data accessibility. Second, an efficient replacement method complements the shared cache to optimize the data miss rate of DCNN processing. Third, we compose dual prefetchers that can deal with DCNN’s memory access patterns, thereby reducing the system’s overall latency. Fourth, we compose a PIM scheduler for PIM core-level autonomous request control. The PIM scheduler relieves the host processor of significant computational loads, achieving the overall latency of the system and reducing the energy consumption. By the performance evaluation based on the trace-driven HMC simulator, our proposed model improves average latency and bandwidth by 38.9 and 27.9 % with only 18.7 % more energy consumption compared with conventional HMC-based PIM systems. Our system also achieves scalable processing performance because when the DCNN becomes deeper, it processes faster than conventional PIM systems.

MEG

Emerging three-dimensional (3D) memory technologies, such as the Hybrid Memory Cube (HMC) and High Bandwidth Memory (HBM), provide high-bandwidth and massive memory-level parallelism. With the growing heterogeneity and complexity of computer systems (CPU cores and accelerators, etc.), efficiently integrating emerging memories into existing systems poses new challenges and requires detailed evaluation in a realistic computing environment. In this article, we propose MEG, an open source, configurable, cycle-exact, and RISC-V-based full-system emulation infrastructure using FPGA and HBM. MEG provides a highly modular hardware design and includes a bootable Linux image for a realistic software flow, so that users can perform cross-layer software-hardware co-optimization in a full-system environment. To improve the observability and debuggability of the system, MEG also provides a flexible performance monitoring scheme to guide the performance optimization. The proposed MEG infrastructure can potentially benefit broad communities across computer architecture, system software, and application software. Leveraging MEG, we present two cross-layer system optimizations as illustrative cases to demonstrate the usability of MEG. In the first case study, we present a reconfigurable memory controller to improve the address mapping of standard memory controller. This reconfigurable memory controller along with its OS support allows us to optimize the address mapping scheme to fully exploit the massive parallelism provided by the emerging three-dimensional (3D) memories. In the second case study, we present a lightweight IOMMU design to tackle the unique challenges brought by 3D memory in providing virtual memory support for near-memory accelerators. We provide a prototype implementation of MEG on a Xilinx VU37P FPGA and demonstrate its capability, fidelity, and flexibility on real-world benchmark applications. We hope MEG fills a gap in the space of publicly available FPGA-based full-system emulation infrastructures, specifically targeting memory systems, and inspires further collaborative software/hardware innovations.

NZESPA: A Near-3D-Memory Zero Skipping Parallel Accelerator for CNNs

Convolutional neural networks (CNNs) are one of the most popular machine learning tools for computer vision. The ubiquitous use in several applications with its high computation-cost has made it lucrative for optimization through accelerated architecture. State-of-the-art has either exploited the parallelism of CNNs, or eliminated computations through sparsity or used near-memory processing (NMP) to accelerate the CNNs. We introduce NMP-fully sparse architecture, which acquires all three capabilities. The proposed architecture is parallel and hence processes the independent CNN tasks concurrently. To exploit the sparsity, the proposed system employs a dataflow, namely, Near-3D-Memory Zero Skipping Parallel dataflow or nZESPA dataflow. This dataflow maintains the compressed-sparse encoding of data that skips all ineffectual zero-valued computations of CNNs. We design a custom accelerator which employs the nZESPA dataflow. The grids of nZESPA modules are integrated into the logic layer of the hybrid memory cube. This integration saves a significant amount of off-chip communications while implementing the concept of NMP. We compare the proposed architecture with three other architectures which either do not exploit sparsity (NMP-dense) or do not employ NMP (traditional-fully sparse) or do not include both (traditional-dense). The proposed system outperforms the baselines in terms of performance and energy consumption while executing CNN inference.

Power-Time Exploration Tools for NMP-Enabled Systems

Recently, dramatic improvements in memory performance have been highly required for data demanding application services such as deep learning, big data, and immersive videos. To this end, the throughput-oriented memory such as high bandwidth memory (HBM) and hybrid memory cube (HMC) has been introduced to provide a high bandwidth. For its effective use, various research efforts have been conducted. Among them, the near-memory-processing (NMP) is a concept that utilizes bandwidth and power consumption by placing computation logic near the memory. In the NMP-enabled system, a processor hierarchy consisting of hosts and NMPs is formed based on the distance from the main memory. In this paper, an evaluation tool is proposed to obtain the optimal design decision considering the power-time trade-off in the processor hierarchy. Every time the operating condition and constraints change, the decision of task-level offloading is dynamically made. For the realistic NMP-enabled system environment, the relationship among HBM, host, and NMP should be carefully considered. Hosts and NMPs are almost hidden from each other and the communications between them are extremely limited. In the simulation results, popular benchmarks and a machine learning application are used to demonstrate power-time trade-offs depending on applications and system conditions.

(Invited) Trends in 3D Markets and Technology

Driven by performance, 3D packaging with through silicon vias (TSVs) has become a reality for memory and advanced logic. New versions are emerging. High Bandwidth Memory (HBM) is one of the most important 3D IC developments in the last 10 years. In the last decade, stacked DRAM with through silicon vias has transitioned from a handful of research programs to rapidly increasing volumes. Tezzaron has been providing small quantities of 3D ICs for high-speed memory applications since 2005. Micron, Samsung, and SK Hynix began producing DRAM stacks with TSVs in late 2014 and early 2015. Micron began shipments of its Hybrid Memory Cube (HMC) in 2015. DRAMs and the logic controller were interconnected with TSVs. HMC was packaged in a BGA and tested before assembly on the board. The HMC has been used in Intel’s Knights Landing. The silicon-on-insulator (SOI) logic layer was fabricated by GLOBALFOUNDRIES (which purchased IBM’s fab) and the memory was fabricated by Micron. Micron used a thermo-compression bond (TCB) process with a non-conductive film (NCF) underfill for its die stacking in the HMC. Thermal issues were a challenge. Today Samsung and SKHynix are providing HBM stacks to the industry and Micron is expected to ship products next year. In February 2018, Samsung announced that its first “three-layer” stacked CMOS image sensors (CIS) were in mass production and found in the Galaxy S9/S9+. This follows Sony’s adoption of three-layer CIS technology one year earlier in February 2017. A third silicon layer in the form of DRAM is added to the pixel array and signal processing layers to significantly improve data readout speeds. These devices are manufactured with 3D IC technologies using TSVs, but Samsung and Sony use different design approaches. Technically speaking, Samsung’s solution is a two-layer image sensor with a stacked DRAM. Sony uses a true three-layer bonded approach. In its three-layer CIS technology, Sony places the DRAM between the pixel array die and the image processor, with all three layers interconnected using TSVs. The pixel array, DRAM, and logic wafers are manufactured with 90nm, 30nm, and 40nm processes, respectively. While logic and memory stacks have remained elusive, Intel and TSMC have introduced new packaging technology with active interposers that are considered 3D. Intel has introduced its Foveros 3D integration technology as a form of heterogeneous system integration. The technology uses a 3D face-to-face stacking process. In the process, logic die are bumped and mounted on an active interposer next to memory or die with other functions. The active interposer can contain active parts of the system, such as the platform controller hub (PCH) that manages I/O for the system. The active interposer is mounted on a package substrate with solder bumps. TSMC has introduced its SoIC with face-to-face stacking and wafer-on-wafer (WoW). These technologies are front-end packaging and use direct or fusion bonding. The SoIC is a wafer with logic, with our without TSVs.

Thermal-aware processing-in-memory instruction offloading

With the advent of die stacking technology and big data applications, Processing-in-memory (PIM) is regaining attention as a promising technology for improving performance and energy efficiency. Although various PIM techniques have been proposed in recent studies for effectively offloading computation from the host, the thermal impacts of PIM offloading have not been fully explored. This paper investigates the thermal constraints of PIM and proposes techniques to enable thermal awareness for efficient PIM offloading. To understand the thermal effects of 3D-stacked designs, we measure the temperature of a real Hybrid Memory Cube (HMC) prototype and observe that compared to conventional DRAM, HMC reaches a significantly higher operating temperature, which causes thermal shutdowns with a passive cooling solution. Even with a commodity-server cooling solution, when in-memory processing is highly utilized, HMC fails to maintain the temperature of the memory dies within the normal operating range. In this paper, we propose a collection of software- and hardware-based techniques to enable thermal-aware PIM offloading by controlling the intensity of PIM offloading at runtime. Our evaluation results show that the proposed techniques achieve up to 1.4 × and 1.37× speedups compared to non-offloading and naïve offloading scenarios.

A Scalable Near-Memory Architecture for Training Deep Neural Networks on Large In-Memory Datasets

Most investigations into near-memory hardware accelerators for deep neural networks have primarily focused on inference, while the potential of accelerating training has received relatively little attention so far. Based on an in-depth analysis of the key computational patterns in state-of-the-art gradient-based training methods, we propose an efficient near-memory acceleration engine called NTX that can be used to train state-of-the-art deep convolutional neural networks at scale. Our main contributions are: (i) a loose coupling of RISC-V cores and NTX co-processors reducing offloading overhead by <inline-formula><tex-math notation="LaTeX">$7\times$</tex-math></inline-formula> over previously published results; (ii) an optimized IEEE 754 compliant data path for fast high-precision convolutions and gradient propagation; (iii) evaluation of near-memory computing with NTX embedded into residual area on the Logic Base die of a Hybrid Memory Cube; and (iv) a scaling analysis to meshes of HMCs in a data center scenario. We demonstrate a <inline-formula><tex-math notation="LaTeX">$2.7\times$</tex-math></inline-formula> energy efficiency improvement of NTX over contemporary GPUs at <inline-formula><tex-math notation="LaTeX">$4.4\times$</tex-math></inline-formula> less silicon area, and a compute performance of 1.2 Tflop/s for training large state-of-the-art networks with full floating-point precision. At the data center scale, a mesh of NTX achieves above 95 percent parallel and energy efficiency, while providing <inline-formula><tex-math notation="LaTeX">$2.1\times$</tex-math></inline-formula> energy savings or <inline-formula><tex-math notation="LaTeX">$3.1\times$</tex-math></inline-formula> performance improvement over a GPU-based system.

GraphH: A Processing-in-Memory Architecture for Large-Scale Graph Processing

Large-scale graph processing requires the high bandwidth of data access. However, as graph computing continues to scale, it becomes increasingly challenging to achieve a high bandwidth on generic computing architectures. The primary reasons include: the random access pattern causing local bandwidth degradation, the poor locality leading to unpredictable global data access, heavy conflicts on updating the same vertex, and unbalanced workloads across processing units. Processing-in-memory (PIM) has been explored as a promising solution to providing high bandwidth, yet open questions of graph processing on PIM devices remain in: 1) how to design hardware specializations and the interconnection scheme to fully utilize bandwidth of PIM devices and ensure locality and 2) how to allocate data and schedule processing flow to avoid conflicts and balance workloads. In this paper, we propose GraphH, a PIM architecture for graph processing on the hybrid memory cube array, to tackle all four problems mentioned above. From the architecture perspective, we integrate SRAM-based on-chip vertex buffers to eliminate local bandwidth degradation. We also introduce reconfigurable double-mesh connection to provide high global bandwidth. From the algorithm perspective, partitioning and scheduling methods like index mapping interval-block and round interval pair are introduced to GraphH, thus workloads are balanced and conflicts are avoided. Two optimization methods are further introduced to reduce synchronization overhead and reuse on-chip data. The experimental results on graphs with billions of edges demonstrate that GraphH outperforms DDR-based graph processing systems by up to two orders of magnitude and $5.12 {\times }$ speedup against the previous PIM design.

Subsystem under 3D-Storage Class Memory on a chip

In this paper, we propose a subsystem architecture under 3D-Storage Class Memory (3D-SCM), termed SuS, to solve the memory and power wall problems. Placing the processing unit under the 3D-SCM achieves high performance. We evaluate SuS using gem5 and the GPGPU-Sim simulator. The simulation results for the central processing unit (CPU)-based SuS architecture reveal a 100% performance improvement and a 73% energy reduction compared to the CPU architecture using dual in-line memory modules and a 4% performance improvement with a 27% energy reduction compared to the CPU architecture using hybrid memory cube. Moreover, the graphics processing unit (GPU)-based SuS architecture simulation results on the neural network benchmark demonstrate performance improvements of 17% and 7% compared to GPU architectures with graphics double data rate series memory and high bandwidth memory, respectively.

An Accelerator for Resolution Proof Checking based on FPGA and Hybrid Memory Cube Technology

Modern Boolean satisfiability solvers can emit proofs of unsatisfiability. There is substantial interest in being able to verify such proofs and also in using them for further computations. In this paper, we present an FPGA accelerator for checking resolution proofs, a popular proof format. Our accelerator exploits parallelism at the low level by implementing the basic resolution step in hardware, and at the high level by instantiating a number of parallel modules for proof checking. Since proof checking involves highly irregular memory accesses, we employ Hybrid Memory Cube technology for accelerator memory. The results show that while the accelerator is scalable and achieves speedups for all benchmark proofs, performance improvements are currently limited by the overhead of transitioning the proof into the accelerator memory.

PIM-WEAVER: A High Energy-efficient, General-purpose Acceleration Architecture for String Operations in Big Data Processing

The ever-growing data volume in big data era drives designers to find more efficient processor architectures both in performance and energy consumption. Among various computing patterns in big data applications, string operations are common but important parts of data processing. However, due to the consideration of generality, current general-purpose CPUs are not efficient in both performance and energy consumption when processing simple and fixed computation patterns of discrete string operations. On the other hand, moving massive data from memory to computing units through NoC, Cache hierarchies, and other memory access data paths is time-consuming and especially energy consuming. Fortunately, emerging technologies, such as Hybrid Memory Cube (HMC), enable the processing-in-memory (PIM) functionality without transferring massive data to remote processing units. In this paper, we propose PIM-WEAVER, a high-efficiency novel acceleration architecture for string processing using PIM mechanism, which is the 3D integration technology that facilitates stacking logic and memory dies in a single package. The PIM-WEAVER is implemented using such technology by integrating string processing units into the real world HMC memory. In PIM-WEAVER, the general-purpose acceleration architecture for string operation is implemented within the memory cube, which can reduce the latency of data transfer and also save energy. We also propose a full-stack solution of programming interface and control mechanism of instruction level. Our comprehensive evaluations using typical string processing algorithms from big data applications show that the PIM-WEAVER gains an average speedup of 14.74x over high-performance Intel processor, and reduces the energy consumption by 82.1% on average, with tiny area overhead.

Ultra-Short-Reach Interconnects for Die-to-Die Links: Global Bandwidth Demands in Microcosm

To meet the ever-increasing demand of global data traffic, the Hybrid Memory Cube (HMC) specification and High-Bandwidth Memory standard call for an aggregate bandwidth of 8 Tb/s in next-generation memory-to-processor links [1], [2]. At the same time, the data rates processed by individual line cards in data center switches will soon go beyond 25 Tb/s [3]. However, as the data rate of serial links over copper interconnects increases, the links appear highly lossy. Further, the power consumption of a wireline transceiver, including all clocking and equalization circuits, increases. Publications suggest a tenfold increase in power consumption with a 30-dB increase in channel loss [4]. Hence, energyefficient communication equipment should be engineered to keep the channel loss as low as possible.

CGAcc: A Compressed Sparse Row Representation-Based BFS Graph Traversal Accelerator on Hybrid Memory Cube

Graph traversal is widely used in map routing, social network analysis, causal discovery and many more applications. Because it is a memory-bound process, graph traversal puts significant pressure on the memory subsystem. Due to poor spatial locality and the increasing size of today’s datasets, graph traversal consumes an ever-larger part of application execution time. One way to mitigate this cost is memory prefetching, which issues requests from the processor to the memory in anticipation of needing certain data. However, traditional prefetching does not work well for graph traversal due to data dependencies, the parallel nature of graphs and the need to move vast amounts of data from memory to the caches. In this paper, we propose a compressed sparse row representation-based graph accelerator on the Hybrid Memory Cube (HMC), called CGAcc. CGAcc combines Compressed Sparse Row (CSR) graph representation with in-memory prefetching and processing to improve the performance of graph traversal. Our approach integrates the prefetching and processing in the logic layer of a 3D stacked Dynamic Random-Access Memory (DRAM) architecture, based on Micron’s HMC. We selected HMC to implement CGAcc because it can provide quite high bandwidth and low access latency. Furthermore, this device has multiple DRAM layers connected to internal logic to control memory access and perform rudimentary computation. Using the CSR representation, CGAcc deploys prefetchers in the HMC to exploit the short transaction latency between the logic and DRAM layers. By doing this, it can also avoid large data movement costs. In the runtime, CGAcc pipelines the prefetching to fetch data from DRAM arrays to improve memory-level parallelism. To further reduce the access latency, several optimized internal caches are also introduced to hold the prefetched data to be Processed In-Memory (PIM). A comprehensive evaluation shows the effectiveness of CGAcc. Experimental results showed that, compared to a conventional HMC main memory equipped with a stream prefetcher, CGAcc achieved an average 3.51× speedup with moderate hardware cost.

Autonomous high‐speed serial link power management depending on required link performance for HMC

Many studies on 3D-stacked dynamic RAMs (DRAMs) have been conducted to overcome the shortcomings of conventional DRAM. The hybrid memory cube (HMC) is one of the most promising 3D-stacked DRAMs, thanks to its high bandwidth and expandable structure. However, a high-speed serial link that interfaces the CPU and HMC consumes significant power, primarily because of the high overhead incurred in synchronising its clock. Although the link provides low-power modes, managing them is very difficult because of their long mode transition times. An autonomous power management method for the high-speed link is proposed. The proposed method determines the optimal number of active links while satisfying the required link performance. Simulations demonstrate that the proposed method reduces link power consumption by an average of 63.06% with a performance degradation of only 1.36%. Therefore, this proposed autonomous link power management is an outstanding option for low-power HMC-based systems.