Memory Energy Consumption Research Articles

Ionic–electronic dynamics in an electrochemical gate stack toward high-speed artificial synapses

Despite their great synaptic potential, the trade-off between programming speed and energy consumption of electrochemical random-access memory (ECRAM) devices are major hindrance to their incorporation into practical applications. In this work, we experimentally study the main limiting factor for high-speed programming of ECRAMs, the ionic current in the gate stack. We use two-terminal structures composed of LiCoO2/Li3PO4/amorphous-Si to represent the ECRAM gate stack (reservoir/electrolyte/channel). We perform electrical characterization including impedance spectroscopy (small-signal) and large-signal transient measurements across nine orders of magnitude in the time domain. We find that at the sub-microseconds range, the current is governed by the energy barrier for Li+ ions at the electrolyte interfaces. After a period of ∼1 μs, ionic migration through the ∼80 nm electrolyte layer dictates the current. At ∼50 μs, the ionic double layer at the interface is fully charged and the gate current drops by several orders of magnitude, indicating that the Li3PO4/Si interface is saturated, and the measured current is dominated by the electronic leakage component. Furthermore, we evaluate ECRAM performance under various pulse parameters. Our predictions show that an aggressively scaled (atomically thin) channel having a low carrier density of ∼1011 cm−2 can be programmed at ∼nanosecond using a gate current of ∼100 A/cm2.

Ultrafast response of Ge2Sb2Te5 nanoparticles: The benefits of low energy amorphization switching with the same read/write speed of bulk memories

We investigate the ultrafast response of crystalline Ge2Sb2Te5 nanoparticles (NPs) below the phase transformation threshold fluence. The observed rapid change of the optical response and the presence of coherent optical phonons are consistent with the relaxation dynamics in bulk Ge2Sb2Te5 films and, more importantly, occur within the same ultrafast timescales. We conclude that the benefit of the lower energy consumption of memories based on Ge-Sb-Te (GST) NPs aggregates, demonstrated by Casarin et al. (2018), occurs with no disadvantage, as the read/write speed can be as fast as in bulk GSTs.

Adaptive Resource Planning for AI Workloads with Variable Real-Time Tasks

AI (Artificial Intelligence) workloads are proliferating in modern real-time systems. As the tasks of AI workloads fluctuate over time, resource planning policies used for traditional fixed real-time tasks should be re-examined. In particular, it is difficult to immediately handle changes in real-time tasks without violating the deadline constraints. To cope with this situation, this paper analyzes the task situations of AI workloads and finds the following two observations. First, resource planning for AI workloads is a complicated search problem that requires much time for optimization. Second, although the task set of an AI workload may change over time, the possible combinations of the task sets are known in advance. Based on these observations, this paper proposes a new resource planning scheme for AI workloads that supports the re-planning of resources. Instead of generating resource plans on the fly, the proposed scheme pre-determines resource plans for various combinations of tasks. Thus, in any case, the workload is immediately executed according to the resource plan maintained. Specifically, the proposed scheme maintains an optimized CPU (Central Processing Unit) and memory resource plan using genetic algorithms and applies it as soon as the workload changes. The proposed scheme is implemented in the open-source simulator SimRTS for the validation of its effectiveness. Simulation experiments show that the proposed scheme reduces the energy consumption of CPU and memory by 45.5% on average without deadline misses.

BIST-Supported Cryogenic Write Trimming With In-MRAM Computing Case Study

In the processor-memory separated von- Neumann computation paradigm, the “memory-wall” effect becomes critical due to large access latency and tremendous amount of data movement. In this work, we pursue cryogenic temperature based memory design and focus on spin-transfer-torque magnetoresistive random access memory (STT-MRAM) at 77-Kelvin (achieved with low-cost liquid nitrogen). Cryogenic compact model and its related cryogenic bit-cell are investigated, based on 77 K experiment data of magnetic tunnel junction (MTJ) and CMOS transistor. Compared with the simulated room temperature results, 70% reduction of sensing latency can be obtained at 0.7 V supply voltage, whereas writing latency and yield performance are significantly degraded. In order to overcome the above drawbacks, a novel joint trimming system is proposed including a build-in- self-test (BIST) block and a string of multi-stage level shifters (LS). A case study is executed with in-MRAM computing (IMC). Results show that cryogenic IMC with write trimming provides performance improvements of 33% on average, and concurrently reduces memory energy consumption by 70% on average. The proposed 77 K cryogenic design method can be further applied to other energy constrained applications.

Optimizing the Energy Efficiency of Unreliable Memories for Quantized Kalman Filtering

This paper presents a quantized Kalman filter implemented using unreliable memories. We consider that both the quantization and the unreliable memories introduce errors in the computations, and we develop an error propagation model that takes into account these two sources of errors. In addition to providing updated Kalman filter equations, the proposed error model accurately predicts the covariance of the estimation error and gives a relation between the performance of the filter and its energy consumption, depending on the noise level in the memories. Then, since memories are responsible for a large part of the energy consumption of embedded systems, optimization methods are introduced to minimize the memory energy consumption under the desired estimation performance of the filter. The first method computes the optimal energy levels allocated to each memory bank individually, and the second one optimizes the energy allocation per groups of memory banks. Simulations show a close match between the theoretical analysis and experimental results. Furthermore, they demonstrate an important reduction in energy consumption of more than 50%.

Energy-Efficient Dataflow Scheduling of CNN Applications for Vector-SIMD DSP

Dataflow-scheduling techniques for convolutional neural networks (CNNs) are extensively studied to minimize the off-chip memory access. However, the efficiencies of the previously proposed techniques are limited because their optimizations only consider the general hardware such as FPGA and GPU. To overcome this limitation, this paper proposes dataflow scheduling for vector-SIMD DSP to minimize the energy consumption for the off-chip memory access. First, the proposed technique attempts to group as many given layers as possible. For grouping the layers, the tiles in different layers are executed in sequence without the off-chip memory access except the first and the last layers in the group. The length of the grouped layers is determined with regard to the minimization of the energy consumption of off-chip memory by estimating the proposed energy model of the off-chip memory. However, grouping the layers results in the additional computation. To minimize this overhead, this paper solves the optimization problem for in the grouped layers. Second, for layers that cannot be grouped, the tiling along the W-axis is not considered, to maximize the size of the overlapped data in consecutive tiles. Consequently, the reuse of the overlapped data in the on-chip buffer is maximized, thereby reducing the energy consumption by the off-chip memory. For evaluation, a cycle-accurate simulation environment is established to measure the energy consumption of the off-chip memory by tracing the data between a vector-SIMD DSP and an off-chip memory. The experimental results show that compared with the baseline tiling and scheduling techniques, the proposed technique reduces the energy consumption by an average of 51% for CNN applications such as Tiny YOLOv2, MobileNetv1, VDSR.

Low-energy motion estimation memory system with dynamic management.

The digital video coding process imposes severe pressure on memory traffic, leading to considerable power consumption related to frequent DRAM accesses. External off-chip memory demand needs to be minimized by clever architecture/algorithm co-design, thus saving energy and extending battery lifetime during video encoding. To exploit temporal redundancies among neighboring frames, the motion estimation (ME) algorithm searches for good matching between the current block and blocks within reference frames stored in external memory. To save energy during ME, this work performs memory accesses distribution analysis of the test zone search (TZS) ME algorithm and, based on this analysis, proposes both a multi-sector scratchpad memory design and dynamic management for the TZS memory access. Our dynamic memory management, called neighbor management, reduces both static consumption—by employing sector-level power gating—and dynamic consumption—by reducing the number of accesses for ME execution. Additionally, our dynamic management was integrated with two previously proposed solutions: a hardware reference frame compressor and the Level C data reuse scheme (using a scratchpad memory). This system achieves a memory energy consumption savings of 99.8% and, when compared to the baseline solution composed of a reference frame compressor and data reuse scheme, the memory energy consumption was reduced by 44.1% at a cost of just 0.35% loss in coding efficiency, on average. When compared with related works, our system presents better memory bandwidth/energy savings and coding efficiency results.

TA-CLOCK: Tendency-Aware Page Replacement Policy for Hybrid Main Memory in High-Performance Embedded Systems

Recently, high-performance embedded systems have adopted phase change memory (PCM) as their main memory because PCMs have attractive advantages, such as non-volatility, byte-addressability, high density, and low power consumption. However, PCMs have disadvantages, such as limited write endurance in each cell and high write latency compared to DRAMs. Therefore, researchers have investigated methods for enhancing the limitations of PCMs. In this paper, we propose a page replacement policy called tendency-aware CLOCK (TA-CLOCK) for the hybrid main memory of embedded systems. To improve the limited write endurance of PCMs, TA-CLOCK classifies the page access tendency of the victim page through access pattern analysis and determines the migration location of the victim page. Through the classification of the page access tendency, TA-CLOCK reduces unnecessary page migrations from DRAMs to PCMs. Unnecessary migrations cause an increase in write operations in PCMs and the energy consumption of the hybrid main memory in embedded systems. Thus, our proposed policy improves the limited write endurance of PCMs and enhances the access latency of the hybrid main memory of embedded systems by classifying the page access tendency. We compared the TA-CLOCK with existing page replacement policies to evaluate its performance. In our experiments, TA-CLOCK reduced the number of write operations in PCMs by 71.5% on average, and it enhanced the energy delay product by 38.3% on average compared with other page replacement policies.

Thermal design power and vectorized instructions behavior

AbstractVectorized instructions were introduced to improve the performance of applications. However, they come at the cost of an increase in the power consumption. As a consequence, processors are designed to limit their frequency when such instructions are used in order to respect the thermal design power limit. In this paper, we study and compare the impact of thermal design power and Simple Instruction Multiple Data (SIMD) instructions on performance, power and energy consumption of processors and memory. The study is performed on three different architectures providing different characteristics and four applications with different profiles (including one application with different phases, each phase having a different profile). The study shows that, because of processor frequency, performance and power consumption are strongly related to thermal design power. It also shows that AVX512 has unexpected behavior regarding processor power consumption, while DRAM power consumption is impacted by SIMD instructions because of the generated memory throughput. Finally, this paper tackles the impact of turboboost which shows equivalent to better performance for all the studied cases while not always decreasing energy consumption.

High-Performance Predictable NVM-Based Instruction Memory for Real-Time Embedded Systems

Worst case execution time and energy consumption are two of the most important design constraints of real-time embedded systems and memory subsystem has a major impact on both of them. Therefore, many recent studies have tried to improve the memory subsystem of embedded systems by using emerging non-volatile memories instead of conventional memories such as SRAM and DRAM. Indeed, the low leakage power dissipation and improved density of emerging non-volatile memories make them prime candidates for replacing the conventional memories. However, accessing these memories imposes performance and energy overhead and using them as the instruction memory could increase the worst case execution time, which would have a negative impact on the system. Furthermore, most previous studies that have tried to address such problems have focused on the data memory and therefore their solutions are not suitable for the instruction memory. In this paper, a new instruction memory architecture for non-volatile memories is proposed which reduces the effective memory access latency by employing memory access interleaving technique. Unlike common instruction access latency improvement techniques such as prefetching which usually increase the worst case execution time of the system, the proposed architecture is predictable and does not increase the worst case execution time of the system. Furthermore, it improves both average case execution time and energy consumption of the system and requires no changes to the application code. The proposed architecture has been evaluated using different applications from MiBench and Malardalen benchmark suites and the results show that compared to previous studies, the proposed architecture can improve the memory energy consumption, the average case execution time, and the worst case execution time of the system by 73, 61, and 27 percent respectively.

RBC: A memory architecture for improved performance and energy efficiency

DRAM-based memory suffers from increasing row buffer conflicts, which causes significant performance degradation and power consumption. As memory capacity increases, the overheads of the row buffer conflict are increasingly worse as increasing bitline length, which results in high row activation and precharge latencies. In this work, we propose a practical approach called Row Buffer Cache (RBC) to mitigate row buffer conflict overheads efficiently. At the core of our proposed RBC architecture, the rows with good spatial locality are cached and protected, which are exempted from being interrupted by the accesses for rows with poor locality. Such an RBC architecture significantly reduces the overheads of performance and energy caused by row activation and precharge, and thus improves overall system performance and energy efficiency. We evaluate RBC architecture using SPEC CPU2006 on a DDR4 memory compared to a commodity baseline memory system. Results show that RBC improves the overall performance by up to 2.24× (16.1% on average) and reduces the memory energy by up to 68.2% (23.6% on average) for single-core simulations. For multi-core simulations, RBC increases the overall performance by up to 1.55× (17% on average) and reduces memory energy consumption by up to 35.4% (21.3% on average).

An air-stable two-dimensional perovskite artificial synapse

Organometal halide perovskites (OHPs) exhibit extraordinary properties in the emulation of synaptic plasticity in artificial synaptic electronic devices. However, the instability of OHPs has hindered their further application. Here, a novel two-terminal artificial synapse based on two-dimensional perovskite (R-MPEA)1.5PbBr3.5(DMSO)0.5 was developed to address this challenge. Essential biological functions including excitatory post-synaptic current, paired-pulse facilitation, spike-number dependent plasticity, spike-voltage dependent plasticity, spike-duration dependent plasticity, and spike-rate dependent plasticity, have been demonstrated in OHP-based synaptic devices. More importantly, this is the first demonstration of an air-stable OHP-based artificial synapse. The OHP-based synaptic device achieved extremely low energy consumption (∼21 fJ per event) in short-term memory and exhibited good stability at ambient atmosphere over 300 h. These results may open a new avenue for next-generation synaptic electronics and neuromorphic computing.

DSM

The number of cores and the capacities of main memory in modern systems have been growing significantly. Specifically, memory scaling, although at a slower pace than computation scaling, provided opportunities for very large DRAMs with Terabytes (TBs) capacity. Consequently, addressing the performance and energy consumption bottlenecks of DRAMs is more important than ever. DRAM memory refresh operation is one of the main contributing factors to the memory overheads, especially for large capacity DRAMs used in modern servers and emerging large-scale data centers. This paper addresses the memory refresh problem by leveraging the fact that most cloud servers host virtualized systems that use similar kernels, libraries, etc. We propose and experimentally evaluate a novel approach that exploits this observation to address the DRAM refresh overhead in such systems. More specifically, in this work, we present DSM, a light-weight hardware extension in memory controller to detect the pages with same content in memory and refresh only one of them and redirect the requests to the others to this page. Our detailed experimental analysis shows that the proposed DSM design can reduce 99th percentile memory access latency by up to 2.01x, and it also reduces the overall memory energy consumption by up to 8.5%.

DSM

The number of cores and the capacities of main memory in modern systems have been growing significantly. Specifically, memory scaling, although at a slower pace than computation scaling, provided opportunities for very large DRAMs with Terabytes (TBs) capacity. Consequently, addressing the performance and energy consumption bottlenecks of DRAMs is more important than ever. DRAM memory refresh operation is one of the main contributing factors to the memory overheads, especially for large capacity DRAMs used in modern servers and emerging large-scale data centers. This paper addresses the memory refresh problem by leveraging the fact that most cloud servers host virtualized systems that use similar kernels, libraries, etc. We propose and experimentally evaluate a novel approach that exploits this observation to address the DRAM refresh overhead in such systems. More specifically, in this work, we present DSM, a light-weight hardware extension in memory controller to detect the pages with same content in memory and refresh only one of them and redirect the requests to the others to this page. Our detailed experimental analysis shows that the proposed DSM design can reduce 99\textsuperscriptth percentile memory access latency by up to 2.01x, and it also reduces the overall memory energy consumption by up to 8.5%.

Cryogenic Memory Architecture Integrating Spin Hall Effect based Magnetic Memory and Superconductive Cryotron Devices

One of the most challenging obstacles to realizing exascale computing is minimizing the energy consumption of L2 cache, main memory, and interconnects to that memory. For promising cryogenic computing schemes utilizing Josephson junction superconducting logic, this obstacle is exacerbated by the cryogenic system requirements that expose the technology’s lack of high-density, high-speed and power-efficient memory. Here we demonstrate an array of cryogenic memory cells consisting of a non-volatile three-terminal magnetic tunnel junction element driven by the spin Hall effect, combined with a superconducting heater-cryotron bit-select element. The write energy of these memory elements is roughly 8 pJ with a bit-select element, designed to achieve a minimum overhead power consumption of about 30%. Individual magnetic memory cells measured at 4 K show reliable switching with write error rates below 10−6, and a 4 × 4 array can be fully addressed with bit select error rates of 10−6. This demonstration is a first step towards a full cryogenic memory architecture targeting energy and performance specifications appropriate for applications in superconducting high performance and quantum computing control systems, which require significant memory resources operating at 4 K.

Hierarchical Hybrid Memory Management in OS for Tiered Memory Systems

The emerging hybrid DRAM-NVM architecture is challenging the existing memory management mechanism at the level of the architecture and operating system. In this paper, we introduce Memos, a memory management framework which can hierarchically schedule memory resources over the entire memory hierarchy including cache, channels, and main memory comprising DRAM and NVM simultaneously. Powered by our newly designed kernel-level monitoring module that samples the memory patterns by combining TLB monitoring with page walks, and page migration engine, Memos can dynamically optimize the data placement in the memory hierarchy in response to the memory access pattern, current resource utilization, and memory medium features. Our experimental results show that Memos can achieve high memory utilization, improving system throughput by around 20.0 percent; reduce the memory energy consumption by up to 82.5 percent; and improve the NVM lifetime by up to 34X.

HyVE: Hybrid Vertex-Edge Memory Hierarchy for Energy-Efficient Graph Processing

High energy consumption of conventional memory modules (e.g., DRAMs) hinders the further improvement of large-scale graph processing’s energy efficiency. The emerging resistive random-access memory (ReRAM) has shown great potential in providing an energy-efficient memory module. However, the performance of ReRAMs suffers from data access patterns with poor locality and large amounts of written data, which are common in graph processing. In this paper, we propose HyVE, a Hybrid Vertex-Edge memory hierarchy for energy-efficient graph processing. In HyVE, we avoid random access and data written to ReRAM modules. HyVE can reduce memory energy consumption by 86.17 percent compared with conventional memory systems. We have also proposed data sharing and bank-level power-gating schemes, which improve the energy efficiency by 1.60x and 1.53x. By analyzing the graph processing model on ReRAMs, we show that ReRAMs are good for read-intensive operations in graph processing (e.g., reading edges), while ReRAM crossbars are not suitable for processing edges because of heavy writing overheads. Our evaluations show that the optimized design achieves two orders of magnitude and 5.90x energy efficiency improvement compared with the CPU-based and conventional memory hierarchy based designs, respectively. Moreover, HyVE achieves 2.83x energy reduction compared with the previous ReRAM-based graph processing architecture.

OCMAS: Online Page Clustering for Multibank Scratchpad Memory

Scratchpad memory (SPM), a software-controlled on-chip memory, is being increasingly used in embedded systems to reduce on-chip memory energy consumption. To further reduce energy consumption, multibank SPM architecture is proposed. In multibank SPM, each bank can be accessed independently, and unused banks can enter the low power mode, thus reducing leakage energy. However, if both frequently and infrequently used data exist in the same bank, the bank will not be able to enter the low power mode, resulting in less energy reduction. To address this issue, we propose online page clustering for multibank SPM (OCMAS) to reduce the leakage energy in multibank SPM. OCMAS groups SPM pages with similar access frequencies into the same bank, allowing banks containing infrequently used data to stay in low power mode longer. We also propose a method to dynamically adjust the thresholds for determining cold pages (pages that contain infrequently used data), so banks that contain cold pages can enter the low power mode with a shorter idle timeout. Compared to conventional timeout-based, periodic drowsy, and bank-based methods, OCMAS can reduce the energy delay product by up to 37.67% (18.14% on average), 39.53% (22.38% on average), and 132.32% (23.34% on average) in 32 KB 4-bank SPM, by up to 25.33% (15.71% on average), 29.87% (15.92% on average), and 72.25% (22.64% on average) in 32 KB 8-bank SPM, by up to 28.99% (13.45% on average), 30.71% (14.74% on average), and 96.67% (19.94% on average) in 16 KB 4-bank SPM, and by up to 30.18% (10.05% on average), 32.13% (11.62% on average), and 65.56% (16.2% on average) in 16 KB 8-bank SPM. The area overhead is approximately 0.72%, which is insignificant.

Tailoring Magnetoelectric Coupling in BiFeO3 /La0.7 Sr0.3 MnO3 Heterostructure through the Interface Engineering.

Electric field control of magnetism ultimately opens up the possibility of reducing energy consumption of memory and logic devices. Electric control of magnetization and exchange bias are demonstrated in all-oxide heterostructures of BiFeO3 (BFO) and La0.7 Sr0.3 MnO3 (LSMO). However, the role of the polar heterointerface on magnetoelectric (ME) coupling is not fully explored. Here, the ME coupling in BFO/LSMO heterostructures with two types of interfaces, achieved by exploiting the interface engineering at the atomic scale, is investigated. It is shown that both magnetization and exchange bias are reversibly controlled by switching the ferroelectric polarization of BFO. Intriguingly, distinctly different modulation behaviors that depend on the interfacial atomic sequence are observed. These results provide new insights into the underlying physics of ME coupling in the model system. This study highlights that designing interface at the atomic scale is of general importance for functional spintronic devices.

Writeback-Aware LLC Management for PCM-Based Main Memory Systems

With the increase in the number of data-intensive applications on today's workloads, DRAM-based main memories are struggling to satisfy the growing data demand capacity. Phase Change Memory (PCM) is a type of non-volatile memory technology that has been explored as a promising alternative for DRAM-based main memories due to its better scalability and lower leakage energy. Despite its many advantages, PCM also has shortcomings such as long write latency, high write energy consumption, and limited write endurance, which are all related to the write operations. In this article, we propose a novel writeback-aware Last Level Cache (LLC) management scheme named WALL to reduce the number of LLC writebacks and consequently improve performance, energy efficiency, and lifetime of a PCM-based main memory system. First, we investigate the writeback behavior of LLC sets and show that writebacks are not uniformly distributed among sets; some sets observe much higher writeback rates than others. We then propose a writeback-aware set-balancing mechanism, which employs the underutilized LLC sets with few writebacks as an auxiliary storage for the evicted dirty lines from sets with frequent writebacks. We also propose a simple and effective writeback-aware replacement policy to avoid the eviction of the dirty blocks that are highly reused after being evicted from the cache. Our experimental results show that WALL achieves an average of 30.9% reduction in the total number of LLC writebacks, compared to the baseline scheme, which uses the LRU replacement policy. As a result, WALL can reduce the memory energy consumption by 23.1% and enhance PCM lifetime by 1.29×, on average, on an 8-core system with a 4GB PCM main memory, running memory-intensive applications.