Hybrid Memory Research Articles

State estimation and frequency stabilization of multi‐area power systems via fault‐alarm approach

AbstractThis work is mainly focused on designing hybrid memory state feedback load frequency control law for a class of multi‐area power systems against transmission delays, parameter uncertainties, actuator faults, and load disturbances via fault‐alarm approach. More specifically, the load disturbances are prompted by the incorporation of abundance amount of renewable energy resources, namely, photo voltaic and wind power. Additionally, the power system takes the effect of actuator faults into account, which is unavoidable and exists at any instant of time. Thus, to stabilize the addressed system, the fault‐alarm approach is exploited to develop the hybrid control law. Particularly, the state dynamics are well estimated by the aid of residual observer to detect the faulty condition of actuators. Also, by following the algorithm of fault detection, the spontaneous alert from the alarm is accomplished. Further, in consideration of appropriate Lyapunov–Krasovskii functional, the stability conditions having linear matrix inequality form are derived and which affirms the asymptotic stability of the power systems. Precisely, the controller gain matrices are obtained by solving the developed sufficient conditions. Eventually, the simulation results of two‐area interconnected power systems are presented to examine the applicability and advantage of the proposed theoretical result.

DLBF: A low overhead wear leveling algorithm for embedded systems with hybrid memory

Due to its attractive characters, nanosecond ranged access latency and higher bit density，phase change memory (PCM) has emerged as a promising main memory candidate for embedded systems in the future. However, relatively lower endurance has limited its practical application. Discussions regarding hybrid memory wear leveling algorithms have dominated research dereliction in recent years. However, most previous wear leveling schemes could not predict the write and read hot pages accurately and always greatly depend on reference-count recording lists to predict the hot of the pages which would bring higher space overhead to the whole system.In this paper, we propose a novel hardware wear leveling algorithm named DLBF to solve the problems mentioned above. It simultaneously considers both page access frequencies with write/read counting bloom filters and memory access recency with dynamic changing write and read thresholds. The proposed algorithm requires only a small amount of space overhead by adopting write and read counting bloom filters. According to experiment results, DLBF evenly distributes write operations among the entire PCM memory space and extends hybrid system lifetime by 1.47 times and 1.46times while compared with bloom filter algorithm and SWL algorithm. Meanwhile, compared with previous algorithms, the proposed algorithm contributes to improving the PCM hit ratio and system bus utilization.

Long short-term memory self-adapting online random forests for evolving data stream regression

Evolving data stream, especially with concept drift is generally accepted as a challenging data type for regression task, because it usually makes machine learning models trained on old data not adapting to new data, and leads to dramatic performance degradation as a result. Moreover, the behavior of a data stream may change in different modes and therefore introduces various concept drifts, e.g., abrupt, incremental, gradual, recurring, even more, complex concept drifts. Although there are some algorithms that can adapt to stationary data streams or a specific type of concept drift in non-stationary data streams, a wide range of practical applications call for machine learning regression models to handle multi-type of data streams. In this work, we propose an online learning strategy called adaptive long and short-term memories online Random Forests regression(ALSM-RFR), where an adaptive memory activation mechanism is designed to make the model switch adaptively between long-term and hybrid (long-term plus short-term) memory modes in the face of stationary data streams or non-stationary data streams with different types of concept drift. In particular, leaf and tree weights in random forests are used to learn information at different timescales, namely, long-term and short-term memories. Moreover, we devise an adaptive memory activation mechanism to formulate the switch decision of memory modes as a classification problem. Numerical experiments show remarkable improvements of the proposed method in the adaptability of stream types and predictive accuracy in data streams across several real datasets and synthetic datasets, compared to the state-of-the-art online approaches. Besides, the convergence and the influence of the parameters involved in our method are evaluated.

Enabling Homomorphically Encrypted Inference for Large DNN Models

The proliferation of machine learning services in the last few years has raised data privacy concerns. Homomorphic encryption (HE) enables inference using encrypted data but it incurs 100x-10,000x memory and runtime overheads. Secure deep neural network (DNN) inference using HE is currently limited by computing and memory resources, with frameworks requiring hundreds of gigabytes of DRAM to evaluate small models. To overcome these limitations, in this paper we explore the feasibility of leveraging hybrid memory systems comprised of DRAM and persistent memory. In particular, we explore the recently-released Intel Optane PMem technology and the Intel HE-Transformer nGraph to run large neural networks such as MobileNetV2 (in its largest variant) and ResNet-50 for the first time in the literature. We present an in-depth analysis of the efficiency of the executions with different hardware and software configurations. Our results conclude that DNN inference using HE incurs on friendly access patterns for this memory configuration, yielding efficient executions.

SwapKV: A Hotness Aware In-memory Key-Value Store for Hybrid Memory Systems

In-memory Key-Value (KV) stores are widely deployed in modern data centers. These systems generally use DRAM as their storage medium, causing huge hardware costs. The emerging persistent memory (PMEM) is a potential substitute for DRAM, which has a lower price and larger capacity, but lower access speed and bandwidth. Many prior studies strive to build hybrid memory systems to retain both the advantages of DRAM and PMEM. However, they are either application agnostic or simply take DRAM as a cache, which are both not efficient for in-memory KV stores. In this paper, we propose SwapKV, a well-designed in-memory KV store for hybrid DRAM-PMEM system. SwapKV has several promising properties. First, SwapKV combines DRAM and PMEM to a uniform memory pool and only stores one copy of data, which maximizes capacity utilization. Second, SwapKV maps all writing operations to DRAM and migrates data to PMEM with large blocks asynchronously, which mitigates the intrinsic inefficiency of PMEM for writing operations. Third, SwapKV maintains the hot data in DRAM through an efficient hotness filtering and data swapping mechanism, which ensures high system throughput and responsiveness. We implement SwapKV and evaluate it under various workload patterns. The results demonstrate that SwapKV improves the throughput by 11\% <formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex>$\sim$</tex> </formula> 41\% compared to the state-of-the-art alternatives.

An Energy-Efficient Hybrid Tunnel FET based STT-MRAM Memory Cell Design at Low VDD

ABSTRACT Perpendicular Magnetic Anisotropy-based Magnetic Tunnel Junction (PMA-MTJ) and Spin Transfer Torque Magnetic Random-Access Memory (STT-MRAM) have attracted wide attention among the next-generation Non-Volatile Memory (NVM) technologies due to low leakage and high-density characteristics for embedded memory architectures. However, STT-MRAM-based memory cells are not energy efficient with CMOS scaling at scaled supply voltages. This paper presents a novel energy-efficient Hybrid TFET (1 T: Hetero-junction TFET, 1 T: Homo-junction TFET)/STT-MRAM cell that explores p-i-n forward current of Tunnel FET. The proposed hybrid memory cell demonstrates ~27.3% energy efficiency over equivalent 1 T FinFET/STT-MRAM cell, 16.75% energy efficiency over equivalent 1 T Homo-junction TFET/STT-MRAM cell and 24.3% energy efficiency over equivalent 1 T Hetero-junction TFET/STT-MRAM cell at VDD = 0.5 V.

Performance Evaluation of Intel Optane Memory for Managed Workloads

Intel Optane memory offers non-volatility, byte addressability, and high capacity. It suits managed workloads that prefer large main memory heaps. We investigate Optane as the main memory for managed (Java) workloads, focusing on performance scalability. As the workload (core count) increases, we note Optane’s performance relative to DRAM. A few workloads incur a slight slowdown on Optane memory, which helps conserve limited DRAM capacity. Unfortunately, other workloads scale poorly beyond a few core counts. This article investigates scaling bottlenecks for Java workloads on Optane memory, analyzing the application, runtime, and microarchitectural interactions. Poorly scaling workloads allocate objects rapidly and access objects in Optane memory frequently. These characteristics slow down the mutator and substantially slow down garbage collection (GC). At the microarchitecture level, load, store, and instruction miss penalties rise. To regain performance, we partition heaps across DRAM and Optane memory, a hybrid that scales considerably better than Optane alone. We exploit state-of-the-art GC approaches to partition heaps. Unfortunately, existing GC approaches needlessly waste DRAM capacity because they ignore runtime behavior. This article also introduces performance impact-guided memory allocation (PIMA) for hybrid memories. PIMA maximizes Optane utilization, allocating in DRAM only if it improves performance. It estimates the performance impact of allocating heaps in either memory type by sampling. We target PIMA at graph analytics workloads, offering a novel performance estimation method and detailed evaluation. PIMA identifies workload phases that benefit from DRAM with high (94.33%) accuracy, incurring only a 2% sampling overhead. PIMA operates stand-alone or combines with prior approaches to offer new performance versus DRAM capacity trade-offs. This work opens up Optane memory to a real-life role as the main memory for Java workloads.

Order, please! Explicit sequence learning in hybrid search in younger and older age

Sequence learning effects in simple perceptual and motor tasks are largely unaffected by normal aging. However, less is known about sequence learning in more complex cognitive tasks that involve attention and memory processes and how this changes with age. In this study, we examined whether incidental and intentional sequence learning would facilitate hybrid visual and memory search in younger and older adults. Observers performed a hybrid search task, in which they memorized four or 16 target objects and searched for any of those target objects in displays with four or 16 objects. The memorized targets appeared either in a repeating sequential order or in random order. In the first experiment, observers were not told about the sequence before the experiment. Only a subset of younger adults and none of the older adults incidentally learned the sequence. The “learners” acquired explicit knowledge about the sequence and searched faster in the sequence compared to random condition. In the second experiment, observers were told about the sequence before the search task. Both younger and older adults searched faster in sequence blocks than random blocks. Older adults, however, showed this sequence-learning effect only in blocks with smaller target sets. Our findings indicate that explicit sequence knowledge can facilitate hybrid search, as it allows observers to predict the next target and restrict their visual and memory search. In older age, the sequence-learning effect is constrained by load, presumably due to age-related decline in executive functions.

History table-based linear analysis method for DRAM-PCM hybrid memory system

This research is to design a history table-based linear analysis method for a DRAM-PCM (phase change memory) hybrid memory system supporting irregular and complex memory accessing patterns in graph processing applications. This linear analysis method can analyze complex memory access patterns in real time through our history table structure, classifying address values requested from the LLC (last-level cache) as the prefetch candidates. Also, the proposed method is to support a prefetching operation from the storage onto the hybrid main memory, causing significant latency losses. Specifically, we proposed a new memory pattern analysis method, which is called a linear analysis based on a history table, to detect any linear memory access patterns from workloads showing irregular accessing characteristics. For the DRAM-PCM hybrid memory, dynamic data that are frequently accessed are stored in the DRAM. Thus, our linear analysis not only categorizes memory access patterns in the form of line patterns but also separates dynamic and static data so that the hybrid memory systems can be managed adaptively and efficiently. Our experiments show that our model can improve performance and energy consumption by 60% and 30%, respectively, compared to those of the traditional DRAM-only memory model. Moreover, our approach can enhance the entire system performance and energy consumption by 11.3%, 6.8%, compared to streamlined prefetcher-based models.

CLU

Convolutional/Deep Neural Networks (CNNs/DNNs) are rapidly growing workloads for the emerging AI-based systems. The gap between the processing speed and the memory-access latency in multi-core systems affects the performance and energy efficiency of the CNN/DNN tasks. This article aims to alleviate this gap by providing a simple and yet efficient near-memory accelerator-based system that expedites the CNN inference. Towards this goal, we first design an efficient parallel algorithm to accelerate CNN/DNN tasks. The data is partitioned across the multiple memory channels (vaults) to assist in the execution of the parallel algorithm. Second, we design a hardware unit, namely the convolutional logic unit (CLU), which implements the parallel algorithm. To optimize the inference, the CLU is designed, and it works in three phases for layer-wise processing of data. Last, to harness the benefits of near-memory processing (NMP), we integrate homogeneous CLUs on the logic layer of the 3D memory, specifically the Hybrid Memory Cube (HMC). The combined effect of these results in a high-performing and energy-efficient system for CNNs/DNNs. The proposed system achieves a substantial gain in the performance and energy reduction compared to multi-core CPU- and GPU-based systems with a minimal area overhead of 2.37%.

HAM: Hotspot-Aware Manager for Improving Communications With 3D-Stacked Memory

Emerging High-Performance Computing (HPC) workloads, such as graph analytics, machine learning, and big data science, are data-intensive. Data-intensive workloads usually present fine-grained memory accesses with limited or no data locality, and thus incur frequent cache misses and low utilization of memory bandwidth. 3D-stacked memory devices such as Hybrid Memory Cube (HMC) and High Bandwidth Memory (HBM) can provide significantly higher bandwidth than conventional memory modules. However, the traditional interfaces and optimization methods for JEDEC DDR devices do not allow to fully exploit the potential performance of 3D-stacked memory with the massive amount of irregular memory accesses of data-intensive applications. In this article, we propose a novel Hotspot-Aware Manager (HAM) infrastructure for 3D-stacked memory devices capable of optimizing memory access streams via request aggregation, hotspot detection, and in-memory prefetching. We present the HAM design and implementation, and simulate it on a system using RISC-V embedded cores with attached HMC devices. We extensively evaluate HAM with over 12 benchmarks and applications representing diverse irregular memory access patterns. The results show that, on average, HAM reduces redundant requests by 37.51 percent and increases the prefetch buffer hit rate by 4.2 times, compared to a baseline streaming prefetcher. On the selected benchmark set, HAM provides performance gains of 21.81 percent in average (up to 34.28 percent), and power savings of 35.07 percent over a standard 3D-stacked memory.

Enabling fast and energy-efficient FM-index exact matching using processing-near-memory

Memory bandwidth and latency constitutes a major performance bottleneck for many data-intensive applications. While high-locality access patterns take advantage of the deep cache hierarchies available in modern processors, unpredictable low-locality patterns cause a significant part of the execution time to be wasted waiting for data. An example of those memory bound applications is the exact matching algorithm based on FM-index, used in some well-known sequence alignment applications. Processing-Near-Memory (PNM) has been proposed as a strategy to overcome the memory wall problem, by placing computation close to data, speeding up memory bound workloads by reducing data movements. This paper presents a performance and energy evaluation of two classes of processor architectures when executing the FM-index exact matching algorithm, as a reference algorithm for exact sequence alignment. One architecture class is processor-centric, based on complex cores and DDR3/4 SDRAM memory technology. The other architecture class is memory-centric, based on simple cores and ultra-high-bandwidth hybrid memory cube (HMC) 3D-stacked memory technologies. The results show that the PNM solution improves performance between 1.26 $$\times$$ and 3.7 $$\times$$ and the energy consumption per operation is reduced between 21 $$\times$$ and 40 $$\times$$ . In addition, a synthetic benchmark RANDOM was developed that mimics the memory access pattern of the FM-index exact matching algorithm, but with a user configurable operational intensity. This benchmark allows us to extend the evaluation to the class of algorithms with similar memory behaviour but running over a range of operational intensity values.

Freezer: A Specialized NVM Backup Controller for Intermittently Powered Systems

The explosion of IoT and wearable devices determined a rising attention toward energy harvesting as source for powering these systems. In this context, many applications cannot afford the presence of a battery because of size, weight, and cost issues. Therefore, due to the intermittent nature of ambient energy sources, these systems must be able to save and restore their state, in order to guarantee progress across power interruptions. In this work, we propose a specialized backup/restore controller that dynamically tracks the memory accesses during the execution of the program. The controller then commits the changes to a snapshot in a nonvolatile memory (NVM) when a power failure is detected. Our approach does not require complex hybrid memories and can be implemented with standard components. Results on a set of benchmarks show an average 8× reduction in backup size. Thanks to our dedicated controller, the backup time is further reduced by more than 100×, with an area and power overhead of only 0.4% and 0.8%, respectively, with respect to a low-end IoT node.

Reliability-aware Garbage Collection for Hybrid HBM-DRAM Memories

Emerging workloads in cloud and data center infrastructures demand high main memory bandwidth and capacity. Unfortunately, DRAM alone is unable to satisfy contemporary main memory demands. High-bandwidth memory (HBM) uses 3D die-stacking to deliver 4–8× higher bandwidth. HBM has two drawbacks: (1) capacity is low, and (2) soft error rate is high. Hybrid memory combines DRAM and HBM to promise low fault rates, high bandwidth, and high capacity. Prior OS approaches manage HBM by mapping pages to HBM versus DRAM based on hotness (access frequency) and risk (susceptibility to soft errors). Unfortunately, these approaches operate at a coarse-grained page granularity, and frequent page migrations hurt performance. This article proposes a new class of reliability-aware garbage collectors for hybrid HBM-DRAM systems that place hot and low-risk objects in HBM and the rest in DRAM. Our analysis of nine real-world Java workloads shows that: (1) newly allocated objects in the nursery are frequently written, making them both hot and low-risk, (2) a small fraction of the mature objects are hot and low-risk, and (3) allocation site is a good predictor for hotness and risk. We propose RiskRelief, a novel reliability-aware garbage collector that uses allocation site prediction to place hot and low-risk objects in HBM. Allocation sites are profiled offline and RiskRelief uses heuristics to classify allocation sites as DRAM and HBM. The proposed heuristics expose Pareto-optimal trade-offs between soft error rate (SER) and execution time. RiskRelief improves SER by 9× compared to an HBM-Only system while at the same time improving performance by 29% compared to a DRAM-Only system. Compared to a state-of-the-art OS approach for reliability-aware data placement, RiskRelief eliminates all page migration overheads, which substantially improves performance while delivering similar SER. Reliability-aware garbage collection opens up a new opportunity to manage emerging HBM-DRAM memories at fine granularity while requiring no extra hardware support and leaving the programming model unchanged.

Resource abstraction and data placement for distributed hybrid memory pool

Emerging byte-addressable non-volatile memory (NVM) technologies offer higher density and lower cost than DRAM, at the expense of lower performance and limited write endurance. There have been many studies on hybrid NVM/DRAM memory management in a single physical server. However, it is still an open problem on how to manage hybrid memories efficiently in a distributed environment. This paper proposes Alloy, a memory resource abstraction and data placement strategy for an RDMA-enabled distributed hybrid memory pool (DHMP). Alloy provides simple APIs for applications to utilize DRAM or NVM resource in the DHMP, without being aware of the hardware details of the DHMP. We propose a hotness-aware data placement scheme, which combines hot data migration, data replication and write merging together to improve application performance and reduce the cost of DRAM. We evaluate Alloy with several micro-benchmark workloads and public benchmark workloads. Experimental results show that Alloy can significantly reduce the DRAM usage in the DHMP by up to 95%, while reducing the total memory access time by up to 57% compared with the state-of-the-art approaches.

Hybrid Memory Device (Memory/Selector) with Scalable and Simple Structure for XNOR‐Based Neural Network Applications

AbstractThis study investigates the electrical behavior of a hybrid memory device with both memory and selector characteristics. Electrical measurements and simulations indicate that the electrical behaviors (nonvolatile characteristics in Al2O3 layers and volatile characteristics in TiO2 layers) are linked to the stability of the conductive filament (CF) used. The binding energy between the Ag atoms in the CF is crucial for achieving nonvolatile or volatile characteristics. Thus, the hybrid memory device exhibits tunable threshold‐voltage characteristics and a consistent off‐state without requiring an additional selector device. Furthermore, the buffer metal layer between the active electrode and oxide layer affects the filament‐formation process in terms of the switching time. Experimental results show that the buffer layer significantly affects ion motion, such as redox reactions and ion migration. Thus, hybrid memory devices with a Zr buffer layer can solve the voltage–time dilemma, enabling fast and low‐voltage switching. Robust XNOR‐based neural network applications are developed using hybrid memory devices in a cross‐point array with characteristics such as scalability, simple structure, and excellent switching characteristics. By carefully considering the on–off ratio and device variability, hybrid memory devices can ensure reliable operation with a high pattern recognition accuracy in XNOR‐based neural neuromorphic hardware systems.

High-performance dataflow computing in hybrid memory systems with UPC++ DepSpawn

Dataflow computing is a very attractive paradigm for high-performance computing, given its ability to trigger computations as soon as their inputs are available. UPC++ DepSpawn is a novel task-based library that supports this model in hybrid shared/distributed memory systems on top of a Partitioned Global Address Space environment. While the initial version of the library provided good results, it suffered from a key restriction that heavily limited its performance and scalability. Namely, each process had to consider all the tasks in the application rather than only those of interest to it, an overhead that naturally grows with both the number of processes and tasks in the system. In this paper, this restriction is lifted, enabling our library to provide higher levels of performance. This way, in experiments using 768 cores the performance improved up to 40.1%, the average improvement being 16.1%.

Monolithic 3D-Based SRAM/MRAM Hybrid Memory for an Energy-Efficient Unified L2 TLB-Cache Architecture

Monolithic 3D (M3D) integration has been emerged as a promising technology for fine-grained 3D stacking. As the M3D integration offers extremely small dimension of via in a nanometer-scale, it is beneficial for small microarchitectural blocks such as caches, register files, translation look-aside buffers (TLBs), etc. However, since the M3D integration requires low-temperature process for stacked layers, it causes lower performance for stacked transistors compared to the conventional 2D process. In contrast, non-volatile memory (NVM) such as magnetic RAM (MRAM) is originally fabricated at a low temperature, which enables the M3D integration without performance degradation. In this paper, we propose an energy-efficient unified L2 TLB-cache architecture exploiting M3D-based SRAM/MRAM hybrid memory. Since the M3D-based SRAM/MRAM hybrid memory consumes much smaller energy than the conventional 2D SRAM-only memory and 2D SRAM/MRAM hybrid memory, while providing comparable performance, our proposed architecture improves energy efficiency significantly. Especially, as our proposed architecture changes the memory partitioning of the unified L2 TLB-cache depending on the L2 cache miss rate, it maximizes the energy efficiency for parallel workloads suffering extremely high L2 cache miss rate. According to our analysis using PARSEC benchmark applications, our proposed architecture reduces the energy consumption of L2 TLB + L2 cache by up to 97.7% (53.6% on average), compared to the baseline with the 2D SRAM-only memory, with negligible impact on performance. Furthermore, our proposed technique reduces the memory access energy consumption by up to 32.8% (10.9% on average), by reducing memory accesses due to TLB misses.

Characterization of Android Memory References and Implication to Hybrid Memory Management

In this article, we analyze Android applications’ memory reference behaviors, and observe that smartphone memory accesses are different from traditional computer systems with respect to the following five aspects: 1) A limited number of hot pages account for a majority of memory writes, and these hot pages have similar logical addresses regardless of application types; 2) The identities of these hot pages are shared library, linker, and stack regions; 3) The memory access behaviors of hot pages do not change significantly as time progresses even after applications finish their launching; 4) The skewness of memory write accesses in Android is extremely stronger than that of desktop systems; 5) In predicting re-reference likelihood of hot pages, temporal locality is better than reference frequency. Based on these observations, we present a new smartphone memory management scheme for DRAM-NVM hybrid memory. Adopting NVM is effective in power-saving of smartphones, but NVM has weaknesses in write operations. Thus, we aim to identify write-intensive pages and place them on DRAM. Unlike previous studies, we prevent migration of pages between DRAM and NVM, which eliminates unnecessary NVM write traffic that accounts for 32-42% of total write traffic. By judiciously managing the admission of hot pages in DRAM, our scheme reduces the write traffic to NVM by 42% on average without performance degradations.

A Survey of Non-Volatile Main Memory Technologies: State-of-the-Arts, Practices, and Future Directions

Non-Volatile Main Memories (NVMMs) have recently emerged as a promising technology for future memory systems. Generally, NVMMs have many desirable properties such as high density, byte-addressability, non-volatility, low cost, and energy efficiency, at the expense of high write latency, high write power consumption, and limited write endurance. NVMMs have become a competitive alternative of Dynamic Random Access Memory (DRAM), and will fundamentally change the landscape of memory systems. They bring many research opportunities as well as challenges on system architectural designs, memory management in operating systems (OSes), and programming models for hybrid memory systems. In this article, we first revisit the landscape of emerging NVMM technologies, and then survey the state-of-the-art studies of NVMM technologies. We classify those studies with a taxonomy according to different dimensions such as memory architectures, data persistence, performance improvement, energy saving, and wear leveling. Second, to demonstrate the best practices in building NVMM systems, we introduce our recent work of hybrid memory system designs from the dimensions of architectures, systems, and applications. At last, we present our vision of future research directions of NVMMs and shed some light on design challenges and opportunities.