Non-volatile Main Memory Research Articles

FSDedup: Feature-Aware and Selective Deduplication for Improving Performance of Encrypted Non-Volatile Main Memory

Enhancing the endurance, performance, and energy efficiency of encrypted Non-Volatile Main Memory (NVMM) can be achieved by minimizing written data through inline deduplication. However, existing approaches applying inline deduplication to encrypted NVMM suffer from substantial performance degradation due to high computing, memory footprint, and index-lookup overhead to generate, store, and query the cryptographic hash (fingerprint). In the preliminary ESD [ 14 ], we proposed the Error Correcting Code (ECC) assisted selective deduplication scheme, utilizing the ECC information as a fingerprint to identify similar data effectively and then leveraging the selective deduplication technique to eliminate a large amount of redundant data with high reference counts. In this article, we proposed FSDedup. Compared with ESD, FSDedup could leverage the prefetch cache to reduce the read overhead during similarity comparison and utilize the cache refresh mechanism to identify further and eliminate more redundant data. Extensive experimental evaluations demonstrate that FSDedup can enhance the performance of the NVMM system further than the ESD. Experimental results show that FSDedup can improve both write and read speed by up to 1.8×, enhance Instructions Per Cycle by up to 1.5×, and reduce energy consumption by up to 2.0×, compared to ESD.

GWalloc: A self-adaptive generational wear-aware allocator for non-volatile main memory

Phase Change Memory (PCM) is considered a promising replacement for DRAM due to its superior performance characteristics such as low leakage power, high integration density, byte addressability and non-volatility. However, PCM’s limited write endurance significantly hinders its wide application. For example, PCM wears out quickly with traditional dynamic memory allocation policy in embedded systems which aggregates lots of writes in few memory blocks. To extend the lifespan of PCM, some wear-aware dynamic memory allocators have been proposed, which generally depend on some fixed parameters to limit the wear of PCM. However, these allocators can be inflexible as it is difficult to specify appropriate values for the required parameters in different scenarios. In this paper, we propose a Self-Adaptive Generational Wear-Aware Allocator (GWalloc). GWalloc divides memory blocks into two generations: the young and the old generation, according to their number of allocation times. GWalloc also dynamically adjusts the system’s wear threshold during allocations so that it can effectively balance the wear degree of PCM and the consumed memory space. The wear threshold restricts the upper wear limit of young memory blocks. Experimental evaluations show that compared with the state-of-the-art wear-aware dynamic memory allocators (NVMalloc, Walloc and UWLalloc), GWalloc improves PCM wear-leveling (evaluated by CV, a wear leveling indicator) by 38.6%, 39.1% and 38.3%, and saves 62.1%, 22.2% and 37.2% memory space overhead.

A Survey of Non-Volatile Main Memory File Systems

A Survey of Non-Volatile Main Memory File Systems

CADEN: Compression-Assisted Adaptive Encoding to Improve Lifetime of Encrypted Nonvolatile Main Memories

Nonvolatile memories are widely considered as a potential replacement of DRAM in main memory due to their high density and low-leakage power consumption. The data in these memories are stored in encrypted form to protect them from data stealing. However, the encryption techniques, on account of their diffusion property impose high randomization in the generated encrypted data. It leads to enormous bit-flips in the memory cells, leading to their early wear out. In this letter, we propose a technique called CADEN that combines our compression technique COMF with an adaptive encoding scheme. The compressed data generated by COMF is encoded in finer granularity using an adaptive encoding approach at low storage overhead. Experimental evaluation shows that CADEN reduces bit-flips and improves lifetime in PCM main memory compared to baseline and existing techniques. In particular, CADEN shows 52% and 57% reduction in bit-flips and energy consumption and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$2.31\times $ </tex-math></inline-formula> improvement in lifetime over baseline.

Zallocator: A High Throughput Write-Optimized Persistent Allocator for Non-Volatile Memory

Non-volatile main memory (NVRAM) is likely to break the bottleneck caused by data transferring between main memory and extern storage, and fundamentally change the way applications do data persistence. We can build persistent data structures directly on NVRAM. To do this correctly and efficiently, we need the support of persistent allocators guaranteeing failure-consistency. However, existing persistent allocators pay huge persistence overhead for failure-consistency. In this article, we present Zallocator, a write-optimized failure-consistency allocator for NVRAM. Zallocator hardly brings write operations to NVRAM. It keeps all the heap management metadata in DRAM, and rebuild them after a crash. Experimental results show that Zallocator achieves a throughput comparable to state-of-the-art transient allocators, and reduces the average number of NVRAM writes per allocation/deallocation to almost zero.

Recoverable mutual exclusion with abortability

Recent advances in non-volatile main memory (NVM) technology have spurred research on algorithms that are resilient to intermittent failures that cause processes to crash and subsequently restart. In this paper we present a Recoverable Mutual Exclusion (RME) algorithm that supports abortability. Our algorithm guarantees FCFS and a strong liveness property: processes do not starve even in runs consisting of infinitely many crashes, provided that a process crashes at most a finite number of times in each of its attempts. On DSM and Relaxed-CC multiprocessors, a process incurs O(min (k, log n)) RMRs in a passage and O(f+ min (k, log n)) RMRs in an attempt, where n is the number of processes that the algorithm is designed for, k is the point contention of the passage or the attempt, and f is the number of times that p crashes during the attempt. On a Strict CC multiprocessor, the passage and attempt complexities are O(n) and O(f+n), respectively. Our algorithm uses only the read, write, and CAS operations, which are commonly supported by multiprocessors. Attiya, Hendler, and Woelfel proved that, with any mutual exclusion algorithm, a process incurs at least varOmega (log n) RMRs in a passage, if the algorithm uses only the read, write, and CAS operations (in: Proc. of the Fortieth ACM Symposium on Theory of Computing, New York, NY, USA, 2008). This lower bound implies that the worst-case RMR complexity of our algorithm is optimal for the DSM and Relaxed CC multiprocessors. This paper is an expanded version of our conference paper as reported by Jayanti and Joshi (in: Atig and Schwarzmann (eds) Networked Systems. Springer International Publishing, Cham, 2019), which presented the first Recoverable Mutual Exclusion (RME) algorithm that supports abortability. This algorithm from our conference paper (in: Atig and Schwarzmann (eds) Networked Systems. Springer International Publishing, Cham, 2019) admits starvation when there are infinitely many aborts in a run. In this paper, we fix this shortcoming and prove the algorithm’s properties by identifying an inductive invariant.

Open-Source Hardware Memory Protection Engine Integrated With NVMM Simulator

With growing on-device IoT processing, security on edge devices becomes increasingly important. Among Trusted Execution Environment (TEE), an open-source RISC-V Keystone TEE is the expected one. However, some issues remain when applying it to various devices: untrusted DRAM, and untrusted path to non-volatile storage. These issues can be resolved by Memory Protection Engine (MPE) based on an integrity tree, and Non-Volatile Main Memory (NVMM), respectively. TEE, MPE, and NVMM must be cooperatively optimized to exploit performance. Despite this demand, there is no widely available platform which enables fast, reliable, and system-wide evaluation. In the paper, we provide an open-source hardware simulator for secure edge devices. We implemented an MPE using SGX-style Integrity Tree on the Keystone compatible RISC-V SoC. Then, we ported the NVMM simulation technique to it. Its whole design was publicized to widely provide a baseline hardware design. The MPE behavior was validated by using micro benchmarks. It revealed that the MPE read/write overhead is <inline-formula><tex-math notation="LaTeX">$2.55\times /4.16\times$</tex-math></inline-formula> on DRAM, and <inline-formula><tex-math notation="LaTeX">$3.05\times /5.40\times$</tex-math></inline-formula> on NVMM, respectively. Besides, we discuss our work's role by comparing with the gem5 considering TEE evaluation time and impact of the protected NVMM.

Efficient In-Memory AES Encryption Implementation Using a General Memristive Logic: Surmounting the data movement bottleneck

Emerging nonvolatile main memory (NVMM) suffers from secure vulnerability due to its nonvolatility. To address this issue, existing methods tend to employ the encryption engine on the CPU side for encryption. However, this incurs large energy and latency overhead due to the massive data movement between the CPU and NVMM. On the other hand, popular encryption algorithms like the Advanced Encryption Standard (AES) usually involve massive bit-level parallelism. As a result, an emerging technology named <i>logic-in-memory</i> (<i>LiM</i>), which leverages the electrical characteristics of nonvolatile devices to enable efficient in-memory Boolean operations in parallel, is a promising solution to eliminating data movement overhead and enables faster and more energy-efficient encryption.

Software-Managed Read and Write Wear-Leveling for Non-Volatile Main Memory

In-memory wear-leveling has become an important research field for emerging non-volatile main memories over the past years. Many approaches in the literature perform wear-leveling by making use of special hardware. Since most non-volatile memories only wear out from write accesses, the proposed approaches in the literature also usually try to spread write accesses widely over the entire memory space. Some non-volatile memories, however, also wear out from read accesses, because every read causes a consecutive write access. Software-based solutions only operate from the application or kernel level, where read and write accesses are realized with different instructions and semantics. Therefore different mechanisms are required to handle reads and writes on the software level. First, we design a method to approximate read and write accesses to the memory to allow aging aware coarse-grained wear-leveling in the absence of special hardware, providing the age information. Second, we provide specific solutions to resolve access hot-spots within the compiled program code (text segment) and on the application stack. In our evaluation, we estimate the cell age by counting the total amount of accesses per cell. The results show that employing all our methods improves the memory lifetime by up to a factor of 955×.

Tnvmalloc: A Thread-Level-Based Wear-Aware Allocator for Nonvolatile Main Memory

The nonvolatile main memory (NVMM) has the advantages of near-DRAM speed, byte-addressability, and persistence, and presents limitations in write durability. The memory allocator, a fundamental data structure of memory management, can effectively mitigate the wear speed, thereby prolonging the NVMM lifetime. Nevertheless, balancing the performance and writing reliability in single and multi-thread scenarios is still an open problem for NVMM allocators. In this paper, we propose a thread-level wear-aware allocator (Tnvmalloc) that divides the NVMM space into multiple management granularities and then dynamically selects the optimal blocks using a wear-leveling strategy based on allocation requests and wear records. Experiments show that the proposed Tnvmalloc provides more than 10 times improvement in wear-leveling than typical allocators Glibc malloc, NVMalloc, and nvm_malloc, which becomes obvious especially in multi-threaded scenarios. Moreover, when allocating large memory blocks, Tnvmalloc achieves three times faster than that of NVMalloc.

Monolithically Integrating Non-Volatile Main Memory over the Last-Level Cache

Many emerging non-volatile memories are compatible with CMOS logic, potentially enabling their integration into a CPU’s die. This article investigates such monolithically integrated CPU–main memory chips. We exploit non-volatile memories employing 3D crosspoint subarrays, such as resistive RAM (ReRAM), and integrate them over the CPU’s last-level cache (LLC). The regular structure of cache arrays enables co-design of the LLC and ReRAM main memory for area efficiency. We also develop a streamlined LLC/main memory interface that employs a single shared internal interconnect for both the cache and main memory arrays, and uses a unified controller to service both LLC and main memory requests. We apply our monolithic design ideas to a many-core CPU by integrating 3D ReRAM over each core’s LLC slice. We find that co-design of the LLC and ReRAM saves 27% of the total LLC–main memory area at the expense of slight increases in delay and energy. The streamlined LLC/main memory interface saves an additional 12% in area. Our simulation results show monolithic integration of CPU and main memory improves performance by 5.3× and 1.7× over HBM2 DRAM for several graph and streaming kernels, respectively. It also reduces the memory system’s energy by 6.0× and 1.7×, respectively. Moreover, we show that the area savings of co-design permits the CPU to have 23% more cores and main memory, and that streamlining the LLC/main memory interface incurs a small 4% performance penalty.

Development of system software on non-volatile main memory

The rapid development of the Internet has led to explosive growth in the total volume of global data storage,leading to the development of data systems from computing intensive to data intensive. How to build a reliableand efficient data storage system has become an urgent problem to be solved in the era of big data. Compared withtraditional disks, non-volatile main memory has the advantages of high performance and byte-addressable characteristics,and these unique advantages provide new opportunities for the construction of efficient storage systems. However, theconstruction of a traditional storage system is not suitable for non-volatile main memory. It cannot give play to theperformance advantage of non-volatile main memory, and it is easy to result in high consistency overhead, low spaceutilization ratio, and low programming security. Therefore, this paper analyzes the challenges for the storage systembased on non-volatile main memory, and summarizes the research development of the spatial management mechanism,new programming model, data structure, file systems, and distributed storage systems. Finally, this paper lists the potential research direction of the storage system based on the non-volatile main memory.

Non-Volatile Main Memory Emulator for Embedded Systems Employing Three NVMM Behaviour Models

Emerging byte-addressable non-volatile memory devices attract much attention. A non-volatile main memory (NVMM) built on them enables larger memory size and lower power consumption than a traditional DRAM main memory. To fully utilize an NVMM, both software and hardware must be cooperatively optimized. Simultaneously, even focusing on a memory module, its micro architecture is still being developed though real non-volatile memory modules, such as Intel Optane DC persistent memory (DCPMM), have been on the market. Looking at existing NVMM evaluation environments, software simulators can evaluate various micro architectures with their long simulation time. Emulators can evaluate the whole system fast with less flexibility in their configuration than simulators. Thus, an NVMM emulator that can realize flexible and fast system evaluation still has an important role to explore the optimal system. In this paper, we introduce an NVMM emulator for embedded systems and explore a direction of optimization techniques for NVMMs by using it. It is implemented on an SoC-FPGA board employing three NVMM behaviour models: coarse-grain, fine-grain and DCPMM-based. The coarse and fine models enable NVMM performance evaluations based on extensions of traditional DRAM behaviour. The DCPMM-based model emulates the behaviour of a real DCPMM. Whole evaluation environment is also provided including Linux kernel modifications and several runtime functions. We first validate the developed emulator with an existing NVMM emulator, a cycle-accurate NVMM simulator and a real DCPMM. Then, the program behavior differences among three models are evaluated with SPEC CPU programs. As a result, the fine-grain model reveals the program execution time is affected by the frequency of NVMM memory requests rather than the cache hit ratio. Comparing with the fine-grain model and the coarse-grain model under the condition of the former's longer total write latency than the latter's, the former shows lower execution time for four of fourteen programs than the latter because of the bank-level parallelism and the row-buffer access locality exploited by the former model.

NVMM-Oriented Hierarchical Persistent Client Caching for Lustre

In high-performance computing (HPC), data and metadata are stored on special server nodes and client applications access the servers’ data and metadata through a network, which induces network latencies and resource contention. These server nodes are typically equipped with (slow) magnetic disks, while the client nodes store temporary data on fast SSDs or even on non-volatile main memory (NVMM). Therefore, the full potential of parallel file systems can only be reached if fast client side storage devices are included into the overall storage architecture. In this article, we propose an NVMM-based hierarchical persistent client cache for the Lustre file system (NVMM-LPCC for short). NVMM-LPCC implements two caching modes: a read and write mode (RW-NVMM-LPCC for short) and a read only mode (RO-NVMM-LPCC for short). NVMM-LPCC integrates with the Lustre Hierarchical Storage Management (HSM) solution and the Lustre layout lock mechanism to provide consistent persistent caching services for I/O applications running on client nodes, meanwhile maintaining a global unified namespace of the entire Lustre file system. The evaluation results presented in this article show that NVMM-LPCC can increase the average read throughput by up to 35.80 times and the average write throughput by up to 9.83 times compared with the native Lustre system, while providing excellent scalability.

A Survey of Non-Volatile Main Memory Technologies: State-of-the-Arts, Practices, and Future Directions

Non-Volatile Main Memories (NVMMs) have recently emerged as a promising technology for future memory systems. Generally, NVMMs have many desirable properties such as high density, byte-addressability, non-volatility, low cost, and energy efficiency, at the expense of high write latency, high write power consumption, and limited write endurance. NVMMs have become a competitive alternative of Dynamic Random Access Memory (DRAM), and will fundamentally change the landscape of memory systems. They bring many research opportunities as well as challenges on system architectural designs, memory management in operating systems (OSes), and programming models for hybrid memory systems. In this article, we first revisit the landscape of emerging NVMM technologies, and then survey the state-of-the-art studies of NVMM technologies. We classify those studies with a taxonomy according to different dimensions such as memory architectures, data persistence, performance improvement, energy saving, and wear leveling. Second, to demonstrate the best practices in building NVMM systems, we introduce our recent work of hybrid memory system designs from the dimensions of architectures, systems, and applications. At last, we present our vision of future research directions of NVMMs and shed some light on design challenges and opportunities.

SWEL-COFAE : Wear Leveling and Adaptive Encoding Assisted Compression of Frequent Words in Non-Volatile Main Memories

Emerging Non-Volatile memories such as Phase Change Memory (PCM) and Resistive RAM are projected as potential replacements of the traditional DRAM-based main memories. However, limited write endurance and high write energy limit their chances of adoption as a mainstream main memory standard. Therefore, developing solutions that enhance the lifetime of these memories while offering a decent system performance has a great impact in building future large capacity and energy-efficient main memories. In this paper, we propose a word-level compression scheme called COMF to reduce bitflips in PCMs by removing the most repeated words from the cache blocks before writing into memory. COMF is augmented with an adaptive granularity based encoding technique to form COFAE, which reduces the bitflips to a further extent. We also propose SWEL-COFAE, which is an intra-line stride-based wear leveling technique to improve lifetime by balancing the bitflip pressure within the cells of the memory lines. Experimental results show that the proposed technique improves lifetime by 103% and reduces bitflips and energy by 60% and 59% respectively over baseline

NVGraph: Enforcing Crash Consistency of Evolving Network Analytics in NVMM Systems

Many complex networks can be modeled as evolving graphs. Existing in-memory graph data structures are designed for DRAM. Thus, they cannot effectively exploit the current and ongoing adoption of emerging non-volatile main memory (NVMM) for two reasons. (1) Ephemeral graph data structures are not crash-consistent for NVMM. (2) NVMM writes and reads and may incur higher latency than DRAM. In this article, we propose a novel persistent evolving graph data structure, named NVG RAPH , for both computing and in-memory storage of evolving graphs in NVMM. We devise NVG RAPH as a multi-version data structure, wherein a minimum of one version of its data is stored in NVMM to provide the desired durability at runtime for failure recovery, and another version is stored in both DRAM and NVMM to reduce the NVMM-induced memory latency. NVG RAPH is also a partitioned data structure. We dynamically transform the layout of NVG RAPH (e.g., changing the size of its partition in DRAM) exploiting network properties and data access patterns of workloads. For the evaluation of NVG RAPH , we implement four representative real-world graph applications: pagerank, BFS, influence maximization, and rumor source detection. The experimental results show that the performance of NVG RAPH is comparable to other in-memory data structure (e.g., CSR and LLAMA) while using 70 percent less DRAM. It scales well up to 10 billion edges and 201 snapshots and supports crash consistency. It offers up to the 21X speedup of execution time compared to the scale-up graph computation approaches (e.g., GraphChi and X-stream).

LosPem

New and emerging types of Persistent Memory (PM) technologies boost the opportunity to improve the performance of storage systems. PM can unify the main memory and secondary storage by incorporating it into legacy computer systems through the memory bus. In recent years, innovative results have been presented that exploit the byte-addressability, low latency, and non-volatility of PM; these have included local PM file systems and PM systems. However, the high overhead of ensuring data consistency has limited the performance of these systems. In this article, we propose LosPem, a novel log-structured framework for persistent memory to address the performance challenge. LosPem utilizes two techniques to accomplish this. Firstly, LosPem deploys efficient hash-indexed linked lists to maintain the log contents to reduce the significant overhead of log content retrieval. Secondly, LosPem improves the transaction throughput by decoupling a transaction into two asynchronous steps and creating a write buffer on Dynamic Random Access Memory (DRAM) write buffer for processing the frequent data writes. The experimental results show that LosPem outperforms Non-volatile Memory Library (NVML), Mnemosyne and Log-structured Non-volatile Main Memory (LSNVMM) by 27%, 1.2x, and 1.0x on a read-intensive workload. On a write-intensive workload, LosPem outperforms NVML, Mnemosyne, and LSNVMM by 1.8x, 1.2x, and 34%, respectively.

Sage

Non-volatile main memory (NVRAM) technologies provide an attractive set of features for large-scale graph analytics, including byte-addressability, low idle power, and improved memory-density. NVRAM systems today have an order of magnitude more NVRAM than traditional memory (DRAM). NVRAM systems could therefore potentially allow very large graph problems to be solved on a single machine, at a modest cost. However, a significant challenge in achieving high performance is in accounting for the fact that NVRAM writes can be much more expensive than NVRAM reads. In this paper, we propose an approach to parallel graph analytics using the Parallel Semi-Asymmetric Model (PSAM) , in which the graph is stored as a read-only data structure (in NVRAM), and the amount of mutable memory is kept proportional to the number of vertices. Similar to the popular semi-external and semi-streaming models for graph analytics, the PSAM approach assumes that the vertices of the graph fit in a fast read-write memory (DRAM), but the edges do not. In NVRAM systems, our approach eliminates writes to the NVRAM, among other benefits. To experimentally study this new setting, we develop Sage , a parallel semi-asymmetric graph engine with which we implement provably-efficient (and often work-optimal) PSAM algorithms for over a dozen fundamental graph problems. We experimentally study Sage using a 48--core machine on the largest publicly-available real-world graph (the Hyperlink Web graph with over 3.5 billion vertices and 128 billion edges) equipped with Optane DC Persistent Memory, and show that Sage outperforms the fastest prior systems designed for NVRAM. Importantly, we also show that Sage nearly matches the fastest prior systems running solely in DRAM, by effectively hiding the costs of repeatedly accessing NVRAM versus DRAM.

В ожидании нативных архитектур СУБД на основе энергонезависимой основной памяти

Many experts in the field of data management believe that the emergence of non-volatile byte-addressable main memory (NVM) available for practical use will lead to the development of a new type of ultra-high-speed database management systems (DBMS) with single-level data storage (native in-NVM DBMS). However, the number of researchers who are actively engaged in research of architectures of native in-NVM DBMS has not increased in recent years. The most active researchers are PhD students that are not afraid of the risks, which, of course, exist in this new area. The second section of the article discusses the state of the art in NVM hardware. The analysis shows that NVM in the DIMM form factor has already become a reality, and that in the near future we can expect the appearance on the market NVM-DIMMs with the speed of conventional DRAM and endurance close to that of hard drives. The third section is devoted to the review of related works, among which the works of young researchers are the most advanced. In the fourth section, we state and justify that the work performed so far in the field of in-NVM DBMS, did not lead to the emergence of a native architecture. This is hampered by the set of limiting factors analyzed by us. In this regard, in the fifth section, we present a sketch of the native architecture of an in-NVM DBMS, the choice of which is influenced only by the goals of simplicity and efficiency. In conclusion, we summarize the article and argues the need for additional research into many aspects of the native architecture of an in-NVM DBMS.