NVHT: An efficient key–value storage library for non-volatile memory

Abstract

Non-Volatile Memory (NVM) promises persistence, byte-addressability and DRAM-like read/write latency. These properties indicate that NVM has the potential to be incorporated with key–value stores to achieve high performance and durability simultaneously. Specifically, data can be stored in NVM inherently without DRAM buffering, which eliminates expensive disk I/Os and data format transformation cost. However, several challenges such as data inconsistency and write endurance arise along with the benefits. We propose a library named NVHT to provide APIs for NVM-based key–value store operations. In NVHT, we introduce non-volatile pointer to solve the dynamic address mapping problem and design a wear-out-aware memory allocator for NVM. The core of NVHT is a novel NVM-friendly hash table structure. NVHT guarantees consistency using a log-based mechanism. The experimental results show that compared with LevelDB and BerkeleyDB running on a DRAM-based file system, NVHT achieves more than 2x and 4x speedup for insert and search operation respectively. Compared with in-memory key–value store system Redis, NVHT still achieves higher transaction performance in terms of random update throughput (up to 1.5x and 2.5x for RDB scheme and AOF scheme, respectively).