LibreKV: A Persistent In-Memory Key-Value Store

Abstract

Emerging Non-Volatile Memory (NVM) possesses unique features including byte-addressability and high density which bring huge opportunities to combine DRAM and NVM in a unified main memory space. And key-value store (KVS) systems play an important role in many applications, such as large-scale websites. Several existing KVS have been proposed for NVMs. However, they have drawbacks such as extra write overhead leading, lower memory utilization and worse endurance. In this paper, we design and develop an NVM-based key-value store system named LibreKV. It specifically targets the hybrid DRAM and NVM memory architecture, leveraging the NVM as the eventual persistent storage medium. It uses both static hash table and dynamic hash tables to achieve a balance between system performance and memory utilization. It adopts a checksum based consistency mechanism to guarantee data consistency and persistent storage on NVM. LibreKV works independently without relying on an underlying file system and simplified the IO stack comparing to the traditional KVS system. Experimental results show that LibreKV outperforms the state-of-the-art KVS systems and achieves better scalability and consistency while with low overhead.