Key-value Store System Research Articles

A Novel Global Key-Value Storage System Based on Kinetic Drives

NoSQL databases are flexible and efficient for many data intensive applications, and the key-value store is one of them. In recent years, a new Ethernet accessed disk drive called the “Kinetic Drive” was developed by Seagate. This new Kinetic Drive is specially designed for key-value stores. Users can directly access data with a Kinetic Drive via its IP address without going through a storage server/layer. With this new innovation, the storage stack and architectures of key-value store systems have been greatly changed. In this paper, we propose a novel global key-value store system based on Kinetic Drives. We explore data management issues including data access, key indexing, data backup, and recovery. We offer scalable solutions with small storage overhead. The performance evaluation shows that our location-aware design and backup approach can reduce the average distance traveled for data access requests.

Anna: A KVS For Any Scale

Modern cloud providers offer dense hardware with multiple cores and large memories, hosted in global platforms. This raises the challenge of implementing high-performance software systems that can effectively scale from a single core to multicore to the globe. Conventional wisdom says that software designed for one scale point needs to be rewritten when scaling up by $10-100\times$ 10 - 100 × [1] . In contrast, we explore how a system can be architected to scale across many orders of magnitude by design. We explore this challenge in the context of a new key-value store system called Anna: a partitioned, multi-mastered system that achieves high performance and elasticity via wait-free execution and coordination-free consistency. Our design rests on a simple architecture of coordination-free actors that perform state update via merge of lattice-based composite data structures. We demonstrate that a wide variety of consistency models can be elegantly implemented in this architecture with unprecedented consistency, smooth fine-grained elasticity, and performance that far exceeds the state of the art.

NVHT: An efficient key–value storage library for non-volatile memory

Non-Volatile Memory (NVM) promises persistence, byte-addressability and DRAM-like read/write latency. These properties indicate that NVM has the potential to be incorporated with key–value stores to achieve high performance and durability simultaneously. Specifically, data can be stored in NVM inherently without DRAM buffering, which eliminates expensive disk I/Os and data format transformation cost. However, several challenges such as data inconsistency and write endurance arise along with the benefits. We propose a library named NVHT to provide APIs for NVM-based key–value store operations. In NVHT, we introduce non-volatile pointer to solve the dynamic address mapping problem and design a wear-out-aware memory allocator for NVM. The core of NVHT is a novel NVM-friendly hash table structure. NVHT guarantees consistency using a log-based mechanism. The experimental results show that compared with LevelDB and BerkeleyDB running on a DRAM-based file system, NVHT achieves more than 2x and 4x speedup for insert and search operation respectively. Compared with in-memory key–value store system Redis, NVHT still achieves higher transaction performance in terms of random update throughput (up to 1.5x and 2.5x for RDB scheme and AOF scheme, respectively).

A distributed in-memory key-value store system on heterogeneous CPU–GPU cluster

In-memory key-value stores play a critical role in many data-intensive applications to provide high-throughput and low latency data accesses. In-memory key-value stores have several unique properties that include (1) data-intensive operations demanding high memory bandwidth for fast data accesses, (2) high data parallelism and simple computing operations demanding many slim parallel computing units, and (3) a large working set. However, our experiments show that homogeneous multicore CPU systems are increasingly mismatched to the special properties of key-value stores because they do not provide massive data parallelism and high memory bandwidth; the powerful but the limited number of computing cores does not satisfy the demand of the unique data processing task; and the cache hierarchy may not well benefit to the large working set. In this paper, we present the design and implementation of Mega-KV, a distributed in-memory key-value store system on a heterogeneous CPU–GPU cluster. Effectively utilizing the high memory bandwidth and latency hiding capability of GPUs, Mega-KV provides fast data accesses and significantly boosts overall performance and energy efficiency over the homogeneous CPU architectures. Mega-KV shows excellent scalability and processes up to 623-million key-value operations per second on a cluster installed with eight CPUs and eight GPUs, while delivering an efficiency of up to 299-thousand operations per Watt (KOPS/W).

LibreKV: A Persistent In-Memory Key-Value Store

Emerging Non-Volatile Memory (NVM) possesses unique features including byte-addressability and high density which bring huge opportunities to combine DRAM and NVM in a unified main memory space. And key-value store (KVS) systems play an important role in many applications, such as large-scale websites. Several existing KVS have been proposed for NVMs. However, they have drawbacks such as extra write overhead leading, lower memory utilization and worse endurance. In this paper, we design and develop an NVM-based key-value store system named LibreKV. It specifically targets the hybrid DRAM and NVM memory architecture, leveraging the NVM as the eventual persistent storage medium. It uses both static hash table and dynamic hash tables to achieve a balance between system performance and memory utilization. It adopts a checksum based consistency mechanism to guarantee data consistency and persistent storage on NVM. LibreKV works independently without relying on an underlying file system and simplified the IO stack comparing to the traditional KVS system. Experimental results show that LibreKV outperforms the state-of-the-art KVS systems and achieves better scalability and consistency while with low overhead.

Weighted Moore–Penrose generalized matrix inverse: MySQL vs. Cassandra database storage system

The research in this paper refers to two areas: programming and data storage (data base) for computing the weighted Moore–Penrose inverse. The main aim of this paper analysis of the execution speed of computing using PHP programming language and data store. The research shows that the speed of execution gives considerable difference between the Procedural programming and Object Oriented PHP language, on the middle layer in the three tier of the web architecture. Also, the research concerning the comparison of relation database system, MySQL and NoSQL, key value store system, ApacheCassandra, on the database layer. The CPU times are compared and discussed.

Mega-KV

In-memory key-value stores play a critical role in data processing to provide high throughput and low latency data accesses. In-memory key-value stores have several unique properties that include (1) data intensive operations demanding high memory bandwidth for fast data accesses, (2) high data parallelism and simple computing operations demanding many slim parallel computing units, and (3) a large working set. As data volume continues to increase, our experiments show that conventional and general-purpose multicore systems are increasingly mismatched to the special properties of key-value stores because they do not provide massive data parallelism and high memory bandwidth; the powerful but the limited number of computing cores do not satisfy the demand of the unique data processing task; and the cache hierarchy may not well benefit to the large working set. In this paper, we make a strong case for GPUs to serve as special-purpose devices to greatly accelerate the operations of in-memory key-value stores. Specifically, we present the design and implementation of Mega-KV, a GPU-based in-memory key-value store system that achieves high performance and high throughput. Effectively utilizing the high memory bandwidth and latency hiding capability of GPUs, Mega-KV provides fast data accesses and significantly boosts overall performance. Running on a commodity PC installed with two CPUs and two GPUs, Mega-KV can process up to 160+ million key-value operations per second, which is 1.4-2.8 times as fast as the state-of-the-art key-value store system on a conventional CPU-based platform.

Jobcast ^|^mdash; Parallel and distributed processing framework Data processing on a cloud style KVS database

In this paper, we propose a new architecture for parallel and distributed processing framework, gJobcasth, which enables data processing on a cloud style KVS database. Nowadays, lots of KVS (as known as Key Value Store) systems exist which achieve high scalability for data spaces among a huge number of computers. Some of KVS implementations use gconsistent hashh algorithm to identify the backend data node to store a pair of key and value. Jobcast also uses consistent hash algorithm for a distribution strategy and has a capability to store key and value pairs into huge number of computers as a KVS system. Furthermore, Jobcast also distributes gjobsh into data nodes for parallel and distributed processing. In this paper, we introduce a basic architecture of Jobcast and evaluate a data processing performance for a typical example.