Key-value Store Research Articles

PoBery: Possibly-complete Big Data Queries with Probabilistic Data Placement and Scanning

In big data query processing, there is a trade-off between query accuracy and query efficiency, for example, sampling query approaches trade-off query completeness for efficiency. In this article, we argue that query performance can be significantly improved by slightly losing the possibility of query completeness, that is, the chance that a query is complete. To quantify the possibility, we define a new concept, Probability of query Completeness (hereinafter referred to as PC). For example, If a query is executed 100 times, PC = 0.95 guarantees that there are no more than 5 incomplete results among 100 results. Leveraging the probabilistic data placement and scanning, we trade off PC for query performance. In the article, we propose PoBery (POssibly-complete Big data quERY), a method that supports neither complete queries nor incomplete queries, but possibly-complete queries. The experimental results conducted on HiBench prove that PoBery can significantly accelerate queries while ensuring the PC. Specifically, it is guaranteed that the percentage of complete queries is larger than the given PC confidence. Through comparison with state-of-the-art key-value stores, we show that while Drill-based PoBery performs as fast as Drill on complete queries, it is 1.7 ×, 1.1 ×, and 1.5 × faster on average than Drill, Impala, and Hive, respectively, on possibly-complete queries.

NVLSM: A Persistent Memory Key-Value Store Using Log-Structured Merge Tree with Accumulative Compaction

Computer systems utilizing byte-addressable Non-Volatile Memory ( NVM ) as memory/storage can provide low-latency data persistence. The widely used key-value stores using Log-Structured Merge Tree ( LSM-Tree ) are still beneficial for NVM systems in aspects of the space and write efficiency. However, the significant write amplification introduced by the leveled compaction of LSM-Tree degrades the write performance of the key-value store and shortens the lifetime of the NVM devices. The existing studies propose new compaction methods to reduce write amplification. Unfortunately, they result in a relatively large read amplification. In this article, we propose NVLSM, a key-value store for NVM systems using LSM-Tree with new accumulative compaction. By fully utilizing the byte-addressability of NVM, accumulative compaction uses pointers to accumulate data into multiple floors in a logically sorted run to reduce the number of compactions required. We have also proposed a cascading searching scheme for reads among the multiple floors to reduce read amplification. Therefore, NVLSM reduces write amplification with small increases in read amplification. We compare NVLSM with key-value stores using LSM-Tree with two other compaction methods: leveled compaction and fragmented compaction. Our evaluations show that NVLSM reduces write amplification by up to 67% compared with LSM-Tree using leveled compaction without significantly increasing the read amplification. In write-intensive workloads, NVLSM reduces the average latency by 15.73%–41.2% compared to other key-value stores.

XStore : Fast RDMA-Based Ordered Key-Value Store Using Remote Learned Cache

RDMA ( Remote Direct Memory Access ) has gained considerable interests in network-attached in-memory key-value stores. However, traversing the remote tree-based index in ordered key-value stores with RDMA becomes a critical obstacle, causing an order-of-magnitude slowdown and limited scalability due to multiple round trips. Using index cache with conventional wisdom—caching partial data and traversing them locally—usually leads to limited effect because of unavoidable capacity misses, massive random accesses, and costly cache invalidations. We argue that the machine learning (ML) model is a perfect cache structure for the tree-based index, termed learned cache . Based on it, we design and implement XStore , an RDMA-based ordered key-value store with a new hybrid architecture that retains a tree-based index at the server to perform dynamic workloads (e.g., inserts) and leverages a learned cache at the client to perform static workloads (e.g., gets and scans). The key idea is to decouple ML model retraining from index updating by maintaining a layer of indirection from logical to actual positions of key-value pairs. It allows a stale learned cache to continue predicting a correct position for a lookup key. XStore ensures correctness using a validation mechanism with a fallback path and further uses speculative execution to minimize the cost of cache misses. Evaluations with YCSB benchmarks and production workloads show that a single XStore server can achieve over 80 million read-only requests per second. This number outperforms state-of-the-art RDMA-based ordered key-value stores (namely, DrTM-Tree, Cell, and eRPC+Masstree) by up to 5.9× (from 3.7×). For workloads with inserts, XStore still provides up to 3.5× (from 2.7×) throughput speedup, achieving 53M reqs/s. The learned cache can also reduce client-side memory usage and further provides an efficient memory-performance tradeoff, e.g., saving 99% memory at the cost of 20% peak throughput.

VSIM: Distributed local structural vertex similarity calculation on big graphs

In many graph analytical applications, the local structural vertex similarity calculation is an essential prerequisite for advanced graph mining. The similarity calculation finds out all the similar vertex pairs whose local structural similarity scores (like the number of common neighbors, and the Jaccard index of adjacency sets) are above a given threshold. The real-world applications use a wide range of similarity thresholds. However, the existing distributed methods for the problem only optimize for either high thresholds (> 0.7) or low thresholds (< 0.1). To overcome the drawback, we propose a new distributed vertex similarity calculation framework VSIM that is efficient under a broad range of thresholds. VSIM processes static undirected graphs with local structural similarity scores that measure the similarity between vertices based on the first-order topology information. VSIM generates a similarity calculation task for every vertex in the graph and conducts all the tasks in parallel on a distributed computing platform along with a distributed key-value store. Each task finds vertices similar to a given center vertex with two task execution modes. The two modes optimize for high and low thresholds, respectively. Each task picks the suitable mode adaptively according to cost estimation models. We also propose an efficient implementation for VSIM on Apache Spark, with three optimization techniques to reduce communication costs and balance workloads on power-law graphs. The experimental evaluation shows that VSIM outperforms the state-of-the-art distributed methods by up to 67x speedup. VSIM can achieve near-linear node scalability in low-threshold and small cache scenarios.

LKSM: Light Weight Key-Value Store for Efficient Application Services on Local Distributed Mobile Devices

With the development of mobile network and corresponding techniques, more and more works focus on providing efficient services based on mobile devices. Furthermore, motivated by IoT, studies of local distributed mobile devices attract attentions of both industry and academia in recent years. However, existing storage systems cannot manage data and support the QoS of mobile services well. This paper presents LKSM, a light weight key-value storage system, which can be deployed on either one node or multiple nodes. To the best of our knowledge, it is the first attempt to propose key-value store in this scenario. We carefully analyze the challenges when designing the system on mobile clusters, and further propose RDS for addressing. With the help of RDS, LKSM achieves the goal of lower latency, better scalability, and higher availability. Furthermore, based on RDS, a novel data management strategy is presented, which successfully avoid energy holes of mobile clusters and achieves the tradeoff between performance and energy. We organize LKSM using a log-structured merge-tree and implement it based on LevelDB, an open source key-value storage system proposed by Google. Experiments on physical smartphones demonstrate that LKSM presents much higher performance compared with the ported LevelDB on mobile devices.

Lock-free Contention Adapting Search Trees

Concurrent key-value stores with range query support are crucial for the scalability and performance of many applications. Existing lock-free data structures of this kind use a fixed synchronization granularity. Using a fixed synchronization granularity in a concurrent key-value store with range query support is problematic as the best performing synchronization granularity depends on a number of factors that are difficult to predict, such as the level of contention and the number of items that are accessed by range queries. We present the first linearizable lock-free key-value store with range query support that dynamically adapts its synchronization granularity. This data structure is called the lock-free contention adapting search tree (LFCA tree). An LFCA tree automatically performs local adaptations of its synchronization granularity based on heuristics that take contention and the performance of range queries into account. We show that the operations of LFCA trees are linearizable, that the lookup operation is wait-free, and that the remaining operations (insert, remove and range query) are lock-free. Our experimental evaluation shows that LFCA trees achieve more than twice the throughput of related lock-free data structures in many scenarios. Furthermore, LFCA trees are able to perform substantially better than data structures with a fixed synchronization granularity over a wide range of scenarios due to their ability to adapt to the scenario at hand.

Concerto: Dynamic Processor Scaling for Distributed Data Systems with Replication

A surge of interest in data-intensive computing has led to a drastic increase in the demand for data centers. Given this growing popularity, data centers are becoming a primary contributor to the increased consumption of energy worldwide. To mitigate this problem, this paper revisits DVFS (Dynamic Voltage Frequency Scaling), a well-known technique to reduce the energy usage of processors, from the viewpoint of distributed systems. Distributed data systems typically adopt a replication facility to provide high availability and short latency. In this type of architecture, the replicas are maintained in an asynchronous manner, while the master synchronously operates via user requests. Based on this relaxation constraint of replica, we present a novel DVFS technique called Concerto, which intentionally scales down the frequency of processors operating for the replicas. This mechanism can achieve considerable energy savings without an increase in the user-perceived latency. We implemented Concerto on Redis 6.0.1, a commercial-level distributed key-value store, demonstrating that all associated performance issues were resolved. To prevent a delay in read queries assigned to the replicas, we offload the independent part of the read operation to the fast-running thread. We also empirically demonstrate that the decreased performance of the replica does not cause an increase of the replication lag because the inherent load unbalance between the master and replica hides the increased latency of the replica. Performance evaluations with micro and real-world benchmarks show that Redis saves 32% on average and up to 51% of energy with Concerto under various workloads, with minor performance losses in the replicas. Despite numerous studies of the energy saving in data centers, to the best of our best knowledge, Concerto is the first approach that considers clock-speed scaling at the aggregate level, exploiting heterogeneous performance constraints across data nodes.

Design of LSM-tree-based Key-value SSDs with Bounded Tails

Key-value store based on a log-structured merge-tree (LSM-tree) is preferable to hash-based key-value store, because an LSM-tree can support a wider variety of operations and show better performance, especially for writes. However, LSM-tree is difficult to implement in the resource constrained environment of a key-value SSD (KV-SSD), and, consequently, KV-SSDs typically use hash-based schemes. We present PinK , a design and implementation of an LSM-tree-based KV-SSD, which compared to a hash-based KV-SSD, reduces 99th percentile tail latency by 73%, improves average read latency by 42%, and shows 37% higher throughput. The key idea in improving the performance of an LSM-tree in a resource constrained environment is to avoid the use of Bloom filters and instead, use a small amount of DRAM to keep/pin the top levels of the LSM-tree. We also find that PinK is able to provide a flexible design space for a wide range of KV workloads by leveraging the read-write tradeoff in LSM-trees.

Penalty- and Locality-aware Memory Allocation in Redis Using Enhanced AET

Due to large data volume and low latency requirements of modern web services, the use of an in-memory key-value (KV) cache often becomes an inevitable choice (e.g., Redis and Memcached). The in-memory cache holds hot data, reduces request latency, and alleviates the load on background databases. Inheriting from the traditional hardware cache design, many existing KV cache systems still use recency-based cache replacement algorithms, e.g., least recently used or its approximations. However, the diversity of miss penalty distinguishes a KV cache from a hardware cache. Inadequate consideration of penalty can substantially compromise space utilization and request service time. KV accesses also demonstrate locality, which needs to be coordinated with miss penalty to guide cache management. In this article, we first discuss how to enhance the existing cache model, the Average Eviction Time model, so that it can adapt to modeling a KV cache. After that, we apply the model to Redis and propose pRedis, Penalty- and Locality-aware Memory Allocation in Redis, which synthesizes data locality and miss penalty, in a quantitative manner, to guide memory allocation and replacement in Redis. At the same time, we also explore the diurnal behavior of a KV store and exploit long-term reuse. We replace the original passive eviction mechanism with an automatic dump/load mechanism, to smooth the transition between access peaks and valleys. Our evaluation shows that pRedis effectively reduces the average and tail access latency with minimal time and space overhead. For both real-world and synthetic workloads, our approach delivers an average of 14.0%∼52.3% latency reduction over a state-of-the-art penalty-aware cache management scheme, Hyperbolic Caching (HC), and shows more quantitative predictability of performance. Moreover, we can obtain even lower average latency (1.1%∼5.5%) when dynamically switching policies between pRedis and HC.

Feasibility of Longest Prefix Matching using Learned Index Structures

This paper revisits longest prefix matching in IP packet forwarding because an emerging data structure, learned index, is recently presented. A learned index uses machine learning to associate key-value pairs in a key-value store. The fundamental idea to apply a learned index to an FIB is to simplify the complex longest prefix matching operation to a nearest address search operation. The size of the proposed FIB is less than half of an existing trie-based FIB while it achieves the computation speed nearly equal to the trie-based FIB. Moreover, the computation speed of the proposal is independent of the length of IP prefixes, unlike trie-based FIBs.

Omega: A Secure Event Ordering Service for the Edge

The edge computing paradigm extends cloud computing with storage and processing capacity close to the edge of the network, which can be materialized by using many fog nodes placed in multiple geographic locations. Fog nodes are likely to be vulnerable to tampering, so it is important to protect the functions they provide from attacks. A key building block of many distributed applications is an ordering service that keeps track of cause-effect dependencies among events and that allows events to be processed in an order that respects causality. This article presents the design and implementation of a secure event ordering service for fog nodes. Our service, named Omega, leverages the availability of a Trusted Execution Environment (TEE), based on SGX technology, to offer fog clients guarantees regarding the order in which events are applied and served, even when fog nodes are compromised. We have also built OmegaKV, a key-value store that uses Omega to offer causal consistency. Experimental results show that the ordering service can be secured without violating the latency constraints of time-sensitive edge applications, despite the overhead associated with using a TEE. Omega introduces an additional latency of approximately 4ms, that contrary to cloud based solutions, allows latency values in the 5ms-30ms range, as required by time-sensitive edge applications.

SwapKV: A Hotness Aware In-memory Key-Value Store for Hybrid Memory Systems

In-memory Key-Value (KV) stores are widely deployed in modern data centers. These systems generally use DRAM as their storage medium, causing huge hardware costs. The emerging persistent memory (PMEM) is a potential substitute for DRAM, which has a lower price and larger capacity, but lower access speed and bandwidth. Many prior studies strive to build hybrid memory systems to retain both the advantages of DRAM and PMEM. However, they are either application agnostic or simply take DRAM as a cache, which are both not efficient for in-memory KV stores. In this paper, we propose SwapKV, a well-designed in-memory KV store for hybrid DRAM-PMEM system. SwapKV has several promising properties. First, SwapKV combines DRAM and PMEM to a uniform memory pool and only stores one copy of data, which maximizes capacity utilization. Second, SwapKV maps all writing operations to DRAM and migrates data to PMEM with large blocks asynchronously, which mitigates the intrinsic inefficiency of PMEM for writing operations. Third, SwapKV maintains the hot data in DRAM through an efficient hotness filtering and data swapping mechanism, which ensures high system throughput and responsiveness. We implement SwapKV and evaluate it under various workload patterns. The results demonstrate that SwapKV improves the throughput by 11\% <formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex>$\sim$</tex> </formula> 41\% compared to the state-of-the-art alternatives.

Viper

Key-value stores (KVSs) have found wide application in modern software systems. For persistence, their data resides in slow secondary storage, which requires KVSs to employ various techniques to increase their read and write performance from and to the underlying medium. Emerging persistent memory (PMem) technologies offer data persistence at close-to-DRAM speed, making them a promising alternative to classical disk-based storage. However, simply drop-in replacing existing storage with PMem does not yield good results, as block-based access behaves differently in PMem than on disk and ignores PMem's byte addressability, layout, and unique performance characteristics. In this paper, we propose three PMem-specific access patterns and implement them in a hybrid PMem-DRAM KVS called Viper. We employ a DRAM-based hash index and a PMem-aware storage layout to utilize the random-write speed of DRAM and efficient sequential-write performance PMem. Our evaluation shows that Viper significantly outperforms existing KVSs for core KVS operations while providing full data persistence. Moreover, Viper outperforms existing PMem-only, hybrid, and disk-based KVSs by 4--18X for write workloads, while matching or surpassing their get performance.

Privacy-Preserving Similarity Search With Efficient Updates in Distributed Key-Value Stores

Privacy-preserving similarity search plays an essential role in data analytics, especially when very large encrypted datasets are stored in the cloud. Existing mechanisms on privacy-preserving similarity search were not able to support secure updates (addition and deletion) efficiently when frequent updates are needed. In this article, we propose a new mechanism to support parallel privacypreserving similarity search in a distributed key-value store in the cloud, with a focus on efficient addition and deletion operations, both executed with sublinear time complexity. If search accuracy is the top priority, we further leverage Yao's garbled circuits and the homomorphic property of Hash-ElGamal encryption to build a secure evaluation protocol, which can obtain the top-R most accurate results without extensive client-side post-processing. We have formally analyzed the security strength of our proposed approach, and performed an extensive array of experiments to show its superior performance as compared to existing mechanisms in the literature. In particular, we evaluate the performance of our proposed protocol with respect to the time it takes to build the index and perform similarity queries. Extensive experimental results demonstrated that our protocol can speedup the index building process by up to 800x with 2 threads and the similarity queries by up to -7x with comparable accuracy, as compared to the state-of-the-art in the literature.

Fast and secure Global-Heap for memory-centric computing

In-memory computing has been widely used to process data quickly in memory, but it is no longer able to cope with the current data explosion. This has occurred because modern computers have structural constraints in terms of both memory capacity and bandwidth, fundamentally limiting the amount of data that can be processed in memory. In order to overcome these limitations, the memory-centric computing concept was proposed, and as an attempt to make this concept feasible, nonvolatile memory, next-generation interconnects, and memory-centric operating system technologies are being actively studied. Although, however, there have been many studies of these concepts, the essential problem of both efficiently and safely using the extremely large memory space has not been completely solved. This paper proposes what is termed Global-Heap which solves the above essential problem, thus supporting the core technology of memory-centric computing. In this technology, a heap-level abstraction which can be switched very lightly between processes is provided in an application-friendly manner as a global resource. Global-Heap also provides an effective means by which safely to use this vast address space based on Intel memory protection keys. We implemented Global-Heap on Linux and developed three useful use cases applicable to data-centric applications. Our primary evaluation shows that Global-Heap enables reorganization-free hashing. In addition, Global-Heap can improve the performance of an in-memory key-value store, Redis, by at least 7.56 times, and it can reduce the running time of graph applications executed in a pipeline approach by at least 48%.

A Distributed Edge-Based Scheduling Technique with Low-Latency and High-Bandwidth for Existing Driver Profiling Algorithms

The gradual increase in latency-sensitive, real-time applications for embedded systems encourages users to share sensor data simultaneously. Streamed sensor data have deficient performance. In this paper, we propose a new edge-based scheduling method with high-bandwidth for decreasing driver-profiling latency. The proposed multi-level memory scheduling method places data in a key-value storage, flushes sensor data when the edge memory is full, and reduces the number of I/O operations, network latency, and the number of REST API calls in the edge cloud. As a result, the proposed method provides significant read/write performance enhancement for real-time embedded systems. In fact, the proposed application improves the number of requests per second by 3.5, 5, and 4 times, respectively, compared with existing light-weight FCN-LSTM, FCN-LSTM, and DeepConvRNN Attention solutions. The proposed application also improves the bandwidth by 5.89, 5.58, and 4.16 times respectively, compared with existing light-weight FCN-LSTM, FCN-LSTM, and DeepConvRNN Attention solutions.

An Elasticity Framework for Distributed Message Queuing Telemetry Transport Brokers

An Elasticity Framework for Distributed Message Queuing Telemetry Transport Brokers

Achieving high throughput and elasticity in a larger-than-memory store

Millions of sensors, mobile applications and machines now generate billions of events. Specialized many-core key-value stores (KVSs) can ingest and index these events at high rates (over 100 Mops/s on one machine) if events are generated on the same machine; however, to be practical and cost-effective they must ingest events over the network and scale across cloud resources elastically. We present Shadowfax, a new distributed KVS based on FASTER, that transparently spans DRAM, SSDs, and cloud blob storage while serving 130 Mops/s/VM over commodity Azure VMs using conventional Linux TCP. Beyond high single-VM performance, Shadowfax uses a unique approach to distributed reconfiguration that avoids any server-side key ownership checks or cross-core coordination both during normal operation and migration. Hence, Shadowfax can shift load in 17 s to improve system throughput by 10 Mops/s with little disruption. Compared to the state-of-the-art, it has 8x better throughput (than Seastar+memcached) and avoids costly I/O to move cold data during migration. On 12 machines, Shadowfax retains its high throughput to perform 930 Mops/s, which, to the best of our knowledge, is the highest reported throughput for a distributed KVS used for large-scale data ingestion and indexing.

Research and design of distributed key-value storage system based on Raft consensus algorithm

With the rapid development of the Internet, data storage technology has gradually become a major research field in the industry. But stand-alone storage has poor performance in terms of consistency or fault tolerance. Therefore, a new type of key-value storage system for unstructured data storage is widely used. This article mainly researches and designs a distributed key-value storage system based on the Raft consensus algorithm.

Optimistic Causal Consistency for Geo-Replicated Key-Value Stores

Causal consistency (CC) is an attractive consistency model for geo-replicated data stores because it hits a sweet spot in the ease-of-programming versus performance trade-off. We present a new approach for implementing CC in geo-replicated data stores, which we call Optimistic Causal Consistency (OCC). OCC's main design goal is to maximize data freshness. The optimism in our approach lies in the fact that the updates replicated to a remote data center are made visible immediately, without checking if their causal dependencies have been received. Servers perform the dependency check needed to enforce CC only upon serving a client operation, rather than on receipt of a replicated data item as in existing systems. OCC offers a significant gain in data freshness, which is of crucial importance for various types of applications, such as real-time systems. OCC's potentially blocking behavior makes it vulnerable to network partitions. We therefore propose a recovery mechanism that allows an OCC system to fall back on a pessimistic protocol to continue operating during network partitions. We implement POCC, the first causally consistent geo-replicated multi-master key-value data store designed to maximize data freshness. We show that POCC improves data freshness, while offering comparable or better performance than its pessimistic counterparts.