Lock-based Concurrency Control Research Articles

A Memory-Disaggregated Radix Tree

Disaggregated memory (DM) is an increasingly prevalent architecture with high resource utilization. It separates computing and memory resources into two pools and interconnects them with fast networks. Existing range indexes on DM are based on B+ trees, which suffer from large inherent read and write amplifications. The read and write amplifications rapidly saturate the network bandwidth, resulting in low request throughput and high access latency of B+ trees on DM. In this paper, we propose that the radix tree is more suitable for DM than the B+ tree due to smaller read and write amplifications. However, constructing a radix tree on DM is challenging due to the costly lock-based concurrency control, the bounded memory-side IOPS, and the complicated computing-side cache validation. To address these challenges, we design SMART , the first radix tree for disaggregated memory with high performance. Specifically, we leverage 1) a hybrid concurrency control scheme including lock-free internal nodes and fine-grained lock-based leaf nodes to reduce lock overhead, 2) a computing-side read-delegation and write-combining technique to break through the IOPS upper bound by reducing redundant I/Os, and 3) a simple yet effective reverse check mechanism for computing-side cache validation. Experimental results show that SMART achieves 6.1 × higher throughput under typical write-intensive workloads and 2.8 × higher throughput under read-only workloads in YCSB benchmarks, compared with state-of-the-art B+ trees on DM.

On the Effects of Transaction Data Access Patterns on Performance in Lock-Based Concurrency Control

Transaction Processing (TP) plays a primary role in the design and implementation of IT applications and services. Many TP systems exploit lock-based concurrency control to guarantee atomicity and isolation of transactions that access shared data. In this article, we show that transaction data access patterns, in particular the order of data accesses along the transaction execution, have a noticeable impact on how lock-based concurrency control affects performance. We show that the performance can remarkably change depending on whether transactions, or a percentage of them, access data items following some common ordering rule or not. We investigate on this aspect and its root causes through an analytical modeling approach, and with the evidence of data gathered through both simulation and the execution of real transactional workloads. Finally, we show how the findings of our study can be easily exploited for improving the performance of common transactional workloads.

Polaris: Enabling Transaction Priority in Optimistic Concurrency Control

Transaction priority is a critical feature for real-world database systems. Under high contention, certain classes of transactions should be given a higher chance to commit than others. Such a prioritization mechanism is commonly implemented in locking-based concurrency control protocols as some lock scheduling mechanisms, but it is rarely supported in the world of optimistic concurrency control. We present Polaris, an optimistic concurrency control protocol that supports multiple priority levels. To enforce priority, Polaris introduces a minimal amount of pessimism through a lightweight reservation mechanism. The protocol is fully optimistic among transactions within the same priority level and preserves the high throughput advantage of optimistic protocols. Our evaluation with YCSB workload shows that Polaris can make the p999 tail latency of high-priority transactions 13x lower than that of low-priority ones. With an abort-aware priority assignment policy, Polaris can deliver 1.9x higher throughput and 17x lower tail latency compared to Silo for high-contention workloads.

Lock-Free High-performance Hashing for Persistent Memory via PM-aware Holistic Optimization

Persistent memory (PM) provides large-scale non-volatile memory (NVM) with DRAM-comparable performance. The non-volatility and other unique characteristics of PM architecture bring new opportunities and challenges for the efficient storage system design. For example, some recent crash-consistent and write-friendly hashing schemes are proposed to provide fast queries for PM systems. However, existing PM hashing indexes suffer from the concurrency bottleneck due to the blocking resizing and expensive lock-based concurrency control for queries. Moreover, the lack of PM awareness and systematical design further increases the query latency. To address the concurrency bottleneck of lock contention in PM hashing, we propose clevel hashing, a lock-free concurrent level hashing scheme that provides non-blocking resizing via background threads and lock-free search/insertion/update/deletion using atomic primitives to enable high concurrency for PM hashing. By exploiting the PM characteristics, we present a holistic approach to building clevel hashing for high throughput and low tail latency via the PM-aware index/allocator co-design. The proposed volatile announcement array with a helping mechanism coordinates lock-free insertions and guarantees a strong consistency model. Our experiments using real-world YCSB workloads on Intel Optane DC PMM show that clevel hashing, respectively, achieves up to 5.7× and 1.6× higher throughput than state-of-the-art P-CLHT and Dash while guaranteeing low tail latency, e.g., 1.9×–7.2× speedup for the p99 latency with the insert-only workload.

Study of Concurrency Control Techniques in Distributed DBMS

Concurrency control focuses on maintaining consistency and integrity of database through synchronized access. The complexity relating to concurrency control in a distributed context is very high as compared to centralized framework due to maintaining consistency within the multiple fragments / copies of the database. This paper consolidates and discusses various lock based concurrency control techniques for Distributed DBMS. The paper also presents a comparative study of various two phase locking based concurrency control techniques.

NeuTu: Software for Collaborative, Large-Scale, Segmentation-Based Connectome Reconstruction.

Reconstructing a connectome from an EM dataset often requires a large effort of proofreading automatically generated segmentations. While many tools exist to enable tracing or proofreading, recent advances in EM imaging and segmentation quality suggest new strategies and pose unique challenges for tool design to accelerate proofreading. Namely, we now have access to very large multi-TB EM datasets where (1) many segments are largely correct, (2) segments can be very large (several GigaVoxels), and where (3) several proofreaders and scientists are expected to collaborate simultaneously. In this paper, we introduce NeuTu as a solution to efficiently proofread large, high-quality segmentation in a collaborative setting. NeuTu is a client program of our high-performance, scalable image database called DVID so that it can easily be scaled up. Besides common features of typical proofreading software, NeuTu tames unprecedentedly large data with its distinguishing functions, including: (1) low-latency 3D visualization of large mutable segmentations; (2) interactive splitting of very large false merges with highly optimized semi-automatic segmentation; (3) intuitive user operations for investigating or marking interesting points in 3D visualization; (4) visualizing proofreading history of a segmentation; and (5) real-time collaborative proofreading with lock-based concurrency control. These unique features have allowed us to manage the workflow of proofreading a large dataset smoothly without dividing them into subsets as in other segmentation-based tools. Most importantly, NeuTu has enabled some of the largest connectome reconstructions as well as interesting discoveries in the fly brain.

D ude T x

Emerging non-volatile memory (NVM) offers non-volatility, byte-addressability, and fast access at the same time. It is suggested that programs should access NVM directly through CPU load and store instructions. To guarantee crash consistency, durable transactions are regarded as a common choice of applications for accessing persistent memory data. However, existing durable transaction systems employ either undo logging , which requires a fence for every memory write, or redo logging , which requires intercepting all memory reads within transactions. Both approaches incur significant overhead. This article presents D ude T x , a crash-consistent durable transaction system that avoids the drawbacks of both undo and redo logging. D ude T x uses shadow DRAM to decouple the execution of a durable transaction into three fully asynchronous steps. The advantage is that only minimal fences and no memory read instrumentation are required. This design enables an out-of-the-box concurrency control mechanism, transactional memory or fine-grained locks, to be used as an independent component. The evaluation results show that D ude T x adds durability to a software transactional memory system with only 7.4%--24.6% throughput degradation. Compared to typical existing durable transaction systems, D ude T x provides 1.7× --4.4× higher throughput. Moreover, D ude T x can be implemented with hardware transactional memory or lock-based concurrency control, leading to a further 1.7× and 3.3× speedup, respectively.

Revisiting transactions in Ada

Classical lock-based concurrency control does not scale with current and foreseen multi-core architectures, opening space for alternative concurrency control mechanisms. The concept of transactions executing concurrently in isolation with an underlying mechanism maintaining a consistent system state was already explored in fault-tolerant and distributed systems, and is currently being explored by transactional memory, this time being used to manage concurrent memory access. In this paper we discuss the use of Software Transactional Memory (STM), and how Ada can provide support for it. Furthermore, we draft a general programming interface to transactional memory, supporting future implementations of STM oriented to real-time systems.

A concurrency control model for PDM systems

In today's manufacturing environment, enterprises having work groups geographically dispersed are not uncommon. A product data management (PDM) system is therefore required for controlling the distribution and maintaining the integrity of the product data throughout its entire lifecycle; the efficiency of a PDM system is greatly affected by the concurrency control method it adopts. The paper proposes a concurrent control model for PDM that can also caters for version management and product architecture. The paper discusses how granularity and versioning are being embedded into a lock-based concurrency control model. The concurrent accessibility of an example product data is explained to illustrate the adjustability according to the actions taken by the users and the architecture of the corresponding entities.

Transactional Memory: The Hardware-Software Interface

As multicore chips become ubiquitous, the need to provide architectural support for practical parallel programming is reaching critical. Conventional lock-based concurrency control techniques are difficult to use, requiring the programmer to navigate through the minefield of coarse-versus fine-grained locks, deadlock, livelock, lock convoying, and priority inversion. This explicit management of concurrency is beyond the reach of the average programmer, threatening to waste the additional parallelism available with multicore architectures. This comprehensive architecture supports nested transactions, transactional handlers, and two-phase commit. The result is a seamless integration of transactional memory with modern programming languages and runtime environments

A high concurrency XPath-based locking protocol for XML databases

Providing efficient access to XML documents becomes crucial in XML database systems. More and more concurrency control protocols for XML database systems were proposed in the past few years. Being an important language for addressing data in XML documents, XPath expressions are the basis of several query languages, such as XQurey and XSLT. In this paper, we propose a lock-based concurrency control protocol, called XLP, for transactions accessing XML data by the XPath model. XLP is based on the XPath model and has the features of rich lock modes, low lock conflict and lock conversion. XLP is also proved to ensure conflict serializability. In sum, there are three major contributions in this paper. The proposed XLP supports most XPath axes, rather than simple path expressions only. Conflict conditions and rules in the XPath model are analyzed and derived. Moreover, a lightweighted lock mode, P-lock, is invented and integrated into XLP for better concurrency.

A multi-granularity locking-based concurrency control in object-oriented database systems

In this paper, a concurrency control scheme to increase concurrency among methods in object-oriented database systems (OODBs) is presented. The author is concerned with all types of access to an object: instance access and class definition access. For instance access, the proposed work has the following characteristics. First, construction of commutativity relation among methods can be automated. Second, it provides more concurrency by taking attributes as locking granularity. Third, deadlocks due to lock escalation can be reduced. Finally, concurrency is increased further with the use of run-time information. For class definition access, the proposed work allows class definition access methods to run concurrently by taking fine granularity. The proposed work also allows more parallelism between class definition access methods and instance access methods. For the performance evaluation of the proposed scheme, a simulation model is constructed. Using this model, the proposed technique is then compared with the two existing techniques. The performance studies show that the proposed scheme is superior to existing works.

An adaptable constrained locking protocol for high data contention environments: correctness and performance

Multiversions of data are used in database systems to increase concurrency and to provide efficient recovery. Data versions improve the concurrency by allowing the concurrent execution of “non-conflicting” read-write lock requests on different versions of data in an arbitrary fashion. A transaction that accesses a data item version, which later diagnosed to lead to an incorrect execution, is aborted. This act is reminiscent of the validation phase in the optimistic concurrency control schemes. Various performance studies suggest that these schemes perform poorly in high data contention environments where the excessive transaction aborts result due to the failed validation. We propose an adaptable constrained two-version two-phase locking (C2V2PL) scheme in which these “non-conflicting” requests are allowed only in a constrained manner. C2V2PL scheme assumes that a lock request failing to satisfy the specific constraints will lead to an incorrect execution and hence, must be either rejected or blocked. This eliminates the need for a separate validation phase. When the contention for data among the concurrent transactions is high, the C2V2PL scheduler in aggressive state rejects such lock requests. The deadlock free nature of C2V2PL scheduler adapts to the low data contention environments by accepting the lock request that have failed the specific constraints but contrary to the assumption, will not lead to an incorrect execution. Thus, C2V2PL scheme improves the potential concurrency due to reduced transaction aborts in this conservative state. We have compared performance of our scheme with other lock-based concurrency control schemes such as two phase locking, Wait-depth locking and Optimistic locking schemes. Our results show increase in throughput and reduced transaction-abort-ratio in case of C2V2PL scheme.

Semantic-based locking technique in object-oriented databases

This paper presents a locking-based concurrency control dealing with three important issues in object-oriented databases: semantics of methods, nested method invocation and referentially shared object. In the proposed scheme, locks are required for the execution of methods instead of atomic operations. This can reduce the locking overhead and deadlocks due to lock escalation. Also, a way of automating commutativity of methods is provided. In addition, concurrency is increased further by use of run-time information. A performance study is conducted by means of simulation using 007 benchmark. Through the simulation, the proposed technique is then compared with the two existing techniques. The performance studies show that the proposed scheme is superior to existing works.

A secure dynamic copy protocol in real-time secure database systems

Concurrency control for real-time secure database systems must satisfy not only logical data consistency but also timing constraints and security requirements associated with transactions. These conflicting natures between timing constraints and security requirements are often resolved by maintaining several versions (or secondary copies) on the same data items. In this paper, we propose a new lock-based concurrency control protocol, Secure Dynamic Copy Protocol, ensuring both conflicting requirements. Our protocol aims for reducing the storage overhead of maintaining secondary copies and minimizing the processing overhead of update history. The main idea of our protocol is to keep a secondary copy only when it is needed to resolve the conflicting read/write operations in real time secure database systems. For doing this, a secondary copy is dynamically created and removed during a transaction's read/write operations. While comparing the existing real-time security protocol, we have also examined the performance characteristics of our protocol through simulation under different workloads. The results show that our protocol consumed less storage and decreased the deadline missing transactions.

Satisfying timing constraints of real-time databases

A real-time database has deadlines for processing transactions. Approximate query processing (AQP) has been presented as a strategy to satisfy these timing constraints by providing approximate answers to queries at the deadline instead of missing deadlines. In order to produce approximate answers, semantic information is maintained about the database and a computational overhead is required. In this paper the performance of AQP is examined to determine its effect on satisfying the timing constraints of real-time databases. Query and update transactions are modeled as periodic tasks with hard deadlines. A lock-based concurrency control is used and the effect of workload characteristics, such as the scheduling algorithm, the number of transactions and the percentage of update transactions, are examined. We compare the number of missed deadlines and approximate answers produced during AQP to the number of missed deadlines occurring during traditional query processing (TQP). Results demonstrate that despite the overhead, fewer deadlines are missed during approximate query processing than during TQP.

An effective class hierarchy concurrency control technique in object-oriented database systems

In this paper, we present a locking-based concurrency control scheme for object-oriented databases (OODBs). Our scheme deals with class hierarchy which is an important property in OODBs. The existing concurrence controls for a class hierarchy perform well only for specific environments. Our scheme is based on so-called special classes and can be used for any applications with less locking overhead than existing works, for both single inheritance and multiple inheritance. In this study, we show the superiority of our scheme over existing works.

Simulation studies on concurrency control in parallel transaction processing systems

Parallel transaction processing (TP) systems have great potential to serve the ever-increasing demands for high transaction processing rate. This potential, however, may not be reached due to the concurrency control (CC) requirements. In this paper a simulation system for shared-nothing parallel TP systems was presented, which aims at, but is not restricted to, the studies of the CC methods. A distributed locking-based CC method called LW DC k (local wait-depth control) was also proposed for overcoming the drawbacks of the widely-used two-phase locking (2PL) CC method. LW DC k was compared with 2PL and the well-known DWDL (distributed wait-depth limited) CC methods based on the presented simulation system. The application of this simulation system demonstrated its effectiveness. Simulation studies indicated also that the proposed method outperforms 2PL and DWDL.

Maintaining temporal consistency: pessimistic vs. optimistic concurrency control

We study the performance of concurrency control algorithms in maintaining temporal consistency of shared data in hard real time systems. In our model, a hard real time system consists of periodic tasks which are either write only, read only or update transactions. Transactions may share data. Data objects are temporally inconsistent when their ages and dispersions are greater than the absolute and relative thresholds allowed by the application. Real time transactions must read temporally consistent data in order to deliver correct results. Based on this model, we have evaluated the performance of two well known classes of concurrency control algorithms that handle multiversion data: the two phase locking and the optimistic algorithms, as well as the rate monotonic and earliest deadline first scheduling algorithms. The effects of using the priority inheritance and stack based protocols with lock based concurrency control are also studied. >

The performance of protocols based on locks with ordered sharing

There is growing evidence that for a wide variety of database workloads and system configurations, locking-based concurrency control outperforms other types of concurrency control strategies. However, in the presence of increased data contention, locking protocols such as two-phase locking perform poorly. In this paper, we analyze a family of locking-based protocols that employ a new relationship between locks called ordered sharing. Using a centralized database simulation model, we demonstrate that these protocols exhibit comparable performance to that of traditional locking-based protocols when data contention is low, and they exhibit superior performance when data contention is high. Furthermore, we show that the performance of these protocols improves as resources become more plentiful. This is particularly significant because the performance of two-phase locking degrades as a result of data contention, not resource contention. Thus, introducing additional resources improves the performance of the proposed protocols, though it does not benefit two-phase locking significantly. >