Lock-free Research Articles

Global Analysis of C Concurrency in High-Level Synthesis

When mapping C programs to hardware, highlevel synthesis (HLS) tools reorder independent instructions, aiming to obtain a schedule that requires as few clock cycles as possible. However, when synthesizing multithreaded C programs, reordering opportunities are limited by the presence of atomic operations (“atomics”), the fundamental concurrency primitives in C. Existing HLS tools analyze and schedule each thread in isolation. In this article, we argue that thread-local analysis is conservative, especially since HLS compilers have access to the entire program. Hence, we propose a global analysis that exploits information about memory accesses by all threads when scheduling each thread. Implemented in the LegUp HLS tool, our analysis is sensitive to sequentially consistent (SC) and weak atomics and supports loop pipelining. Since the semantics of C atomics is complicated, we formally verify that our analysis correctly implements the C memory model using the Alloy model checker. Compared with thread-local analysis, our global analysis achieves a 2.3× average speedup on a set of lock-free data structures and data-flow patterns. We also apply our analysis to a larger application: a lock-free, streamed, and load-balanced implementation of Google's PageRank, where we see a 1.3× average speedup compared with the thread-local analysis.

Asynchronous Non-Blocking Algorithm to Handle Straggler Reduce Tasks in Hadoop System

Hadoop is widely adopted as a big data processing application as it can run on commercial hardware at a reasonable time. Hadoop uses asynchronous blocking concurrency using Thread and Future class. Therefore, in some cases such as network link or hardware failure, a running task may block other tasks from running (the task becomes straggler). Hadoop releases are equipped with algorithms to handle straggler tasks problem. However, the algorithms manage Map and Reduce task similarly, while the straggler root cause might be different for both tasks. In this paper, the Asynchronous Non-Blocking (ANB) method is proposed to improve the performance and avoid the blocking of Reduce task in Hadoop. Instead of using the single queue, our approach uses two queues, i.e. task queue and callback queue. When a task is not ready or detected as a straggler, it is removed from the main task queue and temporarily sent to the callback queue. When the task is ready to run, it will be sent back to the main task queue for running. The performance of the algorithm is compared with rTuner, the latest paper found on handling straggler task in Reduce task. From the comparison, it is shown that ANB consistently gives faster time to complete because any unready tasks will be directly put into the callback queue without blocking other tasks. Furthermore, the overhead time in rTuner is high as it needs to check the straggler status and to find the reason for a task to become straggler.

Lock-Free Parallelization for Variance-Reduced Stochastic Gradient Descent on Streaming Data

Stochastic Gradient Descent (SGD) is an iterative algorithm for fitting a model to the training dataset in machine learning problems. With low computation cost, SGD is especially suited for learning from large datasets. However, the variance of SGD tends to be high because it uses only a single data point to determine the update direction at each iteration of gradient descent, rather than all available training data points. Recent research has proposed variance-reduced variants of SGD by incorporating a correction term to approximate full-data gradients. However, it is difficult to parallelize such variants with high performance and accuracy, especially on streaming data. As parallelization is a crucial requirement for large-scale applications, this article focuses on the parallel setting in a multicore machine and presents LFS-STRSAGA, a lock-free approach to parallelizing variance-reduced SGD on streaming data. LFS-STRSAGA embraces a lock-free data structure to process the arrival of streaming data in parallel, and asynchronously maintains the essential information to approximate full-data gradients with low cost. Both our theoretical and empirical results show that LFS-STRSAGA matches the accuracy of the state-of-the-art variance-reduced SGD on streaming data under sparsity assumption (common in machine learning problems), and that LFS-STRSAGA reduces the model update time by over 98 percent.

Concurrent updates to pages with fixed-size rows using lock-free algorithms

Database systems based on ARIES [11] protocol rely on Write Ahead Logging (WAL) to recover the database in the event of a crash. WAL protocol requires changes to the database are recorded to the transaction log before updating the underlying database page. WAL also mandates that the log record corresponding to the change is persisted to disk before the updated page. While WAL allows updates to the databases using in-place updates or using shadow paging, database systems that perform in-place updates typically latch the page exclusively for the entire duration of log generation and the change on the page. The exclusive latch on the page prevents other threads from modifying the page at the same time, reducing the concurrency, and negatively impacting the throughput of the system. While approaches like Segment-Based recovery [16] attempt to solve the contention by pushing the burden of synchronization to the application along with a proposal for recovering parts of pages, this paper takes a different approach by providing a mechanism to support concurrent updates to certain kinds of pages under a shared latch using lock-free algorithms. The pages are recovered using existing ARIES protocol with a few modifications. This approach significantly boosts the throughput of an ARIES based database system, without any application changes. The paper describes in detail the challenges of implementing the mechanism and how the ARIES concepts like page LSN, logging and checkpoint are handled to support concurrent updates on space maintenance pages in Microsoft SQL Server. The paper also presents the experimental results showcasing the impact of the work.

Fair and trustworthy: Lock-free enhanced tendermint blockchain algorithm

Blockchain Technology is exclusively used to make online transactions secure by maintaining a distributed and decentralized ledger of records across multiple computers. Tendermint is a general-purpose blockchain engine that is composed of two parts; Tendermint Core and the blockchain application interface. The application interface makes Tendermint suitable for a wide range of applications. In this paper, we analyze and improve Practical Byzantine Fault Tolerant (PBFT), a consensus-based Tendermint blockchain algorithm. In order to avoid negative issues of locks, we first propose a lock-free algorithm for blockchain in which the proposal and voting phases are concurrent whereas the commit phase is sequential. This consideration in the algorithm allows parallelism. Secondly, a new methodology is used to decide the size of the voter set which is a subset of blockchain nodes, further investigating the block sensitivity and trustworthiness of nodes. Thirdly, to fairly select the voter set nodes, we employ the random walk algorithm. Fourthly, we imply the wait-freedom property by using a timeout due to which all blocks are eventually committed or aborted. In addition, we have discussed voting conflicts and consensuses issues that are used as a correctness property, and provide some supportive techniques.

The amortized analysis of a non-blocking chromatic tree

A non-blocking chromatic tree is a type of balanced binary search tree where multiple processes can concurrently perform search and update operations. We prove that a certain implementation has amortized cost O(c˙+log⁡n) for each operation, where c˙ is the maximum number of concurrent operations during the execution and n is the maximum number of keys in the tree during the operation. This amortized analysis presents new challenges compared to existing analyses of other non-blocking data structures.

FEAST

We present a lock-free algorithm for concurrent manipulation of a binary search tree (BST) in an asynchronous shared memory system that supports search, insert, and delete operations. In addition to read and write instructions, our algorithm uses (single-word) compare-and-swap (CAS) and bit-test-and-set (BTS) read-modify-write (RMW) instructions, both of which are commonly supported by many modern processors including Intel 64 and AMD64. In contrast to most of the existing concurrent algorithms for a binary search tree, our algorithm is edge-based rather than node-based. When compared to other concurrent algorithms for a binary search tree, modify (insert and delete) operations in our algorithm (a) work on a smaller section of the tree, (b) execute fewer RMW instructions, or (c) use fewer dynamically allocated objects. In our experiments, our lock-free algorithm significantly outperformed all other algorithms for a concurrent binary search tree especially when the contention was high. We also describe modifications to our basic lock-free algorithm so that the amortized complexity of any operation in the modified algorithm can be bounded by the sum of the tree height and the point contention to within a constant factor while preserving the other desirable features of our algorithm.

Step-optimal implementations of large single-writer registers

We present two wait-free algorithms for simulating an ℓ-bit single-writer register from k-bit single-writer registers, for any k≥1. Our first algorithm has Θ(ℓ/k) step complexity for both ▪ and ▪ and uses Θ(4ℓ−k) registers. Our second algorithm has Θ(ℓ/k+(log⁡n)/k) step complexity for both ▪ and ▪, where n is the number of readers, but uses only Θ(nℓ/k) registers. By using the first algorithm when ℓ≤(log⁡n)/2 and the second algorithm when ℓ>(log⁡n)/2, we get a combined implementation with Θ(ℓ/k) step complexity using Θ(nℓ/k) registers which works for any 1≤k<ℓ.We also prove that any implementation with O(ℓ/k) step complexity for ▪ requires Ω(ℓ/k) step complexity for ▪. Reading ℓ bits requires at least ⌈ℓ/k⌉ reads of k-bit registers, so our lower bound shows that our combined implementation is step-optimal.

Design-space evaluation for non-blocking synchronization in Ada: lock elision of protected objects, concurrent objects, and low-level atomics

The current ISO/IEC standard of the Ada programming language does not support non-blocking synchronization. The restriction to locks conflicts with Ada’s design goals for program safety and efficiency , because (1) task failure inside a critical section may incur deadlock, and (2) locks stand in the way of scaling parallel programs on multicore architectures. Increased autonomy of software systems and advances in embedded multicore platforms make non-blocking synchronization a desirable feature for Ada, which is traditionally employed for safety-critical embedded applications in the automotive and aerospace domains. We propose two techniques to support non-blocking synchronization in Ada: (1) Lock elision of Ada’s Hoare-style monitor synchronization construct (called “protected object”) allows method calls of the monitor to overlap in time; inter-thread data conflicts are resolved by underlying hardware transactional memory. (2) Concurrent objects constitute a new programming primitive to encapsulate the complexity of non-blocking synchronization in a language-level construct. We investigate the use of an alternative, low-level API that employs atomic read–modify–write operations in the style of C + + 11 , in conjunction with relaxed memory consistency models. We conduct an extensive experimental evaluation on an x86- and an ARM v8 multicore platform to explore the trade-offs of the proposed designs with respect to programmability, scalability and performance; and evaluate the performance improvements achievable with relaxed memory consistency models. We include a comparison with state-of-the-art blocking synchronization constructs.

Everyone Loves File

Oracle File Storage Service (FSS) is an elastic filesystem provided as a managed NFS service. A pipelined Paxos implementation underpins a scalable block store that provides linearizable multipage limited-size transactions. Above the block store, a scalable B-tree holds filesystem metadata and provides linearizable multikey limited-size transactions. Self-validating B-tree nodes and housekeeping operations performed as separate transactions allow each key in a B-tree transaction to require only one page in the underlying block transaction. The filesystem provides snapshots by using versioned key-value pairs. The system is programmed using a nonblocking lock-free programming style. Presentation servers maintain no persistent local state making them scalable and easy to failover. A non-scalable Paxos-replicated hash table holds configuration information required to bootstrap the system. An additional B-tree provides conversational multi-key minitransactions for control-plane information. The system throughput can be predicted by comparing an estimate of the network bandwidth needed for replication to the network bandwidth provided by the hardware. Latency on an unloaded system is about 4 times higher than a Linux NFS server backed by NVMe, reflecting the cost of replication. FSS has been in production since January 2018 and holds tens of thousands of customer file systems comprising many petabytes of data.

Non-Blocking Synchronization Between Real-Time and Non-Real-Time Applications

Real-time systems where applications with timing requirements coexist with applications without timing constraints are increasingly common. Furthermore, the processors used in desktops, smart phones or embedded devices are mostly multi-core, allowing the execution of applications in parallel. This article presents a set of non-blocking synchronization mechanisms to share data between real-time and non-real-time applications executing in different cores of a shared memory multi-core system. Four typical producer/consumer scenarios have been explored; a) shared data object with real-time reader, b) shared data object with real-time writer, c) shared queue with real-time writer, and d) shared queue with real-time reader. For these scenarios we have developed different non-blocking protocols where the execution of the real-time application is always prioritized over the execution of the non-real-time part. In this way, the real-time applications never have to repeat their operations and, consequently, their execution times are bounded. Furthermore, to reduce the overhead caused by the copies of the information used in the non-blocking algorithms, we have imposed the limitation of a single real-time reader or a single real-time writer in the algorithms developed. Finally, we have evaluated the response times of the developed protocols on a multi-core device with the Android operating system.

Pointer life cycle types for lock-free data structures with memory reclamation

We consider the verification of lock-free data structures that manually manage their memory with the help of a safe memory reclamation (SMR) algorithm. Our first contribution is a type system that checks whether a program properly manages its memory. If the type check succeeds, it is safe to ignore the SMR algorithm and consider the program under garbage collection. Intuitively, our types track the protection of pointers as guaranteed by the SMR algorithm. There are two design decisions. The type system does not track any shape information, which makes it extremely lightweight. Instead, we rely on invariant annotations that postulate a protection by the SMR. To this end, we introduce angels, ghost variables with an angelic semantics. Moreover, the SMR algorithm is not hard-coded but a parameter of the type system definition. To achieve this, we rely on a recent specification language for SMR algorithms. Our second contribution is to automate the type inference and the invariant check. For the type inference, we show a quadratic-time algorithm. For the invariant check, we give a source-to-source translation that links our programs to off-the-shelf verification tools. It compiles away the angelic semantics. This allows us to infer appropriate annotations automatically in a guess-and-check manner. To demonstrate the effectiveness of our type-based verification approach, we check linearizability for various list and set implementations from the literature with both hazard pointers and epoch-based memory reclamation. For many of the examples, this is the first time they are verified automatically. For the ones where there is a competitor, we obtain a speed-up of up to two orders of magnitude.

Evaluating the performance of atomic operations on modern multicore systems

In this work we analyse the efficiency of atomic operations compare-and-swap (CAS), fetch-and-add (FAA), swap (SWP), load and store on modern multicore processors. These operations implemented in hardware as processor instructions are highly demanded in multithreaded programming (design of thread locks and non-blocking data structures). In this article we study the influence of cache coherence protocol, size and locality of the data on the latency of the operations. We developed a benchmark for analyzing the dependencies of throughput and latency on these parameters. We present the results of the evaluation of the efficiency of atomic operations on modern x86-64 processors and give recommendations for the optimizations. Particularly we found atomic operations, which have minimum (load), maximum (“successful CAS”, store) and comparable (“unsuccessful CAS”, FAA, SWP) latency. We showed that the choice of a processor core to perform the operation and the state of cache-line impact on the latency at average 1.5 and 1.3 times respectively. The suboptimal choice of the parameters may increase the throughput of atomic operations from 1.1 to 7.2 times. Our evidences may be used in the design of new and optimization of existing concurrent data structures and synchronization primitives.

A massively parallel CFD/DEM approach for reactive gas-solid flows in complex geometries using unstructured meshes

Despite having been thoroughly described in various simple configurations, the study of gas-fluidized systems in a CFD/DEM (Discrete Element Method) formalism becomes challenging as the computational domain size and complexity rise. For a while, attention has been drawn to the design of physical models for fluid-particles interactions, but a recent challenge for numerical tools has been to take advantage from the increasing power of distributed memory machines, in order to simulate realistic industrial systems. Furthermore, unstructured meshes are appealing for their ability to describe complex geometries and to perform local refinements, but lead to significant coding effort involving sophisticated algorithm. In a attempt to design a numerical tool able to cope with these limitations, the methodology presented here proposes an efficient non-blocking algorithm for massive parallelism management, as well as an exhaustive contact scheme to deal with arbitrarily complex geometries, all to be operated on unstructured meshes. The aim is two-fold: (i) To assist larger scale codes in their endeavor to close the solid stress tensor for example, (ii) to pave the way for complex industrial-scale systems modeling using DEM. The methodology is successfully applied to a pilot-scale fluidized bed gathering 9.6M spherical particles and enables to reach interesting physical times using reasonable computational resources.

Lock-free Data Structures for Data Stream Processing

Processing data in real-time instead of storing and reading from tables has led to a specialization of DBMS into the so-called data stream processing paradigm. While high throughput and low latency are key requirements to keep up with varying stream behavior and to allow fast reaction to incoming events, there are many possibilities how to achieve them. In combination with modern hardware, like server CPUs with tens of cores, the parallelization of stream queries for multithreading and vectorization is a common schema. High degrees of parallelism, however, need efficient synchronization mechanisms to allow good scaling with threads for shared memory access.In this work, we identify the most time-consuming operations for stream processing exemplarily for our own stream processing engine PipeFabric. In addition, we present different design principles of lock-free data structures which are suited to overcome those bottlenecks. We will finally demonstrate how lock-freedom greatly improves performance for join processing and tuple exchange between operators under different workloads. Nevertheless, the efficient usage of lock-free data structures comes with additional efforts and pitfalls, which we also discuss in this paper.

BiloKey : A Scalable Bi-Index Locality-Aware In-Memory Key-Value Store

Fast in-memory key value stores are the keys to building large-scale Internet services. The state-of-the-art solutions mainly focus on optimizing the performance for read-intensive workloads. Nevertheless, a wide range of applications demonstrate a significant amount of updates and range queries, which scale poorly with the current implementations. In this paper, we present BiloKey, a highly scalable in-memory key value store on multi-core machines, significantly outperforming Redis and Memcached for a variety of mixed read and write workloads. To achieve this, BiloKey leverages a fast bi-index comprised by a Hash Table index and a SkipList index, where the former supports feature rich operations including GET, UPDATE and DELETE with $O$O(1) complexity, while the latter supports SCAN with $O$O(log$N$N) complexity. Furthermore, to make the bi-index design scale well, BiloKey adopts three techniques: lazy synchronization for reducing the overhead of maintaining index consistency, lock-free data structure for supporting multi-writers, and locality-aware data parallel processing for preserving the data locality of requests. Compared with two popular in-memory KV stores (i.e., Redis and Memcached), experimental results show that: (1) for write-intensive workloads, BiloKey outperforms Redis and Memcached by 7.8x and 3.7x on average (up to 11.5x and 4.8x), respectively; (2) for scan-intensive workloads, BiloKey achieves an average speedup of 2.3x against Redis; (3) for read-intensive workloads, BiloKey also outperforms Redis and Memcached by 1.2x and 1.8x on average.

Fast Wait-Free Construction for Pool-Like Objects with Weakened Internal Order: Stacks as an Example

This paper focuses on a large class of concurrent data structures that we call pool-like objects (e.g., stack, double-ended queue, and queue). Performance and progress guarantee are two important characteristics for concurrent data structures. In the aspect of performance, weakening the internal order in a pool-like object is an effective technique to reduce the synchronization cost among threads accessing the object, but no objects with weakened internal order provide a progress guarantee as strong as wait-freedom. Meanwhile, wait-free algorithms tend to be inefficient, which is mainly attributed to the helping mechanisms. Based on the philosophy of existing helping mechanisms, a wait-free pool-like object with weakened internal order would suffer from unnecessary process of getting the latest object state and synchronization. This paper takes a state-of-the-art implementation of stacks with weakened internal order as an example, and transforms it into a highly-efficient wait-free stack named WF-TS-Stack. The transformation method includes a helping mechanism with state reuse and a relaxed removal scheme. In addition, we use a simple and effective scheme to further improve the performance of WF-TS-Stack in Non-Uniform Memory Access (NUMA) architectures. Our evaluation with representative benchmarks shows that WF-TS-Stack outperforms its original building blocks by up to 1.45× at maximum concurrency. We also discuss how to yield an efficient double-ended queue (deque) variant of WF-TS-Stack, because deque is a more generalized pool-like object.

Detecting semantic violations of lock-free data structures through C++ contracts

The use of synchronization mechanisms in multithreaded applications is essential on shared-memory multi-core architectures. However, debugging parallel applications to avoid potential failures, such as data races or deadlocks, can be challenging. Race detectors are key to spot such concurrency bugs; nevertheless, if lock-free data structures are used, these may emit a significant number of false positives. In this paper, we present a framework for semantic violation detection of lock-free data structures which makes use of contracts, a novel feature of the upcoming C++20, and a customized version of the ThreadSanitizer race detector. We evaluate the detection accuracy of the framework in terms of false positives and false negatives leveraging some synthetic benchmarks which make use of the SPSC and MPMC lock-free queue structures from the Boost C++ library. Thanks to this framework, we are able to check the correct use of lock-free data structures, thus reducing the number of false positives.

Decoupling lock-free data structures from memory reclamation for static analysis

Verification of concurrent data structures is one of the most challenging tasks in software verification. The topic has received considerable attention over the course of the last decade. Nevertheless, human-driven techniques remain cumbersome and notoriously difficult while automated approaches suffer from limited applicability. The main obstacle for automation is the complexity of concurrent data structures. This is particularly true in the absence of garbage collection. The intricacy of lock-free memory management paired with the complexity of concurrent data structures makes automated verification prohibitive. In this work we present a method for verifying concurrent data structures and their memory management separately. We suggest two simpler verification tasks that imply the correctness of the data structure. The first task establishes an over-approximation of the reclamation behavior of the memory management. The second task exploits this over-approximation to verify the data structure without the need to consider the implementation of the memory management itself. To make the resulting verification tasks tractable for automated techniques, we establish a second result. We show that a verification tool needs to consider only executions where a single memory location is reused. We implemented our approach and were able to verify linearizability of Michael&amp;Scott's queue and the DGLM queue for both hazard pointers and epoch-based reclamation. To the best of our knowledge, we are the first to verify such implementations fully automatically.

Accelerating Wait-Free Algorithms: Pragmatic Solutions on Cache-Coherent Multicore Architectures

Parallelizing performance-critical applications are of critical importance for harnessing the power of multicore processors, which are now ubiquitous. Even though wait-free algorithms offer the appeal of completing each operation of a parallelized application in a finite number of steps, high-performance wait-free algorithms at high levels of concurrency are still rare. In this paper, we demonstrate one primary reason for this inefficiency: existing wait-free algorithms are not optimized for processors’ caches and write buffers, two key components in modern hardware to accelerate memory accesses. As an example, a wait-free multi-producer-single-consumer queue algorithm, which faces common performance problems of wait-free algorithms, is studied in this paper. We accelerate the queue algorithm by (1) allowing producers to buffer enqueue requests in their local buffers and to write them into the shared queue in batch, to exhibit the spatial locality of the program, and (2) eliminating expensive atomic operations, by giving up some degree of internal consistency to avoid write buffers being drained frequently. The outcome is a write-buffer and cache-friendly queue algorithm which is wait-free and efficient on off-the-shelf multicore processors. The experiments show that the optimized queue algorithm outperforms prior queue algorithms on three different architectures ( $\times $ 86, Power8, and ARMv8). On $\times $ 86, it outperforms WFQueue, the state-of-the-art solution, by 2–3 $\times$ , and outperforms CCQueue, the representative of combining solution, by 4–12 $\times $ . Applying the techniques presented in this paper to other wait-free algorithms is straightforward; our queue example demonstrates that these techniques can be applied to other wait-free algorithms while maintaining the control flow of the original algorithms without dramatic changes.