Wait-free Implementation Research Articles

Wait-free implementations in message-passing systems

We study the round complexity of problems in a synchronous, message-passing system with crash failures. We show that if processors start in order-equivalent states, then a logarithmic number of rounds is both necessary and sufficient for them to reach order-inequivalent states. These upper and lower bounds are significant because they establish a complexity threshold below which no nontrivial problem can be solved, but at which certain nontrivial problems do have solutions. This logarithmic lower bound implies a matching lower bound for a variety of decision tasks and concurrent object implementations. In particular, we examine two nontrivial problems for which this lower bound is tight: the strong renaming task, and a wait-free increment register implementation. For each problem, we present a nontrivial algorithm that halts in O( log c) rounds, where c is the number of participating processors.

On the space complexity of randomized synchronization

The “waite-free hierarchy” provides a classification of multiprocessor synchronization primitives based on the values ofnfor which there are deterministic wait-free implementations ofn-process consensus using instances of these objects andread-writeregisters. In a randomized wait-free setting, this classification is degenerate, sincen-process consensus can be solved using onlyO(n) read-writeregisters.In this paper, we propose a classification of synchronization primitives based on thespace complexityof randomized solutions ton-process consensus. Ahistoryless object,such as aread-writeregister, aswapregister, or atest&setregister, is an object whose state depends only on the lost nontrivial operation thate was applied to it. We show that, usinghistorylessobjects, Ω(√n) object instances are necessary to solven-process consensus. This lower bound holds even if the objects have unbounded size and the termination requirement isnondeterministic solo termination, a property strictly weaker than randomized wait-freedom.We then use this result to related the randomized space complexity of basic multiprocessor synchronization primitives such asshared counters, fetch&addregisters, andcompare&swapregisters. Viewed collectively, our results imply that there is a separation based on space complexity for synchronization primitives in randomized computation, and that this separation differs from that implied by the deterministic “wait-free hierarchy.”

Randomized naming using wait-free shared variables

A naming protocol assigns unique names (keys) to every process out of a set of communicating processes. We construct a randomized wait-free naming protocol using wait-free atomic read/write registers (shared variables) as process intercommunication primitives. Each process has its own private register and can read all others. The addresses/names each one uses for the others are possibly different: Processes p and q address the register of process r in a way not known to each other. For n processes and e > 0, the protocol uses a name space of size (1 + e)n and O(n log n log log n) running time (read/writes to shared bits) with probability at least 1-o(1), and O(nlog2n) overall expected running time. The protocol is based on the wait-free implementation of a novel α-Test&SetOnce object that randomly and fast selects a winner from a set of q contenders with probability at least α in the face of the strongest possible adaptive adversary.

Fault-tolerant wait-free shared objects

Wait-free implementations of shared objects tolerate the failure of processes, but not the failure of base objects from which they are implemented. We consider the problem of implementing shared objects that tolerate the failure of both processes and base objects.We identify two classes of object failures:responsiveandnonresponsive. With responsive failures, a faulty object responds to every operation, but its responses may be incorrect. With nonresponsive failures, a faulty object may also “hang” without responding. In each class, we definecrash, omission,andarbitrarymodes of failure.We show that all responsive failure modes can be tolerated. More precisely, for all responsive failure modes ℱ, object typesT, andt≥ 0, we show how to implement a shared object of typeTwhich ist-tolerant for ℱ. Such an object remains correct and wait-free even if up totbase objects fail according to ℱ. In contrast to responsive failures, we show that even the most benign non-responsive failure mode cannot be tolerated. We also show that randomization can be used to circumvent this impossibility result.Graceful degradationis a desirable property of fault-tolerant implementations: the implemented object never fails more severely than the base objects it is derived from, even if all the base objects fail. For several failure modes, we show wheter this property can be achieved, and, if so, how.

Robust wait-free hierarchies

The problem of implementing a shared object of one type from shared objects of other types has been extensively researched. Recent focus has mostly been onwait-free implementations, which permit every process to complete its operations on implemented objects, regardless of the speeds of other processes. It is known that shared objects of different types have differing abilities to support wait-free implementations. It is therefore natural to want to arrange types in a hierarchy that reflects their relative abilities to support wait-free implementations. In this paper, we formally define robustness and other desirable properties of hierarchies. Roughly speaking, a hierarchy is robust if each type is “stronger” than any combination of lower level types. We study two specific hierarchies: one, that we callhrmin which the level of a type is based on the ability of anunboundednumber of objects of that type, and another hierarchy, that we callhr1, in which a type's level is based on the ability of afixednumber of objects of that type. We prove that resource bounded hierarchies, such ashr1and its variants, are not robust. We also establish the unique importance ofhrm: every nontrivial robust hierarchy, if one exists, is necessarily a “coarsening” ofhrm.

A methodology for implementing highly concurrent data objects

A concurrent object is a data structure shared by concurrent processes. Conventional techniques for implementing concurrent objects typically rely on critical sections ; ensuring that only one process at a time can operate on the object. Nevertheless, critical sections are poorly suited for asynchronous systems: if one process is halted or delayed in a critical section, other, nonfaulty processes will be unable to progress. By contrast, a concurrent object implementation is lock free if it always guarantees that some process will complete an operation in a finite number of steps, and it is wait free if it guarantees that each process will complete an operation in a finite number of steps. This paper proposes a new methodology for constructing lock-free and wait-free implementations of concurrent objects. The object's representation and operations are written as stylized sequential programs, with no explicit synchronization. Each sequential operation is atutomatically transformed into a lock-free or wait-free operation using novel synchronization and memory management algorithms. These algorithms are presented for a multiple instruction/multiple data (MIMD) architecture in which n processes communicate by applying atomic read, write, load_linked, and store_conditional operations to a shared memory.

Using Smalltalk for wait-free implementation of highly-concurrent objects

In this paper I describe wait-free implementations of concurrent objects and show, with the help of an example, how the design of the Smalltalk language can allow a concurrent version of Smalltalk to easily support highly-concurrent objects. Specifically, I explain how a universal wait-free algorithm written in Smalltalk can allow arbitrary concurrent-objects to be easily implemented and used in concurrent applications.

Atomic snapshots of shared memory

This paper introduces a general formulation of atomic snapshot memory , a shared memory partitioned into words written ( updated ) by individual processes, or instantaneously read ( scanned ) in its entirety. This paper presents three wait-free implementations of atomic snapshot memory. The first implementation in this paper uses unbounded (integer) fields in these registers, and is particularly easy to understand. The second implementation uses bounded registers. Its correctness proof follows the ideas of the unbounded implementation. Both constructions implement a single-writer snapshot memory, in which each word may be updated by only one process, from single-writer, n -reader registers. The third algorithm implements a multi-writer snapshot memory from atomic n -writer, n -reader registers, again echoing key ideas from the earlier constructions. All operations require Θ( n 2 ) reads and writes to the component shared registers in the worst case. — Authors' Abstract

Beyond atomic registers: bounded wait-free implementations of nontrivial objects

We define a class of operations called pseudo read-modify-write (PRMW) operations, and show that nontrivial shared data objects with such operations can be implemented in a bounded wait-free manner from atomic registers. A PRMW operation is similar to a “true” read-modify-write (RMW) operation in that it modifies the value of a shared variable based upon the original value of that variable. However, unlike an RMW operation, a PRMW operation does not return the value of the variable that it modifies. We consider a class of shared data objects that can either be read, written, or modified by an associative, commutative PRMW operation, and show that any object in this class can be implemented without waiting from atomic registers. The implementations that we present are polynomial in both space and time and thus are an improvement over previously published ones, all of which have unbounded space complexity.

A Methodology for Implementing Highly Concurrent Data Objects

A concurrent object is a data structure shared by concurrent processes. Conventional techniques for implementing concurrent objects typically rely on critical sections; ensuring that only one process at a time can operate on the object. Nevertheless, critical sections are poorly suited for asynchronous systems: if one process is halted or delayed in a critical section, other, nonfaulty processes will be unable to progress. By contrast, a concurrent object implementation is lock free if it always guarantees that some process will complete an operation in a finite number of steps, and it is wait free if it guarantees that each process will complete an operation in a finite number of steps. This paper proposes a new methodology for constructing lock-free and wait-free implementations of concurrent objects. The object's representation and operations are written as stylized sequential programs, with no explicit synchronization. Each sequential operation is atutomatically transformed into a lock-free or wait-free operation using novel synchronization and memory management algorithms. These algorithms are presented for a multiple instruction/multiple data (MIMD) architecture in which n processes communicate by applying atomic read, write, load_linked, and store_conditional operations to a shared memory.

Wait-free synchronization

A wait-free implementation of a concurrent data object is one that guarantees that any process can complete any operation in a finite number of steps, regardless of the execution speeds of the other processes. The problem of constructing a wait-free implementation of one data object from another lies at the heart of much recent work in concurrent algorithms, concurrent data structures, and multiprocessor architectures. First, we introduce a simple and general technique, based on reduction to a concensus protocol, for proving statements of the form, “there is no wait-free implementation of X by Y.” We derive a hierarchy of objects such that no object at one level has a wait-free implementation in terms of objects at lower levels. In particular, we show that atomic read/write registers, which have been the focus of much recent attention, are at the bottom of the hierarchy: thay cannot be used to construct wait-free implementations of many simple and familiar data types. Moreover, classical synchronization primitives such as test&set and fetch&add , while more powerful than read and write , are also computationally weak, as are the standard message-passing primitives. Second, nevertheless, we show that there do exist simple universal objects from which one can construct a wait-free implementation of any sequential object.

A methodology for implementing highly concurrent data structures

A concurrent object is a data structure shared by concurrent processes. Conventional techniques for implementing concurrent objects typically rely on critical sections : ensuring that only one process at a time can operate on the object. Nevertheless, critical sections are poorly suited for asynchronous systems: if one process is halted or delayed in a critical section, other, non-faulty processes will be unable to progress. By contrast, a concurrent object implementation is non-blocking if it always guarantees that some process will complete an operation in a finite number of steps, and it is wait-free if it guarantees that each process will complete an operation in a finite number of steps. This paper proposes a new methodology for constructing non-blocking and wait-free implementations of concurrent objects. The object's representation and operations are written as stylized sequential programs, with no explicit synchronization. Each sequential operation is automatically transformed into a non-blocking or wait-free operation using novel synchronization and memory management algorithms. These algorithms are presented for a multiple instruction/multiple data (MIMD) architecture in which n processes communicate by applying read, write , and compare&swap operations to a shared memory.

Taking concurrency seriously (position paper)

I'd like to propose a challenge to language designers interested in concurrency: how well do your favorite constructs support highly-concurrent data structures? For example, consider a real-time system consisting of a pool of sensor and actuator processes that communicate via a priority queue in shared memory. Processes execute asynchronously. When a sensor process detects a condition requiring a response, it records the condition, assigns it a priority, and places the record in the queue. Whenever an actuator process becomes idle, it dequeues the highest priority item from the queue and takes appropriate action. The conventional way to prevent concurrent queue operations from interfering is to execute each operation as a critical section : only one process at a time is allowed to access the data structure. As long as one process is executing an operation, any other needing to access the queue must wait. Although this approach is widely used, it has significant drawbacks. Similar concerns arise even in systems not subject to real-time demands or failures. For example, process execution speeds may vary considerably if processors are multiplexed among multiple processes. If a process executing in a critical region takes a page fault, exhausts its quantum, or is swapped out, then other runnable processes needing to use that resource will be unable to make progress. An implementation of a concurrent object is wait-free if it guarantees that any process will complete any operation within a fixed number of steps, independent of the level of contention and the execution speeds of the other processes. To construct a wait-free implementation of the shared priority queue, we must break each enqueue or dequeue operation into a non-atomic sequence of atomic steps, where each atomic step is a primitive operation directly supported by the hardware, such as read, write, or fetch-and-add . To show that such an implementation is correct. It is necessary to show that (1) each operation's sequence of primitive steps has the desired effect (e.g., enqueuing or dequeuing an item) regardless of how it is interleaved with other concurrent operations, and (2) that each operation terminates within a fixed number of steps regardless of variations in speed (including arbitrary delay) of other processes. Support for wait-free synchronization requires genuinely new language constructs, not just variations on conventional approaches such as semaphores, monitors, tasks, or message-passing. I don't know what these constructs look like, but in this position paper, I would like to suggest some research directions that could lead, directly or indirectly, to progress in this area. We need to keep up with work in algorithms. To pick just one example, we now know that certain kinds of wait-free synchronization, e.g., implementing a FIFO queue from read/write registers, require randomized protocols in which processes flip coins to choose their next steps [3, 1]. The implications of such results for language design remain unclear, but suggestive. We also need to pay more attention to specification. Although transaction serializability has become widely accepted as the basic correctness condition for databases and certain distributed systems, identifying analogous properties for concurrent objects remains an active area of research [2].