Asynchronous Message-passing Systems Research Articles

Dynamic Byzantine Broadcast in Asynchronous Message-Passing Systems

The <i>reconfiguration</i> problem is considered a key challenge in distributed systems, especially in <i>dynamic</i> asynchronous message-passing systems. To keep the data reliability and availability in long-lived systems, any protocols should support reconfigurations, to dynamically add resources, or remove old and slow machines with newer faster ones. Previous results in reconfigurations either rely on consensus, or study the problem restricted to crash failures only. However, it is difficult to argue that real-world systems experience crash failures only. In this paper, we study the dynamic reconfiguration problem in fully asynchronous message-passing systems with Byzantine faults. We first specify dynamic Byzantine broadcast, and then specify a clean and explicit liveness condition. We show that dynamic Byzantine broadcast is solvable by presenting a dynamic Byzantine consistent broadcast algorithm and a dynamic Byzantine reliable broadcast algorithm.

Set-constrained delivery broadcast: A communication abstraction for read/write implementable distributed objects

This paper introduces a new communication abstraction, called Set-Constrained Delivery Broadcast (SCD-broadcast), whose aim is to provide its users with an appropriate abstraction level when they have to implement objects or distributed tasks in an asynchronous message-passing system prone to process crash failures. This abstraction allows each process to broadcast messages and deliver a sequence of sets of messages in such a way that, if a process delivers a set of messages including a message m and later delivers a set of messages including a message m′, no process delivers first a set of messages including m′ and later a set of message including m.After having presented an algorithm implementing SCD-broadcast, the paper investigates its programming power and its computability limits. On the “power” side it presents SCD-broadcast-based algorithms, which are both simple and efficient, building objects (such as snapshot and conflict-free replicated data types), and distributed tasks. On the “computability limits” side it shows that SCD-broadcast and read/write registers are computationally equivalent.

On the Versatility of Bracha’s Byzantine Reliable Broadcast Algorithm

G. Bracha presented in 1987 a simple and efficient reliable broadcast algorithm for [Formula: see text]-process asynchronous message-passing systems, which tolerates up to [Formula: see text] Byzantine processes. Following an idea recently introduced by Hirt, Kastrato and Liu-Zhang (OPODIS 2020), instead of considering the upper bound on the number of Byzantine processes [Formula: see text], the present short article considers two types of Byzantine behavior: the ones that can prevent the safety property from being satisfied, and the ones that can prevent the liveness property from being satisfied (a Byzantine process can exhibit only one or both types of failures). This Byzantine differentiated failure model is captured by two associated upper bounds denoted [Formula: see text] (for safety) and [Formula: see text] for liveness). The article shows that only the threshold values used in the predicates of Bracha’s algorithm must be modified to obtain an algorithm that works with this differentiated Byzantine failure model.

Emulating a Shared Register in a System That Never Stops Changing

Emulating a shared register can mask the intricacies of designing algorithms for asynchronous message-passing systems subject to crash failures, since it allows them to run algorithms designed for the simpler shared-memory model. Typically such emulations replicate the value of the register in multiple servers and require readers and writers to communicate with a majority of servers. The success of this approach for static systems, where the set of nodes (readers, writers, and servers) is fixed, has motivated several similar emulations for dynamic systems, where nodes may enter and leave. However, existing emulations need to assume that the system eventually stops changing for a long enough period or that the system size is bounded. This paper presents the first emulation of a register supporting any number of readers and writers in a crash-prone system that can withstand nodes continually entering and leaving and imposes no upper bound on the system size. The algorithm works as long as the number of nodes entering and leaving during a fixed time interval is at most a constant fraction of the system size at the beginning of the interval, and as long as the number of crashed nodes in the system is at most a constant fraction of the current system size. The paper includes a lower bound on the fraction of correct nodes that is strictly larger than the fraction sufficient to solve the problem in the static case.

Implementing Snapshot Objects on Top of Crash-Prone Asynchronous Message-Passing Systems

In asynchronous crash-prone read/write shared-memory systems there is the notion of a snapshot object, which simulates the behavior of an array of single-writer/multi-reader (SWMR) shared registers that can be read atomically. Processes in the system can access the object invoking (any number of times) two operations, denoted $\mathsf {write}()$ and $\mathsf {snapshot}()$ . A process invokes $\mathsf {write}()$ to update the value of its register in the array. When it invokes $\mathsf {snapshot}()$ , the process obtains the values of all registers, as if it read them simultaneously. It is known that a snapshot object can be implemented on top of SWMR registers, tolerating any number of process failures. Snapshot objects provide a level of abstraction higher than individual SWMR registers, and they simplify the design of applications. Building a snapshot object on an asynchronous crash-prone message-passing system has similar benefits. The object can be implemented by using the known simulations of a SWMR shared memory on top of an asynchronous message-passing system (if less than half the processes can crash), and then build a snapshot object on top of the simulated SWMR memory. This paper presents an algorithm that implements a snapshot object directly on top of the message-passing system, without building an intermediate layer of a SWMR shared memory. To the authors knowledge, the proposed algorithm is the first providing such a direct construction. The algorithm is more efficient than the indirect solution, yet relatively simple.

Time-efficient read/write register in crash-prone asynchronous message-passing systems

The atomic register is one of the most basic and useful object of computing science, and its simple read-write semantics is appealing when programming distributed systems. Hence, its implementation on top of crash-prone asynchronous message-passing systems has received a lot of attention. It was shown that having a strict minority of processes that may crash is a necessary and sufficient requirement to build an atomic register on top of a crash-prone asynchronous message-passing system. This paper visits the notion of a fast implementation of an atomic register, and presents a new time-efficient asynchronous algorithm that reduces latency in many cases: a write operation always costs a round-trip delay, while a read operation costs a round-trip delay in favorable circumstances (intuitively, when it is not concurrent with a write). When designing this algorithm, the design spirit was to be as close as possible to the original algorithm proposed by Attiya, Bar-Noy, and Dolev.

Gracefully degrading consensus and k-set agreement in directed dynamic networks

We study distributed agreement in synchronous directed dynamic networks, where an omniscient message adversary controls the presence/absence of communication links. We prove that consensus is impossible under a message adversary that guarantees weak connectivity only, and introduce eventually vertex-stable source components (VSSCs) as a means for circumventing this impossibility: A VSSC(k,d) message adversary guarantees that, eventually, there is an interval of d consecutive rounds where every communication graph contains at most k strongly connected components consisting of the same processes (with possibly varying interconnect topology), which have no incoming links from outside processes. We present a consensus algorithm that works correctly under a VSSC(1,4E+2) message adversary, where E is the dynamic network depth. Our algorithm maintains local estimates of the communication graphs, and applies techniques for detecting network stability and univalent system configurations. Several related impossibility results and lower bounds, in particular, that neither a VSSC(1,E−1) message adversary nor a VSSC(2,∞) one allow to solve consensus, reveal that there is not much hope to deal with (much) stronger message adversaries here.However, we show that gracefully degrading consensus, which degrades to general k-set agreement in case of unfavorable network conditions, allows to cope with stronger message adversaries: We provide a k-universal k-set agreement algorithm, where the number of system-wide decision values k is not encoded in the algorithm, but rather determined by the actual power of the message adversary in a run: Our algorithm guarantees at most k decision values under a VSSC(n,d)+MAJINF(k) message adversary, which combines VSSC(n,d) (with some small value of d, ensuring termination) with some information flow guarantee MAJINF(k) between certain VSSCs (ensuring k-agreement). Since related impossibility results reveal that a VSSC(k,d) message adversary is too strong for solving k-set agreement and that some information flow between VSSCs is mandatory for this purpose as well, our results provide a significant step towards the exact solvability/impossibility border of general k-set agreement in directed dynamic networks.Finally, we relate (the eventually-forever-variants of) our message adversaries to failure detectors. It turns out that even though VSSC(1,∞) allows to solve consensus and to implement the Ω failure detector, it does not allow to implement Σ. This contrasts the fact that, in asynchronous message-passing systems with a majority of process crashes, (Σ,Ω) is a weakest failure detector for solving consensus. Similarly, although the message adversary VSSC(n,d)+MAJINF(k) allows to solve k-set agreement, it does not allow to implement the failure detector Σk, which is known to be necessary for k-set agreement in asynchronous message-passing systems with a majority of process crashes. Consequently, it is not possible to adapt failure-detector-based algorithms to work in conjunction with our message adversaries.

Randomized k-set agreement in crash-prone and Byzantine asynchronous systems

k-Set agreement is a central problem of fault-tolerant distributed computing. Considering a set of n processes, where up to t may commit failures, let us assume that each process proposes a value. The problem consists in defining an algorithm such that each non-faulty process decides a value, at most k different values are decided, and the decided values satisfy some context-depending validity condition. Algorithms solving k-set agreement in synchronous message-passing systems have been proposed for different failure models (mainly process crashes, and process Byzantine failures). Differently, k-set agreement cannot be solved in failure-prone asynchronous message-passing systems when t≥k. To circumvent this impossibility an asynchronous system must be enriched with additional computational power.Assuming t≥k, this paper presents two distributed algorithms that solve k-set agreement in asynchronous message-passing systems where up to t processes may commit crash failures (first algorithm) or more severe Byzantine failures (second algorithm). To circumvent k-set agreement impossibility, this article considers that the underlying system is enriched with the computability power provided by randomization. Interestingly, the algorithm that copes with Byzantine failures is signature-free, and ensures that no value proposed only by Byzantine processes can be decided by a non-faulty process. Both algorithms share basic design principles.

Trading off t-Resilience for Efficiency in Asynchronous Byzantine Reliable Broadcast

This paper presents a simple and efficient reliable broadcast algorithm for asynchronous message-passing systems made up of n processes, among which up to [Formula: see text] may behave arbitrarily (Byzantine processes). This algorithm requires two communication steps and n2 − 1 messages. When compared to Bracha’s algorithm, which is resilience optimal ([Formula: see text]) and requires three communication steps and [Formula: see text] messages, the proposed algorithm shows an interesting tradeoff between communication efficiency and t-resilience.

Atomic Read/Write Memory in Signature-Free Byzantine Asynchronous Message-Passing Systems

This article presents a signature-free distributed algorithm which builds an atomic read/write shared memory on top of a fully connected peer-to-peer n-process asynchronous message-passing system in which up to t

Signature-free asynchronous Byzantine systems: from multivalued to binary consensus with $$t&lt;n/3$$ t &lt; n / 3 , $$O(n^2)$$ O ( n 2 ) messages, and constant time

This paper presents a new algorithm that reduces multivalued consensus to binary consensus in an asynchronous message-passing system made up of n processes where up to t may commit Byzantine failures. This algorithm has the following noteworthy properties: it assumes t < n/3 (and is consequently optimal from a resilience point of view), uses $$O(n^2)$$O(n2) messages, has a constant time complexity, and uses neither signatures nor additional computational power (such as random numbers, failure detectors, additional scheduling assumption, or additional synchrony assumption). The design of this reduction algorithm relies on two new all-to-all communication abstractions. The first one allows the non-faulty processes to reduce the number of proposed values to c, where c is a small constant. The second communication abstraction allows each non-faulty process to compute a set of (proposed) values satisfying the following property: if the set of a non-faulty process is a singleton containing value v, the set of any non-faulty process contains v. Both communication abstractions have an $$O(n^2)$$O(n2) message complexity and a constant time complexity. The reduction of multivalued Byzantine consensus to binary Byzantine consensus is then a simple sequential use of these communication abstractions. To the best of our knowledge, this is the first asynchronous message-passing algorithm that reduces multivalued consensus to binary consensus with $$O(n^2)$$O(n2) messages and constant time complexity (measured with the longest causal chain of messages) in the presence of up to t < n/3 Byzantine processes, and without using cryptography techniques. Moreover, this reduction algorithm uses a single instance of the underlying binary consensus, and tolerates message re-ordering by Byzantine processes.

Read/write shared memory abstraction on top of asynchronous Byzantine message-passing systems

This paper is on the construction and use of a shared memory abstraction on top of an asynchronous message-passing system in which up to t processes may commit Byzantine failures. This abstraction consists of arrays of n single-writer/multi-reader atomic registers, where n is the number of processes. These registers enable Byzantine tolerance by recording the whole history of values written to each one of them. A distributed algorithm building such a shared memory abstraction is first presented. This algorithm assumes t<n/3, which is shown to be a necessary and sufficient condition for such a construction. Hence, the algorithm is resilient-optimal.Then the paper presents distributed objects built on top of this read/write shared memory abstraction, which cope with Byzantine processes. As illustrated by these objects, the proposed shared memory abstraction is motivated by the fact that, for a lot of problems, algorithms are simpler to design and prove correct in a shared memory system than in a message-passing system.

Signature-Free Asynchronous Binary Byzantine Consensus with t &lt; n/3, O(n2) Messages, and O(1) Expected Time

This article is on broadcast and agreement in asynchronous message-passing systems made up of n processes, and where up to t processes may have a Byzantine Behavior. Its first contribution is a powerful, yet simple, all-to-all broadcast communication abstraction suited to binary values. This abstraction, which copes with up to t &lt; n /3 Byzantine processes, allows each process to broadcast a binary value, and obtain a set of values such that (1) no value broadcast only by Byzantine processes can belong to the set of a correct process, and (2) if the set obtained by a correct process contains a single value v , then the set obtained by any correct process contains v . The second contribution of this article is a new round-based asynchronous consensus algorithm that copes with up to t &lt; n /3 Byzantine processes. This algorithm is based on the previous binary broadcast abstraction and a weak common coin. In addition to being signature-free and optimal with respect to the value of t , this consensus algorithm has several noteworthy properties: the expected number of rounds to decide is constant; each round is composed of a constant number of communication steps and involves O ( n 2) messages; each message is composed of a round number plus a constant number of bits. Moreover, the algorithm tolerates message reordering by the adversary (i.e., the Byzantine processes).

Intrusion-Tolerant Broadcast and Agreement Abstractions in the Presence of Byzantine Processes

A process commits a Byzantine failure when its behavior does not comply with the algorithm it is assumed to execute. Considering asynchronous message-passing systems, this paper presents distributed abstractions, and associated algorithms, that allow non-faulty processes to correctly cooperate, despite the uncertainty created by the net effect of asynchrony and Byzantine failures. These abstractions are broadcast abstractions (namely, no-duplicity broadcast, reliable broadcast, and validated broadcast), and agreement abstraction (namely, consensus). While no-duplicity broadcast and reliable broadcast are well-known one-to-all communication abstractions, validated broadcast is a new all-to-all communication abstraction designed to address agreement problems. After having introduced these abstractions, the paper presents an algorithm implementing validated broadcast on top of reliable broadcast. Then the paper presents two consensus algorithms, which are reductions of multivalued consensus to binary consensus. The first one is a generic algorithm, that can be instantiated with unreliable broadcast or no-duplicity broadcast, while the second is a consensus algorithm based on validated broadcast. Finally, a third algorithm is presented that solves the binary consensus problem. This algorithm is a randomized algorithm based on validated broadcast and a common coin. The presentation of all the abstractions and their algorithms is done incrementally.

Special issue on “Combining Compositionality and Concurrency”: part 2

This is the second part of the special issuemotivated by theworkshop “25Years of Combining Compositionality and Concurrency” that we organised in August 2013 in Konigswinter near Bonn, Germany. The first part was published in Acta Informatica (2015) 52(1):3–106. For an introduction to the topic see our editorial there. This second part comprises four papers. The paper Richer Interface Automata with Optimistic and Pessimistic Compatibility by Gerald Luttgen, Walter Vogler, and Sascha Fendrich addresses component-based reasoning for concurrent systems via modal interface automata (MIA). The authors study optimistic and pessimistic MIA variants with internal must-transitions. They define conjunction and disjunction on interfaces as well as mechanisms for extending alphabets. The paper Compositional Verification of Asynchronous Concurrent Systems using CADP by Hubert Garavel, Frederic Lang, and Radu Mateescu investigates how various compositional techniques help to fight the state-space explosion in the verification of asynchronous message-passing systems. The techniques are implemented in the toolbox CADP (Construction and Analysis of Distributed Processes). The authors compare the different approaches using a common case study. The paper Denotational Fixed-Point Semantics for Constructive Scheduling of Synchronous Concurrency by Joaquin Aguado, Michael Mendler, Reinhard von Hanxleden, and Insa Fuhrmann defines an abstract semantic domain and an associated denotational semantics for synchronous programs. The semantics induces three notions of constructiveness, among

Compositional verification of asynchronous concurrent systems using CADP

During the last decades, concurrency theory successfully developed salient concepts to formally model and soundly reason about distributed and parallel systems. In practice, however, most attempts at analyzing large systems face severe complexity issues, especially state explosion, which prevents to exhaustively enumerate reachable state spaces. Compositionality is the most promising approach to fight state explosion. In this article, we focus on finite-state verification techniques for asynchronous message-passing systems, highlighting the existence of multiple, diverse compositional techniques such as: compositional model generation, semi-composition and projection, automatic generation of projection interfaces, formula-dependent model generation, and partial model checking. These approaches have been implemented in the framework of the CADP (Construction and Analysis of Distributed Processes) software toolbox and applied to large-scale, industrial systems. A key point is the ability to combine several compositional techniques, as no single technique is sufficient to address all kinds of systems.

Practically stabilizing SWMR atomic memory in message-passing systems

A fault-tolerant and practically stabilizing simulation of an atomic register is presented. The simulation works in asynchronous message-passing systems, and allows a minority of processes to crash. The simulation stabilizes in a practically stabilizing manner, by reaching a long execution in which it runs correctly. A key element in the simulation is a new combinatorial construction of a bounded labeling scheme accommodating arbitrary labels, including those not generated by the scheme itself.

Natural proofs for asynchronous programs using almost-synchronous reductions

We consider the problem of provably verifying that an asynchronous message-passing system satisfies its local assertions. We present a novel reduction scheme for asynchronous event-driven programs that finds almost-synchronous invariants - invariants consisting of global states where message buffers are close to empty. The reduction finds almost-synchronous invariants and simultaneously argues that they cover all local states. We show that asynchronous programs often have almost-synchronous invariants and that we can exploit this to build natural proofs that they are correct. We implement our reduction strategy, which is sound and complete, and show that it is more effective in proving programs correct as well as more efficient in finding bugs in several programs, compared to current search strategies which almost always diverge. The high point of our experiments is that our technique can prove the Windows Phone USB Driver written in P [9]correct for the responsiveness property, which was hitherto not provable using state-of-the-art model-checkers.

Beyond Lamport's Happened-before

The coordination of a sequence of actions, to be performed in a linear temporal order in a distributed system, is studied. While in asynchronous message-passing systems such ordering of events requires the construction of message chains based on Lamport's happened-before relation, this is no longer true in the presence of time bounds on message delivery. Given such bounds, the mere passage of time can provide information about the occurrence of events at remote sites, without the need for explicit confirmation. A new causal structure called the centipede is introduced, and it is shown that centipedes must exist in every execution where linear ordering of actions is ensured. Centipedes capture the subtle interplay between the explicit information obtained via message chains, and the indirectly derived information gained by the passage of time, given the time bounds. Centipedes are defined using two relations. One is called syncausality, a slight generalisation of the happened-before relation. The other is a novel bound guarantee relation among events, that is based on the bounds on message transmission. In a precise sense, centipedes play a role in the synchronous setting analogous to that played by message chains in asynchronous systems. Our study is based on a knowledge-based analysis of distributed coordination. Temporally linear coordination is reduced to nested knowledge (knowledge about knowledge). Obtaining nested knowledge of a spontaneous event is, in turn, shown to require the existence of an appropriate centipede.

Efficient distributed snapshots in an anonymous asynchronous message-passing system

We present a global snapshot algorithm with concurrent initiators, with termination detection in an anonymous asynchronous distributed message-passing system having FIFO channels. In anonymous systems, process identifiers are not available and an algorithm cannot use process identifiers in its operation. Such systems arise in several domains due to a variety of reasons. In the proposed snapshot algorithm for anonymous systems, each instance of algorithm initiation is identified by a random number (nonce); however, this is not used as an address in any form of communication. In the algorithm, each process can determine an instant when the local snapshot recordings at all the processes have terminated. This is a challenging problem when an algorithm cannot use process identifiers and a process does not know the number of processes in the system or the diameter of the network and cannot use a predefined topology overlay on the network, because there is no easy way to identify the global termination condition. The message complexity of our algorithm is (cn2), where c is the number of concurrent initiators and n is the number of processes in the system, which is much better than that of the algorithm by Chalopin et al. (2012) [6]. Further, the algorithm by Chalopin et al. also requires knowledge of the network diameter.