Synchronous Distributed System Research Articles

Detecting causality in the presence of Byzantine processes: The case of synchronous systems

Detecting causality or the “happens before” relation between events in a distributed system is a fundamental building block for distributed applications. It was recently proved that this problem cannot be solved in an asynchronous distributed system in the presence of Byzantine processes, irrespective of whether the communication mechanism is via unicasts, multicasts, or broadcasts. In light of this impossibility result, we turn attention to synchronous systems and examine the possibility of solving the causality detection problem in such systems. In this paper, we prove that causality detection between events can be solved in the presence of Byzantine processes in a synchronous distributed system. We prove the result by providing two algorithms. The first algorithm uses the Replicated State Machine (RSM) approach and vector clocks. The second algorithm is round-based and uses matrix clocks. The RSM-based algorithm can also run deterministically in partially synchronous systems.

Lea-TN: leader election algorithm considering node and link failures in a torus network

Torus network topology offers many advantages such as higher speed, lower latency, better fairness, and lower energy consumption. For these kinds of benefits, nowadays, it is used to construct many parallel and distributed systems like IBM Blue Gene, IBM Sequoia, Mira, and Sugon TC8600. In parallel and distributed computing, multiple nodes act together to carry out large tasks fast. Hence, coordination is indispensable among these nodes to complete such tasks. A possible coordination method could be to elect a leader among the nodes. Along with coordination, the elected leader node also manages other activities such as task allocation, result aggregation, clock synchronization, proficient resource sharing management, and communication among the system’s nodes. In this paper, we propose a lower bound $$\varOmega (N\log _3 N)$$ of message complexity on a comparison-based leader election for a 2D torus network (where N is the number of nodes in the network). Next, we sketch a new leader election algorithm (Lea-TN) considering both the node and link failures for a 2D torus network. This Lea-TN is a deterministic and robust algorithm that elects a leader for a synchronous distributed system. The algorithm chooses a leader, even when there are some link or node failures in the system. We consider the number of non-faulty links and the subsisting nodes’ failure rate to elect a reliable leader. We introduce new patterns for sending messages that help reduce the number of exchanged messages and the execution time of the election process. The proposed algorithm (Lea-TN) enables a node to identify its link failures during the election also. Further, we simulate the Lea-TN algorithm and compare its performance with that of the well-known existing algorithms.

Near-optimal self-stabilising counting and firing squads

Consider a fully-connected synchronous distributed system consisting of n nodes, where up to f nodes may be faulty and every node starts in an arbitrary initial state. In the synchronous C-counting problem, all nodes need to eventually agree on a counter that is increased by one modulo C in each round for given C>1. In the self-stabilising firing squad problem, the task is to eventually guarantee that all non-faulty nodes have simultaneous responses to external inputs: if a subset of the correct nodes receive an external “go” signal as input, then all correct nodes should agree on a round (in the not-too-distant future) in which to jointly output a “fire” signal. Moreover, no node should generate a “fire” signal without some correct node having previously received a “go” signal as input. We present a framework reducing both tasks to binary consensus at very small cost. For example, we obtain a deterministic algorithm for self-stabilising Byzantine firing squads with optimal resilience f<n/3, asymptotically optimal stabilisation and response time O(f), and message size O(log f). As our framework does not restrict the type of consensus routines used, we also obtain efficient randomised solutions.

Enhanced Bully Algorithm for Leader Node Election in Synchronous Distributed Systems

In distributed computing systems, if an elected leader node fails, the other nodes of the system need to elect another leader. The bully algorithm is a classical approach for electing a leader in a synchronous distributed computing system. This paper presents an enhancement of the bully algorithm, requiring less time complexity and minimum message passing. This significant gain has been achieved by introducing node sets and tie breaker time. The latter provides a possible solution to simultaneous elections initiated by different nodes. In comparison with the classical algorithm and its existing modifications, this proposal generates minimum messages, stops redundant elections, and maintains fault-tolerant behaviour of the system.

Implementing a Regular Register in an Eventually Synchronous Distributed System Prone to Continuous Churn

Due to their capability to hide the complexity generated by the messages exchanged between processes, shared objects are one of the main abstractions provided to developers of distributed applications. Implementations of such objects, in modern distributed systems, have to take into account the fact that almost all services, implemented on top of distributed infrastructures, are no longer fully managed due to either their size or their maintenance cost. Therefore, these infrastructures exhibit several autonomic behaviors in order to, for example, tolerate failures and continuous arrival and departure of nodes (churn phenomenon). Among all the shared objects, the register object is a fundamental one. Several protocols have been proposed to build fault resilient registers on top of message-passing system, but, unfortunately, failures are not the only challenge in modern distributed systems and new issues arise in the presence of churn. This paper addresses the construction of a multiwriter/multireader regular register in an eventually synchronous distributed system affected by the continuous arrival/departure of participants. In particular, a general protocol implementing a regular register is proposed and feasibility conditions associated with the arrival and departure of the processes are given. The protocol is proved correct under the assumption that a constraint on the churn is satisfied.

Message and time efficient consensus protocols for synchronous distributed systems

For a synchronous distributed system of n processes with up to t potential and f actual crash failures, where ( t < n - 1 , f ≤ t ) , the time lower bound for a protocol to achieve consensus is min ( t + 1 , f + 2 ) rounds. Currently, most researches in this field focus on the time efficiency of consensus protocols. This paper proposes consensus protocols for synchronous distributed systems that achieve both message and time efficiency. Based on an early stopping consensus protocol for synchronous distributed system with crash failures, we propose a rotating coordinator scheme that significantly reduces message complexity. However, this protocol is not time efficient because it requires min ( t + 1 , f + 3 ) rounds to reach consensus. Thus, to achieve both time and message efficiency, we propose another protocol in which ( t + 1 ) coordinators are used to send messages in each round. Furthermore, we show that the proposed consensus protocol with crash failures can be revised to be more message-efficient with orderly crash failures. When a process is able to send more than one message to another in a round, we propose an optimal message efficient early stopping consensus protocol for synchronous distributed systems with orderly crash failures.

Fully distributed three-tier active software replication

Keeping strongly consistent the state of the replicas of a software service deployed across a distributed system prone to crashes and with highly unstable message transfer delays (e.g., the Internet), is a real practical challenge. The solution to this problem is subject to the FLP impossibility result, and thus there is a need for long enough periods of synchrony with time bounds on process speeds and message transfer delays to ensure deterministic termination of any run of agreement protocols executed by replicas. This behavior can be abstracted by a partially synchronous computational model. In this setting, before reaching a period of synchrony, the underlying network can arbitrarily delay messages and these delays can be perceived as false failures by some timeout-based failure detection mechanism leading to unexpected service unavailability. This paper proposes a fully distributed solution for active software replication based on a three-tier software architecture well-suited to such a difficult setting. The formal correctness of the solution is proved by assuming the middle-tier runs in a partially synchronous distributed system. This architecture separates the ordering of the requests coming from clients, executed by the middle-tier, from their actual execution, done by replicas, i.e., the end-tier. In this way, clients can show up in any part of the distributed system and replica placement is simplified, since only the middle-tier has to be deployed on a well-behaving part of the distributed system that frequently respects synchrony bounds. This deployment permits a rapid timeout tuning reducing thus unexpected service unavailability

Performing work in broadcast networks

We consider the problem of how to schedule t similar and independent tasks to be performed in a synchronous distributed system of p stations communicating via multiple-access channels. Stations are prone to crashes whose patterns of occurrence are specified by adversarial models. Work, defined as the number of the available processor steps, is the complexity measure. We consider only reliable algorithms that perform all the tasks as long as at least one station remains operational. It is shown that every reliable algorithm has to perform work Omega(t + p root t) even when no failures occur. An optimal deterministic algorithm for the channel with collision detection is developed, which performs work O(t + p root t). Another algorithm, for the channel without collision detection, performs work O(t + p root t + p min {f, t}), where f < p is the number of failures. This algorithm is proved to be optimal, provided that the adversary is restricted in failing no more than f stations. Finally, we consider the question if randomization helps against weaker adversaries for the channel without collision detection. A randomized algorithm is developed which performs the expected minimum amount O(t + p root t) of work, provided that the adversary may fail a constant fraction of stations and it has to select failure-prone stations prior to the start of an execution of the algorithm.

A bivalency proof of the lower bound for uniform consensus

Bivalency argument is a widely-used technique that employs forward induction to show impossibility results and lower bounds related to consensus. However, for a synchronous distributed system of n processes with up to t potential and f actual crash failures, applying bivalency argument to prove the lower bound for reaching uniform consensus is still an open problem. In this paper, we address this problem by presenting a bivalency proof that the lower bound for reaching uniform consensus is ( f + 2 ) -rounds where 0 ⩽ f ⩽ t − 2 .

A Note on Set Agreement with Omission Failures

This paper considers the k-set agreement problem in a synchronous distributed system model with send-omission failures in which at most f processes can fail by send-omission. We show that, in a system of n+1 processes (n+1) f), no algorithm can solve k-set agreement in ⌊ fk ⌋ rounds. Our lower bound proof uses topological techniques to characterize subsets of executions of our model. The characterization has a surprisingly regular structure which leads to a simple and succinct proof. We also show that the lower bound is tight by exhibiting a new algorithm that solves k-set agreement in ⌊ fk ⌋ + 1 rounds.

Three‐tier replication for FT‐CORBA infrastructures

AbstractEnforcing strong replica consistency among a set of replicas of a service deployed across an asynchronous distributed system in the presence of crash failures is a real practical challenge. If each replica runs the consistency protocol bundled with the actual service implementation, this target cannot be achieved, as replicas need to be located over a partially synchronous distributed system to solve the distributed agreement problems underlying strong replica consistency.A three‐tier architecture for software replication enables the separation of the replication logic, i.e. protocols and mechanisms necessary for managing software replication, from both clients and server replicas. The replication logic is embedded in a middle‐tier that confines the need of partial synchrony and thus frees replica deployment.In this paper we first introduce the basic concepts underlying three‐tier replication. Then we present the interoperable replication logic (IRL) architecture, a fault‐tolerant CORBA compliant infrastructure. IRL exploits a three‐tier approach to replicate stateful deterministic CORBA objects and allows object replicas to run on object request brokers from different vendors. A description of an IRL prototype developed in our department is proposed along with an extensive performance analysis. Copyright © 2003 John Wiley &amp; Sons, Ltd.

Mastering agreement problems in distributed systems

Nonblocking atomic commitment is a well-known agreement problem of practical interest to distributed-systems engineers and designers. The authors present this problem as a case study in the context of synchronous and asynchronous distributed systems to survey recent notions about failure detection. They also show how systems engineers can use these findings to gain deeper insight into the challenges they encounter.

The Distributed Firing Squad Problem

The distributed firing squad problem is defined in the context of a synchronous distributed system where the correct processors operate in lock-step synchrony but do not share a global clock. If one or more correct processors receive a command to start a firing squad synchronization, then at some future time all correct processors must “fire” (formally, enter a special state) at exactly the same step. For various fault models, upper and lower bounds are proved on the number of faulty processors that can be tolerated and on the number of rounds of communication required between the reception of the start command and firing. For example, if a firing squad protocol is resilient to t fail-stop faults, then at least $t + 1$ rounds are necessary and sufficient. For the case of Byzantine faults with authentication where the faulty processors can take steps in between the synchronous steps of the correct processors, the firing squad problem can be solved in $t + 5$ rounds, provided that $n > 3t$, where n is the number of processors and t is the number of faults, and the problem cannot be solved at all if $n \leqq 3t$. Moreover, in the case that $n \leqq 3t$, the impossibility of a firing squad protocol holds even for a weaker “timing fault model” where all processors generate messages correctly according to the protocol, but the faulty processors can affect the system by slightly slowing down or speeding up messages.