Byzantine Fault-tolerant Protocols Research Articles

Reliable multicast using remote direct memory access (RDMA) over a passive optical cross-connect fabric enhanced with wavelength division multiplexing (WDM)

It has been well studied that reliable multicast enables consistency protocols, including Byzantine Fault Tolerant protocols, for distributed systems. However, no transport-layer reliable multicast is used today due to limitations with existing switch fabrics and transport-layer protocols. In this paper, we introduce a layer-4 (L4) transport based on remote direct memory access (RDMA) datagram to achieve reliable multicast over a shared optical medium. By connecting a cluster of networking nodes using a passive optical cross-connect fabric enhanced with wavelength division multiplexing, all messages are broadcast to all nodes. This mechanism enables consistency in a distributed system to be maintained at a low latency cost. By further utilizing RDMA datagram as the L4 protocol, we have achieved a low-enough message loss-ratio (better than one in 68 billion) to make a simple Negative Acknowledge (NACK)-based L4 multicast practical to deploy. To our knowledge, it is the first multicast architecture able to demonstrate such low message loss-ratio. Furthermore, with this reliable multicast transport, end-to-end latencies of eight microseconds or less (< 8us) have been routinely achieved using an enhanced software RDMA implementation on a variety of commodity 10G Ethernet network adapters.

Generalized Paxos Made Byzantine (and Less Complex)

One of the most recent members of the Paxos family of protocols is Generalized Paxos. This variant of Paxos has the characteristic that it departs from the original specification of consensus, allowing for a weaker safety condition where different processes can have a different views on a sequence being agreed upon. However, much like the original Paxos counterpart, Generalized Paxos does not have a simple implementation. Furthermore, with the recent practical adoption of Byzantine fault tolerant protocols in the context of blockchain protocols, it is timely and important to understand how Generalized Paxos can be implemented in the Byzantine model. In this paper, we make two main contributions. First, we attempt to provide a simpler description of Generalized Paxos, based on a simpler specification and the pseudocode for a solution that can be readily implemented. Second, we extend the protocol to the Byzantine fault model, and provide the respective correctness proof.

Scalable Byzantine Consensus via Hardware-Assisted Secret Sharing

The surging interest in blockchain technology has revitalized the search for effective Byzantine consensus schemes. In particular, the blockchain community has been looking for ways to effectively integrate traditional Byzantine fault-tolerant (BFT) protocols into a blockchain consensus layer allowing various financial institutions to securely agree on the order of transactions. However, existing BFT protocols can only scale to tens of nodes due to their $O(n^2)$ message complexity. In this paper, we propose FastBFT, a fast and scalable BFT protocol. At the heart of FastBFT is a novel message aggregation technique that combines hardware-based trusted execution environments (TEEs) with lightweight secret sharing primitives. Combining this technique with several other optimizations (i.e., optimistic execution, tree topology and failure detection), FastBFT achieves low latency and high throughput even for large scale networks. Via systematic analysis and experiments, we demonstrate that FastBFT has better scalability and performance than previous BFT protocols.

Scalable Byzantine fault-tolerant state-machine replication on heterogeneous servers

When provided with more powerful or extra hardware, state-of-the-art Byzantine fault-tolerant (BFT) replication protocols are unable to effectively exploit the additional computing resources: on the one hand, in settings with heterogeneous servers existing protocols cannot fully utilize servers with higher performance capabilities. On the other hand, using more servers than the minimum number of replicas required for Byzantine fault tolerance in general does not lead to improved throughput and latency, but instead actually degrades performance. In this paper, we address these problems with Omada, a BFT system architecture that is able to benefit from additional hardware resources. To achieve this property while still providing strong consistency, Omada first parallelizes agreement into multiple groups and then executes the requests handled by different groups in a deterministic order. By varying the number of requests to be ordered between groups as well as the number of groups that a replica participates in between servers, Omada offers the possibility to individually adjust the resource usage per server. Moreover, the fact that not all replicas need to take part in every group enables the architecture to exploit additional servers.

Byzantine fault-tolerant and semantic-driven consensus protocol

To provide fault tolerance, modern distributed storage systems use specialized network topologies and consensus protocols that create high overheads. The main disadvantage of existing specialized topologies is a difficulty to implement an efficient data placement that takes into account locality of the data. In scientific problems very often it is necessary to provide the semantic proximity of the data at the nodes of the distributed system. Core drawback of modern consensus protocols is that most of them use messaging through broadcasting, which is impossible for large amounts of data and storage nodes. With more stringent requirements for a fault tolerance, e.g. requirement for resistance to Byzantine errors [1], ensuring of data localization becomes nearly impossible. The purpose of this paper is to implement a distributed consensus protocol that preserves data localization and is resistant to Byzantine errors. Theoretical tasks: a) consider how to implement distributed services based on finite state machines; b) consider how to ensure Byzantine fault tolerance; c) consider methods for organizing the semantic proximity of data. Practical tasks: a) develop ways to jointly ensure the Byzantine fault tolerance and locality of data; b) study the shortcomings of the joint provision of Byzantine fault tolerance and locality of data; c) develop a protocol of Byzantine consensus using semantic links between data; d) analyze the performance of the developed protocol within the framework of applied tasks. The object of the study is a distributed data storage system with the requirement for Byzantine fault tolerance. The subject of this study is the provision of Byzantine fault tolerance in conjunction with the use of semantic links between the data. The subject of this study has never been the subject of a special scientific study. Separate scientific foundations of this problematic set in the present study were developed in separate works on Byzantine fault tolerance, local hash methods and dimensional reduction.

On Choosing Server- or Client-Side Solutions for BFT

Byzantine Fault Tolerant (BFT) protocols have the ability to work correctly even when up to a threshold f of system servers are compromised. This makes them appealing for the construction of critical systems connected to the Internet, which are constantly a target for cyber attacks. BFT protocols differ based on the kind of application, deployment settings, performance, access control mechanisms, number of servers in the system, and protocol implementation. The large number of protocols present in the literature and their differences make it difficult for a system builder to choose the solution that best satisfies the requirements of the system that he wants to build. In particular, the main difference among BFT protocols lies in their system models: server-side versus client-side. In the server-side model each client relies on the system to consistently order and replicate updates, while in the client-side model each client actively participates in the protocol. In this article, we classify BFT protocols as server-side or client-side. We analyze the trade-offs between the two models, describe systems that use these models and the trade-offs they choose, highlight the research gaps, and provide guidelines to system builders in order to choose the solution that best satisfies their needs.

A Method of Parallelizing Consensuses for Accelerating Byzantine Fault Tolerance

We propose a new method that accelerates asynchronous Byzantine Fault Tolerant (BFT) protocols designed on the principle of state machine replication. State machine replication protocols ensure consistency among replicas by applying operations in the same order to all of them. A naive way to determine the application order of the operations is to repeatedly execute the BFT consensus to determine the next executed operation, but this may introduce inefficiency caused by waiting for the completion of the previous execution of the consensus protocol. To reduce this inefficiency, our method allows parallel execution of the consensuses while keeping consistency of the consensus results at the replicas. In this paper, we also prove the correctness of our method and experimentally compare it with the existing method in terms of latency and throughput. The evaluation results show that our method makes a BFT protocol three or four times faster than the existing one when some machines or message transmissions are delayed.

Hash-based Byzantine fault tolerant agreement with enhanced view consistency

This paper presents an authentication-based Byzantine fault tolerant agreement protocol that is capable of detecting the tentative failures in transaction processing systems. The protocol employs a cryptographic hash function to detect crash as well as a few more types of commonly faced arbitrary behaviours manifested by the participating server replicas in the system. We show that the use of hash functions can successfully handle various types of arbitrary faults. In addition, it suggests a preemptive mechanism to proactively detect the malicious replicas. The protocol avoids frequent view changes that contributes to the liveness of the system and ensures continuity in transaction processing.

Byzantine Fault-Tolerant Consensus in Wireless Ad Hoc Networks

Wireless ad hoc networks, due to their inherent unreliability, pose significant challenges to the task of achieving tight coordination among nodes. The failure of some nodes and momentary breakdown of communications, either of accidental or malicious nature, should not result in the failure of the entire system. This paper presents an asynchronous Byzantine consensus protocol-called Turquois-specifically designed for resource-constrained wireless ad hoc networks. The key to its efficiency is the fact that it tolerates dynamic message omissions, which allows an efficient utilization of the wireless broadcasting medium. The protocol also refrains from computationally expensive public-key cryptographic during its normal operation. The protocol is safe despite the arbitrary failure of f <; n/3 nodes from a total of n nodes, and unrestricted message omissions. Progress is ensured in rounds where the number of omissions is σ ≤ [n-t/2] (n - k - t) + k - 2, where k is the number of nodes required to terminate and t ≤ f is the number of nodes that are actually faulty. These characteristics make Turquois the first consensus protocol that simultaneously circumvents the FLP and the Santoro-Widmayer impossibility results, which is achieved through randomization. Finally, the protocol was prototyped and subject to a comparative performance evaluation against two well-known Byzantine fault-tolerant consensus protocols. The results show that, due to its design, Turquois outperforms the other protocols by more than an order of magnitude as the number of nodes in the system increases.

Adaptive request batching for byzantine replication

Castro and Liskov proposed in 1999 a successful solution for byzantine fault-tolerant replication, named PBFT, which overcame performance drawbacks of earlier byzantine faulttolerant replication protocols. Other proposals extended PBFT with further optimizations, improving PBFT performance in certain conditions. One of the key optimizations of PBFT-based protocols is the use a request batching mechanism. If the target distributed system is dynamic, that is, if its underlying characteristics change dynamically, such as workload, channel QoS, network topology, etc., the configuration of the request batching mechanism must follow the dynamics of the system or it may not yield the desired performance improvement. This paper addresses this challenge by proposing an innovative solution to the dynamic configuration of request batching parameters inspired on feedback control theory. In order to evaluate its efficiency, the proposed solution is simulated in various scenarios and compared with the original version used in the PBFT-family protocols.

Trustworthy Coordination of Web Services Atomic Transactions

The Web Services Atomic Transactions (WS-AT) specification makes it possible for businesses to engage in standard distributed transaction processing over the Internet using Web Services technology. For such business applications, trustworthy coordination of WS-AT is crucial. In this paper, we explain how to render WS-AT coordination trustworthy by applying Byzantine Fault Tolerance (BFT) techniques. More specifically, we show how to protect the core services described in the WS-AT specification, namely, the Activation service, the Registration service, the Completion service and the Coordinator service, against Byzantine faults. The main contribution of this work is that it exploits the semantics of the WS-AT services to minimize the use of Byzantine Agreement (BA), instead of applying BFT techniques naively, which would be prohibitively expensive. We have incorporated our BFT protocols and mechanisms into an open-source framework that implements the WS-AT specification. The resulting BFT framework for WS-AT is useful for business applications that are based on WS-AT and that require a high degree of dependability, security, and trust.

Prime: Byzantine Replication under Attack

Existing Byzantine-resilient replication protocols satisfy two standard correctness criteria, safety and liveness, even in the presence of Byzantine faults. The runtime performance of these protocols is most commonly assessed in the absence of processor faults and is usually good in that case. However, faulty processors can significantly degrade the performance of some protocols, limiting their practical utility in adversarial environments. This paper demonstrates the extent of performance degradation possible in some existing protocols that do satisfy liveness and that do perform well absent Byzantine faults. We propose a new performance-oriented correctness criterion that requires a consistent level of performance, even with Byzantine faults. We present a new Byzantine fault-tolerant replication protocol that meets the new correctness criterion and evaluate its performance in fault-free executions and when under attack.

Antiquity

Antiquity is a wide-area distributed storage system designed to provide a simple storage service for applications like file systems and back-up. The design assumes that all servers eventually fail and attempts to maintain data despite those failures. Antiquity uses a secure log to maintain data integrity, replicates each log on multiple servers for durability, and uses dynamic Byzantine fault-tolerant quorum protocols to ensure consistency among replicas. We present Antiquity's design and an experimental evaluation with global and local testbeds. Antiquity has been running for over two months on 400+ PlanetLab servers storing nearly 20,000 logs totaling more than 84 GB of data. Despite constant server churn, all logs remain durable.