Fault-tolerant Replication Research Articles

Optimizing Distributed Query Processing in Heterogeneous Multi-Cloud Environments: A Framework for Dynamic Data Sharding and Fault-Tolerant Replication

Optimizing Distributed Query Processing in Heterogeneous Multi-Cloud Environments: A Framework for Dynamic Data Sharding and Fault-Tolerant Replication

Reliable and adaptive computation offload strategy with load and cost coordination for edge computing

There are several important factors to consider in edge computing systems including latency, reliability, power consumption, and queue load. Task replication requires additional energy costs in mobile edge offloading scenarios based on master-slave replication for fault tolerance. Excessive task offloading may lead to a sharp increase in the total energy consumption of the system including replication costs. Conversely, new tasks cannot enter the waiting queue and are lost, resulting in reliability issues. This paper proposes an adaptive task offloading strategy for balancing the edge node queue load and offloading cost (Lyapunov and Differential Evolution based Offloading schedule strategy, LDEO). The LDEO strategy innovatively customizes the Lyapunov drift-plus-penalty function by incorporating replication redundancy offloading costs to establish a balance model between the queue load and offloading cost. The LDEO strategy computes the optimal offloading decisions with dynamic adjustment characteristics by integrating a low-complexity differential evolution method, aiming to find the optimal balance point that minimizes the offloading cost while maintaining reliability performance. The experimental results show that compared with the existing strategies, LDEO strategy effectively reduces the redundancy of fault tolerance cost and the waiting time under the condition of ensuring that the task will not be discarded over time. It stabilizes the queue length in a reasonable range, controls the waiting time and loss rate of tasks, reduces the extra energy consumption paid by replication redundancy, and effectively realizes the optimal balance under multiple conditions.

Towards Log-Less, Fine-Granular State Machine Replication

State machine replication is used to increase the availability of a service such as a data management system while ensuring consistent access to it. State-of-the-art implementations are based on a command log to gain linear write access to storage and avoid repeated transmissions of large replicas. However, the command log requires non-trivial state management such as allocation and pruning to prevent unbounded growth.By introducing in-place replicated state machines that do not use command logs, the log overhead can be avoided. Instead, replicas agree on a sequence of states, and former states are directly overwritten. This method enables the consistent, fault-tolerant replication of basic data management primitives such as counters, sets, or individual locks with little to no overhead. It matches the properties of fast, byte-addressable, non-volatile memory particularly well, where it is no longer necessary to rely on sequential access for good performance. Our approach is especially well suited for small states and fine-granular distributed data management as it occurs in key-value stores, for example.

Topology-aware virtual machine replication for fault tolerance in cloud computing systems

Topology-aware virtual machine replication for fault tolerance in cloud computing systems

On Byzantine fault tolerance in multi-master Kubernetes clusters

Docker container virtualization technology is being widely adopted in cloud computing environments because of its lightweight and efficiency. However, it requires adequate control and management via an orchestrator. As a result, cloud providers are adopting the open-access Kubernetes platform as the standard orchestrator of containerized applications. To ensure applications’ availability in Kubernetes, the latter uses Raft protocol’s replication mechanism. Despite its simplicity, Raft assumes that machines fail only when shutdown. This failure event is rarely the only reason for a machine’s malfunction. Indeed, software errors or malicious attacks can cause machines to exhibit Byzantine (i.e. random) behavior and thereby corrupt the accuracy and availability of the replication protocol. In this paper, we propose a Kubernetes multi-Master Robust (KmMR) platform to overcome this limitation. KmMR is based on the adaptation and integration of the BFT-SMaRt fault-tolerant replication protocol into Kubernetes environment. Unlike Raft protocol, BFT-SMaRt is resistant to both Byzantine and non-Byzantine faults. Experimental results show that KmMR is able to guarantee the continuity of services, even when the total number of tolerated faults is exceeded. In addition, KmMR provides on average a consensus time 1000 times shorter than that achieved by the conventional platform (with Raft), in such condition. Finally, we show that KmMR generates a small additional cost in terms of resource consumption compared to the conventional platform.

STC: Sub-packetization tunable codes for fast recovery

Abstract In distributed cyber-physical systems, the data should be stored in a reliable and available manner in the face of multiple kinds of failures. Considering that the storage space is an expensive part, erasure codes are storage-efficient alternatives to replication for fault tolerance. However, traditional erasure codes, e.g. Reed–Solomon (RS) codes, impose a huge burden on disk I/O and network resources for recovering unavailable data in case of node failures. In this paper, we introduce a new set of erasure codes called Sub-packetization Tunable Codes (STC). STC can reduce the amount of required information during recovering a failed block, while still allowing a significant reduction in storage overhead. The customized recovery method in STC needs each block to be sub-packetized, and the repair bandwidth decreases as the number of sub-packets increases. However, high sub-packetization may result in some extra costs. In STC, the sub-packetization level of our codes is tunable so that storage systems can weigh the balance between sub-packetization costs and repair bandwidth. In addition, our STC can be constructed as a family of double regenerating codes (DRC) to improve the repair performance in hierarchical distributed situations, whose idea is to trade the abundant inner-rack bandwidth for constrained cross-rack bandwidth.

Scalable Byzantine fault-tolerant state-machine replication on heterogeneous servers

When provided with more powerful or extra hardware, state-of-the-art Byzantine fault-tolerant (BFT) replication protocols are unable to effectively exploit the additional computing resources: on the one hand, in settings with heterogeneous servers existing protocols cannot fully utilize servers with higher performance capabilities. On the other hand, using more servers than the minimum number of replicas required for Byzantine fault tolerance in general does not lead to improved throughput and latency, but instead actually degrades performance. In this paper, we address these problems with Omada, a BFT system architecture that is able to benefit from additional hardware resources. To achieve this property while still providing strong consistency, Omada first parallelizes agreement into multiple groups and then executes the requests handled by different groups in a deterministic order. By varying the number of requests to be ordered between groups as well as the number of groups that a replica participates in between servers, Omada offers the possibility to individually adjust the resource usage per server. Moreover, the fact that not all replicas need to take part in every group enables the architecture to exploit additional servers.

SieveQ: A Layered BFT Protection System for Critical Services

Firewalls play a crucial role in assuring the security of today's critical infrastructures, forming a first line of defense by being placed strategically at the front-end of the networks. Sometimes, however, they have exploitable weaknesses, allowing an adversary to bypass them in different ways. Therefore, their design should include improved resilience capabilities to allow them to operate correctly in highly adverse environments. This paper proposes SieveQ, a message queue service that protects and regulates the access to critical systems, in a way similar to an application-level firewall. SieveQ achieves fault and intrusion tolerance by employing an architecture based on two filtering layers, enabling efficient removal of invalid messages at early stages and decreasing the costs associated with Byzantine Fault-Tolerant (BFT) replication of previous solutions. Our experimental evaluation shows that SieveQ improves existing replicated-firewalls resilience in the presence of corrupted messages by faulty nodes. Furthermore, it accommodates high loads, as it is able to handle sixteen times more security events per second than what was processed by the Security Information and Event Management (SIEM) infrastructure employed in the 2012 Summer Olympic Games.

MITRA

Replication is often considered a cost-effective solution for building dependable systems with off-the-shelf hardware. Replication software is usually designed to tolerate crash faults, but Byzantine (or arbitrary) faults such as software bugs are well-known to affect transactional database management systems (DBMSs) as many other classes of software. Despite the maturity of replication technology, Byzantine fault-tolerant replication of databases remains a challenging problem. The paper presents MITRA, a middleware for replicating DBMSs and making them tolerant to Byzantine faults. MITRA is designed to offer transparent replication of off-theshelf DBMSs with replicas from different vendors.

Efficient Randomized Byzantine Fault-Tolerant Replication Based on Special Valued Coin Tossing

Efficient Randomized Byzantine Fault-Tolerant Replication Based on Special Valued Coin Tossing

Adaptive request batching for byzantine replication

Castro and Liskov proposed in 1999 a successful solution for byzantine fault-tolerant replication, named PBFT, which overcame performance drawbacks of earlier byzantine faulttolerant replication protocols. Other proposals extended PBFT with further optimizations, improving PBFT performance in certain conditions. One of the key optimizations of PBFT-based protocols is the use a request batching mechanism. If the target distributed system is dynamic, that is, if its underlying characteristics change dynamically, such as workload, channel QoS, network topology, etc., the configuration of the request batching mechanism must follow the dynamics of the system or it may not yield the desired performance improvement. This paper addresses this challenge by proposing an innovative solution to the dynamic configuration of request batching parameters inspired on feedback control theory. In order to evaluate its efficiency, the proposed solution is simulated in various scenarios and compared with the original version used in the PBFT-family protocols.

Efficient Byzantine Fault-Tolerance

We present two asynchronous Byzantine fault-tolerant state machine replication (BFT) algorithms, which improve previous algorithms in terms of several metrics. First, they require only 2f+1 replicas, instead of the usual 3f+1. Second, the trusted service in which this reduction of replicas is based is quite simple, making a verified implementation straightforward (and even feasible using commercial trusted hardware). Third, in nice executions the two algorithms run in the minimum number of communication steps for nonspeculative and speculative algorithms, respectively, four and three steps. Besides the obvious benefits in terms of cost, resilience and management complexity-fewer replicas to tolerate a certain number of faults-our algorithms are simpler than previous ones, being closer to crash fault-tolerant replication algorithms. The performance evaluation shows that, even with the trusted component access overhead, they can have better throughput than Castro and Liskov's PBFT, and better latency in networks with nonnegligible communication delays.

A Comparative Study of System-Level Energy Management Methods for Fault-Tolerant Hard Real-Time Systems

Low energy consumption and fault tolerance are often key objectives in the design of real-time embedded systems. However, these objectives are at odds, and there is a trade-off between them. Real-time systems usually use system level energy reduction methods, i.e., dynamic voltage scaling (DVS) and dynamic power management (DPM). Also hard real-time systems often use replication to achieve fault tolerance. In this paper, we investigate the impact of system level energy reduction methods on both the reliability and energy consumption of hard real-time systems which use replication for fault tolerance. In this analysis, we have considered four various existing energy management methods: 1) Classic DPM, 2) Classic DVS, 3) Postponement method: a variation of DPM which is only applicable to replicated systems, and 4) Hybrid method: a combination of Postponement and DVS. Based on the comparative study, we have provided guidelines so that a designer can decide which energy management method is more suitable for a given application. For example, we have shown that when reliability is the main concern, the postponement method is the most preferable. However, when the energy consumption is the primary concern, the hybrid method may be more appropriate.

Prime: Byzantine Replication under Attack

Existing Byzantine-resilient replication protocols satisfy two standard correctness criteria, safety and liveness, even in the presence of Byzantine faults. The runtime performance of these protocols is most commonly assessed in the absence of processor faults and is usually good in that case. However, faulty processors can significantly degrade the performance of some protocols, limiting their practical utility in adversarial environments. This paper demonstrates the extent of performance degradation possible in some existing protocols that do satisfy liveness and that do perform well absent Byzantine faults. We propose a new performance-oriented correctness criterion that requires a consistent level of performance, even with Byzantine faults. We present a new Byzantine fault-tolerant replication protocol that meets the new correctness criterion and evaluate its performance in fault-free executions and when under attack.

Aspect-oriented development of cluster computing software

In complex software systems, modularity and readability tend to be degraded owing to inseparable interactions between concerns that are distinct features in a program. Such interactions result in tangled code that is hard to develop and maintain. Aspect-Oriented Programming (AOP) is a powerful method for modularizing source code and for decoupling cross-cutting concerns. A decade of growing research on AOP has brought the paradigm into many exciting areas. However, pioneering work on AOP has not flourished enough to enrich the design of distributed systems using the refined AOP paradigm. This article investigates three case studies that cover time-honored issues such as fault-tolerant computing, network heterogeneity, and object replication in the cluster computing community using the AOP paradigm. The aspects that we define here are simple, intuitive, and reusable. Our intensive experiences show that (i) AOP can improve the modularity of cluster computing software by separating the source code into base and instrumented parts, and (ii) AOP helps developers to deploy additional features to legacy cluster computing software without harming code modularity and system performance.

A Robust Byzantine Fault-Tolerant Replication Technique for Peer-to-Peer Content Distribution

Problem statement: In peer-to-peer networks, Byzantine fault tolerance refers to the capability of a system to tolerate Byzantine faults. It can be achieved by replicating the server and by ensuring all server replicas reach an agreement on the input despite Byzantine faulty replicas and clients. Since malicious attacks and software errors can cause faulty nodes to exhibit Byzantine behavior, Byzantine-fault-tolerant algorithms are increasingly important. Approach: In the study, we wish to develop a robust Byzantine Fault-Tolerance Replication (BFTR) technique for peer-to-peer content distribution systems which contains fault detection and fault recovery. It is based on collaborative monitoring of each node to detect the occurrence of a fault. Already we proposed a QoS based overlay network architecture (QIRM) involving an intelligent replica placement algorithm to improve the network utilization of the P2P system. Results: By simulation results, we show that the proposed technique involves less overhead and recovery time with increased accuracy. Conclusion/Recommendations: Here the result obtained is that BFTR Technique is much efficient than the QIRM with respect to packet drop ratio, average end-to-end delay, throughput and overhead.

A dependability layer for large-scale distributed systems

Ensuring dependability in large-scale distributed systems represents an important research subject today. Despite the fact that many projects obtained valuable results in this domain, no acceptable solution was yet found that could integrate all the requirements for designing a dependable system and that could exploit all the capabilities of modern systems. We present a unitary and aggregate approach to ensuring reliability, availability, safety and security of distributed systems. Starting from the proposed architecture, we present implementation details for two solutions designed to ensure fault tolerance, using virtualisation and container-based replication of services. We also present an approach to enhance security using combined modern security models in large-scale distributed systems. The results and implementation details can serve as a methodology to assist distributed infrastructures in adopting such a middleware layer designed to enforce dependability in large-scale distributed systems.

Steward: Scaling Byzantine Fault-Tolerant Replication to Wide Area Networks

This paper presents the first hierarchical byzantine fault-tolerant replication architecture suitable to systems that span multiple wide-area sites. The architecture confines the effects of any malicious replica to its local site, reduces message complexity of wide-area communication, and allows read-only queries to be performed locally within a site for the price of additional standard hardware. We present proofs that our algorithm provides safety and liveness properties. A prototype implementation is evaluated over several network topologies and is compared with a flat byzantine fault-tolerant approach. The experimental results show considerable improvement over flat byzantine replication algorithms, bringing the performance of byzantine replication closer to existing benign fault-tolerant replication techniques over wide area networks.

DepSpace

The tuple space coordination model is one of the most interesting coordination models for open distributed systems due to its space and time decoupling and its synchronization power. Several works have tried to improve the dependability of tuple spaces through the use of replication for fault tolerance and access control for security. However, many practical applications in the Internet require both fault tolerance and security. This paper describes the design and implementation of DepSpace, a Byzantine fault-tolerant coordination service that provides a tuple space abstraction. The service offered by DepSpace is secure, reliable and available as long as less than a third of service replicas are faulty. Moreover, the content-addressable confidentiality scheme developed for DepSpace bridges the gap between Byzantine fault-tolerant replication and confidentiality of replicated data and can be used in other systems that store critical data.

Efficient task replication and management for adaptive fault tolerance in Mobile Grid environments

Fault tolerant Grid computing is of vital importance as the Grid and Mobile computing worlds converge to the Mobile Grid computing paradigm. We present an efficient scheme based on task replication, which utilizes the Weibull reliability function for the Grid resources so as to estimate the number of replicas that are going to be scheduled in order to guarantee a specific fault tolerance level for the Grid environment. The additional workload that is produced by the replication is handled by a resource management scheme which is based on the knapsack formulation and which aims to maximize the utilization and profit of the Grid infrastructure. The proposed model has been evaluated through simulation and has shown its efficiency for being used in a middleware approach in future mobile Grid environments.