Automatic Failover Research Articles

Real-Time rejuvenation scheduling for cloud systems with virtualized software spares

With the increasing popularity of cloud services, there is a growing demand for high reliability and availability of cloud computing. As viable solutions, virtualized software spares and rejuvenation scheduling have been used to maintain highly reliable software platforms and combat Mandelbugs in cloud systems. However, developing real-time rejuvenation schedules for software components with dynamic reliability models has been a challenging task. In this paper, we propose a hybrid approach that integrates preventive and automatic failover strategies to mitigate the harmful effects of Mandelbugs. The approach allows selecting reliability models based on the state of virtualized software components, performing reliability calculations for Software SPare (SSP) gates with up to two virtual hot spares, and scheduling software rejuvenation in real time for cloud systems. Furthermore, the use of Dynamic Fault Tree (DFT) analysis supports the compositional modeling of complex and interconnected systems, alleviating the problem of state-space explosion. Finally, we present a case study of a cloud system with virtualized software spares to demonstrate how rejuvenation schedules can be generated and updated in real time.

Closed-loop Orchestration for Cloud-native Mobile IPv6

With the advent of Network Function Virtualization (NFV) and Software-Defined Networking (SDN), every network service type faces significant challenges induced by novel requirements. Mobile IPv6, the well-known IETF standard for network-level mobility management, is not an exemption. Cloud-native Mobile IPv6 has acquired several new capabilities due to the technological advancements of NFV/SDN evolution. This paper presents how automatic failover and scaling can be envisioned in the context of cloud-native Mobile IPv6 with closed-loop orchestration on the top of the Open Network Automation Platform. Numerical results are also presented to indicate the usefulness of the new operational features (failover, scaling) driven by the cloud-native approach and highlight the advantages of network automation in virtualized and softwarized environments.

Database resource integration of shared cloud platform based on RAC architecture

This paper focuses on the core problem of database resource integration mode. The theme of this paper is the construction and implementation of database resource integration model, and some legal and commercial constraints related to it, so the content of the research involves the integration of technology, integration system, the integration of the status analysis and other aspects. The basic concept, classification, structure system and basic principles of Oracle RAC are discussed. Combined with business needs, a solution based on Oracle RAC is proposed. We implemented the planning and deployment of shared storage, file storage and network setup based on Oracle RAC database. The automatic failover without user intervention is realized, which greatly improves the reliability of the database, and has a certain guiding significance for the application and construction of the enterprise database. First, the research status of database resources integration at home and abroad is expounded through literature analysis. Secondly, the model of database resource integration is put forward, and its realization is discussed. Thirdly, the status of database resource integration is analyzed. Finally, the problems to be solved are put forward in the end. Through the research of this topic, it provides some reference and basis for the database resource integration model. At the same time, it also makes a paving for the maximum elimination of information isolated island and promoting the sharing of database resources.

Modeling Impact of Human Errors on the Data Unavailability and Data Loss of Storage Systems

Data storage systems (DSSs) and their availability play a crucial role in contemporary datacenters. Despite using mechanisms such as automatic failover in datacenters, the role of human agents and consequently their destructive errors is inevitable. Due to very large number of disk drives used in exascale datacenters and their high failure rates, the disk subsystem in storage systems has become a major source of data unavailability (DU) and data loss (DL) initiated by human errors. In this paper, we investigate the effect of incorrect disk replacement service (IDRS) on the availability and reliability of DSSs. To this end, we analyze the consequences of IDRS in a disk array, and conduct Monte Carlo simulations to evaluate DU and DL during mission time. The proposed modeling framework can cope with different storage array configurations and data object survivability , representing the effect of system-level redundancies such as remote backups and mirrors. In the proposed framework, the model parameters are obtained from industrial and scientific reports alongside field data, which have been extracted from a datacenter operating with 70 storage racks. The results show that ignoring the impact of IDRS leads to unavailability underestimation by up to three orders of magnitude. Moreover, our study suggests that by considering the effect of human errors, the conventional beliefs about the dependability of different redundant array of independent disks (RAID) mechanisms should be revised. The results show that $\text{RAID}1$ can result in lower availability compared to $\text{RAID}5$ in the presence of human errors. The results also show that employing automatic fail-over policy (using hot spare disks) can reduce the drastic impacts of human errors by two orders of magnitude.

Scalable and fail-safe deployment of the ATLAS Distributed Data Management system Rucio

This contribution details the deployment of Rucio, the ATLAS Distributed Data Management system. The main complication is that Rucio interacts with a wide variety of external services, and connects globally distributed data centres under different technological and administrative control, at an unprecedented data volume. It is therefore not possible to create a duplicate instance of Rucio for testing or integration. Every software upgrade or configuration change is thus potentially disruptive and requires fail-safe software and automatic error recovery. Rucio uses a three-layer scaling and mitigation strategy based on quasi-realtime monitoring. This strategy mainly employs independent stateless services, automatic failover, and service migration. The technologies used for deployment and mitigation include OpenStack, Puppet, Graphite, HAProxy and Apache. In this contribution, the interplay between these components, their deployment, software mitigation, and the monitoring strategy are discussed.

Rollback-Recovery for Middleboxes

Network middleboxes must offer high availability, with automatic failover when a device fails. Achieving high availability is challenging because failover must correctly restore lost state (e.g., activity logs, port mappings) but must do so quickly (e.g., in less than typical transport timeout values to minimize disruption to applications) and with little overhead to failure-free operation (e.g., additional per-packet latencies of 10-100s of us). No existing middlebox design provides failover that is correct, fast to recover, and imposes little increased latency on failure-free operations. We present a new design for fault-tolerance in middleboxes that achieves these three goals. Our system, FTMB (for Fault-Tolerant MiddleBox), adopts the classical approach of "rollback recovery" in which a system uses information logged during normal operation to correctly reconstruct state after a failure. However, traditional rollback recovery cannot maintain high throughput given the frequent output rate of middleboxes. Hence, we design a novel solution to record middlebox state which relies on two mechanisms: (1) 'ordered logging', which provides lightweight logging of the information needed after recovery, and (2) a `parallel release' algorithm which, when coupled with ordered logging, ensures that recovery is always correct. We implement ordered logging and parallel release in Click and show that for our test applications our design adds only 30$\mu$s of latency to median per packet latencies. Our system introduces moderate throughput overheads (5-30%) and can reconstruct lost state in 40-275ms for practical systems.

Open source firewall alternatives

Most enterprises look no further than the major commercial vendors when it comes to planning future firewalls. But open source firewalls - often poo-pooed as overly simplistic - can now be used in a much wider range of situations and are better suited for complex and demanding environments. Traditional firewalls have provided security and network engineers real enterprise functionality: high throughput, stateful filtering, ease of management, automatic failover, and highly reliable operation. For larger enterprises, these the cost of these firewalls are easily outweighed by their benefits. But for many the price may be burdensome or the feature set may simply be more than is needed. But due to the work of large numbers of developers, the open source firewall alternatives have become serious competitors to their commercial brethren. Open source firewalls still may not be for everyone, but they are certainly beginning to make a difference. When most enterprises start examining options for firewalls, they instinctively consider the major commercial firewall vendors. Product offerings such as Cisco's Pix, Checkpoint's Firewall-1, and FortiNet's FortiGate all bubble to the surface. These firewalls have historically provided security and network engineers real enterprise functionality: high throughput, stateful filtering, ease of management, automatic failover, and highly reliable operation. For larger enterprises, these the cost of these firewalls are easily outweighed by their benefits. However, for many the price may be burdensome or the feature set may simply be more than is needed.

Experience with autonet

Abstract Autonet is a self-configuring, switch-based local-area network with automatic routing of individual Ethernet-style packets. The twelve ports on each switch may be connected to host controllers or other switch ports in arbitrary configurations. A distributed control program running in the switches monitors the state of the network, determines deadlock-free routes for all sources and destinations, and sets up hardware forwarding tables for each switch. A link is a single wire of cable-television coax up to 100 m long, passing data at 100 Mbit/s simultaneously in each direction. Functionally, Autonet looks like Ethernet, but it is faster, easier to manage, and more available. Autonet has excellent performance and can grow easily and economically by adding redundant links and switches to provide increased throughput. Automatic reconfiguration and failover enable the network to operate in any configuration, to defined itself against intermittent components, and to be repaired while still in operation.