Fast Failure Research Articles

CubicRing: Exploiting Network Proximity for Distributed In-Memory Key-Value Store

In-memory storage has the benefits of low I/O latency and high I/O throughput. Fast failure recovery is crucial for large-scale in-memory storage systems, bringing network-related challenges, including false detection due to transient network problems, traffic congestion during the recovery, and top-of-rack switch failures. In order to achieve fast failure recovery, in this paper, we present CubicRing, a distributed structure for cube-based networks, which exploits network proximity to restrict failure detection and recovery within the smallest possible one-hop range. We leverage the CubicRing structure to address the aforementioned challenges and design a network-aware in-memory key-value store called MemCube. In a 64-node 10GbE testbed, MemCube recovers 48 GB of data for a single server failure in 3.1 s. The 14 recovery servers achieve 123.9 Gb/s aggregate recovery throughput, which is 88.5% of the ideal aggregate bandwidth and several times faster than RAMCloud with the same configurations.

Conceptual modelling of the Failure management process in the manufacturing industry

Companies in the production industry struggle with a fast and sustainable failure elimination. In the past, organisational models for Failure management were in the focus of research. But these approaches often miss a consideration of interdependencies within the production process and the Failure management. Furthermore, they miss a definition of the necessary relations. The purpose of this paper is to understand the challenges of a better integration and design of the Failure management into the production process by developing a concept of a model. This paper describes the elements, connections and targets of Failure management in relation to production systematics in a stock and flow model. Therefore, sub-models for Failure management and the production process are derived from literature and conducted research projects. The model allows detecting critical relations for an effective Failure management. An improvement or establishment of these relations supports companies to develop their Failure management towards a better integration into the production process and leads to a faster, sustainable and more efficient elimination of failures. The provided model allows a deep understanding of the Failure management process. It exceeds the state of the art towards the understanding of intra-system relations. It allows an understanding of the necessary mechanisms of action to enable an organisation to improve its production performance by failure-based learning.

Modified Polymer Matrix Nano Biocomposite for Bone Repair and Replacement- Radiological Study

Synthetic biomaterials for bone repair and substitute must be yield to critical criteria , one of them, their mechanical properties should not be very less than that for natural bone to prevent fast failure, nor much higher to prevent stress shielding effect which led to fast implant collapse. Other important issue, it should not cause inflammation in implantation region. According to these criteria, many researchers investigated several biomaterials. Titanium dioxide (TiO2) reinforced polyetheretherketone (PEEK) biocomposite is considered a promising biomaterials, because of their superior properties and good biocompatibility with host tissue. Alumina (Al2O3), considered a bio inert ceramic used in this work to modify mechanical properties. Hot pressing technique adopted in this work with pressing pressure of 50 MPa at 370,380,390, and400 C° compounding temperatures to produce different compositions implants. Animal model used to study inflammation behaviour for implants as a comparison with control group. X-Ray radiological test were carried out for both, implants and control regions. Mechanical testing shows good values, similar to natural bone. No inflammation observed in injury area.

Fault tolerance in TCAM-limited software defined networks

In Software Defined Networks (SDNs), while a proactive fault tolerance based on the local rerouting approach enables fast failure recovery, it requires to install forwarding rules for the backup paths in the switch Ternary Content Addressable Memory (TCAM) in advance. Since the TCAM size is limited and forwarding rules are long, using large number of forwarding rules for backup paths leads to frequent eviction of flows at the switches. To guarantee the bandwidth requirement for a flow upon a link failure, bandwidth reservation is also required on the backup path. Inefficient backup bandwidth usage will cause rejection of increased number of flows. In this paper, we address the problem of proactive fault tolerance in SDNs, considering the above challenges by adopting the local rerouting approach. We develop an optimization programming formulation that determines the set of backup paths to protect a flow while minimizing the number of additional rules and bandwidth required for the backup paths. Since this problem is computationally prohibitive, we develop two heuristic algorithms, namely Forward Local Rerouting (FLR) and Backward Local Rerouting (BLR) to compute backup paths so as to improve TCAM and bandwidth usage efficiency. We propose a flexible adaptive failure recovery framework that takes the decision of using FLR or BLR based on the network state. We evaluate the proposed algorithms through simulations and compare their performance with the optimal solution. The results show that the proposed algorithms perform very close to the optimal solution and reduce the number of additional rules required to protect a flow by up to 75% compared to the approaches that do not consider the limited size of TCAM. The results also show that the proposed algorithms are effective in terms of backup bandwidth efficiency and meet the carrier-grade requirement with the recovery time below 50 ms.

Risk Assessment Method of UHV AC/DC Power System under Serious Disasters

Based on the theory of risk assessment, the risk assessment method for an ultra-high voltage (UHV) AC/DC hybrid power system under severe disaster is studied. Firstly, considering the whole process of cascading failure, a fast failure probability calculation method is proposed, and the whole process risk assessment model is established considering the loss of both fault stage and recovery stage based on Monte Carlo method and BPA software. Secondly, the comprehensive evaluation index system is proposed from the aspects of power system structure, fault state and economic loss, and the quantitative assessment of system risk is carried out by an entropy weight model. Finally, the risk assessment of two UHV planning schemes are carried out and compared, which proves the effectiveness of the research work.

Recursive SDN for Carrier Networks

Control planes for global carrier networks should be programmable and scalable. Neither traditional control planes nor new SDN-based control planes meet both of these goals. Here we propose a framework for recursive routing computations that combines the best of SDN (programmability through centralized controllers) and traditional networks (scalability through hierarchy) to achieve these two desired properties. Through simulation on graphs of up to 10,000 nodes, we evaluate our design's ability to support a variety of unicast routing and traffic engineering solutions, while incorporating a fast failure recovery mechanism based on network virtualization.

Feedback-Based Path Failure Detection and Buffer Blocking Protection for MPTCP

A multipath TCP (MPTCP) is a promising protocol that has been standardized by the Internet Engineering Task Force to support multipath operations in the transport layer. However, although the MPTCP can provide multiple transmission paths and aggregate the bandwidth of multiple paths, it does not consistently achieve more throughput (goodput) nor a greater connection resilience. Currently, the MPTCP is vulnerable to path failure or underperforming subflows, which cause transmission interruption or throughput (goodput) degradation. Unfortunately, there is no exact rule for declaring a path failure or preventing the usage of underperforming subflows in the MPTCP. In this paper, we propose a novel path failure detection method referred to as feedback-based path failure (FPF) detection. In addition, we propose a new decision method called buffer blocking protection (BBP) to address the underperforming subflows for the MPTCP. Measurement results indicate that the FPF detection reduces transmission interruption time by the fast path failure decision, which can prevent duplicate transmission interruption events and unnecessary retransmissions. Furthermore, the FPF detection is sufficiently robust in terms of packet loss and the delay difference between paths. The results additionally show that the BBP method prevents goodput degradation due to underperforming subflows. Consequently, the MPTCP with the BBP method can at least achieve the throughput performance of a single Transmission Control Protocol (TCP), which uses the best path regardless of the delay difference between paths.

Fast failure detection and recovery in SDN with stateful data plane

SUMMARYWhen dealing with node or link failures in software‐defined networking (SDN), the network capability to establish an alternative path depends on controller reachability and on the round‐trip times between controller and involved switches. Moreover, current SDN data plane abstractions for failure detection, such as OpenFlow “Fast‐failover,” do not allow programmers to tweak switches' detection mechanism, thus leaving SDN operators relying on proprietary management interfaces (when available) to achieve guaranteed detection and recovery delays. We propose SPIDER, an OpenFlow‐like pipeline design that provides (i) a detection mechanism based on switches' periodic link probing and (ii) fast reroute of traffic flows even in the case of distant failures, regardless of controller availability. SPIDER is based on stateful data plane abstractions such as OpenState or P4, and it offers guaranteed short (few milliseconds or less) failure detection and recovery delays, with a configurable trade‐off between overhead and failover responsiveness. We present here the SPIDER pipeline design, behavioral model, and analysis on flow tables' memory impact. We also implemented and experimentally validated SPIDER using OpenState (an OpenFlow 1.3 extension for stateful packet processing) and P4, showing numerical results on its performance in terms of recovery latency and packet loss.

Automatized failure analysis of tungsten coated TSVs via scanning acoustic microscopy

In 3D integrated microelectronics, the failure analysis of through silicon vias (TSVs) represents a highly demanding task. In this study, defects in tungsten coated TSVs were analysed using scanning acoustic microscopy (SAM). Here, the focus lay on the realization of an automatized failure detection method towards rapid learning. We showed that by using a transducer of 100MHz center frequency, established with an acoustical objective (AO), it is possible to detect defects within the TSVs. In order to interpret our analysis, we performed acoustic wave propagation simulations based on the elastodynamic finite integration technique (EFIT). In addition, high resolution X-ray computed tomography (XCT) was performed which corroborated the SAM analysis. In order to go towards automatized defect detection, firstly the commercially available software “WinSAM8” was enhanced to perform scans at defined working distances automatically. Secondly, a pattern recognition algorithm was successfully applied using “Python” to the SAM scans in order to distinguish damaged TSVs from defect-free TSVs. Besides the potential for automatized failure detection in TSVs, the SAM approach exhibits the advantages of fast and non-destructive failure detection, without the need for special preparation of the sample.

Mg2si Anode with Greatly Enhanced Cycle Life

Lithium-ion batteries (LIBs) with silicon anode have a potential application in electric vehicles and energy storage systems owing to the high capacity of silicon, however, the volume change and thus-resulted fast capacity decay or electrode failure greatly restricts its practical use. Using Si-M composite instead of pure Si has been proved to be an efficient strategy to minimize the capacity fade. Among these Si composites, magnesium silicide (Mg2Si) attracts much attention due to the natural abundance of Mg and cost concern. More importantly, Mg and Si can accommodate Li respectively which brings a higher specific capacity than other Si-M composite. Despite the above advantages, similar to pure Si anode, Mg2Si anode also suffers from the large volume change during the cycling. In this work, we propose using fluoroethylene carbonate (FEC) or vinylene carbonate (VC) additive to enhance the cycling life of Mg2Si anode and greatly improved cycling stability has been achieved on Mg2Si anode. Experiment Mg2Si (Adamas Beta, 98%) and acetylene black (ACB) powder were vacuum dried at 80°C for 12h and then ball milled for 3h to get Mg2Si/ACB mixture (mass ratio of 5:3). The electrode contained 80% above mixture and 20% PVDF binder. 1M LiPF6 in EC and DMC mixture solvent (volume ratio of 1:1) was used as the electrolyte. The cyclic performance was evaluated on LAND CT2001A system between 0.01 and 2.0V with a current of 100mA/g. The impedance spectra were collected on Autolab electrochemical working station. The surface information of the cycled electrode was obtained through SEM observation and XPS measurement. Results and discussions Figure 1 shows the cycling performances of Mg2Si in bare electrolyte or the electrolyte with VC or FEC addition. Rapid capacity fading was found on Mg2Si in bare electrolyte, such as lower than 300 mAh/g after several cycles. In comparison, in FEC-added electrolyte, Mg2Si still released 400 mAh/g capacity after 100 cycles. In VC-added electrolyte, similar improvement was also observed, but not as notable as in FEC-added electrolyte. SEM observation supplied the SEI information of the Mg2Si electrodes cycled in electrolyte with or without additives. In bare electrolyte, the thinnest and most porous SEI layer was found on Mg2Si. While in FEC or VC-added electrolyte, the SEI layer was much thicker and denser, and VC addition led to the most compact SEI layer on Mg2Si anode. EIS impedance further confirmed this difference by the biggest diameter shown by Mg2Si anode in VC-added electrolyte. To understand the reason of the different cycling behavior of Mg2Si, XPS analysis was conducted. Given the fixed amount of LiPF6 in all tested electrolytes, the F-P bond could be used as a reference, and the amount of Li-F could be determined in a semi-quantitative model. The results revealed the highest content of LiF in the SEI formed on Mg2Si in FEC-added electrolyte and the lowest content of LiF in the SEI formed in bare electrolyte. Combined with previous research and some additional experiments, we think FEC could promote the LiF accumulation in the outer layer rather than the inner layer of SEI and reduce the LiF content in the bulk Mg2Si anode, and this eventually brings some protection effect on Mg2Si. Conclusions FEC addition in electrolyte greatly enhanced the cycling stability of Mg2Si, and around 400mAh/g capacity could be obtained after 100cycles. Detailed analysis revealed the protection effect of FEC on Mg2Si anode. Figure 1

Fast breaker failure backup protection for HVDC grids

High voltage direct current (HVDC) grid protection must clear dc faults within a time-frame of milliseconds to avoid damages to power electronic components due to fast rising dc fault currents. If the breaker associated with the primary protection fails, backup must be provided to clear the faults which would otherwise persist. Backup relaying algorithms originally developed for ac systems delay the detection of primary protection failure until the expected primary fault clearance instant. Application of similar algorithms to HVDC grids results in a long fault clearing time, which requires converters and breakers to withstand unrealistically large currents. To reduce the fault clearing time, this paper proposes two fast backup relaying algorithms which detect local and remote breaker failure before the expected primary fault clearance instant. The proposed algorithms use thresholds on the voltage and current measurements to detect breaker failure. They are evaluated using a four-terminal HVDC grid test system implemented in PSCAD. The study results show that the proposed algorithms can reliably detect breaker failure within few milliseconds after fault inception. Furthermore, the algorithms provide faster backup protection compared to methods based on ac backup protection philosophy. The resulting decreased fault clearance time reduces the maximum dc fault current and consequently, leads to lower required ratings for HVDC grid equipment.

Enabling fast failure recovery in shared Hadoop clusters: Towards failure-aware scheduling

Hadoop emerged as the de facto state-of-the-art system for MapReduce-based data analytics. The reliability of Hadoop systems depends in part on how well they handle failures. Currently, Hadoop handles machine failures by re-executing all the tasks of the failed machines (i.e., executing recovery tasks). Unfortunately, this elegant solution is entirely entrusted to the core of Hadoop and hidden from Hadoop schedulers. The unawareness of failures therefore may prevent Hadoop schedulers from operating correctly towards meeting their objectives (e.g., fairness, job priority) and can significantly impact the performance of MapReduce applications. This paper presents Chronos, a failure-aware scheduling strategy that enables an early yet smart action for fast failure recovery while still operating within a specific scheduler objective. Upon failure detection, rather than waiting an uncertain amount of time to get resources for recovery tasks, Chronos leverages a lightweight preemption technique to carefully allocate these resources. In addition, Chronos considers data locality when scheduling recovery tasks to further improve the performance. We demonstrate the utility of Chronos by combining it with Fifo and Fair schedulers. The experimental results show that Chronos recovers to a correct scheduling behavior within a couple of seconds only and reduces the job completion times by up to 55% compared to state-of-the-art schedulers.

Electrical evaluation of crack generation in SiNx and SiOxNy thin-film encapsulation layers for OLED displays

By measuring leakage current density, we detected crack generation in silicon nitride (SiNx) and silicon oxynitride (SiOxNy) thin-film encapsulation layers, and correlated with the films’ water vapor permeability characteristics. After repeated bending cycles, both the changes in water vapor transmission rate and leakage current density were directly proportional to the crack density. Thick SiNx films had better water vapor barrier characteristics in their pristine state, but cyclic loading led to fast failure. Varying the atomic concentration of the SiOxNy films affected their bending reliability. We attribute these differences to changes in the shape of the crack tip as the oxygen content varies.

Investigation of possible wellbore cement failures during hydraulic fracturing operations

Abstract We model and assess the possibility of shear failure along the vertical well by using the Mohr–Coulomb failure model and employing a rigorous coupled flow-geomechanic analysis. To this end, we take various values of cohesion between the well casing and the surrounding cement to represent different quality levels of cementing operation (low cohesion corresponds to low-quality cement and/or incomplete cementing). The simulation results show that there is very little fracturing when the cement is of high quality. Conversely, incomplete cementing and/or weak cement can cause significant shear failure and evolution of long fractures/cracks along the vertical well. Specifically, low cohesion between the well and cemented areas can cause significant shear failure along the well, while high cohesion does not cause shear failure. The Biot and thermal dilation coefficients strongly affect shear failure along the well casing, and low Young's modulus causes fast failure propagation. Still, for the high quality of the cementing job, failure propagates very little. When the hydraulic fracturing pressure is high or when permeability increases significantly, low cohesion of the cement can cause fast propagation of shear failure and of the resulting fracture/crack, but a high-quality cement with no weak zones exhibits limited shear failure that is only concentrated near the bottom of the vertical part of the well. Thus, high-quality cement and complete cementing along the vertical well appears to be the strongest protection against shear failure of the wellbore cement and, consequently, against contamination hazards to drinking water aquifers during hydraulic fracturing operations.

A Multicast Tree Management Method Supporting Fast Failure Recovery and Dynamic Group Membership Changes in OpenFlow Networks

A Multicast Tree Management Method Supporting Fast Failure Recovery and Dynamic Group Membership Changes in OpenFlow Networks

Design and Implementation of Path Failure Detection and Recovery Mechanism for Multihomed HIMALIS Network

We have designed the Heterogeneity Inclusion and Mobility Adaptation through Locator ID Separation (HIMALIS) network architecture to natively support sessions transfer from one link to another independently of the network and transport layer protocols. However, lack of a prompt path failure detection mechanism in HIMALIS cannot well utilize multiple links to protect against path failures. In this paper, we propose a complete path recovery mechanism that consists of path failure detection, lively path exploration, and path recovery processes. In particular, the failure detection mechanism is based on monitoring of data sending and receiving instances in end hosts by employing two timers, probe timer and keepalive timer. In case of path failure caused by gateway's upstream link down, the gateway provides the link failure notification to the host for expediting the detection process. We also present an overview of implementation of the mechanism. The implemented functions are verified by using both TCP and UDP applications and evaluated in a 21-node-scale network-emulation environment. The results show that the proposed mechanism provides fast failure detection, about 7 s which is much faster than the failure detection carried out based on TCP retransmission timeout which may take 5 min in the worst case, under both site- and host-multihoming configurations. The results also show that the proposed mechanism takes only about 200 ms to detect gateway's upstream link down.

Anomalously deep and fast failure of copper and bronze under the action of the corrosion products existing on them

When the corrosion products on copper and bronze are in close contact with a diluted aqueous solution of hydrochloric acid and atmospheric oxygen, they rapidly transform into effective metal (alloy) oxidizers, which provide rapid and deep metal consumption. The metal can be almost fully consumed in a reasonable technological time provided the accumulated solid phase of the products is periodically removed.

Experimental and statistical investigation on the failure of contaminated elastohydrodynamic lubrication rolling contact

This paper deals with the failure of an elastohydrodynamic lubrication rolling contact, operating in highly contaminated media. To achieve this goal, an original experimental device was built and then successfully used to investigate the effects of three operating parameters, called contaminant concentration ( C), rotational speed ( V), and load ( Q) on the evolution of wear, W, and arithmetic mean surface roughness, Ra. The influence of the above mentioned parameters is statistically explored using the analysis of variance (ANOVA). Mathematical models for wear and surface roughness were also established using the response surface methodology by employing multiple regression method. From this experimental research, it is clearly shown that the contaminant concentration is the major factor leading to the fast failure of machine elements. Finally, the obtained results justify and confirm the lifetime reduction of mechanisms in highly contaminated media.

Economical Comparison of Cryogenic vs. Traditional Turning of Ti-6Al-4V: A Case Study

Due to their poor machinability, titanium alloys need to be worked at low cutting speeds to prevent a fast tool failure caused by the very high temperatures that are reached at the tool-chip interface. As demonstrated by previous works by the authors, an improvement of productivity for Titanium alloys can be obtained by adopting cryogenic cooling during the machining operations.The present work shows the features of a toolholder specifically designed for cryogenic adduction in turning operations, following Hong’s design guidelines. The paper compares tool life results between traditional and cryogenic rough turning by adopting Grade 5 titanium as the working material. Rough turning is economically more relevant to the machining industry, especially in the aerospace field where generally a large quantity of rough material has to be removed due to the very high buy-to-fly ratio of aerospace components. A full factorial experimental plan was performed basing on typical rough turning parameters. Machining outputs such as forces, roughness, temperatures, friction coefficients were calculated in order to define statistical differences between cryogenic cooling method using the special toolholder and traditional oil water emulsion cooling system.Furthermore, thanks to tool life results the Taylor’s law for cryogenic and traditional cases was calculated and an hypothetical production scenario for Ti6Al4V parts was analysed. An analytical model to calculate production costs and time was built for both cooling methods. A 4-turrets turning centre was considered and cooling methods and costs per hour of machine tool were taken into account in a cost model. The results show that the benefits in terms of tool life offered by liquid nitrogen cooling allows to improve productivity by adopting higher optimal cutting parameters. This improvement, coupled to an increase of tool life, is very significant and allows not only to reduce time of production but also to cover the major costs of liquid nitrogen and have a slight reduction of machining total costs.

Integrated, Distributed Traffic Control in Multidomain Networks

In this paper, we put forward an integrated traffic control structure and the associated control laws for multidomain networks. This control structure performs per-edge-to-edge-based multinext-hop or multipath rate adaptation and load balancing among domain edge nodes in a multidomain network. This control structure is underpinned by a large family of distributed control laws, with provable convergence and optimality properties. With any user-defined global design objective, a set of control laws can be selected from this family of control laws that track an operational point where the global design objective is achieved, while providing traffic engineering (TE) and fast failure recovery (FFR) features for class-of-service (CoS)-aware flow aggregates. The structure allows the user to have full control over how the domains should be created and whether to use point-to-multipoint and/or point-to-point multipath. The flexibility and versatility of the control structure makes it an ideal theoretical underpinning for the development of integrated traffic control solutions for large-scale networking systems, in particular, software-defined networks in which the data plane is fully programmable via a well-defined south-bound interface, such as OpenFlow. The simulation testing demonstrates the viability of the solution in providing TE, FFR, and CoS features.