Runtime Faults Research Articles

Energy Attentive and Pre-fault Recognize Mechanism for Distributed Wireless Sensor Network Using Fuzzy Logic Approach

Wireless sensor networks (WSNs) have been transforming over recent years with development in the design of smart real-time applications. However, it presents numerous challenges in terms of fault-tolerant communication, low latency, scalability, and transmission efficiency. It is extremely difficult for WSNs to detect runtime faults since they're unaware of the internal processes at work within the sensor node. As a result, valuable sensed information cannot reach its destination and performance starts degrading. Towards this objective, the proposed mechanism applies a novel pre-fault detection mechanism based on a fuzzy rule-based method for multilevel transmission in distributed sensor networks. The proposed mechanism uses a fuzzy rule set to make routing decisions. A fuzzy decision rule set is proposed to perform routing based on the fuzzy fault count status of a node. The proposed mechanism assists in identifying the fault in advance and determining the optimal routing path to save energy and improve network performance. In accordance with the node fault status, the data transmission rate is finalized to prevent further energy consumption. The results demonstrated that the proposed mechanism performed well on judgment evaluation metrics like the energy dissipation ratio, throughput, packet loss rate and communication delay.

Trapping Runtime Faults from Web Service Fault Ontology

The ontology offers a clear considerate of the runtime faults in web services and helps to share this common understanding with users and applications. This paper presents Web Service Fault Ontology and to trap the runtime faults from the Web Services Fault Ontology. Web Service Fault Ontology has been developed to represent the different types of faults that can occur during the interactions between service users, service publishers and service registries: publishing, discovery, binding and execution of web services. Ontology has been proposed to define the intended behavior of web services from the service provider. A sample web service application was developed for testing the proposed model.

Model-Based Monitoring and Adaptation of Pacemaker Behavior Using Hierarchical Fuzzy Colored Petri-Nets

A pacemaker is an embedded device that is sited in the chest to regulate irregular heartbeats known as arrhythmias. Since such devices are directed by software, a software failure may cause a serious hazard such as patient death. Runtime monitoring and adaptation of the device software behavior offers a solution for preventing device hazards. We have already obtained some experiences from monitoring medical devices like: 1) insulin pump using fuzzy Petri net and 2) cardiac pacemaker using colored Petri-net (CPN) and hierarchical fuzzy CPN (HFCPN). However, these studies did not present an adaptation method for software runtime faults. This paper, extending our previous work, presents an automatic runtime method for continuous verification of the behavior of the implanted pacemaker using a software agent. The autonomy and the intelligence characteristics of the software agent are used to control the behavior of the cardiac pacemaker software by drawing inferences from a knowledge base where HFCPN is used by the agent as the inference engine. Compared to a flat inference engine, the HFCPN is able to cover the concurrent states initiated by input fuzzy values and improve the running time for finding a suitable rule by up to 92%. In addition, the intelligent software agent checks the runtime operation accuracy of the pacemaker software in vital and unexpected situations, and redirects the software decision if it finds an unacceptable value. To demonstrate the HFCPN’s behavior and decision-making in different situations, three different scenarios are presented.

Towards runtime verification of collaborative embedded systems

A group of collaborative embedded systems does not depend on a central authority to operate in uncertain environments. By engaging in various negotiation protocols, the participants assign roles, schedule tasks, and combine their world views for more resilient perception and planning. To verify functional correctness, critical components can be tested with simulation-based methods, but the possibility of runtime faults still necessitates online monitoring. In this work, we characterize and address the runtime verification problem in the context of collaborative embedded systems. We present a case study based on industrial transport robots and model the main operating procedure, a distributed bidding protocol. The key properties that must hold for functional correctness turn out to comprise multiple semantic concepts that cannot be jointly expressed with any single formalism. To address this issue, we identify three specification languages that are particularly suitable for monitoring of collaborative embedded systems: Certifying distributed algorithms, trace expressions, and real-valued temporal logic. We show how each of them can be used to capture a subset of the relevant properties and outline a way of integrating them into a common framework.

FIJI: Fault InJection Instrumenter

FPGAs are increasingly used in safety-critical applications (e.g., in aerospace and automotive engineering). Safety standards stipulate that implemented countermeasures against run-time faults such as detection and isolation of affected components, automatic reconfiguration, and redundancy mechanisms must be adequately verified. To that end, fault injection tests by various means have been established as a suitable method.For such tests, faults can be provoked by radiation, simulation, or manipulating the design, for example, by inserting additional logic or manipulating the synthesis flow. This work briefly summarizes the various fault injection approaches with a focus on methods that are capable of stressing critical nets of a design running on actual hardware without requiring to re-synthesize. While the state-of-the-art tools can work with complex designs, they often lack controllability of the exact timing of the injection events (which is important to track the system’s response on faults in a logic simulation) and/or use a high amount of FPGA resources. To overcome these issues, we propose a resource-saving netlist-based fault injection framework Fault InJection Instrumenter (FIJI) that can target individual nets at test runtime. This paper presents FIJI’s work flow, implementation details, and an evaluation in terms of FPGA resources, timing impact, and performance during instrumentation and test execution. The FIJI framework has been made publicly available by the authors under an open-source license.

Energy-efficient scheduling with reliability guarantee in embedded real-time systems

Embedded systems have been widely applied in many areas such as aerospace, intelligent transportation, smart grid, and, medical care. However, limited energy supply and runtime faults are two challenges that hinder the embedded system to be even more pervasively applied. In this paper a greedy energy-efficient (GEE) algorithm is proposed to improve the energy efficiency under the constraint of reliability. As GEE may lead to more preemptions with reduced execution frequency, this paper analyzes the preemption during the execution and proposes a series of constraints to reduce the frequency of preemption. This reduces both the energy consumption and execution time. To further balance the execution frequency, two energy-efficient algorithms are presented based the processor utilization. Finally, considering the fact that the actual execution time (AET) of many tasks may be less than the worst-case execution time (WCET), a global dynamic scheduling algorithm is proposed to maximize the energy efficiency while ensuring the reliability. The experimental results show that the proposed techniques have significant improvements of energy efficiency with guaranteed system reliability.

Causality and Temporal Dependencies in the Design of Fault Management Systems

Reasoning about causes and effects naturally arises in the engineering of safety-critical systems. A classical example is Fault Tree Analysis, a deductive technique used for system safety assessment, whereby an undesired state is reduced to the set of its immediate causes. The design of fault management systems also requires reasoning on causality relationships. In particular, a fail-operational system needs to ensure timely detection and identification of faults, i.e. recognize the occurrence of run-time faults through their observable effects on the system. Even more complex scenarios arise when multiple faults are involved and may interact in subtle ways. In this work, we propose a formal approach to fault management for complex systems. We first introduce the notions of fault tree and minimal cut sets. We then present a formal framework for the specification and analysis of diagnosability, and for the design of fault detection and identification (FDI) components. Finally, we review recent advances in fault propagation analysis, based on the Timed Failure Propagation Graphs (TFPG) formalism.

A runtime fault survival method for deployed software during production runs

Runtime memory faults during production run should be more thoroughly addressed because they severely affect system availability. This paper proposes a method for mitigating memory faults during production runs of deployed software, thereby ensuring normal system operation until patches to fix the faults are delivered. Furthermore, the method helps enhance debugging efficiency by providing accurate on-site fault information used by developers to release timely patches. The core of the method is to offer information tagging to identify runtime faults and a fault survival algorithm to provide differentiated fault mitigation according to the runtime state. We implemented ROPHE on a Linux 2.6 platform and conducted an empirical study of representative Linux applications. The results show that the average fault-handling rate among the applications is 35.75%, whereas the RemOte runtime Protection for High-risk Error ROPHE greatly improves capacity to an average of 91.94%. Specifically, the fault-handling rates of the applications ranged widely from 7.32% to 62.96%, while ROPHE provided fault-survival rates in the relatively narrow range of 82.35-97.44%. The experimental results show that the proposed method guarantees the same level of reliability for all applications regardless of their individual fault handling capacity. Copyright © 2016 John Wiley & Sons, Ltd.

Optimizing Checkpoint Restart with Data Deduplication

The increasing scale, such as the size and complexity, of computer systems brings more frequent occurrences of hardware or software faults; thus fault-tolerant techniques become an essential component in high-performance computing systems. In order to achieve the goal of tolerating runtime faults, checkpoint restart is a typical and widely used method. However, the exploding sizes of checkpoint files that need to be saved to external storage pose a major scalability challenge, necessitating the design of efficient approaches to reducing the amount of checkpointing data. In this paper, we first motivate the need of redundancy elimination with a detailed analysis of checkpoint data from real scenarios. Based on the analysis, we apply inline data deduplication to achieve the objective of reducing checkpoint size. We use DMTCP, an open-source checkpoint restart package, to validate our method. Our experiment shows that, by using our method, single-computer programs can reduce the size of checkpoint file by 20% and distributed programs can reduce the size of checkpoint file by 47%.

Run Time Fault Detection System for Web Service Composition and Execution

Composite web services are a group of web services which together offer service functionality. Faults can occur during composition or execution of any of the services that make up the composite services. Run-time monitoring of web service execution enables real-time detection of faults concurrent with the service execution. This paper extends our previous work on fault detection of single web services to detect faults in composite web services. Fault handler has been proposed to detect faults during composition of web services for static, semi dynamic and dynamic compositions. Policies have been used to denote the intended run-time behavior of web services. Web service execution has been monitored against these policies to detect run-time faults. The proposed work has been tested using a sample web service application and the results are presented

Intrinsic fault tolerance of multilevel Monte Carlo methods

Monte Carlo (MC) and multilevel Monte Carlo (MLMC) methods applied to solvers for Partial Differential Equations with random input data are proved to exhibit intrinsic failure resilience. Sufficient conditions are provided for non-recoverable loss of a random fraction of MC samples not to fatally damage the asymptotic accuracy versus work of a MC simulation. Specifically, the convergence behavior of MLMC methods on massively parallel hardware with runtime faults is analyzed mathematically and investigated computationally. Our mathematical model assumes node failures which occur uncorrelated of MC sampling and with general sample failure statistics on the different levels and which also assume absence of checkpointing, i.e., we assume irrecoverable sample failures with complete loss of data. Modifications of the MLMC with enhanced resilience are proposed. The theoretical results are obtained under general statistical models of CPU failure at runtime. Particular attention is paid to node failures with so-called Weibull failure models on massively parallel stochastic finite volume computational fluid dynamics simulations are discussed. We discuss the resilience of massively parallel stochastic Finite Volume computational fluid dynamics simulations.

A low overhead, fault tolerant and congestion aware routing algorithm for 3D mesh-based Network-on-Chips

Nowadays, three dimensional Network-On-Chips (NOCs) have emerged as most efficient and scalable communication structures for complex and high performance System-on-Chips (SOCs). These structures are so susceptible to manufacturing and runtime faults. Thus fault tolerant routing is essential to increase reliability and performance of NOC-based SOCs. In this paper, we propose FT-DyXYZ, an adaptive fault tolerant routing to tolerate permanent faulty links in 3D mesh based NOCs that uses proximity congestion information to balance traffic. Our routing achieves fault tolerance without using routing tables, redundancy or global information of paths and faults. To evaluate performance of our routing, we compared it with fault tolerant planar-adaptive routing in terms of average packet latency, throughput and reliability. Simulation results demonstrate significant improvement of saturation injection rate, average throughput and reliability of our routing under synthetic traffic patterns.

A system-level approach to adaptivity and fault-tolerance in NoC-based MPSoCs: The MADNESS project

Modern embedded systems increasingly require adaptive run-time management of available resources. One method for supporting adaptivity is to implement run-time application mapping. The system may adapt the mapping of the applications in order to accommodate the current workload conditions, to balance the computing load for efficient resource utilization, to meet quality of service agreements, to avoid thermal hot-spots, and to reduce power consumption. As the possibility of experiencing run-time faults becomes increasingly relevant with deep-sub-micron technology nodes, in the scope of the MADNESS project, we focused particularly on the problem of graceful degradation by dynamic remapping in presence of run-time faults.In this paper, we summarize the major results achieved in the MADNESS project regarding the system adaptivity and fault-tolerant processing. We report the results of the integration between platform level and middleware level support for adaptivity and fault-tolerance. Two case studies demonstrate the survival ability of the system via a low-overhead process migration mechanism and by taking near optimal remapping decisions at run-time.

Web Service Diagnoser Model for managing faults in web services

Reliability is an important criterion to facilitate extensive deployment of web service technology for commercial business applications. Run-time monitoring and fault management of web services are essential to ensure uninterrupted and continuous availability of web services. This paper presents WISDOM (Web Service Diagnoser Model) a generic architecture for detecting faults during execution of web services. Policies have been proposed to describe the intended behavior of web services and faulty behavior would be detected as deviations or inconsistencies with respect to the specified behavior. The model proposes the use of monitoring components in service registries and service providers to detect run-time faults during publishing, discovery, binding and execution of web services. An independent fault diagnoser is proposed to coordinate the individual monitoring components and also act as a repository for the specified web service policies. The proposed model has been tested with a sample web service application and the results obtained are presented.

On the Optimization of GLite-Based Job Submission

A Grid is a very dynamic, complex and heterogeneous system, whose reliability can be adversely conditioned by several different factors such as communications and hardware faults, middleware bugs or wrong configurations due to human errors. As the infrastructure scales, spanning a large number of sites, each hosting hundreds or thousands of hosts/resources, the occurrence of runtime faults following job submission becomes a very frequent and phenomenon. Therefore, fault avoidance becomes a fundamental aim in modern Grids since the dependability of individual resources spread upon widely distributed computing infrastructures and often used outside of their native organizational boundaries, cannot be guaranteed in any systematic way. Accordingly, we propose a simple job optimization solution based on a user-driven fault avoidance strategy. Such strategy starts from the introduction within the grid information system of several on-line service-monitoring metrics that can be used as specific hints to the workload management system for driving resource discovery operations according to a fault-free resource-scheduling plan. This solution, whose main goal is to minimize the execution time by avoiding execution failures, demonstrated to be very effective in incrementing both the user perceivable quality and the overall grid performance.

Experience with fault injection experiments for FMEA

AbstractFailure Modes and Effects Analysis (FMEA) is a widely used system and software safety analysis technique that systematically identifies failure modes of system components and explores whether these failure modes might lead to potential hazards. In practice, FMEA is typically a labor‐intensive team‐based exercise, with little tool support. This article presents our experience with automating parts of the FMEA process, using a model checker to automate the search for system‐level consequences of component failures. The idea is to inject runtime faults into a model based on the system specification and check if the resulting model violates safety requirements, specified as temporal logical formulas. This enables the safety engineer to identify if a component failure, or combination of multiple failures, can lead to a specified hazard condition. If so, the model checker produces an example of the events leading up to the hazard occurrence which the analyst can use to identify the relevant failure propagation pathways and co‐effectors. The process is applied on three medium‐sized case studies modeled with Behavior Trees. Performance metrics for SAL model checking are presented. Copyright © 2011 John Wiley & Sons, Ltd.

Address Translation Aware Memory Consistency

Computer systems with virtual memory are susceptible to design bugs and runtime faults in their address translation systems. Detecting bugs and faults requires a clear specification of correct behavior. A new framework for address translation aware memory consistency models addresses this need.

An MPSoC-Based QAM Modulation Architecture with Run-Time Load-Balancing

QAM is a widely used multilevel modulation technique, with a variety of applications in data radio communication systems. Most existing implementations of QAM-based systems use high levels of modulation in order to meet the high data rate constraints of emerging applications. This work presents the architecture of a highly parallel QAM modulator, using MPSoC-based design flow and design methodology, which offers multirate modulation. The proposed MPSoC architecture is modular and provides dynamic reconfiguration of the QAM utilizing on-chip interconnection networks, offering high data rates (more than 1 Gbps), even at low modulation levels (16-QAM). Furthermore, the proposed QAM implementation integrates a hardware-based resource allocation algorithm that can provide better throughput and fault tolerance, depending on the on-chip interconnection network congestion and run-time faults. Preliminary results from this work have been published in the Proceedings of the 18th IEEE/IFIP International Conference on VLSI and System-on-Chip (VLSI-SoC 2010). The current version of the work includes a detailed description of the proposed system architecture, extends the results significantly using more test cases, and investigates the impact of various design parameters. Furthermore, this work investigates the use of the hardware resource allocation algorithm as a graceful degradation mechanism, providing simulation results about the performance of the QAM in the presence of faulty components.

Specifying and dynamically verifying address translation-aware memory consistency

Computer systems with virtual memory are susceptible to design bugs and runtime faults in their address translation (AT) systems. Detecting bugs and faults requires a clear specification of correct behavior. To address this need, we develop a framework for AT-aware memory consistency models. We expand and divide memory consistency into the physical address memory consistency (PAMC) model that defines the behavior of operations on physical addresses and the virtual address memory consistency (VAMC) model that defines the behavior of operations on virtual addresses. As part of this expansion, we show what AT features are required to bridge the gap between PAMC and VAMC. Based on our AT-aware memory consistency specifications, we design efficient dynamic verification hardware that can detect violations of VAMC and thus detect the effects of design bugs and runtime faults, including most AT related bugs in published errata.

Specifying and dynamically verifying address translation-aware memory consistency

Computer systems with virtual memory are susceptible to design bugs and runtime faults in their address translation (AT) systems. Detecting bugs and faults requires a clear specification of correct behavior. To address this need, we develop a framework for AT-aware memory consistency models. We expand and divide memory consistency into the physical address memory consistency (PAMC) model that defines the behavior of operations on physical addresses and the virtual address memory consistency (VAMC) model that defines the behavior of operations on virtual addresses. As part of this expansion, we show what AT features are required to bridge the gap between PAMC and VAMC. Based on our AT-aware memory consistency specifications, we design efficient dynamic verification hardware that can detect violations of VAMC and thus detect the effects of design bugs and runtime faults, including most AT related bugs in published errata.