Fault-tolerant Computer Systems Research Articles

Performability analysis for degradable computer systems

Degradable performance of fault-tolerant computer systems has given rise to considerable interest in mathematical models for combined evaluation of performance and reliability. Most of these models are based upon Markov processes. Several methods have been proposed for the computation of the probability distribution of performability upon an interval of time [0, t]. In this paper, we present a new algorithm based on the uniformization technique to compute this distribution for block degradable models. The main advantage of this method is its low polynomial computational complexity and its numerical stability, since it only deals with a nonincreasing sequence of positive numbers bounded by 1. This important property allows us to determine new truncation steps which improve the execution time of the algorithm. We apply this method to a degradable computer system.

An analytical model for a parallel fault–tolerant computing system

We present an analytical model of a parallel computing system. Since the probability of fault occurrence is non-negligible, the model takes into consideration fault–tolerance issues, by combining results obtained from a performance model with a fault/repair model. To this purpose, the system performance must be evaluated under several different configurations, caused by the occurrence of faults and repairs. This requires efficient solution techniques of the performance model. The model we adopt is based on an extended queueing network. The queueing network includes a fork/join subnetwork with finite capacity, and three different blocking models to manage saturation condition: blocking before service (BBS), Repetitive Service or Blocking After Service. We prove that the underlying Markov process has a particular structure suitable for efficient solution. To show a possible use of such a model, we present numerical results for a particular maintenance policy, looking for the optimal trade-off between the frequency of service interruption due to repair operations and the need of avoiding excessive performance degradation.

A separable method for incorporating imperfect fault-coverage into combinatorial models

This paper presents a new method for incorporating imperfect FC (fault coverage) into a combinatorial model. Imperfect FC, the probability that a single malicious fault can thwart automatic recovery mechanisms, is important to accurate reliability assessment of fault-tolerant computer systems. Until recently, it was thought that the consideration of this probability necessitated a Markov model rather than the simpler (and usually faster) combinatorial model. SEA, the new approach, separates the modeling of FC failures into two terms that are multiplied to compute the system reliability. The first term, a simple product, represents the probability that no uncovered fault occurs. The second term comes from a combinatorial model which includes the covered faults that can lead to system failure. This second term can be computed from any common approach (e.g. fault tree, block diagram, digraph) which ignores the FC concept by slightly altering the component-failure probabilities. The result of this work is that reliability engineers can use their favorite software package (which ignores the FC concept) for computing reliability, and then adjust the input and output of that program slightly to produce a result which includes FC. This method applies to any system for which: the FC probabilities are constant and state-independent; the hazard rates are state-independent; and an FC failure leads to immediate system failure.

SACI_FTR_LOADER: A Dynamic Loader for the On-Board Computer of the First Brazilian Microssatelite

This paper describes the conception and implementation of a dynamic loader, called SACI_FTR_LOADER, which will be used to upload programs from a ground station onto the onboard computer (called Trisputer) of the first Brazilian microssatelite. The satellite, named SACI (Satellite for Scientific Applications), will be launched by the middle of the next year. The loader is part of the real-time operating system designed to control the resources of the machine, which also incorporates software fault tolerance techniques. The Trisputer is a fault-tolerant multiprocessor computer system which was designed having in mind the high reliability requirements associated with the mission. This goal was achieved by introducing fault tolerance at both hardware and software levels. The on-board computer consists of three processing modules, each one based on a Transputer T-805-30 INMOS microprocessor

Performance and dependability evaluation of scalable massively parallel computer systems with conjoint simulation

Computer systems are becoming more and more a part of our daily life; business and industry rely on their service, and the health of human beings depends on their correct functioning. Computer systems used for critical tasks have to be carefully designed and tested during the early design stage, the prototype phase, and their operational life. Methods and tools are required to support and facilitate this vital task. In this article, we tackle the issue of system-level performance and dependability analysis of fault-tolerant scalable computer systems. A modeling methodology called “Conjoint Simulation” is presented, which is based on the parti tioning of the system model and the combination of several modeling techniques. Object-oriented model construction and process-based simulation are applied for architecture and workload modeling, and timed Petri nets are the core modeling technique representing the failure scenarios and repair policies. Splitting the overall model and exploiting appropriate modeling techniques ease the development, maintenance, and extensibility of large-scale and complex simulation models. Furthermore, techniques are provided for hierarchical model construction, object-oriented workload modeling, and simulated error injection in order to perform combined performance and dependability analysis.

Tight steady-state availability bounds using the failure distance concept

Continuous-time Markov chains are commonly used for dependability modeling of repairable fault-tolerant computer systems. Realistic models of non-trivial fault-tolerant systems often have very large state spaces. An attractive approach for dealing with the largeness problem is the use of pruning methods with error bounds. Several such methods for computing steady-state availability bounds have been proposed recently. This paper presents a new method which exploits the failure distance concept to bound more efficiently the behavior in the non-generated state space. It is proved that the bounding method gives tighter bounds than previous methods. Numerical analysis shows that the new bounds can be significantly tighter.

Optimal repair of degrading computers under asymmetric information and incentive incompatibility

This paper presents the optimal repair policy of a firm's fault-tolerant computer system where asymmetric information and incentive incompatibility problems in allocating the computing resources are prevalent. Traditional maintenance optimization theory focuses on optimization models for repair, replacement, and inspection of machines (computers) subject to breakdowns or deterioration, without considering the allocation of such resources. Likewise, performance evaluation literature of fault-tolerant computers concentrates on deriving reliability and availability measures. This paper is a first attempt to jointly examine both the resource allocation and repair problems of a firm's fault-tolerant computer system. This paper presents the optimal repair policy of a firm's computer system in the presence of asymmetric information and incentive incompatibility. It applies the resource allocation model developed by Cheng et al. [Cheng, H. K., Freimer, M., Richmond, W., Sumita, U., Optimal allocation and backup of computer resources under asymmetric information and incentive incompatibility. European Journal of Operational Research, 1992, 91, 411–426] to analyze the impact of asymmetric information and incentive incompatibility on the repair policy of a firm's fault-tolerant computer system whose capacity degrades over time. A general methodology is presented for deriving the optimal repair policy for the firm's computer system by integrating the results from the resource allocation problem and the stochastic model describing the degrading capacity behavior of the computer system. The optimal repair policy is demonstrated via numerical results that offer useful insights. In particular, the conventional wisdom of having a largest asymptotic expected computer capacity does not guarantee a desirable long-run minimum cost to the organization.

Performability measure for acyclic Markovian models

Continuous-time Markov processes with a finite-state space are generally considered to model degradable fault-tolerant computer systems. The finite space is partitioned as ∪ m i=1 B i , where B i stands for the set of states which corresponds to the configuration where the system has a performance level (or reward rate) equal to τ i . The performability Y t is defined as the accumulated reward over a mission time [0, t]. In this paper, a renewal equation is established for the performability measure and solved for both “standard” and uniform acyclic models. Two closed form expressions for the performability measure are derived for the two types of models. Furthermore, an algorithm with a low polynomial computational complexity is presented and applied to a degradable computer system.

Fault tolerant computer system with shadow virtual processor

Fault tolerant computer system with shadow virtual processor

Hardware fault latency: Problem formulation and solution

A fault does not necessarily cause an immediately effective error, thereby leading to a system failure. Many experiments have shown that most hardware faults do not cause immediately detectable errors. In fact, it has been found that a significant proportion of the faults injected during the experiments remained latent, i.e. went undetected. In this paper we call this phenomenon the latency problem. An analytical model is developed in the paper to study the effects of the latency problem on fault-tolerant computer systems. Some of the results of our sensitivity analysis are presented and discussed.

Performability modelling tools and techniques

Over the last decade considerable effort has been put in the development of techniques to assess the performance and the dependability of computer and communication systems in an integrated way. This so-called performability modelling becomes especially useful when the system under study can operate partially, which is for instance the case for fault-tolerant computer systems and distributed systems. Modelling techniques are a fundamental prerequisite for actually doing performability analysis. A prerequisite of a more practical but not less important nature is the availability of software tools to support the modelling techniques and to allow system designers to incorporate the new techniques in the design process of systems. Since performability modelling requires many aspects of a system to be specified, high requirements should be posed on performability modelling tools. Moreover, these tools should be structured such that the models can be specified at a level that is easy to understand for a system designer, and that the mathematical aspects are hidden as much as possible. The output of the tool should also be such that it can be understood with only limited knowledge of the underlying mathematical model. We have developed a new, fairly general modelling tool framework that can be used as a guide to assess the usability and structure of performability modelling tools. After briefly reviewing the mathematical aspects of performability modelling we discuss this framework. We then discuss 12 recently developed tools (Metaphor, Numas, Metasan, Metfac, Save, Sharpe, SPNP, Tangram, Penpet, UltraSAN, Surf-2, DyQNtool +) that can all be used for some aspects of performability modelling and analysis. We assess among other things their structure, their capabilities in terms of measures that can be obtained, and the used modelling formalism. We also discuss directions for future work in the field of performability modelling tools.

Hypervisor-based fault tolerance

Protocols to implement a fault-tolerant computing system are described. These protocols augment the hypervisor of a virtual-machine manager and coordinate a primary virtual machine with its backup. No modifications to the hardware, operating system, or application programs are required. A prototype system was constructed for HP's PA-RISC instruction-set architecture. Even though the prototype was not carefully tuned, it ran programs about a factor of 2 slower than a bare machine would.

Fault-tolerant computer system design

Fault-Tolerant Computer System Design by Dhiraj K. Pradhan 550 pp. $72 Prentice Hall Upper Saddle River, N.J. 1996 ISBN 0-13-057887-8

PERFORMABILITY EVALUATION OF FAULT-TOLERANT COMPUTER SYSTEMS USING DYQNTOOL+

For fault-tolerant computer systems (FTCS) supporting critical applications, it is of key importance to be able to answer the question of whether they indeed fulfill the quality of service requirements of their users. In particular, answers related to the combined performance and dependability of the FTCS are important. To facilitate these so-called performability studies, we present DYQNTOOL+, a performability evaluation tool based on the dynamic queuing network concept, that allows for a combined modeling of system performance and dependability. Different from other performability evaluation tools, DYQNTOOL+ combines two different modeling paradigms, i.e., queuing networks and stochastic Petri nets, for respectively the performance and the dependability aspects of the system under study. The mutual relations between these two model parts, such as workload-induced failures and performance decreases due to failures, are explicitly modeled as well. By the above choice for such a combination of modeling paradigms, the modeling can be done in greater detail, thereby often revealing system behavior that cannot be revealed otherwise. We present the dynamic queuing network modeling approach and its implementation in DYQNTOOL+, as well as illustrate its usage by addressing a number of examples.

Fault injection: a method for validating computer-system dependability

A fault tolerant computer system's dependability must be validated to ensure that its redundancy has been correctly implemented and the system will provide the desired level of reliable service. Fault injection-the deliberate insertion of faults into an operational system to determine its response offers an effective solution to this problem. We survey several fault injection studies and discuss tools such as React (Reliable Architecture Characterization Tool) that facilitate its application.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

A combinatorial approach to modeling imperfect coverage

A new algorithm combines a coverage model with a combinatorial model to compute system unreliability. Its advantage is that for a class of computer systems, it is simpler than current algorithms. The method applies to combinatorial models which can generate cutsets for the system. This set of cutsets is augmented with cutsets representing the uncovered failures of the system. The resulting set is manipulated by combining standard multi-state and sum-of-disjoint products solution techniques. It is possible to compute the exact unreliability of the system using this algorithm. If the size of the system and the time required for the analysis become prohibitive, however, the solution can be truncated and bounds on the system unreliability computed. The authors' algorithm is important because it adapts standard combinatorial solution techniques to a problem that was previously thought to require a Markov solution. The ability to model a fault-tolerant computer system completely within a combinatorial model allows results to be calculated more quickly and accurately, and thus to impact system design. This new technology is easily integrated into existing design/analysis methodologies. Coverage provides a more accurate picture of system behavior, and gives more faith in reliability estimates. >

Unified approach to synchronous and asynchronous approximate agreement in the presence of hybrid faults

An important problem in fault-tolerant distributed computer systems is maintaining agreement between nonfaulty processes in the presence of undiagnosed faults. Approximate agreement defines a condition in which it is not necessary for the agreed values to be numerically identical. Rather, processes need only agree with each other to within a predefined numerical tolerance. Convergent voting algorithms which achieve approximate agreement have been studied in the context of two classes of systems, synchronous and asynchronous. Studies have also addressed both completely connected and partially connected systems. Together, the two properties of synchrony and connectivity yield 4 different voting domains. In all studies to date, each voting domain has been treated as a separate problem. This paper: shows that for at least one broad family of voting algorithms, the 4 domains are special cases of a more general convergent voting problem; analyzes convergent voting under the 3-mode hybrid fault model of Thambidurai and Park; and presents a set of unifying relations applicable to all 4 voting domains. These relations are used to specify voting algorithms which optimize fault-tolerance, convergence rate, or computational overhead in any given voting domain. The task of designing a voting algorithm for a particular fault-tolerant system is thus greatly simplified.

FERRARI: a flexible software-based fault and error injection system

A major step toward the development of fault-tolerant computer systems is the validation of the dependability properties of these systems. Fault/error injection has been recognized as a powerful approach to validate the fault tolerance mechanisms of a system and to obtain statistics on parameters such as coverages and latencies. This paper describes the methodology and guidelines for the design of flexible software based fault and error injection and presents a tool, FERRARI, that incorporates the techniques. The techniques used to emulate transient errors and permanent faults in software are described in detail. Experimental results are presented for several error detection techniques, and they demonstrate the effectiveness of the software-based error injection tool in evaluating the dependability properties of complex systems.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Fair distribution of concerns in design and evaluation of fault-tolerant distributed computer systems

In analysing the fault tolerance capabilities of distributed computer system designs, the functionally or physically replaceable components have usually been modelled as one of the two extreme types with respect to their failure symptoms (faulty output behaviour): the fail-silent unit (FSU) model at the simplest end, and the malicious unit (MaU) model, also called the Byzantine unit model, at the other end. The basic weaknesses of these models for use in practical system design and evaluation are pointed out. The FSU model is justifiable for handling most simple parts of a system, but not so for other complex parts. The MaU model is misleading in that it tends to draw attention to events of negligible occurrence probabilities while taking attention away from events of higher occurrence probabilities. It is also pointed out that the state-of-the-art in analytic modelling and evaluation of fault-tolerant distributed computer systems has a vast weakly characterized region in the domain of conceivable component models enclosed by the two extreme models. The main constructive proposition made with respect to advancing the state-of-the-art is to establish scientific procedures for fair distribution of concerns over possible occurrences of anomalous events during system design and validation. A direction for obtaining such a fair modelling procedure relying on extensive probabilistic reasoning is proposed. The principle may have broad applicability, extending much beyond the area of fault-tolerant computing.

The Design of the Liahona and the Purpose of the Second Spindle

Abstract The Liahona was given by the Lord as a communications device for Lehi to determine the appropriate direction of travel. This device contained two pointers, only one of which was necessary to provide directional information. But the Liahona was more than just a simple compass in function, for it additionally required faith for correct operation. Since a single pointer always "points" in some direction, the additional pointer was necessary to indicate whether or not the first pointer could be relied upon. This proposed purpose for the second pointer conforms to a well-established engineering principle used in modern fault-tolerant computer systems called "voting," in which two identical process states are compared and declared correct if they are the same, and incorrect if they are different. Hence the second pointer, when coincident with the first, would indicate proper operation, and when orthogonal, would indicate nonoperation.