Software Failure Probability Research Articles

Demonstrating software reliability using possibly correlated tests: Insights from a conservative Bayesian approach

AbstractThis paper presents Bayesian techniques for conservative claims about software reliability, particularly when evidence suggests the software's executions are not statistically independent. We formalise informal notions of “doubting” that the executions are independent, and incorporate such doubts into reliability assessments. We develop techniques that reveal the extent to which independence assumptions can undermine conservatism in assessments, and identify conditions under which this impact is not significant. These techniques – novel extensions of conservative Bayesian inference (CBI) approaches – give conservative confidence bounds on the software's failure probability per execution. With illustrations in two application areas – nuclear power‐plant safety and autonomous vehicle (AV) safety – our analyses reveals: (1) the confidence an assessor should possess before subjecting a system to operational testing. Otherwise, such testing is futile – favourable operational testing evidence will eventually decrease one's confidence in the system being sufficiently reliable; (2) the independence assumption supports conservative claims sometimes; (3) in some scenarios, observing a system operate without failure gives less confidence in the system than if some failures had been observed; (4) building confidence in a system is very sensitive to failures – each additional failure means significantly more operational testing is required, in order to support a reliability claim.

Software Reliability Prediction using Correlation Constrained Multi-Objective Evolutionary Optimization Algorithm

Software reliability frameworks are extremely effective for estimating the probability of software failure over time. Numerous approaches for predicting software dependability were presented, but neither of those has shown to be effective. Predicting the number of software faults throughout the research and testing phases is a serious problem. As there are several software metrics such as object-oriented design metrics, public and private attributes, methods, previous bug metrics, and software change metrics. Many researchers have identified and performed predictions of software reliability on these metrics. But none of them contributed to identifying relations among these metrics and exploring the most optimal metrics. Therefore, this paper proposed a correlation- constrained multi-objective evolutionary optimization algorithm (CCMOEO) for software reliability prediction. CCMOEO is an effective optimization approach for estimating the variables of popular growth models which consists of reliability. To obtain the highest classification effectiveness, the suggested CCMOEO approach overcomes modeling uncertainties by integrating various metrics with multiple objective functions. The hypothesized models were formulated using evaluation results on five distinct datasets in this research. The prediction was evaluated on seven different machine learning algorithms i.e., linear support vector machine (LSVM), radial support vector machine (RSVM), decision tree, random forest, gradient boosting, k-nearest neighbor, and linear regression. The result analysis shows that random forest achieved better performance.

Research on the software reliability quantitative evaluation of nuclear power plant digital control system based on non-homogeneous poisson process model

With the application of digital technology, the software used in various systems has become increasingly important, as well as complex. This increase in complexity and scale has led to an increase in the probability of software failure. Quantitative reliability evaluation of the digital instrumentation and control system software used in nuclear power plants represent a significant challenge. This study uses a non-homogeneous Poisson process model for quantitative evaluation of the software reliability of a nuclear power plant safety digital control system (DCS), which is the research object. To analyse the research object, a software test environment is established, and the research object is tested to obtain the software failure data. The software reliability growth model, which is widely used in other industries, is combined with data on nuclear power plant DCS software defects that obey a non-homogeneous Poisson process, to construct a non-homogeneous Poisson process model. The formulas for software reliability indicators such as total failures, remaining failures, failure rate (failures per week), reliability, and time to next failure(s) are derived. The software failure data obtained from the test are used to determine the variables of the model based on maximum likelihood estimation; the estimates are then used to quantitatively evaluate software reliability-related indicators. Finally, the minimum correlation error method is used to analyse the error by comparing the actual and model prediction results. The model can evaluate the current reliability level of software and predict the impact of subsequent reliability test results. The findings from this study can provide data support for evaluating the reliability of nuclear power plant DCS software. Therefore, this study can be significant for further research on the reliability of nuclear power plant DCS software.

Bayesian Software Failure Probability based Time-Invariant Tanimoto Random Testing for Software Quality Management

Bayesian Software Failure Probability based Time-Invariant Tanimoto Random Testing for Software Quality Management

Optimization of software development life cycle quality for NPP safety software based on a risk-cost model

As I&C systems in NPPs are being replaced with digital-based systems, the need for cost-effective safety-critical software engineering plans has been received increased attention. This paper proposes a framework for optimizing software development and V&V qualities by incorporating and estimating various risk-cost factors related to SDLC processes. The model determines the optimal software development or V&V quality that gives the minimum value of an objective function which includes three risk-cost factors: 1) software quality control cost; 2) software defect fixing cost; and 3) the cost from NPP accident consequence considering software failure probability. We present a case study using OPR-1000 plant model. The study results showed important SDLC phases where the risk-cost factors can be minimized by achieving high software quality. The proposed method is expected to be useful to software developers and decision makers for assisting NPP software project and quantifying risk from software life cycle on NPP safety.

Tuning lock-based multicore program based on sliding windows to tolerate data race

Because in-house debugging and test are difficult to discover all potential data races in multicore programs, it is necessary and significant to tolerate the potential data races in the production-run phase to secure the correct execution. However, the existing tolerating methods are limited to some kinds of data races. This paper proposes a new data-race tolerating approach, which can detect and adjust the data races whether it is in the protection of critical section or lack of protection to improve the correctness of multicore programs. It uses sliding windows to accommodate the memory instructions in critical section or recent memory instructions lack of protection and detects the potential data races which are more likely to cause errors. Then, by delaying the critical reversion points, data races are adjusted to reduce the probability of software failure. To implement the tolerating approach, the current multicore processor need not change its original cache coherence protocol and just adds very little hardware. Simulation results show that it brings low hardware, low bandwidth overhead, and negligible slowdown.

Development of a Bayesian belief network model for software reliability quantification of digital protection systems in nuclear power plants

As the instrumentation and control (I&C) systems in nuclear power plants (NPPs) have been replaced with digital-based systems, the need has emerged to not only establish a basis for incorporating software behavior into digital I&C system reliability models, but also to quantify the software reliability used in NPP digital protection systems. Therefore, a Bayesian belief network (BBN) model which estimates the number of faults in a software considering its software development life cycle (SDLC) is developed in this study. The model structure and parameters are established based on the information applicable to safety-related systems and expert elicitation. The evidence used in the model was collected from three stages of expert elicitation. To assess the feasibility of using BBN in NPP digital protection software reliability quantification, the BBN model was applied to the Integrated Digital Protection System–Reactor Protection System and estimated the number of defects at each SDLC phase and further assessed the software failure probability. The developed BBN model can be employed to estimate the reliability of deployed safety-related NPP software and such results can be used to evaluate the quality of the digital I&C systems in addition to estimating the potential reactor risk due to software failure.

Input-Domain Software Testing for Failure Probability Estimation of Safety-Critical Applications in Consideration of Past Input Sequence

Software failure probability quantification is an important aspect of digital system reliability assessment. Several quantification methods currently available in the software reliability field have characteristics unsuitable for application to safety-critical software. In this paper, a software test framework in consideration of input trajectory is developed, and a software failure probability quantification method is also suggested. The test input cases consist of the states and present inputs, where input trajectory is represented by the state. To obtain the input domain, which represents realistic plant behavior, digital system characteristics and plant dynamics are considered. This allows software failure probability to be estimated by using the result of each representative test case, thus reducing testing efforts. The proposed framework was applied to a nuclear power plant reactor protection system as an example to show its effectiveness. The method provides a practical and relatively simple way to test software and estimate software failure probability.

Reactor protection system software test-case selection based on input-profile considering concurrent events and uncertainties

Recently, the input-profile-based testing for safety critical software has been proposed for determining the number of test cases and quantifying the failure probability of the software. Input-profile of a reactor protection system (RPS) software is the input which causes activation of the system for emergency shutdown of a reactor. This paper presents a method to determine the input-profile of a RPS software which considers concurrent events/transients. A deviation of a process parameter value begins through an event and increases owing to the concurrent multi-events depending on the correlation of process parameters and severity of incidents. A case of reactor trip caused by feedwater loss and main steam line break is simulated and analyzed to determine the RPS software input-profile and estimate the number of test cases. The different sizes of the main steam line breaks (e.g., small, medium, large break) with total loss of feedwater supply are considered in constructing the input-profile. The uncertainties of the simulation related to the input-profile-based software testing are also included. Our study is expected to provide an option to determine test cases and quantification of RPS software failure probability.

Reliability modeling of safety-critical network communication in a digitalized nuclear power plant

The Engineered Safety Feature-Component Control System (ESF-CCS), which uses a network communication system for the transmission of safety-critical information from group controllers (GCs) to loop controllers (LCs), was recently developed. However, the ESF-CCS has not been applied to nuclear power plants (NPPs) because the network communication failure risk in the ESF-CCS has yet to be fully quantified. Therefore, this study was performed to identify the potential hazardous states for network communication between GCs and LCs and to develop quantification schemes for various network failure causes. To estimate the risk effects of network communication failures in the ESF-CCS, a fault-tree model of an ESF-CCS signal failure in the containment spray actuation signal condition was developed for the case study. Based on a specified range of periodic inspection periods for network modules and the baseline probability of software failure, a sensitivity study was conducted to analyze the risk effect of network failure between GCs and LCs on ESF-CCS signal failure. This study is expected to provide insight into the development of a fault-tree model for network failures in digital I&C systems and the quantification of the risk effects of network failures for safety-critical information transmission in NPPs.

Estimation of reactor protection system software failure probability considering undetected faults

Over the years, the probabilistic safety assessment (PSA) of digital reactor protection systems (RPSs) has been a field of research because of the ambiguity over the mechanism of the software failure and quantification of the failure probability. Currently, the software failure probability in the PSA of a nuclear power plant (NPP) is determined mostly based on expert elicitation. Although a number of software reliability assessment methods are available, none of them appears to be compatible to assess the failure probability of safety grade software in an NPP. Quantification of the software reliability becomes challenging, especially when the software reveals no test failures. This paper proposes an integrated approach for an estimation of the probability of failure on demand of safety critical software considering its undetected residual faults. In the proposed approach, the software reliability can be quantified by incorporating the BBN modeling approach, test results, and operational experiences. A case study is also presented by employing the proposed method. The proposed approach is expected to be helpful for a probabilistic safety analysis of a digital I&C system.

A Simple Model of the Software Failure Rate

In the paper a very simple model of a software failure rate is derived. This model is not intended to compute the failure rate or failure probability of software under realistic conditions. It is rather used to show the influences of measures for software quality assurance and of measures required by software safety standards on this failure rate, i.e. a tendency. In particular, it is shown that testing is important to reduce software failure rate and the effect is the larger, the closer testing is to exhaustive testing. Verification is also important and it needs to be done by an independent person or party to give a good effect. Another result is that software failure rate grows with the size of the software, the frequency the software is used and, the fraction of the software that is used per demand cycle. As a result, it is recommended to implement the measures with more consequence and rigidity, the larger the part of the software that is used, the larger the software itself and the more frequently it is used.

A conservative bound for the probability of failure of a 1-out-of-2 protection system with one hardware-only and one software-based protection train

Redundancy and diversity have long been used as means to obtain high reliability in critical systems. While it is easy to show that, say, a 1-out-of-2 diverse system will be more reliable than each of its two individual “trains”, assessing the actual reliability of such systems can be difficult because the trains cannot be assumed to fail independently. If we cannot claim independence of train failures, the computation of system reliability is difficult, because we would need to know the probability of failure on demand (pfd) for every possible demand. These are unlikely to be known in the case of software. Claims for software often concern its marginalpfd, i.e. average across all possible demands. In this paper we consider the case of a 1-out-of-2 safety protection system in which one train contains software (and hardware), and the other train contains only hardware equipment. We show that a useful upper (i.e. conservative) bound can be obtained for the system pfd using only the unconditional pfd for software together with information about the variation of hardware failure probability across demands, which is likely to be known or estimatable. The worst-case result is obtained by “allocating” software failure probability among demand “classes” so as to maximize system pfd.

Software Failure Probability Assessment by Bayesian Inference

This paper describes a method of estimating the probability of failure for trip-functioning software of a fully digitalized reactor protection system. The Bayesian inference is used to estimate and update the probability of software failure along the software development life cycle. At the requirements and design phases, the probability of software failure is estimated from qualitative quality information based on a specific verification and validation process. This probability of failure is updated at the implementation/testing phases, based on the test data for trip functions implemented by software.

Sensitivity analysis on the effect of software-induced common cause failure probability in the computer-based reactor trip system unavailability

The reactor trip system has been digitized in advanced nuclear power plants, since the programmable nature of computer based systems has a number of advantages over non-programmable systems. However, software is still vulnerable to common cause failure (CCF). Residual software faults represent a CCF concern, which threat the implemented achievements.This study attempts to assess the effectiveness of so-called defensive strategies against software CCF with respect to reliability. Sensitivity analysis has been performed by re-quantifying the models upon changing the software failure probability. Importance measures then have been estimated in order to reveal the specific contribution of software CCF in the trip failure probability.The results reveal the importance and effectiveness of signal and software diversity as applicable strategies to ameliorate inefficiencies due to software CCF in the reactor trip system (RTS). No significant change has been observed in the rate of RTS failure probability for the basic software CCF greater than 1×10−4. However, the related Fussell–Vesley has been greater than 0.005, for the lower values.The study concludes that consideration of risk associated with the software based systems is a multi-variant function which requires compromising among them in more precise and comprehensive studies.

A Comparative Template for Reliable Requirement Specification

ABSTRACT For documenting the groundwork pitch of requirements, many types of requirement specification template are available containing some specific requirement parameter which eventually recites the root requirement for constructing the design, but they are not good enough to make the template reliable requirement specification. In this paper a reliable requirement specification template has been proposed having some reliable parameter which will assess the reliability of the individual requirement before finalizing the requirement documentation which reflects the reliability of requirements at early stage of software development. Keywords Software Reliability, requirement specification 1. INTRODUCTION Many years of the effort have gone into study of hardware reliability particularly in the area of complex electronic assemblies; however the research scholars and practitioners are giving attention to software reliability estimation as the major research area since last two decades. [3] Since the software reliability becomes the most important characteristics regarding computer system so it is necessary to measure and manage the reliability of the computer software, for that purpose many software reliability models have been proposed for the study of software reliability but as we know that most of the models are made for estimating reliability at testing phase which comes into later stage of Software Life Cycle (SLC). The failure of software depends on rooted defects that didn‟t come into face at early stage of requirement gathering which carried on in the further phases of development and testing and these defects will greatly affect the reliability of the software. If the reliable requirements are gathered at initial stage then the probability of software failure (program behavior doesn‟t meet the user‟s requirement within the given time of execution i.e. failure totally depends on the leakage of user oriented concept) will certainly be decrease specifically. For giving weightage to early Requirement Suzanne and James Robertson proposed a „Quality Gateway to find requirements related defects as early as they can be identified and hence prevent them from being incorporated in the design and implementation.‟ For that purpose they have suggested Volere specification template for software specification [1, 2]

AN OVERVIEW OF RISK QUANTIFICATION ISSUES FOR DIGITALIZED NUCLEAR POWER PLANTS USING A STATIC FAULT TREE

Risk caused by safety-critical instrumentation and control (I&C) systems considerably affects overall plant risk. As digitalization of safety-critical systems in nuclear power plants progresses, a risk model of a digitalized safety system is required and must be included in a plant safety model in order to assess this risk effect on the plant. Unique features of a digital system cause some challenges in risk modeling. This article aims at providing an overview of the issues related to the development of a static fault-tree-based risk model. We categorize the complicated issues of digital system probabilistic risk assessment (PRA) into four groups based on their characteristics: hardware module issues, software issues, system issues, and safety function issues. Quantification of the effect of these issues dominates the quality of a developed risk model. Recent research activities for addressing various issues, such as the modeling framework of a software-based system, the software failure probability and the fault coverage of a self monitoring mechanism, are discussed. Although these issues are interrelated and affect each other, the categorized and systematic approach suggested here will provide a proper insight for analyzing risk from a digital system.

Input-profile-based software failure probability quantification for safety signal generation systems

The approaches for software failure probability estimation are mainly based on the results of testing. Test cases represent the inputs, which are encountered in an actual use. The test inputs for the safety-critical application such as a reactor protection system (RPS) of a nuclear power plant are the inputs which cause the activation of protective action such as a reactor trip. A digital system treats inputs from instrumentation sensors as discrete digital values by using an analog-to-digital converter. Input profile must be determined in consideration of these characteristics for effective software failure probability quantification. Another important characteristic of software testing is that we do not have to repeat the test for the same input value since the software response is deterministic for each specific digital input. With these considerations, we propose an effective software testing method for quantifying the failure probability. As an example application, the input profile of the digital RPS is developed based on the typical plant data. The proposed method in this study is expected to provide a simple but realistic mean to quantify the software failure probability based on input profile and system dynamics.

Software Failure Probability Quantification for System Risk Assessment

Risk caused by safety-critical I&C systems considerably affects overall plant risk. Software failures in digitalized I&C systems must be considered as the cause of risk. As digitalization of safety-critical systems progresses, the need for software failure probability quantification increases. For the software of safety-critical systems, very high reliability is required. This article aims at providing an overview of promising software failure probability quantification models for this kind of safety-critical system: The software reliability growth model (SRGM), the input-domain-based test model (IDBT), and the validation/verification quality model (VVQM). In order to accommodate the characteristics of safety-critical systems, a more effective framework of practical risk assessment applications is necessary. In this article, we propose the combined use of SRGM&VVQM for a more systematic and traceable method of the failure probability quantification of safety-critical software.

An analysis of safety-critical digital systems for risk-informed design

This paper quantitatively presents the results of a case study which examines the fault tree analysis framework of the safety of digital systems. The case study is performed for the digital reactor protection system of nuclear power plants. The broader usage of digital equipment in nuclear power plants gives rise to the need for assessing safety and reliability because it plays an important role in proving the safety of a designed system in the nuclear industry. We quantitatively explain the relationship between the important characteristics of digital systems and the PSA result using mathematical expressions. We also demonstrate the effect of critical factors on the system safety by sensitivity study and the result which is quantified using the fault tree method shows that some factors remarkably affect the system safety. They are the common cause failure, the coverage of fault tolerant mechanisms and software failure probability.