Software Reliability Growth Research Articles

Optimal time for management review during testing process: an approach using S-curve two-dimensional software reliability growth model

PurposeThe software development team reviews the testing phase to assess if the reliability growth of software is as per plan and requirement and gives suggestions for improvement. The objective of this study is to determine the optimal review time such that there is enough time to make judgments about changes required before the scheduled release.Design/methodology/approachTesting utilizes majority of time and resources, assures reliability and plays a critical role in release and warranty decision-making reviews necessary. A very early review during testing may not give useful information for analyzing or improving project performance, and a very late review may delay product delivery and lead to opportunity loss for developers. Therefore, it is assumed that the optimal time for review is in the later stage of testing when the fault removal rate starts to decline. The expression for this time point is determined using the S-curve 2-D software reliability growth model (SRGM).FindingsThe methodology has been illustrated using the real-life fault datasets of Tandem computers and radar systems resulting in optimal review time of 14 weeks and 26 months, respectively, which is neither very early in testing nor very near to the scheduled release. The developer can make changes (more resources or postpone release) to expedite the process.Originality/valueMost of the literature studies focus on determination of optimal testing or release time to achieve considerable reliability within the budget, but in this study, the authors determine the optimal review time during testing using SRGM to ensure the considerable reliability at release.

Optimum bug fixing rate and bug fixing time detection by software reliability modelling

PurposeSoftware reliability growth models (SRGMs) are used to assess and predict reliability of a software system. Many of these models are effective in predicting future failures unless the software evolves.Design/methodology/approachThis objective of this paper is to identify the best path for rectifying the BFT (bug fixing time) and BFR (bug fixing rate). Moreover, the flexible software project has been examined while materializing the BFR. To enhance the BFR, the traceability of bug is lessened by the version tag virtue in every software deliverable component. The release time of software build is optimized with the utilization of mathematical optimization mechanisms like ‘software reliability growth’ and ‘non-homogeneous Poisson process methods.’FindingsIn current market scenario, this is most essential. The automation and variation of build is also resolved in this contribution. Here, the software, which is developed, is free from the bugs or defects and enhances the quality of software by increasing the BFR.Originality/valueIn current market scenario, this is most essential. The automation and variation of build is also resolved in this contribution. Here, the software, which is developed, is free from the bugs or defects and enhances the quality of software by increasing the BFR.

Discrete Time-Space Stochastic Mathematical Modelling for Quantitative Description of Software Imperfect Fault-Debugging with Change-Point

Statistics and stochastic-process theories, along with the mathematical modelling and the respective empirical evidence support, describe the software fault-debugging phenomenon. In software-reliability engineering literature, stochastic mathemat-ical models based on the non-homogeneous Poisson process (NHPP) are employed to measure and boost reliability too. Since reliability evolves on account of the running of computer test-run, NHPP type of discrete time-space models, or difference-equation, is superior to their continuous time-space counterparts. The majority of these models assume either a constant, monotonically increasing, or decreasing fault-debugging rate under an imperfect fault-debugging environment. However, in the most debugging scenario, a sudden change may occur to the fault-debugging rate due to an addition to, deletion from, or modification of the source code. Thus, the fault-debugging rate may not always be smooth and is subject to change at some point in time called change-point. Significantly few studies have addressed the problem of change-point in discrete-time modelling approach. The paper examines the combined effects of change-point and imperfect fault-debugging with the learning process on software-reliability growth phenomena based on the NHPP type of discrete time-space modelling approach. The performance of the proposed modelling approach is compared with other existing approaches on an actual software-reliability dataset cited in literature. The findings reveal that incorporating the effect of change-point in software-reliability growth modelling enhances the accuracy of software-reliability assessment because the stochastic character-istics of the software fault-debugging phenomenon alter at the change-point.

Finding the best bug fixing rate and bug fixing time using software reliability modelling

This article is mainly focused on finding the best possible way to rectify bug fixing rate (BFR) and bug fixing time (BFT). Further, versatile software projects have been verified when materialising the bug fixing rate. To increase the bug fixing rate, bug traceability is reduced by virtue of version tag in each and every component of a software deliverable. Software build release time is optimised by using mathematical optimisation techniques such as software reliability growth and non-homogeneous Poisson process models. This is very much essential in the present market scenario. The build inconsistency and automation are also rectified in this erudite research work. The developed software is free from defects and improves the software quality by increasing bug fixing rate.

Finding the best bug fixing rate and bug fixing time using software reliability modelling

This article is mainly focused on finding the best possible way to rectify bug fixing rate (BFR) and bug fixing time (BFT). Further, versatile software projects have been verified when materialising the bug fixing rate. To increase the bug fixing rate, bug traceability is reduced by virtue of version tag in each and every component of a software deliverable. Software build release time is optimised by using mathematical optimisation techniques such as software reliability growth and non-homogeneous Poisson process models. This is very much essential in the present market scenario. The build inconsistency and automation are also rectified in this erudite research work. The developed software is free from defects and improves the software quality by increasing bug fixing rate.

Lognormal Process Software Reliability Modeling with Testing-Effort

We propose a software reliability growth model with testing-effort based on a continuous-state space stochastic process, such as a lognormal process, and conduct its goodness-of-fit evaluation. We also discuss a parameter estimation method of our model. Then, we derive several software reliability assessment measures by the probability distribution of its solution process, and compare our model with existing continuous-state space software reliability growth models in terms of the mean square error and the Akaike’s information criterion by using actual fault count data.

On Way to Acquiring Reliability Growth in Software Systems

Reliability of a software system has been one of the driving forces for the various software engineering processes and methodologies that led to their evolution and sophistication. The concerns for reliability of a software system surface very early on during the development phases of the software system. When it comes to acquisition of reliability, we should not immediately get model-oriented; instead every minutia of software development life cycle should be given its due. This paper outlines the areas where reliability needs to pick up. This paper underlines the fact that it is only through carefully controlled and carefully applied software engineering process that software reliability growth can be achieved. It emphasizes on the acquisition of reliability of software systems as soon as the conception phase. We trace the reliability concerns from the early stage to the fully functional stage. The terms “hard reliability” and “soft reliability” are used.

Bayesian updating of optimal release time for software systems

In this paper, a Bayesian method dealing with software reliability growth with consideration of the learning effect is proposed to determine an optimal release time for software systems with regard to the testing cost and experts' prior judgments. Such an approach is able to devise an appropriate software-debugging scheme which has the best arrangement of available resources and personnel with a minimal software testing cost when lacking sufficient information for decision making. Past research on software reliability emphasized the estimation of the number of cumulative software errors or the software reliability with respect to a specific time period, yet it neglected the determination of software release time with consideration of the software testing cost, meaning that existing approaches are not entirely practical. Accordingly, the proposed method is concerned with the evaluation of the software testing cost incurred during the testing period based on experts' prior judgments and the software testing data collected within a given duration, and is thus characterized by its practicality as well as meaningfulness with consideration of the learning effect. Finally, a numerical example is given to verify the effectiveness of the proposed approach, and sensitivity and risk analyses are performed on this example.

Response Modeling Methodology Validating Evidence from Engineering and the Sciences

AbstractModeling a response in terms of the factors that affect it is often required in quality applications. While the normal scenario is commonly assumed in such modeling efforts, leading to the application of linear regression analysis, there are cases when the assumptions underlying this scenario are not valid and alternative approaches need to be pursued, like the normalization of the data or generalized linear modeling. Recently, a new response modeling methodology (RMM) has been introduced, which seems to be a natural generalization of various current scientific and engineering mainstream models, where a monotone convex (concave) relationship between the response and the affecting factor (or a linear combination of factors) may be assumed. The purpose of this paper is to provide the quality practitioner with a survey of these models and demonstrate how they can be derived as special cases of the new RMM. A major implication of this survey is that RMM can be considered a valid approach for quality engineering modeling and, thus, may be conveniently applied where theory‐based models are not available or the goodness‐of‐fit of current empirically‐derived models is unsatisfactory. A numerical example demonstrates the application of the new RMM to software reliability‐growth modeling. The behavior of the new model when the systematic variation vanishes (there is only random variation) is also briefly explored. Copyright © 2003 John Wiley & Sons, Ltd.

Software reliability growth with test coverage

"Software test-coverage measures" quantify the degree of thoroughness of testing. Tools are now available that measure test-coverage in terms of blocks, branches, computation-uses, predicate-uses, etc. that are covered. This paper models the relations among testing time, coverage, and reliability. An LE (logarithmic-exponential) model is presented that relates testing effort to test coverage (block, branch, computation-use, or predicate-use). The model is based on the hypothesis that the enumerable elements (like branches or blocks) for any coverage measure have various probabilities of being exercised; just like defects have various probabilities of being encountered. This model allows relating a test-coverage measure directly with defect-coverage. The model is fitted to 4 data-sets for programs with real defects. In the model, defect coverage can predict the time to next failure. The LE model can eliminate variables like test-application strategy from consideration. It is suitable for high reliability applications where automatic (or manual) test generation is used to cover enumerables which have not yet been tested. The data-sets used suggest the potential of the proposed model. The model is simple and easily explained, and thus can be suitable for industrial use. The LE model is based on the time-based logarithmic software-reliability growth model. It considers that: at 100% coverage for a given enumerable, all defects might not yet have been found.

A software-reliability growth model for N-version programming systems

This paper presents a NHPP-based SRGM (software reliability growth model) for NVP (N-version programming) systems (NVP-SRGM) based on the NHPP (nonhomogeneous Poisson process). Although many papers have been devoted to modeling NVP-system reliability, most of them consider only the stable reliability, i.e., they do not consider the reliability growth in NVP systems due to continuous removal of faults from software versions. The model in this paper is the first reliability-growth model for NVP systems which considers the error-introduction rate and the error-removal efficiency. During testing and debugging, when a software fault is found, a debugging effort is devoted to remove this fault. Due to the high complexity of the software, this fault might not be successfully removed, and new faults might be introduced into the software. By applying a generalized NHPP model into the NVP system, a new NVP-SRGM is established, in which the multi-version coincident failures are well modeled. A simplified software control logic for a water-reservoir control system illustrates how to apply this new software reliability model. The s-confidence bounds are provided for system-reliability estimation. This software reliability model can be used to evaluate the reliability and to predict the performance of NVP systems. More application is needed to validate fully the proposed NVP-SRGM for quantifying the reliability of fault-tolerant software systems in a general industrial setting. As the first model of its kind in NVP reliability-growth modeling, the proposed NVP SRGM can be used to overcome the shortcomings of the independent reliability model. It predicts the system reliability more accurately than the independent model and can be used to help determine when to stop testing, which is a key question in the testing and debugging phase of the NVP system-development life cycle.

Effect of code coverage on software reliability measurement

Existing software reliability-growth models often over-estimate the reliability of a given program. Empirical studies suggest that the over-estimations exist because the models do not account for the nature of the testing. Every testing technique has a limit to its ability to reveal faults in a given system. Thus, as testing continues in its region of saturation, no more faults are discovered and inaccurate reliability-growth phenomena are predicted from the models. This paper presents a technique intended to solve this problem, using both time and code coverage measures for the prediction of software failures in operation. Coverage information collected during testing is used only to consider the effective portion of the test data. Execution time between test cases, which neither increases code coverage nor causes a failure, is reduced by a parameterized factor. Experiments were conducted to evaluate this technique, on a program created in a simulated environment with simulated faults, and on two industrial systems that contained tenths of ordinary faults. Two well-known reliability models, Goel-Okumoto and Musa-Okumoto, were applied to both the raw data and to the data adjusted using this technique. Results show that over-estimation of reliability is properly corrected in the cases studied. This new approach has potential, not only to achieve more accurate applications of software reliability models, but to reveal effective ways of conducting software testing.

Reliability analysis of systems based on software and human resources

Safety-critical systems require an assessment activity to verify that they are able to perform their functions in specified use environments. This activity benefits from evaluation methods that consider these systems as a whole and not as the simple sum of their parts. Indeed, analysis of accidents involving such systems has shown that they are rarely due to the simple failure of one of their components. Accidents are the outcome of a composite causal scenario where human, software, and hardware failures combine in a complex pattern. Unfortunately, dependability analysis and evaluation of safety critical systems are usually based on techniques and methods that consider human and computer separately, and whose results can hardly be integrated. The analogies between the processes of: (1) software-reliability growth due to testing and the related fault removal; (2) improvement of man-machine interface due to preliminary operative feedback; and (3) improvement of operator performance due to learning activity; all suggest a common evaluation approach. Only the first one of these processes is currently modeled using mathematical methods. This paper extends these methods to study the reliability-growth process of other system components: operator and man-machine interfaces. To study the feasibility of the approach, this paper analyzes the results of an experiment in which the reliability of a system is evaluated using trend analysis and reliability-growth models. The evaluation concerns the graphic man-machine interface and the operators, and could easily be extended to the software control system. The experimental results show that trend analysis and reliability-growth models could be complementary to the qualitative evaluation performed within the cognitive science approach. They could offer a quantitative support especially when their information is based on analysis of average values. In this case they could assist several decisions during operator training and especially during interface design, when comparing the effect of different possible interfaces on operator behavior. Moreover they can support the share of the same tools and of the related know-how between the fields of human and software dependability.

Bayes inference for S-shaped software-reliability growth models

Bayes inference for a nonhomogeneous Poisson process with an S-shaped mean value function is studied. In particular, the authors consider the model of Ohba et al. (1983), and its generalization to a class of gamma distribution growth curves. Two Gibbs sampling approaches are proposed to compute the Bayes estimates of the mean number of errors remaining and the current system reliability. One algorithm is a Metropolis within Gibbs algorithm, The other is a stochastic substitution algorithm with data augmentation. Model selection based on the posterior Bayes factor is studied. A numerical example with simulated data is given.

Reliability growth in the probability and possibility contexts

In this paper we focus on the software reliability growth. All reliability characteristics are described in terms of probability and possibility measures. The mathematical models of the reliability growth are based on the systems of integral and functional equations.

Software-reliability growth model: primary-failures generate secondary-faults under imperfect debugging : Z. Zeephongsekul, G. Xia and S. Kumar. IEEE Transactions on Reliability, 43, 3, 408 (September 1994)

Software-reliability growth model: primary-failures generate secondary-faults under imperfect debugging : Z. Zeephongsekul, G. Xia and S. Kumar. IEEE Transactions on Reliability, 43, 3, 408 (September 1994)

Sensitivity of reliability-growth models to operational profile errors vs. testing accuracy [software testing

This paper investigates: 1) the sensitivity of reliability-growth models to errors in the estimate of the operational profile (OP); and 2) the relation between this sensitivity and the testing accuracy for computer software. The investigation is based on the results of a case study in which several reliability-growth models are applied during the testing phase of a software system. The faults contained in the system are known in advance; this allows measurement of the software reliability-growth and comparison with the estimates provided by the models. Measurement and comparison are repeated for various OPs, thus giving information about the effect of a possible error in the estimate of the OP. The results show that: 1) the predictive accuracy of the models is not heavily affected by errors in the estimate of the OP; and 2) this relation depends on the accuracy with which the software system has been tested.

Optimal release policy for hyper-geometric distribution software-reliability growth model

The hyper-geometric distribution software-reliability growth model (HGDM) can estimate the number of initial faults in a software program. An important problem in software development is to determine when to stop testing and then release the software. This paper mainly investigates the optimal software release policies which minimize the mean total software cost and satisfy the software-reliability requirement based on the HGDM. The optimal software release times are determined and shown to be finite. A numerical example illustrates these optimal software release policies.

Use of failure-intensity models in the software-validation phase for telecommunications

Telephone switching systems require the use of more complex and bulky software. It is therefore important for France Telecom to take an interest in software quality before the operational phase. Trend tests show the evolution of detected failures during the validation phase. When these tests show a tendency towards improvement, we can apply software-reliability growth models. This paper describes experiments conducted at CNET, Lannion A Center, concerning the use of failure-intensity models in the software-validation phase of 3 telecommunication products. The experiments show that the logarithmic model is the most accurate. This paper also addresses future perspectives in data collection and use of reliability models.

Reliability growth of fault-tolerant software

Reliability growth of fault-tolerant software