Traditional Software Reliability Growth Models Research Articles

Recursive approach for multiple step-ahead software fault prediction through long short-term memory (LSTM)

The advancement of technologies demands a sustainable solution. To ensure the software system’s sustainability, diminishing the software faults before the implementation requires utmost attention, along with an effective procedure to predict the faults. A software system’s maximum number of faults can be neutralized if it can be predicted at the earliest possible time. Therefore, we applied Long short-term memory (LSTM) to predict the faults of multi-time stamps ahead using a recursive approach. The Min-Max scaler and one of the power transformation methods, Box-Cox are used to normalize the software fault data. The traditional software reliability growth models (SRGMs) are also used to predict faults. The performance of the LSTM and SRGMs models are compared based on their prediction accuracy evaluation. The observed prediction error of LSTM models is much lower than the SRGMs.

Software Reliability Prediction through Encoder-Decoder Recurrent Neural Networks

With the growing demand for high reliability and safety software, software reliability prediction has attracted more and more attention to identifying potential faults in software. Software reliability growth models (SRGMs) are the most commonly used prediction models in practical software reliability engineering. However, their unrealistic assumptions and environment-dependent applicability restrict their development. Recurrent neural networks (RNNs), such as the long short-term memory (LSTM), provide an end-to-end learning method, have shown a remarkable ability in time-series forecasting and can be used to solve the above problem for software reliability prediction. In this paper, we present an attention-based encoder-decoder RNN called EDRNN to predict the number of failures in the software. More specifically, the encoder-decoder RNN estimates the cumulative faults with the fault detection time as input. The attention mechanism improves the prediction accuracy in the encoder-decoder architecture. Experimental results demonstrate that our proposed model outperforms other traditional SRGMs and neural network-based models in terms of accuracy.

Deep Cross-Project Software Reliability Growth Model Using Project Similarity-Based Clustering

Software reliability is an essential characteristic for ensuring the qualities of software products. Predicting the potential number of bugs from the beginning of a development project allows practitioners to make the appropriate decisions regarding testing activities. In the initial development phases, applying traditional software reliability growth models (SRGMs) with limited past data does not always provide reliable prediction result for decision making. To overcome this, herein, we propose a new software reliability modeling method called a deep cross-project software reliability growth model (DC-SRGM). DC-SRGM is a cross-project prediction method that uses features of previous projects’ data through project similarity. Specifically, the proposed method applies cluster-based project selection for the training data source and modeling by a deep learning method. Experiments involving 15 real datasets from a company and 11 open source software datasets show that DC-SRGM can more precisely describe the reliability of ongoing development projects than existing traditional SRGMs and the LSTM model.

Simulated Software Testing Process and Its Optimization Considering Heterogeneous Debuggers and Release Time

Most traditional software reliability growth models (SRGMs) assume immediate fault correction upon detection and therefore only consider fault detection process (FDP). In order to be more realistic, some researchers have tried to incorporate fault correction process (FCP) and fault introduction process (FIP) into the software reliability models. However, it is still difficult to incorporate into the analytical software reliability models some other factors, such as the different fault detection and correction capabilities of debuggers. In this paper, a simulation approach is proposed to model FDP, FIP, and FCP together considering debuggers with different contributions to fault detection rate, different fault correction rate and different fault introduction rate. Besides, this paper also constructed a cost calculation method to optimize the testing design including debuggers assignment and software release time. Some numerical examples are provided to illustrate the proposed model. The results show that the trends of FDP, FCP and FIP are consistent with the intuition to the practice of software testing, and the optimal testing resources allocation and the optimal release time can be obtained according to the proposed model.

Optimal test activity allocation for covariate software reliability and security models

Traditional software reliability growth models enable quantitative assessment of the software testing process by characterizing the fault detection in terms of testing time or effort. However, the majority of these models do not identify specific testing activities underlying fault discovery and thus can only provide limited guidance on how to incrementally allocate effort. Although there are several novel studies focused on covariate software reliability growth models, they are limited to model development, application, and assessment.This paper presents a non-homogeneous Poisson process software reliability growth model incorporating covariates based on the discrete Cox proportional hazards model. An efficient and stable expectation conditional maximization algorithm is applied to identify the model parameters. An optimal test activity allocation problem is formulated to maximize fault discovery. The proposed method is illustrated through numerical examples on two data sets.

Entropy Based Software Reliability Analysis of Multi-Version Open Source Software

The number of issues fixed in the current release of the software is one of the factors which decides the next release of the software. The source code files get changed during fixing of these issues. The uncertainty arises due to these changes is quantified using entropy based measures. We developed a Non-Homogeneous Poisson Process model for Open Source Software to understand the fixing of issues across releases. Based on this model, optimal release-updating using entropy and maximizing the active user's satisfaction level subject to fixing of issues up to a desired level, is investigated as well. The proposed models have been validated on five products of the Apache open source project. The optimal release time estimated from the proposed model is close to the observed release time at different active user's satisfaction levels. The proposed decision model can assist management to appropriately determine the optimal release-update time. The proposed entropy based model for issues estimation shows improvement in performance for 21 releases out of total 23 releases, when compared with well-known traditional software reliability growth models, namely GO model <xref ref-type="bibr" rid="ref1" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">[1]</xref> and S-shaped model <xref ref-type="bibr" rid="ref2" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">[2]</xref> . The proposed model is also found statistically significant.

Improving software reliability estimation using multi-layer neural-network combination model

Software reliability growth models (SRGM) can be used to derive quantitative information about the quality of the product and also to estimate the software testing time needed to achieve target reliability. The modelling performance of SRGMs depends on the nature of the failure data set. In this paper, we propose a multi-layer feed-forward artificial neural network (ANN) combination model using two widely used generalised SRGMs as base models with back-propagation training algorithm. The proposed ANN-based model is compared with the traditional SRGMs and a few ANN based software reliability models. The performance comparison seems to confirm that, the goodness of fit and the predictive validity of proposed model appears to be better than that of reliability models compared. The proposed ANN-based model provides consistent performance for both exponential, and S-shaped growth of mean value function witnessed in software projects.

NHPP-based software reliability model considering testing effort and multivariate fault detection rate

In recent decades, many software reliability growth models (SRGMs) have been proposed for the engineers and testers in measuring the software reliability precisely. Most of them is established based on the non-homogeneous Poisson process (NHPP), and it is proved that the prediction accuracy of such models could be improved by adding the describing of characterization of testing effort. However, some research work indicates that the fault detection rate (FDR) is another key factor affects final software quality. Most early NHPP-based models deal with the FDR as constant or piecewise function, which does not fit the different testing stages well. Thus, this paper first incorporates a multivariate function of FDR, which is bathtub-shaped, into the NHPP-based SRGMs considering testing effort in order to further improve performance. A new model framework is proposed, and a stepwise method is used to apply the framework with real data sets to find the optimal model. Experimental studies show that the obtained new model can provide better performance of fitting and prediction compared with other traditional SRGMs.

Toward a Unified Approach to Software Reliability Modeling under Imperfect Debugging

Many software reliability growth models (SRGMs) have been developed to estimate some useful measures such as the mean value function, number of remaining faults, and failure detection rate. Most of these models have focused on the failure detection process and not given equal priority to modeling the fault correction process. But, most latent software errors may remain uncorrected for a long time even after they are detected, which increases their impact. The remaining software faults are often one of the most unreliable reasons for software quality. Therefore, we develop a general framework of the modeling of the failure detection and fault correction processes. Furthermore, it is assumed that a detected fault is immediately removed and is perfectly repaired with no new faults being introduced for the traditional SRGMs. In reality, it is impossible to remove all faults from the fault correction process and have a fault-free effect on the software development environment. In order to relax this perfect debugging assumption, we introduce the possibility of imperfect debugging phenomenon. Finally, numerical examples are shown to illustrate the results of the unified approach for integration of the detection and correction process under imperfect debugging.

Software, Hardware, and Procedure Reliability by Testing and Verification: Evidence of Learning Trends

Software, hardware, and procedure reliability modeling and prediction have enjoyed ample attention by the research community over the past decades. However, significantly less research has been dedicated to studying the contribution of the human element of reliability engineering. This paper applies several mathematical models from the theory of learning in search of evidence for learning trends during the testing phase. To demonstrate the utility of the approach, learning models such as the Stevens-Savin equation are applied to several widely studied datasets from the historical literature. In several cases, no evidence of learning was detected, which suggests that additional faults may have eluded testers and been included in the final product. We also discuss how models that are based on learning theory can be used to predict the number of failures remaining. The results illustrate that learning models can predict the number of failures remaining in software more accurately than traditional software reliability growth models that were based on the nonhomogeneous Poisson process. Thus, the proposed techniques produce predictions like existing reliability models, but also offer additional guidance to managers concerned with the thoroughness of product and procedure testing.

AN NHPP SOFTWARE RELIABILITY GROWTH MODEL WITH IMPERFECT DEBUGGING AND ERROR GENERATION

In this paper, we propose a non-homogeneous Poisson process (NHPP) based software reliability growth model (SRGM) in the presence of modified imperfect debugging and fault generation phenomenon. The testing team may not be able to remove a fault perfectly on observation of a failure due to the complexity of software systems and incomplete understanding of software, and the original fault may remain, or get replaced by another fault causing error generation. We have proposed an exponentially increasing fault content function and constant fault detection rate. The total fault content of the software for our proposed model increases rapidly at the beginning of the testing process. It grows gradually at the end of the testing process because of increasing efficiency of the testing team with testing time. We use the maximum likelihood estimation method to estimate the unknown parameters of the proposed model. The applicability of our proposed model and comparisons with established models in terms of goodness of fit and predictive validity have been presented using five known software failure data sets. Experimental results show that the proposed model gives a better fit to the real failure data sets and predicts the future behavior of software development more accurately than the traditional SRGMs.

Software reliability analysis and assessment using queueing models with multiple change-points

Over the past three decades, many software reliability growth models (SRGMs) have been proposed, and they can be used to predict and estimate software reliability. One common assumption of these conventional SRGMs is to assume that detected faults will be removed immediately. In reality, this assumption may not be reasonable and may not always occur. During debugging, developers need time to reproduce the failure, identify the root causes of faults, fix them, and then re-run the software. From some experiments or observations, the fault correction rate may not be a constant and could be changed at certain points as time proceeds. Consequently, in this paper, we will investigate and study how to apply queueing models to describe the fault detection and correction processes during software development. We propose an extended infinite server queueing model with multiple change-points to predict and assess software reliability. Experimental results based on real failure data show that the proposed model can depict the change of fault correction rates and predict the behavior of software development more accurately than traditional SRGMs.

Software reliability modeling based on grey system theory

Grey theory is applied to the study of software reliability modeling.Through modeling with history failure data,the next-step failure time was predicted.To verify the accuracy of the Grey Model(GM),the authors applied it to a group of generally accepted data,and compared the results with that of traditional software reliability growth models by computing the SRE measure.The experimental results show that in most cases,the GM ranks first as far as the short-term predictability is concerned.

Better reliability assessment and prediction through data clustering

This paper presents a new approach to software reliability modeling by grouping data into clusters of homogeneous failure intensities. This series of data clusters associated with different time segments can be directly used as a piecewise linear model for reliability assessment and problem identification, which can produce meaningful results early in the testing process. The dual model fits traditional software reliability growth models (SRGMs) to these grouped data to provide long-term reliability assessments and predictions. These models were evaluated in the testing of two large software systems from IBM. Compared with existing SRGMs fitted to raw data, our models are generally more stable over time and produce more consistent and accurate reliability assessments and predictions.

Software reliability prediction incorporating information from a similar project

Although there are many models for the prediction of software reliability using the failure data collected during testing, the estimation is usually inaccurate, especially at the early stages of the testing phase, and hence many practitioners are hesitant to use software reliability models. On the other hand, the traditional software reliability growth models do not make use of information from earlier or similar projects. For example, software systems today are usually an improvement or modification of an earlier version or at least within the same application domain, which implies that some information should be available from similar projects. In this paper we study some approaches for the estimation of software reliability by incorporating information from a similar project. In particular, we use the Goel–Okumoto model and assume the same value of the fault detection rate. The other parameter is then estimated based on the available testing data. For an actual set of data, our approach provides much more stable estimates and when the traditional maximum likelihood estimates exist and are reasonable, our results are very close to that from a statistical point of view. In addition, our approach does not require numerical algorithm to update the estimate and hence it is convenient to use.