Optimal Software Release Time Research Articles

A NHPP Software Reliability Growth Model Considering Learning Process and Number of Residual Faults

A large number of software reliability growth models have been proposed to analyze the reliability of software application during the testing phase. With the increasing demand to deliver high-quality software, more accurate software reliability models are required to estimate the optimal software release time and the cost of testing efforts. In this paper we firstly demonstrate that the G-O model based on NHPP doesn’t need to consider imperfect debugging and new mistakes introduction during the debugging process. Then on the basis of G-O model a flexible and accurate SRGM is proposed, which considers the time-dependent fault detection rate. And in this model the value of fault detection rate is not only calculated with the number of faults remaining in the software system, but also calculated with human’s learning process during testing phase. It is more realistic. Moreover, the experiment results show that the proposed model fits the public failure data better and can provide more accurate software reliability prediction compared with other existing models.

Risk-based software release policy under parameter uncertainty

The determination of the optimal release time is a significant problem in the software development process. Most existing research on this problem is based on the assumption that the model parameters are either known or can be accurately estimated. Due to the uncertainties associated with parameter estimation created by the very limited amount of software failure data that is generally available, the accuracy of the optimum release time determined by traditional approaches is questionable. When the mean value of the optimal release time is used, for example, there is only a 50 per cent chance that the reliability target is met at the time of release. In this paper, an optimal software release policy under parameter uncertainty is studied. To take parameter uncertainty into consideration, an optimal risk-based software release time determination approach is introduced. Application examples are given to illustrate this approach and simulation studies are carried out. The presented results can help management to consider multiple risk levels in order to reach a more reasonable decision.

Analysis of Software Reliability Modeling Considering Testing Compression Factor and Failure-to-Fault Relationship

This paper is an attempt to relax and improve the assumptions regarding software reliability modeling. To approximate reality much more closely, we take into account the concepts of testing compression factor and the quantified ratio of faults to failures in the modeling. Numerical examples based on real failure data show that the proposed framework has a fairly good prediction capability. Further, we also address the optimal software release time problem and conduct a detailed sensitivity analysis through the proposed model.

Editorial: Software testing is an elephant

Editorial: Software testing is an elephant

A Study of Uncertainty in Software Cost and Its Impact on Optimal Software Release Time

For a software development project, management often faces the dilemma of when to stop testing the software and release it for operation, which requires careful decision making as it has great impact on both software reliability and project cost. In most existing research on the optimal software release problem, the cost considered was the Expected Cost (EC) of the project. However, what concerns management is the Actual Cost (AC) of the project rather than the EC. Treatment (such as minimization) of the EC may not ensure the desired low level of the AC due to the uncertainty (variability) involved in the AC. In this paper, we study the uncertainty in software cost and its impact on optimal software release time in detail. The uncertainty is quantified by the variance of the AC and several risk functions. A risk-control approach to the optimal software release problem is proposed. New formulations of the problem which are extensions of current formulations are developed and solution procedures are established. Several examples are presented. Results reveal that it seems crucial to take into account the uncertainty in software cost in the optimal software release problem; otherwise, unsafe decisions may be reached which could be a false dawn to management.

The determination of optimal software release times at different confidence levels with consideration of learning effects

AbstractAs most software reliability models do not clearly explain the variance in the mean value function of cumulative software errors, they might not be effective in deducing the confidence interval regarding the mean value function. In such cases, software developers cannot estimate the possible risk variation in software reliability by using the randomness of the mean value function, thus reducing the decision‐making reliability when determining an optimal software release time. In this paper, the method of stochastic differential equations is used to build a software reliability model, which is validated based on practical data previously used in six published papers. Moreover, the estimation of the parameters of the proposed model, which can be defined as the autonomous error‐detected factor and the learning factor, is also illustrated, and the results of model validation empirically confirm that the proposed model is able to account for a fairly large portion of the variance of the mean value function. Additionally, the confidence intervals of the mean value function regarding software faults are employed to assist software developers in determining the optimal release times at different confidence levels. Finally, a numerical example is given to verify the effectiveness of the proposed model. Copyright © 2008 John Wiley & Sons, Ltd.

Enhancing and measuring the predictive capabilities of testing-effort dependent software reliability models

Software testing is necessary to accomplish highly reliable software systems. If the project manager can conduct well-planned testing activities, the consumption of related testing-resources will be cost-effective. Over the past 30 years, many software reliability growth models (SRGMs) have been proposed to estimate the reliability growth of software, and they are mostly applicable to the late stages of testing in software development. Thus far, it appears that most SRGMs do not take possible changes of testing-effort consumption rates into consideration. However, in some cases, the policies of testing-resource allocation could be changed or adjusted. Thus, in this paper, we will incorporate the important concept of multiple change-points into Weibull-type testing-effort functions. The applicability and performance of the proposed models are demonstrated through two real data sets. Experimental results show that the proposed models give a fairly accurate prediction capability. Finally, based on the proposed SRGM, constructive rules are developed for determining optimal software release times.

Reliability growth modeling and optimal release policy under fuzzy environment of an N-version programming system incorporating the effect of fault removal efficiency

Failure of a safety critical system can lead to big losses. Very high software reliability is required for automating the working of systems such as aircraft controller and nuclear reactor controller software systems. Fault-tolerant softwares are used to increase the overall reliability of software systems. Fault tolerance is achieved using the fault-tolerant schemes such as fault recovery (recovery block scheme), fault masking (N-version programming (NVP)) or a combination of both (Hybrid scheme). These softwares incorporate the ability of system survival even on a failure. Many researchers in the field of software engineering have done excellent work to study the reliability of fault-tolerant systems. Most of them consider the stable system reliability. Few attempts have been made in reliability modeling to study the reliability growth for an NVP system. Recently, a model was proposed to analyze the reliability growth of an NVP system incorporating the effect of fault removal efficiency. In this model, a proportion of the number of failures is assumed to be a measure of fault generation while an appropriate measure of fault generation should be the proportion of faults removed. In this paper, we first propose a testing efficiency model incorporating the effect of imperfect fault debugging and error generation. Using this model, a software reliability growth model (SRGM) is developed to model the reliability growth of an NVP system. The proposed model is useful for practical applications and can provide the measures of debugging effectiveness and additional workload or skilled professional required. It is very important for a developer to determine the optimal release time of the software to improve its performance in terms of competition and cost. In this paper, we also formulate the optimal software release time problem for a 3VP system under fuzzy environment and discuss a the fuzzy optimization technique for solving the problem with a numerical illustration.

Incorporating Fault Debugging Activities Into Software ReliabilityModels: A Simulation Approach

A large number of software reliability growth models have been proposed to analyse the reliability of a software application based on the failure data collected during the testing phase of the application. To ensure analytical tractability, most of these models are based on simplifying assumptions of instantaneous & perfect debugging. As a result, the estimates of the residual number of faults, failure rate, reliability, and optimal software release time obtained from these models tend to be optimistic. To obtain realistic estimates, it is desirable that the assumptions of instantaneous & perfect debugging be amended. In this paper we discuss the various policies according to which debugging may be conducted. We then describe a rate-based simulation framework to incorporate explicit debugging activities, which may be conducted according to the different debugging policies, into software reliability growth models. The simulation framework can also consider the possibility of imperfect debugging in conjunction with any of the debugging policies. Further, we also present a technique to compute the failure rate, and the reliability of the software, taking into consideration explicit debugging. An economic cost model to determine the optimal software release time in the presence of debugging activities is also described. We illustrate the potential of the simulation framework using two case studies.

SOFTWARE RELIABILITY ISSUES UNDER OPERATIONAL AND TESTING CONSTRAINTS

Software reliability plays an important role in assuring the quality of a software. To ensure software reliability, the software is tested thoroughly during the testing phase. The time invested in the testing phase or the optimal software release time depends on the level of reliability to be achieved. There are two different concepts related to software reliability, viz., testing reliability and operational reliability. In this paper, we compare both types of software reliabilities to determine the optimal testing time of the software so as to minimize the total expected software maintenance cost. We consider a software has a number of clusters of modules, each having a different number of errors and a different failure rate. A hyperexponential model is employed for analyzing software reliability growth. Parameter estimation using the maximum likelihood estimation technique is also discussed. Numerical illustrations are taken to explore the effect of various parameters on reliability and maintenance cost. It is noticed that the operational reliability concept should be adopted for the software testing time problem.

Sensitivity analysis in optimal software release time problems

In the modeling of software reliability and in the decision-making using software reliability models, there are a number of parameters whose values are either assumed or estimated based on early data. These parameters are usually inaccurate and thus it is important to study the sensitivity of the estimation. In this paper, the parameter sensitivity problem is studied from two different points of view. First, in addition to point estimation that is commonly used in data analysis, interval estimation is recommended to quantify the errors in the estimated parameters. Second, with the example of sensitivity analysis for the release time determination problem that usually involves both software reliability model parameters and software cost model parameters, a statistical design of experiments technique is employed. This method can not only estimate the effect of separate change of each model parameter, but also estimate the interaction effects when parameters are changed simultaneously. It is useful information for decision maker so that important parameters can be estimated more accurately.

A study of the effect of imperfect debugging on software development cost

It is widely recognized that the debugging processes are usually imperfect. Software faults are not completely removed because of the difficulty in locating them or because new faults might be introduced. Hence, it is of great importance to investigate the effect of the imperfect debugging on software development cost, which, in turn, might affect the optimal software release time or operational budget. In this paper, a commonly used cost model is extended to the case of imperfect debugging. Based on this, the effect of imperfect debugging is studied. As the probability of perfect debugging, termed testing level here, is expensive to be increased, but manageable to a certain extent with additional resources, a model incorporating this situation is presented. Moreover, the problem of determining the optimal testing level is considered. This is useful when the decisions regarding the test team composition, testing strategy, etc., are to be made for more effective testing.

Determination of the optimal software release time based on proportional hazards software reliability growth models

The software reliability models to describe the reliability growth phenomenon are formulated by any stochastic point process with state‐dependent or time‐dependent intensity function. On the other hand, to deal with the environmental data, which consists of covariates influencing times to software failure, it may be useful to apply the Cox’s proportional hazards model for assessing the software reliability. In this paper, we review the proportional hazards software reliability models and discuss the problem to determine the optimal software release time under the expected total software cost criterion. Numerical examples are devoted to examine the dependence of the covariate structure in both the software reliability prediction and the optimal software release decision.

A software cost model with warranty and risk costs

In this paper, a cost model with warranty cost, time to remove each error detected in the software system, and risk cost due to software failure is developed. A software reliability model based on non-homogeneous Poisson process is used. The optimal release policies to minimize the expected total software cost are discussed. A software tool is also developed using Excel and Visual Basic to facilitate the task of determining the optimal software release time. Numerical examples are provided to illustrate the results.

A software cost model with warranty cost, error removal times and risk costs

In this paper, a software cost model with a warranty period and cost, a cost to remove each error detected in the software and a risk cost due to software failure is developed. A software reliability model based on the nonhomogeneous Poisson process is used. The optimal release policies which minimize the expected total software costs are presented. A software tool is also developed using Excel and Visual Basic that facilitates the task of determining the optimal software release time. Numerical examples are provided to illustrate the results.

An optimal software release problem under cost rate criterion: artificial neural network approach

The determination of the release schedule for a new software product is the most important issue for designing and controlling a software development process. In fact, the optimal software release problem based on some software reliability growth models has been studied by many authors. In this paper, we propose a new method to estimate the optimal software release time under an alternative cost criterion. More precisely, two kinds of artificial neural networks are used to estimate the fault‐detection time observed in both testing and operation phases. As a cost criterion, we adopt the expected cost rate (the expected total software cost per unit testing time). Then, it is shown that the optimization problem to obtain the optimal release time can be reduced to a graphical one to minimize the tangent slope from a point to an (estimated) empirical curve in two‐dimensional space. Through numerical examples using actual fault‐detection time data, it is illustrated that the method proposed is a very useful device to estimate the optimal software release time precisely.

A software cost model with error removal times and risk costs

A cost model subject to times to remove errors in the software system and risk cost due to software failure is developed. A software reliability model based on the non-homogeneous Poisson process is used. The optimal release policies to minimize the expected total software cost are discussed. A software is also developed using Excel and Visual Basic tools to facilitate the task of determining the optimal software release time. Numerical examples are provided to illustrate the results.

Optimal Software Release Policies with Debugging Time Lag

This paper presents a new stochastic model for determining the optimal release time for a computer software in testing phase, taking account of the debugging time lag. In the earlier works, most of software release models were considered, but it was assumed that an error detected can be removed instantaneously. In other words, none discussed quantitatively the effect of the software maintenance action in the optimal software release time. Main purpose of this work is to relate the optimal software release policy with the arrival-service process on the software operation phase by users. We use the Non-Homogeneous Poisson Process (NHPP) type of software reliability growth models as the software error detection phenomena and obtain the optimal software release policies minimizing the expected total software costs. As a result, the usage circumstance of a software in operation phase gives a monotone effect to the software release planning.

Optimal release policy for hyper-geometric distribution software-reliability growth model

The hyper-geometric distribution software-reliability growth model (HGDM) can estimate the number of initial faults in a software program. An important problem in software development is to determine when to stop testing and then release the software. This paper mainly investigates the optimal software release policies which minimize the mean total software cost and satisfy the software-reliability requirement based on the HGDM. The optimal software release times are determined and shown to be finite. A numerical example illustrates these optimal software release policies.

A dynamic software release model

The determination of optimal software release times constitutes an interesting decision making problem which involves the stochastic structure of the underlying software reliability model, as well as various cost parameters. There is an apparent tradeoff between testing the software further to improve its reliability, and releasing it for operational use to decrease the costs. We propose and analyze in depth a new dynamic model with sufficient generality. After each failure, a debugging activity, possibly imperfect, is undertaken and a decision is made regarding the duration of additional testing. If no failure is observed during this time, then the software is released. Otherwise, the failure is debugged and the decision process is repeated in a dynamic fashion. The problem is formulated using dynamic programming and interesting characterizations of the optimal release policy are presented. The dynamic solution procedure is demonstrated by some numerical illustrations.