Software Fault Count Data Research Articles

Are Infinite-Failure NHPP-Based Software Reliability Models Useful?

In the literature, infinite-failure software reliability models (SRMs), such as Musa-Okumoto SRM (1984), have been demonstrated to be effective in quantitatively characterizing software testing processes and assessing software reliability. This paper primarily focuses on the infinite-failure (type-II) non-homogeneous Poisson process (NHPP)-based SRMs and evaluates the performances of these SRMs comprehensively by comparing with the existing finite-failure (type-I) NHPP-based SRMs. In more specific terms, to describe the software fault-detection time distribution, we postulate 11 representative probability distribution functions that can be categorized into the generalized exponential distribution family and the extreme-value distribution family. Then, we compare the goodness-of-fit and predictive performances with the associated 11 type-I and type-II NHPP-based SRMs. In numerical experiments, we analyze software fault-count data, collected from 16 actual development projects, which are commonly known in the software industry as fault-count time-domain data and fault-count time-interval data (group data). The maximum likelihood method is utilized to estimate the model parameters in both NHPP-based SRMs. In a comparison of the type-I with the type-II, it is shown that the type-II NHPP-based SRMs could exhibit better predictive performance than the existing type-I NHPP-based SRMs, especially in the early stage of software testing.

Burr-type NHPP-based software reliability models and their applications with two type of fault count data

In this paper, we summarize the so-called Burr-type software reliability models (SRMs) based on the non-homogeneous Poisson process (NHPP) and comprehensively evaluate the model performances by comparing them with the existing NHPP-based SRMs. Two kinds of software fault count data are considered; fault-detection time-domain data and fault-detection time-interval data (group data). For 8 data sets in each fault count type, we estimate the model parameters by means of the maximum likelihood estimation and evaluate the performance metrics in terms of goodness-of-fit and prediction. It is shown that the Burr-type NHPP-based SRMs could show the better performances than the existing NHPP-based SRMs in many cases. The main contribution of the paper consists in suggesting that the Burr-type NHPP-based SRMs should be the possible candidates for selecting the best SRM in terms of goodness-of-fit and predictive performances.

Open Source Software Reliability Model with the Decreasing Trend of Fault Detection Rate

Abstract Software reliability assessment methods have been changed from closed to open source software (OSS). Although numerous new approaches for improving OSS reliability are formulated, they are not used in practice due to their inaccuracy. A new proposed model considering the decreasing trend of fault detection rate is developed in this study to effectively improve OSS reliability. We analyse the changes of the instantaneous fault detection rate over time by using real-world software fault count data from two actual OSS projects, namely, Apache and GNOME, to validate the proposed model performance. Results show that the proposed model with the decreasing trend of fault detection rate has better fitting and predictive performance than the traditional closed source software and other OSS reliability models. The proposed model for OSS can further accurately fit and predict the failure process and thus can assist in improving the quality of OSS systems in real-world OSS projects.

Optimal Release Time Estimation of Software System using Box-Cox Transformation and Neural Network

The determination of the software release time for a new software product is the most critical issue for designing and controlling software development processes. This paper presents an innovative technique to predict the optimal software release time using a neural network. In our approach, a three-layer perceptron neural network with multiple outputs is used, where the underlying software fault count data are transformed into the Gaussian data by means of the well-known Box-Cox power transformation. Then the prediction of the optimal software release time, which minimizes the expected software cost, is carried out using the neural network. Numerical examples with four actual software fault count data sets are presented, where we compare our approach with conventional Non-Homogeneous Poisson Process (NHPP) -based Software Reliability Growth Models (SRGMs).

A Neuro-Based Software Fault Prediction with Box-Cox Power Transformation

Software fault prediction is one of the most fundamental but significant management techniques in software dependability assessment. In this paper we concern the software fault prediction using a multilayer-perceptron neural network, where the underlying software fault count data are transformed to the Gaussian data, by means of the well-known Box-Cox power transformation. More specially, we investigate the long-term behavior of software fault counts by the neural network, and perform the multi-stage look ahead prediction of the cumulative number of software faults detected in the future software testing. In numerical examples with two actual software fault data sets, we compare our neural network approach with the existing software reliability growth models based on nonhomogeneous Poisson process, in terms of predictive performance with average relative error, and show that the data transformation employed in this paper leads to an improvement in prediction accuracy.

An Analysis of Software Reliability Assessment with Neuro-Fuzzy based Expert Systems

Abstract We propose software reliability assessment methods by using neuro-fuzzy based systems and their effectiveness in assessing the Software Reliability. Also, we make a comparison between the neural networks based software reliability growth model and the fuzzy logic based software reliability growth models based on a homogeneous Poisson process applied to software reliability assessment of the entire system composed of several software components. Moreover, we analyze software fault count data to show numerical examples of software reliability assessment with the implementation of Neuro-Fuzzy systems based approach. Furthermore, we investigate the performance of an efficient software reliability assessment methods in this context. Also, we had shown the implementation of the approach by using Java programming language with some programs. We used the normalized root mean square error (NRMSE) as evaluation criteria. The experiments show that the non-parametric models are superior when compared to the parametric models in their ability to provide an accurate estimate when historical data is missing. A comparison among the neural network and fuzzy logic models are provided.

Interval Estimation Method for Decision Making in Wavelet-Based Software Reliability Assessment

Recently, the wavelet-based estimation method has gradually been becoming popular as a new tool for software reliability assessment. The wavelet transform possesses both spatial and temporal resolution which makes the wavelet-based estimation method powerful in extracting necessary information from observed software fault data, in global and local points of view at the same time. This enables us to estimate the software reliability measures in higher accuracy. However, in the existing works, only the point estimation of the wavelet-based approach was focused, where the underlying stochastic process to describe the software-fault detection phenomena was modeled by a non-homogeneous Poisson process. In this paper, we propose an interval estimation method for the wavelet-based approach, aiming at taking account of uncertainty which was left out of consideration in point estimation. More specifically, we employ the simulation-based bootstrap method, and derive the confidence intervals of software reliability measures such as the software intensity function and the expected cumulative number of software faults. To this end, we extend the well-known thinning algorithm for the purpose of generating multiple sample data from one set of software-fault count data. The results of numerical analysis with real software fault data make it clear that, our proposal is a decision support method which enables the practitioners to do flexible decision making in software development project management.

Estimating Software Intensity Function Based on Translation-Invariant Poisson Smoothing Approach

Because the software failure occurrence process is well-modeled by a non-homogeneous Poisson process, it is of great interest to estimate accurately the software intensity function of Non-Homogeneous Poisson Process (NHPP)-based software reliability models (SRM) from observed software-fault count data. In recent years, wavelet-based techniques have been well established in Poisson intensity estimation because of their technical advantages of computational cost and accuracy. The approach enables us to carry out the analysis of a software debugging process in a nonparametric way. In this paper, we propose an applied Haar wavelet-based approach which is without an approximate data transformation, for software reliability assessment. In a numerical study with real software-fault count data, we compare the proposed estimation method with the previously used data transformation-based estimation method, as well as conventional maximum likelihood estimation and least squares estimation methods. Furthermore, we conduct sensitivity analysis of the resolution level, which affects the estimation accuracy of the proposed method. We also estimate some predictive measures such as software reliability.

NONPARAMETRIC BOOTSTRAP INTERVAL ESTIMATION OF SOFTWARE RELIABILITY AND ITS APPLICATION TO AN OPTIMAL RELEASE PROBLEM BASED ON A DISCRETIZED MODEL

We discuss a simulation-based interval estimation method of software reliability and cost-optimal software release time based on a nonparametric bootstrap method via a discretized software reliability growth model. In general, it is very difficult to derive probability distributions of model parameters, software reliability assessment measures, and optimal software release time analytically even if we assume an asymptotic theory assuming a large number of samples. The bootstrap method overcomes the problem above by obtaining the probability distributions of the parameter based on the simulation-based approach. This paper includes numerical examples for confirming that our bootstrap approach yields simulation-based probability distributions of model parameters, software reliability assessment measures, and cost-optimal software release time from actual software fault-count data.

Wavelet Shrinkage Estimation for Non-Homogeneous Poisson Process Based Software Reliability Models

We develop a novel estimation approach for quantitative software reliability by means of wavelet-based technique, where the underlying software reliability model is described by a non-homogeneous Poisson process. Our approach involves some advantages over the commonly used techniques such as maximum likelihood estimation: 1) the wavelet shrinkage estimation enables us to carry out the time-series analysis with high speed and accuracy requirements; and 2) The wavelet shrinkage estimation is classified into a non-parametric estimation without specifying a parametric form of the software intensity function. We consider data-transform-based wavelet shrinkage estimation with four kinds of thresholding schemes for empirical wavelet coefficients to estimate the software intensity function. In numerical experiments with real software-fault count data, we show that our wavelet-based estimation methods can provide better goodness-of-fit performance than not only the conventional maximum likelihood estimation and least squares estimation but also the local likelihood estimation method, in many cases, in spite of their non-parametric nature. Furthermore, we investigate the predictive performance of the proposed methods by employing the so-called one-stage look-ahead prediction method, and estimate some predictive measures such as software reliability.

Reliability modeling and analysis for open source cloud computing

At present, cloud computing is attracting attention as a network service to share the computing resources, such as networks, servers, storage, applications, and services. Also, many open source softwares are developed in all parts of the world, e.g. Firefox, Apache HTTP server, Linux, Android, etc. This article focuses on a cloud computing environment by using open source software such as OpenStack and Eucalyptus because of the unify management of data and low cost. This article proposes a new approach to software reliability assessment based on the stochastic differential equations and hierarchical Bayesian modeling in order to consider the interesting aspect of the network status of a cloud computing environment. Also, actual software fault-count data to show numerical examples of software reliability assessment considering the characteristics of network environment is analyzed.

A Comparative Study of Data Transformations for Wavelet Shrinkage Estimation with Application to Software Reliability Assessment

In our previous work, we proposed wavelet shrinkage estimation (WSE) for nonhomogeneous Poisson process (NHPP)-based software reliability models (SRMs), where WSE is a data-transform-based nonparametric estimation method. Among many variance-stabilizing data transformations, the Anscombe transform and the Fisz transform were employed. We have shown that it could provide higher goodness-of-fit performance than the conventional maximum likelihood estimation (MLE) and the least squares estimation (LSE) in many cases, in spite of its non-parametric nature, through numerical experiments with real software-fault count data. With the aim of improving the estimation accuracy of WSE, in this paper we introduce other three data transformations to preprocess the software-fault count data and investigate the influence of different data transformations to the estimation accuracy of WSE through goodness-of-fit test.

Component-Oriented Reliability Analysis Based on Hierarchical Bayesian Model for an Open Source Software

The successful experience of adopting distributed development models in such open source projects includes GNU/Linux operating system, Apache HTTP server, Android, BusyBox, and so on. The open source project contains special features so-called software composition by which several geographically-dispersed compo-nents are developed in all parts of the world. We propose a method of component-oriented reliability as-sessment based on hierarchical Bayesian model and Markov chain Monte Carlo methods. Especially, we fo-cus on the fault-detection rate for each component reported to the bug tracking system. We can assess the reliability for the whole open source software system by using the confidence interval for each component. Also, we analyze actual software fault-count data to show numerical examples of reliability assessment for OSS.

Performance evaluation of reliability assessment method based on stochastic differential equation model for a large-scale open source solution

At present, many open source softwares are developed in all parts of the world, i.e, Firefox, Apache HTTP server, Linux, Android, etc. Network technologies have made rapid progress with the dissemination of computer systems in all areas. These network technologies become increasingly more complex in a wide sphere. Especially, open source software systems which serve as key components of critical infrastructures in the society are still ever-expanding now. At present, a large scale open source solution by using several open source softwares is now attracting attention as the next-generation software development paradigm. In this paper, we propose a new approach to software reliability assessment by using the stochastic differential equations. Also, we analyze actual software fault-count data to show numerical examples of software reliability assessment considering component collision for several open source softwares. Especially, we show several results of performance evaluation for sensitivity analysis in terms of our model parameters. Moreover, we compare our model with the conventional model based on stochastic differential equations in terms of goodness-of-fit for actual data. We show that the proposed model can assist improvement of quality for the large scale open source solution.

Measuring reliability growth of software by considering fault dependency, debugging time Lag functions and irregular fluctuation

The progress of software testing is influenced by various uncertainty factors like effort expenditure, skill of test personal, testing tool, defect density, irregular state of open source project, and irregular state of software fault-report phenomena on the bug tracking system. Hence, there is an irregular fluctuation in fault detection/removal rate during testing phase. In software, the independence of failures can hardly be assumed and dependency of faults can also be considered as one of the factor for getting irregular fluctuation. In Literature, various software reliability growth models have been developed by considering fault dependency with various debugging time lag functions. But, none of the models have incorporated irregular fluctuation in their fault detection rate. Therefore, in this paper fault dependency based software reliability growth models have been developed by applying an It ∧o type Stochastic Differential Equations in order to incorporate (i) the irregular fluctuation in the fault detection process due to various uncertainty factor during testing phase and (ii) irregular state of software fault-report phenomena on the bug tracking system. The proposed stochastic differential equation based fault dependency models have been validated using (i) open source software fault count data where software fault-report phenomena on the bug tracking system keep an irregular state and (ii) a fault counting data with minor, major and critical faults. The proposed models have been compared with the existing fault dependency models. Various comparison criteria results for goodness of fit have also been presented in the paper.

AN EMBEDDED OSS RELIABILITY AND OPTIMIZATION ANALYSIS INCORPORATING IMPERFECT DEBUGGING

As a result of the technological progress, software development environment has changed into development paradigm based on client/server systems by using network computing technologies. Network technologies have made rapid progress with the dissemination of computer systems in all areas. These network technologies become increasingly more complex in a wide sphere. Especially, open source software systems which serve as key components of critical infrastructures in the society are still ever-expanding now. In this paper, we propose a method of software reliability assessment based on stochastic differential equations. Especially, we derive several assessment measures in terms of imperfect debugging. Also, we analyze actual software fault-count data to show numerical examples of software reliability assessment for an embedded open source software. Further, it has been necessary to manage the software development process in terms of reliability, effort, and release time. Then, we find the optimal release time based on the total expected software maintenance effort.

An Empirical Study of Stochastic Differential Equation Models Based on Component Importance Level for Open Source Software

Network technologies become increasingly more complex in a wide sphere. Especially, open source software systems which serve as key components of critical infrastructures in the society are still ever-expanding now. In this paper, we propose a new approach to software reliability assessment by creating a fusion of neural network and stochastic differential equations based on component importance levels. Also, we analyze actual software fault-count data to show numerical examples of software reliability assessment considering component importance levels for an open source software. Moreover, we compare the goodness-of-fit of the proposed models with the conventional software reliability growth model for OSS.

A COMPONENT-ORIENTED RELIABILITY ASSESSMENT METHOD FOR OPEN SOURCE SOFTWARE

Software development environment has been changing into new development paradigms such as concurrent distributed development environment and the so-called open source project by using network computing technologies. Especially, an OSS (open source software) system which serves as key components of critical infrastructures in the society is still ever-expanding now. In case of considering the effect of the debugging process on an entire system in the development of a method of reliability assessment for the OSS, it is necessary to grasp the deeply-intertwined factors, such as programming path, size of each component, skill of fault reporter, and so on. In order to consider the effect of each software component on the reliability of an entire system, we propose a new approach to user-oriented software reliability assessment by creating a fusion of neural network and software reliability growth modeling. In this paper, we show application examples of component-oriented software reliability assessment based on neural network and software reliability growth modeling for the OSS. Also, we analyze actual software fault count data to show numerical examples of software reliability assessment for the OSS. Moreover, we develop the testing management tool for OSS.

Comparison of Software Reliability Assessment Methods for Open Source Software and Reliability Assessment Tool

We propose software reliability assessment methods for concurrent distributed system development by using the Analytic Hierarchy Process. Also, we make a comparison between the inflection S-shaped software reliability growth model and the other models based on a nonhomogeneous Poisson process applied to reliability assessment of the entire system composed of several software components. Moreover, we analyze software fault count data to show numerical examples of software reliability assessment for the open source project. Furthermore, we investigate an efficient software reliability assessment method for the open source system development. Also, we develop the software testing-management tool for open source software system by using Java programming language. We show several performance examples of the software testing-management tool for the open source software.

A Software Reliability Assessment Tool for Distributed Software Development Projects

At present, software development environment has been changing into new development paradigm. Moreover, a software development paradigm based on an open source project is rapidly spreading. In this paper, we propose a software reliability assessment tool incorporating flexible software reliability growth models based on a nonhomogeneous Poisson process and stochastic differential equations for distributed development environment by using Java programming language. Especially, we incorporate the method of reliability assessment for open source software development into the software reliability assessment tool. Also, we analyze actual software fault count data to show numerical examples of software reliability assessment for the open source project by using the tool.