Software Reliability Assessment Research Articles

Developing an Innovative Imperfect Debugging Software Reliability Growth Model with Enhanced Testing Coverage Strategies

In this paper, we introduce an innovative Software Reliability Growth Model (SRGM) designed to tackle the pivotal challenges associated with software reliability in the contemporary digital landscape, where the prevalence of online systems is ubiquitous. This SRGM integrates Imperfect Debugging (ID), Testing Coverage (TC), Testing Effort (TE), and error generation into a cohesive framework. Employing a sigmoid function to encapsulate TE, it incorporates three distinct TC functions: Delayed S-shaped, Exponential, and Logistic. This model relies on foundational assumptions, including the proportionality of fault detection rates to remaining faults, the introduction of new faults during debugging, and the intricate connection between fault detection and code coverage. The Mean Value Function (MVF) is computed through these differential equations, and the resultant MVFs are systematically tabulated for all models. An examination of the sigmoid TE function and the Weibull TE function across diverse datasets, utilizing a range of goodness-of-fit criteria including Mean Square Error (MSE), Pham’s Criterion (PC), Predictive Risk Ratio (PRR), Bayesian Information Criterion (BIC), and Akaike’s Information Criterion (AIC), reveals the superior performance of the sigmoid TE function over the Weibull counterpart across various datasets and evaluation criteria. In conclusion, this paper introduces a groundbreaking SRGM that seamlessly integrates ID, TC, and TE, offering valuable insights for assessing software reliability in the dynamic landscape of modern digital systems.

Long-term software fault prediction with wavelet shrinkage estimation

Wavelet shrinkage estimation received considerable attentions to estimate stochastic processes such as a non-homogeneous Poisson process in a non-parametric way, and was applied to software reliability estimation/prediction. However, it lacks the prediction ability for unknown future patterns in long term and penalizes assessing the software reliability in practice. In this paper, we focus on the long-term prediction of the number of software faults detected in the testing phase and propose many novel long-term prediction methods based on the wavelet shrinkage estimation. The fundamental idea is to adopt both the denoised fault-count data and prediction values, and to minimize several kinds of loss functions to make effective predictions. We also develop an automated wavelet-based software reliability assessment tool, W-SRAT2, which is a drastic extension of the existing tool, W-SRAT, by adding those prediction algorithms. In numerical experiments with 6 actual software development project data, we investigate the predictive performance of our long-term prediction approaches, which consist of 2,640 combinations, and compare them with the common software reliability growth models with the maximum likelihood estimation. It is shown that our wavelet shrinkage estimation/prediction methods outperform the existing software reliability growth models.

An updated software reliability model using the shanker model and failure data

AbstractSoftware developers' goal is to develop reliable and superior software. Due to the fact that software errors frequently generate large societal or financial losses, software reliability is essential. Software reliability growth models are a widely used technique for software reliability assessment. This study examines various nonhomogeneous Poisson process models with the newly developed software reliability distribution and evaluates the unknown model parameters based on frequentist and Bayesian methods of estimation. Finally, we conduct evaluations on real datasets using a variety of evaluation criteria to compare the results of previous software reliability growth models and show how the proposed model may be applied under both approaches in a practical setting. According to this study, the innovative model's mean square error, R2, , bias, predicted relative variation, Theil statistic, and mean error of prediction values show the lowest values under the Bayesian approach for data sets II to IV, and both approaches perform well for data set I. These implementation findings demonstrate the effectiveness of our specific approach based on our examination of failure data.

Software reliability assessment: An architectural and component impact analysis

Software reliability assessment: An architectural and component impact analysis

Nonhomogeneous Markov Process Modeling for Software Reliability Assessment

Nonhomogeneous Markov Process Modeling for Software Reliability Assessment

Deep Learning Based on Fine Tuning with Application to the Reliability Assessment of Similar Open Source Software

Recently, many open-source products have been used under the situations of general software development, because the cost saving and standardization. Therefore, many open-source products are gathering attention from many software development companies. Then, the reliability/quality of open-source products becomes very important factor for the software development. This paper focuses on the reliability/quality evaluation of open-source products. In particular, the large quantity fault data sets recorded on Bugzilla of open-source products is used in many open-source development projects. Then, the large amount of data sets of software faults is recorded on the Bugzilla. This paper proposes the reliability/quality evaluation approach based on the deep machine learning by using the large quantity fault data on the Bugzilla. Moreover, the large quantity fault data sets are analyzed by the deep machine learning based on the fine-tuning.

Testing coverage based software reliability assessment incorporating effort expenditure and error generation

Testing coverage based software reliability assessment incorporating effort expenditure and error generation

Modeling the Role of Testing Coverage in the Software Reliability Assessment

To assure the reliability and quality of the final product, testing is an essential and crucial part in the software development cycle. During this process, fault correction/detection activities are carried out to increase the reliability of the software. The non-homogeneous Poisson Process (NHPP) is the basis of the investigated software reliability growth models (SRGMs), which are based on the supposition that the number of faults found is affected by the amount of code covered during testing and that the amount of code covered during testing depends on the testing effort expended. This research takes into consideration several testing coverage functions: exponential, delayed S-shaped and logistic distributions, to propose three SRGMs that are based on testing efforts. For testing effort expenditure Weibull distribution has been employed. Two real failure datasets have been utilised to validate the proposed models, and their performance is evaluated using four goodness-of-fit metrics, including predictive ratio risk (PRR), coefficient of determination (R^2 ), predictive power (PP) and mean square error (MSE). Sensitivity analysis of cost requirement-based release time of software for exponential function has been done by using a genetic algorithm, which minimized the overall cost of the software subject to the requirement for reliability.

Open-source Software Reliability Modeling with Stochastic Impulsive Differential Equations

In reality, sudden updates of software, attacks of hackers, influence of the Internet market, etc. can cause a surge in the number of open-source software (OSS) faults (this moment is the time when impulse occurs), which results in impulsive phenomenon. For the existing software reliability models, dynamic process of software fault is considered to be continuous when assessing reliability, but continuity of the process can be disrupted with appearance of random impulses. Thus, to more accurately assess software reliability, we proposed an OSS reliability model with SIDE. In the model, dynamic process of software fault is divided into a continuous and a skipped part, described the continuous part of the process with SDE, and described destruction of the continuity caused by unpredictable random events with random impulses. Finally, the proposed model is verified with two datasets from real OSS project, and the results show that the proposed model is more in line with reality and has better fitting effect than the existing models.

A Bayesian model averaging method for software reliability modeling and assessment

AbstractThe software reliability modeling is of great significance in improving software quality and managing the software development process. However, the existing methods are not able to accurately model software reliability improvement behavior because existing single model methods rely on restrictive assumptions and combination models cannot well deal with model uncertainties. In this article, we propose a Bayesian model averaging (BMA) method to model software reliability. First, the existing reliability modeling methods are selected as the candidate models, and the Bayesian theory is used to obtain the posterior probabilities of each reliability model. Then, the posterior probabilities are used as weights to average the candidate models. Both Markov Chain Monte Carlo (MCMC) algorithm and the Expectation–Maximization (EM) algorithm are used to evaluate a candidate model's posterior probability and for comparison purpose. The results show that the BMA method has superior performance in software reliability modeling, and the MCMC algorithm performs better than EM algorithm when they are used to estimate the parameters of BMA method.

Development of a Method for Software Reliability Assessment using Neural Networks

Software reliability growth models (SRGMs) are used to estimate future failure rates and the number of residual defects in software. SRGM helps reliability software engineers make test termination decisions. Although more than 250 traditional SRGMs have been proposed for reliability assessment, research to develop more reliable models is still ongoing. Recently, new methods based on neural networks have been developed to improve the accuracy of software reliability assessment.The article develops a neural network algorithm for assessing software reliability. To do this, factors affecting software reliability and covering its life cycle were assessed. Based on them, sets for training a three-layer neural network were created.

Multi-release software reliability assessment: testing coverage-based approach

Software development companies continue to improve their products to stay up with the market's growing needs by adding new features and fixing previously identified bugs. Software reliability growth models (SRGMs) with testing effort function are the incredibly valuable and have been widely utilised by software engineers. Many researchers have built SRGMs that incorporate the concept testing coverage in the model building for the multi-release software system. The proposed model intends to give three different models of multi-release software reliability modelling with testing coverage function. Three models describe testing coverage by exponential function, delayed S-shaped function and logistic function respectively, with the testing effort function assumed to be Weibull in nature. Real-world data is used to estimate parameters in order to validate the proposed SRGM of four releases from Tandem Computers and the model goodness-of-fit is assessed. The results suggest that the proposed model matches the failure data effectively.

Are Infinite-Failure NHPP-Based Software Reliability Models Useful?

In the literature, infinite-failure software reliability models (SRMs), such as Musa-Okumoto SRM (1984), have been demonstrated to be effective in quantitatively characterizing software testing processes and assessing software reliability. This paper primarily focuses on the infinite-failure (type-II) non-homogeneous Poisson process (NHPP)-based SRMs and evaluates the performances of these SRMs comprehensively by comparing with the existing finite-failure (type-I) NHPP-based SRMs. In more specific terms, to describe the software fault-detection time distribution, we postulate 11 representative probability distribution functions that can be categorized into the generalized exponential distribution family and the extreme-value distribution family. Then, we compare the goodness-of-fit and predictive performances with the associated 11 type-I and type-II NHPP-based SRMs. In numerical experiments, we analyze software fault-count data, collected from 16 actual development projects, which are commonly known in the software industry as fault-count time-domain data and fault-count time-interval data (group data). The maximum likelihood method is utilized to estimate the model parameters in both NHPP-based SRMs. In a comparison of the type-I with the type-II, it is shown that the type-II NHPP-based SRMs could exhibit better predictive performance than the existing type-I NHPP-based SRMs, especially in the early stage of software testing.

CAFI: A Configurable location-Aware Fault Injection technique for software reliability assessment against soft errors

The rapid development of processor manufacturing technologies has encouraged most designers to highlight the processors' resilience to errors. Regarding embedded systems in harsh environments, temporary faults, such as single-event upsets (SEUs) or single-event multiple upsets (SEMUs), induced by radiation effects on a memory cell or a combinational logic circuit, can cause perturbations in the system's behavior, which in turn can cause catastrophic consequences. In this context, Fault Injection (FI) has been successfully applied as a mature method to assess reliability against faults and reveal the system's deficiencies to be protected cost-effectively. Although high-level software FI techniques are less accurate, the high accuracy of fault injection at a low level usually comes at the expense of desirable characteristics related to portability, flexibility, and intrusiveness. Hence, this paper proposes a Configurable location-Aware Fault Injection (CAFI) technique. CAFI is a software-based technique that facilitates emulating different kinds of transient hardware faults, e.g., SEUs and SEMUs, at the software level. Therefore, it supports flexibility in conducting reliability assessment studies with varying fault models. It operates at the binary code level and exploits a timing-based mechanism to inject faults at run-time in a negligible intrusive manner. CAFI can inject faults at different granularity levels to maximize fault activation. In detail, a fine-grained into a specific instruction field and a coarse-grained into the whole system's software. CAFI requires negligible modifications to the target software under test and allows it to run at near-native speed. The effectiveness of CAFI is evaluated by conducting many fault injection experiments applying to different real-world benchmark programs. Moreover, the accuracy of CAFI is quantified with respect to high-level software fault injection. For this aim, a practical prototype of CAFI is implemented on the x86 architecture employing an Intel core i7 processor with 16GB RAM. Based on a total of 108,000 fault injection experiments, the rate of program crashes caused by CAFI is slightly more than 18% higher than that caused by high-level software fault injection. On the other hand, there are no significant differences between the silent data corruptions (SDCs), results obtained by CAFI, and high-level software fault injections, making CAFI applicable for studying crash- and SDC-causing errors.

A Comprehensive Analysis of Proportional Intensity-Based Software Reliability Models with Covariates

This paper focuses on the so-called proportional intensity-based software reliability models (PI-SRMs), which are extensions of the common non-homogeneous Poisson process (NHPP)-based SRMs, and describe the probabilistic behavior of software fault-detection process by incorporating the time-dependent software metrics data observed in the development process. The PI-SRM is proposed by Rinsaka et al. in the paper “PISRAT: Proportional Intensity-Based Software Reliability Assessment Tool” in 2006. Specifically, we generalize this seminal model by introducing eleven well-known fault-detection time distributions, and investigate their goodness-of-fit and predictive performances. In numerical illustrations with four data sets collected in real software development projects, we utilize the maximum likelihood estimation to estimate model parameters with three time-dependent covariates (test execution time, failure identification work, and computer time-failure identification), and examine the performances of our PI-SRMs in comparison with the existing NHPP-based SRMs without covariates. It is shown that our PI-STMs could give better goodness-of-fit and predictive performances in many cases.

Reliability Assessment for Open-Source Software Using Deterministic and Probabilistic Models

Nowadays, computer software plays a significant role in all fields of our life. Essentially open-source software provides economic benefits for software companies such that it allows building new software without the need to create it from scratch. Therefore, it is extremely used, and accordingly, open-source software’s quality is a critical issue and one of the top research directions in the literature. In the development cycles of the software, checking the software reliability is an important indicator to release software or not. The deterministic and probabilistic models are the two main categories of models used to assess software reliability. In this paper, we perform a comparative study between eight different software reliability models: two deterministic models, and six probabilistic models based on three different methodologies: perfect debugging, imperfect debugging, and Gompertz distribution. We evaluate the employed models using three versions of a standard open-source dataset which is GNU’s Not Unix Network Object Model Environment projects. We evaluate the employed models using four evaluation criteria: sum of square error, mean square error, R-square, and reliability. The experimental results showed that for the first version of the open-source dataset SRGM-4 based on imperfect debugging methodology achieved the best reliability result, and for the last two versions of the open-source dataset SRGM-6 based on Gompertz distribution methodology achieved the best reliability result in terms of sum of square error, mean square error, and R-square.

A Novel Method for Software Reliability Assessment via Neuro-Fuzzy System

Nowadays, the utilization of software engineering in various areas of technology is remarkably increased. As a matter of fact, it is used in many critical applications such as eye surgery, autopilot systems of airplanes, centralized traffic control (CTC), and so on. Therefore, the reliability of software is very important, and it plays an essential role in the lifetime of the software. Software reliability is one of the main characteristics of software quality. Moreover, the rapid assessment of the reliability of the application is essential during the software life cycle. In this paper, Iuse the neuro-fuzzy methods to assess the software's reliability in order to cope with uncertainties in measuring the actual parameters of the software. By designing neuro-fuzzy inference systems and applying four parameters of the ISO/IEC 9126quality model(i.e., the maturity of software, fault-tolerant, recoverability, and reliability compliance) and finding the parameters of a fuzzy system by exploiting approximation techniques from neural networks, Ipresent an integrated assessment model for evaluation of software reliability. The case study used in this paper to evaluate the proposed method is the software income tax calculator. By applying the input parameters, Iobserve that the software reliability is 0.65. software reliability in our proposed method is more exact than software reliability in the fuzzy multi-criteria and fuzzy method because The weights of the input parameters have been set by experts and software developers, and simulations are carried out using MATLAB tool (ANFIS). Simulations confirm that the proposed method provides acceptable results.

Prototype of 3D Reliability Assessment Tool Based on Deep Learning for Edge OSS Computing

We focus on an estimation method based on deep learning in terms of fault correction time for the operation reliability assessment of open-source software (OSS) under the environment of an edge computing service. Then, we discuss fault severity levels in order to consider the difficulty of fault correction. We use a deep feedforward neural network in order to estimate fault correction times. In particular, we consider the characteristics of fault trends by using three-dimensional graphs. Therefore, we can increase the recognizability of the proposed method based on deep learning for large-scale fault data from the standpoint of fault severity levels under edge OSS operation.

Semantic Feature Driven Consensus-Based Model for Software Reliability Assessment: A Reusability Sensitive Verification Paradigm

Semantic Feature Driven Consensus-Based Model for Software Reliability Assessment: A Reusability Sensitive Verification Paradigm

Unified framework to assess software reliability and determine optimal release time in presence of fault reduction factor, error generation and fault removal efficiency

Unified framework to assess software reliability and determine optimal release time in presence of fault reduction factor, error generation and fault removal efficiency