Reliability Growth Research Articles

Testing coverage based NHPP software reliability growth modeling with testing effort and change-point

This paper investigates a software reliability growth model based on non-homogeneous Poisson process that includes coverage testing and change-points. The model integrate effort spent into a model to evaluate software reliability based on testing coverage. The amount of testing effort spent follows the Weibull distribution, while the amount of coverage is modeled by logistic, delayed S-shaped and exponential functions. Additionally, we look into the cost requirement-based software release time for exponential functions with a reliability constraint. We introduce the genetic algorithm, which is a powerful tool for dealing with search and optimization issues. For the purpose of testing the model's goodness of fit, two real failure datasets are used, and the model’s performance is analyzed based on goodness-of-fit metrics. By comparing the proposed model to perfect debugging models that have been described in the literature, it is demonstrated that the proposed model demonstrates notable advancements in this regard. The results of our analysis indicate that a software reliability estimation model should include change-points in a testing process if they exist.

A system reliability demonstration test design method based on subsystem test data

AbstractReliability demonstration test (RDT) is the test to verify that the reliability of the product meets the reliability requirements during product development and is usually required to be performed to determine whether it can be accepted. It is essential for complex systems with higher costs. However, the traditional test plans such as the plan in the current RDT standard GJB‐899A have the problem of too long test time or too high risk. In addition, the test data of the complex system itself is usually less, but the test data of the subsystem is easier to obtain. In this paper, the research object is the system, and the main research content is how to use the reliability growth data in the development stage of the subsystem to design the system RDT. First, based on the reliability growth model, the test data in the development stage of the subsystems are modeled to obtain the reliability of each subsystem. The Bayesian theory is used to obtain the lifetime distribution of the system. Then, according to the definition of the two types of risks in the RDT, the life distribution of the product is integrated into the two types of risks, and the RDT plans of the system are designed. Finally, a simulation case is used to verify our RDT design method, and the results show that compared with the traditional plan, our RDT plan designed has a shorter test time and lower test risk.

Statistical Approaches Used in Studies Evaluating the Reliability of Autonomous Vehicles Based on Disengagements and Reaction Times

The rapid development of autonomous vehicle (AV) technology highlights the critical importance of enhancing the reliability of these vehicles. Due to the need to test the reliability of AVs, since 2014, the California Department of Motor Vehicles has permitted autonomous vehicle manufacturers to establish an AV Testing program, enabling them to test automated systems on the transportation network. With this, studies on the reliability of AVs have increased rapidly. The most emphasized issues regarding the reliability of AVs have been disengagements, accidents, and reaction times. In this study, disengagements and reaction times are categorized and explained in detail according to the data type, company, period, and statistical method. The data used in the studies cover the years 2014-2020. When examining studies on the reliability of AVs, until 2018, inferences were generally made using real data and descriptive statistics, particularly with methods such as correlation analysis and calculation of disengagements per mile, which investigates the relationship between distance traveled and disengagements. However, since 2018, machine learning has gained importance in evaluating AV reliability. It has been observed that regression, classification, and decision trees were frequently used during this period. Techniques such as deep transfer learning, text mining, and natural language processing also stand out. Furthermore, Software Reliability Growth Models are used to measure software reliability, playing an essential role in evaluating, analyzing, and improving the performance and reliability of AVs. This study aims to reveal the development and diversity of the statistical methods used to determine AV reliability. Additionally, this study aims to guide and provide insights to researchers in the field about the statistical approaches they can utilize.

A multi-step mathematical model-based predictive strategy for software release timing during testing stage

Mathematically precise modeling is important to be established to accurately examine the quantitative relationship between software testing and software reliability. Software testing process is complex since it is concerned with various factors such as test case execution, defect debugging, tester expertise, test case selection, and so forth. For this reason, it is required to be meticulous in formulating the software testing process in a manner which is mathematically concise. The software release life cycle or sequential release timeline, referring to the process related to the development, testing and distribution of a software product comprises several critical stages, and the length of this particular life cycle reveals variations depending on different factors like the type of product, the intended use of it, industry security, general standards and compliance. One consideration software engineers have is related to the release date of the software so that future commitments about the software’s release time can be formulated beforehand. In view of these aspects, a multi-step strategy for predicting software release dates is proposed in the current study along with the following stages: firstly, the proposed technique selects the utmost reliability growth model that very well fits the observed test data halfway through the testing period, and then employs it to forecast the probable date of release. This technique entails approximating the unknown parameters of suitable Software Reliability Growth Models (SRGMs). Finally, the chosen SRGM is used to forecast the release date of the software under test by fitting it to available fault data. The proposed method is straightforward and applied to test on a total of ten actual datasets collected from the literature. The results of the proposed technique reveal that in the majority of the situations, nearly exact approximation of date of release can be made halfway through the testing period. Moreover, the proposed method’s performance is also compared to that of a number of previous strategies present in the literature. The outcomes obtained by our study demonstrate that the proposed strategy may be used to forecast the release date of software in practical situations.

A software reliability growth model for open-source software using sine cosine algorithm

A software reliability growth model for open-source software using sine cosine algorithm

AI learning-driven optimization of microgrid systems for rural electrification and economic empowerment

Artificial Intelligence (AI) and machine learning (ML) are transforming the landscape of rural electrification through their application in microgrid systems. Microgrids, localized networks that can operate independently or in conjunction with the main grid, offer a viable solution for delivering reliable electricity to rural areas. AI-driven optimization enhances the performance and efficiency of these microgrids, addressing the unique challenges faced in remote regions and contributing to economic empowerment. AI technologies, including machine learning algorithms and data analytics, play a critical role in optimizing microgrid systems. By employing predictive maintenance, AI can forecast equipment failures and schedule timely repairs, thereby minimizing downtime and reducing maintenance costs. Additionally, AI enhances energy management through precise load forecasting, optimizing energy distribution, and integrating renewable energy sources like solar and wind. This not only balances energy supply and demand but also mitigates the variability associated with renewable sources. The impact of AI-driven microgrid optimization on rural electrification is profound. Improved system reliability and efficiency result in more stable energy supplies, which is crucial for supporting local businesses, educational facilities, and healthcare services in remote areas. Economically, the reduction in energy costs and the creation of new job opportunities further empower rural communities. Socially, reliable electricity enhances the quality of life, enabling better educational outcomes and promoting economic development. Successful implementations of AI in microgrid systems have demonstrated significant benefits. For instance, AI-optimized microgrids in rural communities have shown improved performance and stability, leading to greater energy access and economic benefits. However, challenges remain, including technical complexities, financial constraints, and regulatory hurdles. Addressing these challenges through continued research, investment, and policy support is essential for scaling AI-driven solutions and achieving broader impacts. AI-driven optimization of microgrid systems offers a powerful tool for advancing rural electrification and economic empowerment. By leveraging AI technologies, we can enhance energy reliability, reduce costs, and foster economic growth in underserved regions, paving the way for a more equitable and sustainable future. Keywords: AI Learning-Driven, Microgrid Systems, Rural Electrification, Economic Empowerment.

Modeling the impact of noise and uncertain operational environment on software release decisions considering testing coverage-based SRGM

The operational environment of software differs from the debugging environment. Therefore, the study explores the impact of irregular consumption by diverse users on the development cost of software, reliability of software, and release decisions. To accomplish this, a release model has been formulated considering logistic testing coverage-based reliability growth model built as a stochastic process including the error generation. The stochastic nature has been captured by considering the noise factor due to irregular fluctuations occurring during testing while usage uncertainty has been captured by introducing a constant parameter in the cost function of the operational phase. It is assumed that the testing phase cost is affected by the noise and the operation phase cost is affected by the severity of noise which is the result of uncertain usage by users. The model was evaluated against a real failure dataset. The release model creates a trade-off between software development cost, release timing, and reliability aspirations. This study contributes to software reliability literature and provides insights to practitioners to make software release decisions. The sensitivity analysis results give information about various aspects during the operational phase that affect the overall development cost.

Retraction Note: A novel machine learning approach for software reliability growth modelling with pareto distribution function

Retraction Note: A novel machine learning approach for software reliability growth modelling with pareto distribution function

Advancing software reliability with time series insights: A non‐autoregressive ANN approach

AbstractSoftware reliability is a critical factor in assessing the health of software and identifying defects. Software reliability growth models (SRGM) are used to estimate the occurrence of software faults. There are various parameterized and non‐parameterized models of SRGM. These models effectively predict fault occurrence for limited testing conditions. To resolve this problem various neural and artificial neural network (ANN) models are proposed. A problem while using ANN is over‐fitting and under‐fitting. Non‐autoregressive time series models, including ANN variants, offer promising solutions to address under‐fitting issues in SRGM, providing enhanced predictive capabilities for fault occurrence across diverse testing conditions. This study proposes a modified version with a Bayesian regularization technique to address over‐fitting. This modification aims to enhance the suitability of the Bayesian regularization framework for nonlinear autoregressive (NAR) models by carefully adjusting regularization parameters. Comprehensive testing with real‐world software failure datasets is conducted to evaluate the effectiveness of the proposed approach. The results demonstrate that our modified approach improved generalization capabilities and increased prediction accuracy. The NAR‐ANN model exhibits a lower mean squared error of 0.12935 and a higher value of 0.99853.

Software reliability prediction: A machine learning and approximation Bayesian inference approach

AbstractReliability growth models are commonly categorized into two primary groups: parametric and non‐parametric models. Parametric models, known as Software Reliability Growth Models (SRGM) rely on a set of hypotheses that can potentially affect the accuracy of model predictions, while non‐parametric models (such as neural networks) can predict the model solely based on training data without any assumptions regarding the model itself. In this paper, we propose several methods to enhance prediction accuracy in software reliability context. More specifically, we, on one hand, introduce two gradient‐based techniques for estimating parameters of classical SRGMs. On the other, we propose methods involving LSTM Encoder–Decoder and Bayesian approximation within Langevin Gradient and Variational inference neural networks. To evaluate our proposed models' performance, we compare them with various neural network‐based software reliability models using three real‐world software failure datasets and utilizing the Mean Square Error (MSE) as a model comparison criterion. The experimental results indicate that our proposed non‐parametric models outperform most classical parametric and non‐parametric models.

Modeling Reliability-Driven Software Release Strategy Considering Testing Effort with Fault Detection and Correction Processes: A Control Theoretic Approach

In this paper, we have introduced a software reliability growth model (SRGM) that integrates a Weibull testing effort function (TEF) within the software fault detection process (FDP) and fault correction process (FCP), effectively capturing the intricacies of software testing. Leveraging the least squared estimation method, we estimated both model parameters and the TEF, leading to a notably improved fit with the dataset and enhancing our comprehension of software reliability dynamics. Our approach also involved employing a reliability-based method to identify the optimal time for software release, further reinforcing the robustness of our model. Subsequently, we devised an optimal control model to minimize testing efforts throughout the software development life cycle. This work provides a comprehensive methodological framework for computing the total cost while ensuring debugging costs follow a learning curve pattern. Through numerical simulations involving hypothetical values and the Weibull TEF, we observed that once the software achieves the desired reliability for release, subsequent costs gradually decline to zero. In essence, both testing and future costs converge to zero post-release, reducing instantaneous expenses.

A System-Level Reliability Growth Model for Efficient Inspection Planning of Offshore Wind Farms

Fatigue damage can lead to failures of structural systems. To reduce the failure risk and enhance the reliability of structural systems, inspection and maintenance interventions are required, and it is important to develop an efficient inspection strategy. This study, for the first time, develops a system-level reliability growth model to establish efficient inspection planning. System-level reliability growth is defined as an increase in the percentage of the system reliability index with and without inspection. The probabilistic S-N approach is used to obtain the reliability index without inspection. Moreover, advanced risk analysis and Bayesian inference techniques are used to obtain the reliability index with inspection. The optimal inspection planning is obtained by maximizing system-level reliability growth. This model is applied to an offshore wind farm. The results show that inspection efficiency can be improved by increasing the number of repair objects in response to a ‘detection’ inspection outcome, changing the inspection object for each inspection, and increasing the inspection quality. The maximum system-level reliability growth gained from one additional inspection decreases as the number of inspections increases. This study quantifies the inspection efficiency of offshore wind farms by explicit system-level reliability growth computation, offering valuable insights for promoting sustainable energy solutions.

Stochastic software reliability growth modelling with fault introduction and change point

Stochastic software reliability growth modelling with fault introduction and change point

Developing an Innovative Imperfect Debugging Software Reliability Growth Model with Enhanced Testing Coverage Strategies

In this paper, we introduce an innovative Software Reliability Growth Model (SRGM) designed to tackle the pivotal challenges associated with software reliability in the contemporary digital landscape, where the prevalence of online systems is ubiquitous. This SRGM integrates Imperfect Debugging (ID), Testing Coverage (TC), Testing Effort (TE), and error generation into a cohesive framework. Employing a sigmoid function to encapsulate TE, it incorporates three distinct TC functions: Delayed S-shaped, Exponential, and Logistic. This model relies on foundational assumptions, including the proportionality of fault detection rates to remaining faults, the introduction of new faults during debugging, and the intricate connection between fault detection and code coverage. The Mean Value Function (MVF) is computed through these differential equations, and the resultant MVFs are systematically tabulated for all models. An examination of the sigmoid TE function and the Weibull TE function across diverse datasets, utilizing a range of goodness-of-fit criteria including Mean Square Error (MSE), Pham’s Criterion (PC), Predictive Risk Ratio (PRR), Bayesian Information Criterion (BIC), and Akaike’s Information Criterion (AIC), reveals the superior performance of the sigmoid TE function over the Weibull counterpart across various datasets and evaluation criteria. In conclusion, this paper introduces a groundbreaking SRGM that seamlessly integrates ID, TC, and TE, offering valuable insights for assessing software reliability in the dynamic landscape of modern digital systems.

An SRGM Using Fault Removal Efficiency and Correction Lag Function

Software reliability growth models (SRGMs) have shown remarkable research since the early 1970s. Usually, growth models are used throughout the software testing process to determine the failure pattern, consumption of testing efforts, total expected cost and reliability. Many existing models start with the underlying premise that defects are found and immediately fixed. But in reality, initially faults are detected, isolated and then corrected. This study enhances the SRGM by including the detection and correction procedure to address this discrepancy, considering the correction lag function and fault removal efficiency. In addition, this study also examines the overall anticipated cost and reliability associated with the suggested model. Furthermore, numerical outcomes are shown using actual datasets to verify the validity of the proposed model.

Long-term software fault prediction with wavelet shrinkage estimation

Wavelet shrinkage estimation received considerable attentions to estimate stochastic processes such as a non-homogeneous Poisson process in a non-parametric way, and was applied to software reliability estimation/prediction. However, it lacks the prediction ability for unknown future patterns in long term and penalizes assessing the software reliability in practice. In this paper, we focus on the long-term prediction of the number of software faults detected in the testing phase and propose many novel long-term prediction methods based on the wavelet shrinkage estimation. The fundamental idea is to adopt both the denoised fault-count data and prediction values, and to minimize several kinds of loss functions to make effective predictions. We also develop an automated wavelet-based software reliability assessment tool, W-SRAT2, which is a drastic extension of the existing tool, W-SRAT, by adding those prediction algorithms. In numerical experiments with 6 actual software development project data, we investigate the predictive performance of our long-term prediction approaches, which consist of 2,640 combinations, and compare them with the common software reliability growth models with the maximum likelihood estimation. It is shown that our wavelet shrinkage estimation/prediction methods outperform the existing software reliability growth models.

Modeling Software Release Time and Software Patch Release Time Based on Testing Effort and Warranty

In this world of software technology, our dependency on software’s is increasing continuously. As a result, software industries are working hard to develop highly reliable software and to meet the expectation of customers. Generally, software companies release software early in market to take gain market share, but rigorous software testing is required for early release software to ensure reliability of software and meet the customer’s expectations. This requires a huge amount of resources, and it increases financial burden on the company, consequently, decreases the overall profit of company. Further, late release due to prolong testing of a software may improves reliability but results into a loss of market opportunity cost or may not be fulfil the customer’s aspirations. As a result, to stay competitive, companies release software early and release patches later to fix the bugs, improve the functionality of software, and to update the software. Software industries are improving the performance or usability of software by releasing patches which may increase the consumption of testing effort and consequently increase in cost. On the other hand, software firms also provide warranty on their products. To address the above said issues, we have developed a testing effort-based software reliability growth model, which incorporates warranty policy and estimates the optimal software release and patch time with the objective to minimise the total testing cost. Further, we have used Genetic Algorithm (GA) to estimate optimum software release and patch time. A numerical illustration has been presented on a real time data set to validate the proposed model.

Empowering software reliability: Leveraging efficient fault detection and removal efficiency

Advancements in science and technology have led to the widespread use of computer systems in various applications, emphasizing the importance of software reliability. Software failures can have severe consequences, making thorough testing crucial. Software reliability growth models (SRGMs) play a significant role in enhancing reliability by predicting improvement over time. This article introduces a comprehensive approach to software reliability that incorporates a dynamic fault detection rate, along with fault removal efficiency. The fault detection rate measures the rate at which faults are identified during testing, reflecting the effectiveness of the testing process. By incorporating this dynamic component, the model provides a more accurate estimation of software reliability and enables adaptive testing strategies and resource allocation. Achieving a high fault detection rate is desirable, but organizations must consider the cost implications and strike a balance between reliability and time-to-market constraints. This article extends the analysis to calculate the optimal release time and optimal warranty period that minimize development costs, subject to the desired reliability. By considering these factors, development teams can make informed decisions regarding the timing of software release and the duration of the warranty period, optimizing both reliability and cost.

Understanding the Impact of Changes in Application Characteristics on SRGM

Due to the increasing reliance on technology in nearly every industry over the past three decades, it has become necessary to evaluate the performance of a software product prior to its formal release in the market. The properties of a software application, such as its complexity and lines of code, are subject to change over time as a result of factors namely the testing environment, allocation of resources, testing efficiency, and testing team’s expertise. The assumption of constant Fault Detection Rate (FDR) may not accurately anticipate the potential number of bugs correctly. Keeping all these considerations in mind, a framework is developed to incorporate change point in the development of a testing effort-based Software Reliability Growth Model (SRGM) that takes into account the effect of application characteristics under both perfect and imperfect debugging settings. In addition, these outcomes are compared to the model without a change point. The proposed model is validated on two real-life software fault datasets. The results demonstrate that the proposed model performs better than the model without a change point.

A Method of Reliability Assessment Based on Trend Analysis for Open Source Software

Software reliability growth model (SRGM) is used as one of the reliability assessment methods to assess the software reliability. In SRGM, the degree of reliability growth may fluctuate greatly according to change in the internal state of the software. It is called the change point (CP). Several researchers proposed the SRGM considering CP. In the open source software (OSS), there are many projects that continue development even after the software is released. Therefore, major updates with breaking changes may occur in it. The major updates can be a factor that causes a CP because it greatly changes the internal state of the OSS. This paper focuses on the relationship between CP and software updates. We collect OSS fault data from a bug tracking system. Moreover, we examine the behavior of SRGM before and after software updates. Furthermore, we discuss the applicability of SRGM for CP in OSS. Also, we compare the proposed model based on CP with the model without CP. As a result, we have confirmed that the SRGM can evaluate the reliability in the environment with major updates. Moreover, the proposed method performs better than without considering CP model. Especially, the exponential model’s mean value function is the suitable method to assess the OSS reliability for the proposed method.