Software Fault Research Articles

Software reliability prediction: A machine learning and approximation Bayesian inference approach

AbstractReliability growth models are commonly categorized into two primary groups: parametric and non‐parametric models. Parametric models, known as Software Reliability Growth Models (SRGM) rely on a set of hypotheses that can potentially affect the accuracy of model predictions, while non‐parametric models (such as neural networks) can predict the model solely based on training data without any assumptions regarding the model itself. In this paper, we propose several methods to enhance prediction accuracy in software reliability context. More specifically, we, on one hand, introduce two gradient‐based techniques for estimating parameters of classical SRGMs. On the other, we propose methods involving LSTM Encoder–Decoder and Bayesian approximation within Langevin Gradient and Variational inference neural networks. To evaluate our proposed models' performance, we compare them with various neural network‐based software reliability models using three real‐world software failure datasets and utilizing the Mean Square Error (MSE) as a model comparison criterion. The experimental results indicate that our proposed non‐parametric models outperform most classical parametric and non‐parametric models.

Development of partial least squares regression with discriminant analysis for software bug prediction

Many prediction models and approaches have been introduced during the past decades that try to forecast bugged code elements based on static source code metrics, change and history metrics, or both. However, there is still no universal best solution to this problem, as most suitable features and models vary from dataset to dataset and depend on the context in which we use them. Therefore, novel approaches and further studies on this topic are highly necessary.In this paper, we employ a chemometric approach – Partial Least Squares with Discriminant Analysis (PLS-DA) – for predicting bug prone Classes in Java programs using static source code metrics. PLS-DA is successfully applied within the field of chemometrics, but to our best knowledge, it has never been used before in the software maintenance domain for predicting bugs. In addition, we have used rigorous statistical treatments and evaluation for representing the software engineering results.We show that our PLS-DA based prediction model achieves superior performances compared to the state-of-the-art approaches (i.e. F-measure of 0.44–0.47 at 90% confidence level) when no data re-sampling applied and comparable to others when applying up-sampling on the largest open bug dataset, while training the model is significantly faster, thus finding optimal parameters is much easier. In terms of completeness, which measures the amount of bugs contained in the Java Classes predicted to be defective, PLS-DA outperforms every other algorithm: it found 69.3% and 79.4% of the total bugs with no re-sampling and up-sampling, respectively.

Development of Hardware-in-the-Loop Simulation Test Bed to Verify and Validate Power Management System for LNG Carriers

Liquefied natural gas carrier (LNGC) orders are increasing owing to marine environment regulations. The complexity of the integrated system applied to shipbuilding and software errors have increased with the high degree of automation. Direct on-site inspection methods are associated with high costs and safety risks, whereas software-based simulations rely heavily on the accuracy of the models of power system components. Hardware-in-the-loop simulation (HILS) can be utilized for designing and testing intricate real-time embedded systems. Specifically, HILS offers a reliable means of evaluating power management system (PMS) performance for LNGCs, which are high-value vessels commonly used in offshore plants. This study proposes a PMS–HIL test bed comprising a power supply unit, consumer, simulation control console, and main switchboard. The proposed HILS test bed utilizes the real equipment data of the shipbuilding industry to replicate the conditions associated with actual LNGCs. The proposed system is verified and validated through a software acceptance test procedure. Additionally, load-sharing, load-dependent start, blackout prevention, and preferential tests are performed for the PMS function evaluation. Test results indicate that the proposed system has great potential for conventional PMS commissioning. Therefore, it exhibits the potential to replace traditional factory acceptance tests. Additional development of the system will be conducted for ship automation, utilizing PMS control and an energy management system.

BjEnet: a fast and accurate software bug localization method in natural language semantic space

bjEnet: a fast and accurate software bug localization method in natural language semantic space

FSBOA: feature selection using bat optimization algorithm for software fault detection

Feature selection (FS) plays a crucial role in software fault prediction (SFP), aiming to identify a subset of relevant and discriminative features from a large pool of software metrics. It serves as a critical preprocessing step in building accurate fault prediction models, enabling the identification of potential software faults early in the development life cycle and facilitating effective resource allocation for testing and maintenance activities. The study's objective is to determine how well the bat optimization algorithm (BOA) can extract the features that are most important for correctly predicting software flaws, improve the accuracy of fault prediction, reduce the dimensionality of the feature space, and mitigate the risk of overfitting, thereby enabling more effective resource utilization and better allocation of testing efforts. The forecasting models underwent testing and training utilizing a collection of software metrics, with the datasets undergoing evaluation using several different FS algorithms. An assessment was conducted by contrasting the effectiveness of multiple optimization algorithms, including evolutionary methods such as FS employing genetic algorithm (FSGA), FS employing differential evolution (FSDE), and swarm-based techniques such as FS employing ant colony optimization (FSACO), FS employing particle swarm optimization (FSPSO), FS employing firefly algorithm (FSFA), and FS employing binary grey wolf optimization algorithm (FSBGWO) in relation to FS employing bat optimization algorithm (FSBAO). The results obtained from FSBAO approach demonstrate the effectiveness in solving FS optimization problems with at most accuracy of 98.92%. Furthermore, the experimental results have been statistically validated for the greater efficiency of the proposed FSBAO algorithm. This study's findings have crucial implications for developing a software failure prediction models that is more accurate and efficient.

Using sum product networks to predict defects in software systems

Using sum product networks to predict defects in software systems

Fault localization by abstract interpretation and its applications

Fault localization aims to automatically identify the cause of an error in a program by localizing the error to a relatively small part of the program. In this paper, we present a novel technique for automated fault localization via error invariants inferred by abstract interpretation. An error invariant for a location in an error program over-approximates the reachable states at the given location that may produce the error, if the execution of the program is continued from that location. Error invariants can be used for statement-wise semantic slicing of error programs and for obtaining concise error explanations. We use an iterative refinement sequence of backward–forward static analyses by abstract interpretation to compute error invariants, which are designed to explain why an error program violates a particular assertion.Furthermore, we present a practical application of the fault localization technique for automatic repair of programs. Given an erroneous program, we first use the fault localization to automatically identify statements relevant for the error, and then repeatedly mutate the expressions in those relevant statements until a correct program that satisfies all assertions is found. All other statements classified by the fault localization as irrelevant for the error are not mutated in the program repair process. This way, we significantly reduce the search space of mutated programs without losing any potentially correct program, and so locate a repaired program much faster than a program repair without fault localization.We have developed a prototype tool for automatic fault localization and repair of C programs. We demonstrate the effectiveness of our approach to localize errors in realistic C programs, and to subsequently repair them. Moreover, we show that our approach based on combining fault localization and code mutations is significantly faster that the previous program repair approach without fault localization.

Advancements in Predictive Models for Software Defects: A Comprehensive Exploration

Advancements in Predictive Models for Software Defects: A Comprehensive Exploration

Modeling Reliability-Driven Software Release Strategy Considering Testing Effort with Fault Detection and Correction Processes: A Control Theoretic Approach

In this paper, we have introduced a software reliability growth model (SRGM) that integrates a Weibull testing effort function (TEF) within the software fault detection process (FDP) and fault correction process (FCP), effectively capturing the intricacies of software testing. Leveraging the least squared estimation method, we estimated both model parameters and the TEF, leading to a notably improved fit with the dataset and enhancing our comprehension of software reliability dynamics. Our approach also involved employing a reliability-based method to identify the optimal time for software release, further reinforcing the robustness of our model. Subsequently, we devised an optimal control model to minimize testing efforts throughout the software development life cycle. This work provides a comprehensive methodological framework for computing the total cost while ensuring debugging costs follow a learning curve pattern. Through numerical simulations involving hypothetical values and the Weibull TEF, we observed that once the software achieves the desired reliability for release, subsequent costs gradually decline to zero. In essence, both testing and future costs converge to zero post-release, reducing instantaneous expenses.

Hybrid YSGOA and neural networks based software failure prediction in cloud systems

In the realm of cloud computing, ensuring the dependability and robustness of software systems is paramount. The intricate and evolving nature of cloud infrastructures, however, presents substantial obstacles in the pre-emptive identification and rectification of software anomalies. This study introduces an innovative methodology that amalgamates hybrid optimization algorithms with Neural Networks (NN) to refine the prediction of software malfunctions. The core objective is to augment the purity metric of our method across diverse operational conditions. This is accomplished through the utilization of two distinct optimization algorithms: the Yellow Saddle Goat Fish Algorithm (YSGA), which is instrumental in the discernment of pivotal features linked to software failures, and the Grasshopper Optimization Algorithm (GOA), which further polishes the feature compilation. These features are then processed by Neural Networks (NN), capitalizing on their proficiency in deciphering intricate data patterns and interconnections. The NNs are integral to the classification of instances predicated on the ascertained features. Our evaluation, conducted using the Failure-Dataset-OpenStack database and MATLAB Software, demonstrates that the hybrid optimization strategy employed for feature selection significantly curtails complexity and expedites processing

Bug numbers matter: An empirical study of effort‐aware defect prediction using class labels versus bug numbers

AbstractPrevious research have utilized public software defect datasets such as NASA, RELINK, and SOFTLAB, which only contain class label information. Most effort‐aware defect prediction (EADP) studies are carried out around these datasets. However, EADP studies typically relying on predicted bug number (i.e., considering modules as effort) or density (i.e., considering lines of code as effort) for ranking software modules. To explore the impact of bug number information in constructing EADP models, we access the performance degradation of the best‐performing learning‐to‐rank methods when using class labels instead of bug numbers for training. The experimental results show that using class labels instead of bug numbers in building EADP models results in an decrease in the detected bugs when module is considering as effort. When effort is LOC, using class labels to construct EADP models can lead to a significant increase in the initial false alarms and a significant increase in the modules that need to be inspected. Therefore, we recommend not only the class labels but also the bug number information should be disclosed when publishing software defect datasets, in order to construct more accurate EADP models.

Predicting Software Defect Complexity and Accuracy using Bug Tracking and Clustering

Many open sources, free and commercial bug tracking tools have been developed and are currently under development. There are number of issues are related to software projects are daily increasing and the developers are started to use bug tracking systems in that order to manages the bug reports. The industry needs that the criteria to select the best system tool among the available set of system tools which will helps to fix and track the progressive report of bug fixes. While, collection of useful information from the large and not organized set of these reports is still difficult problem because there are various bug tracking systems are provide the data via many resources like web interfaces. We try to present these comprehensive classification criteria to manage the reviews for available tools and propose a new modified tool for the bug tracking and reporting system. It also helps in reporting the bugs which are founded by that process, assigning the bug to the developer for monitoring and fixing the progress of bug fixing by various graphical/charting facility and status updates. It also providing the reliability of bug prediction and tries to find the bugs for complexity measurements, and allows to distributing.

Software Fault Localization Based on Weighted Association Rule Mining and Complex Networks

Software fault localization technology aims to identify suspicious statements that cause software failures, which is crucial for ensuring software quality. Spectrum-based software fault location (SBFL) technology calculates the suspiciousness of each statement by analyzing the correlation between statement coverage information and execution results in test cases. SBFL has attracted increasing attention from scholars due to its high efficiency and scalability. However, existing SBFL studies have shown that a large number of statements share the same suspiciousness, which hinders software debuggers from quickly identifying the location of faulty statements. To address this challenge, we propose an SBFL model based on weighted association rule mining and complex networks: FL-WARMCN. The algorithm first uses Jaccard to measure the distance between passing and failing test cases, and applies it as the weight of passing test cases. Next, FL-WARMCN calculates the initial suspiciousness of each statement based on the program spectrum data. Then, the FL-WARMCN model utilizes a weighted association rule mining algorithm to obtain the correlation relationships between statements and models the network based on this. In the network, the suspiciousness of statements is used as node weights, and the correlation between statements is used as edge weights. We chose the eigenvector centrality that takes into account the degree centrality of statements and the importance of neighboring statements to calculate the importance of each statement, and used it as a weight to incorporate into the weighted suspiciousness calculation of the statement. Finally, we applied the FL-WARMCN model for experimental validation on the Defects4J dataset. The results showed that the model was significantly superior to other baselines. In addition, we analyzed the impact of different node and edge weights on model performance.

BERT-based cross-project and cross-version software defect prediction

BERT-based cross-project and cross-version software defect prediction

Systematic Review of Bespoke Techniques of Software Fault Prediction: Machine Learning and Conventional Techniques

Systematic Review of Bespoke Techniques of Software Fault Prediction: Machine Learning and Conventional Techniques

A comparative study of software defect binomial classification prediction models based on machine learning

A comparative study of software defect binomial classification prediction models based on machine learning

Software defect prediction using global and local models

Software defect prediction using global and local models

Enhancing Program Synthesis with Large Language Models Using Many-Objective Grammar-Guided Genetic Programming

The ability to automatically generate code, i.e., program synthesis, is one of the most important applications of artificial intelligence (AI). Currently, two AI techniques are leading the way: large language models (LLMs) and genetic programming (GP) methods—each with its strengths and weaknesses. While LLMs have shown success in program synthesis from a task description, they often struggle to generate the correct code due to ambiguity in task specifications, complex programming syntax, and lack of reliability in the generated code. Furthermore, their generative nature limits their ability to fix erroneous code with iterative LLM prompting. Grammar-guided genetic programming (G3P, i.e., one of the top GP methods) has been shown capable of evolving programs that fit a defined Backus–Naur-form (BNF) grammar based on a set of input/output tests that help guide the search process while ensuring that the generated code does not include calls to untrustworthy libraries or poorly structured snippets. However, G3P still faces issues generating code for complex tasks. A recent study attempting to combine both approaches (G3P and LLMs) by seeding an LLM-generated program into the initial population of the G3P has shown promising results. However, the approach rapidly loses the seeded information over the evolutionary process, which hinders its performance. In this work, we propose combining an LLM (specifically ChatGPT) with a many-objective G3P (MaOG3P) framework in two parts: (i) provide the LLM-generated code as a seed to the evolutionary process following a grammar-mapping phase that creates an avenue for program evolution and error correction; and (ii) leverage many-objective similarity measures towards the LLM-generated code to guide the search process throughout the evolution. The idea behind using the similarity measures is that the LLM-generated code is likely to be close to the correct fitting code. Our approach compels any generated program to adhere to the BNF grammar, ultimately mitigating security risks and improving code quality. Experiments on a well-known and widely used program synthesis dataset show that our approach successfully improves the synthesis of grammar-fitting code for several tasks.

PMTT: Parallel multi-scale temporal convolution network and transformer for predicting the time to aging failure of software systems

Software aging is one of the significant factors affecting the reliability and availability of long-running software systems, such as Android, Cloud systems, etc. The time to aging failure (TTAF) prediction for software systems plays a crucial role in proactive rejuvenation scheduling through machine learning or statistical analysis techniques, due to its ability to determine when to perform rejuvenation to mitigate the aging effects. However, software aging characterization is relatively complicated, and only fitting the variations for a single aging indicator cannot grasp the comprehensive degradation process across different case systems; moreover, since software systems often exhibit long and short-term inherent degradation characteristics, existing prediction models possess a poor ability for modeling both global and local information simultaneously. To tackle the above problems, a novel TTAF prediction framework based on the parallel multi-scale temporal convolution network and transformer (named PMTT) is proposed, by mapping various system running indicators reflecting the software aging to TTAF. PMTT possesses the following distinctive characteristics. First, a local feature extraction module that contains multiple channel TCNs with different scales is developed to extract inherent local information from the raw input. Second, in a parallel manner, a global feature extraction module integrating transformer blocks is built to extract global information representation synchronously using the self-attention mechanism. Afterward, high-level global–local features extracted from different channels are fused, and TTAF is estimated through two fully connected regression layers using the fused features. The proposed PMTT has been compared to seven competitors using run-to-failure data collected from Android and OpenStack systems. The experiments have demonstrated the superiority of PMTT, showing an average improvement of 11.2%, 9.0%, and 9.3% in performance across three evaluation metrics compared with the optimal baseline model.

Testing Error Handling Code With Software Fault Injection and Error-Coverage-Guided Fuzzing

Real-world programs require error handling code to handle various kinds of possible errors. However, these errors just infrequently occur due to special conditions, so error handling code is difficult to test. Coverage-guided fuzzing and software fault injection (SFI) are two common techniques that can test error handling code, but they still have major limitations. Specifically, existing fuzzing approaches generate program inputs guided by code coverage, but many occasional errors (such as insufficient memory) are unrelated to inputs, and code coverage cannot effectively reflect the execution contexts of these errors; existing SFI approaches often inject single or random faults, without exploring fault space or using program feedback. In this paper, we propose a new fuzzing framework named EH-Fuzz, to effectively test error handling code. EH-Fuzz uses a context-sensitive SFI-based fuzzing approach to explore fault space and perform fault injection, guided by a new metric named <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">error coverage</i> . We evaluate EH-Fuzz on 9 user-level programs and 6 kernel-level modules, and find 45 new real bugs, 31 of which have been confirmed and fixed. We compare EH-Fuzz to existing fuzzing approaches (including AFL, AFL++, Syzkaller, FIZZER and FIFUZZ), and EH-Fuzz finds many real bugs missed by these approaches with higher testing coverage.