Software Fault Research Articles

Deep Hybrid Intelligence: CNN-LSTM for Accurate Software Bug Prediction

This research aimed to explore the potential of applying deep learning to software bug prediction. The study utilized various data preprocessing techniques that were essential in preparing the data for analysis, using a set of commonly available software bug reports and related metrics. In the data collection and preprocessing phase, the dataset was filtered to focus on critical software metrics, scaled for consistency, and additional techniques such as feature engineering and standardization were employed to enhance data variability. In order to analyze the effectiveness of the model in predicting software faults, the dataset was split so that it could be used for testing and training purposes. Several deep learning models, include CNN and LSTM architectures, were developed utilizing the preprocessed dataset in order to enhance the performance of the models. Subsequently, a hybrid ensemble technique was employed, combining the prediction outcomes of the best-performing individual models to form an ensemble model. Using test datasets, each model's performance was assessed using common assessment measures including precision, F1 score, accuracy, and recall. The ensemble models outperformed individual models in bug prediction, as demonstrated by higher accuracy and F1 scores. The final model achieved an accuracy of 96%, which was considered highly satisfactory for predicting software defects.

A neighborhood rough sets-based ensemble method, with application to software fault prediction

Software fault prediction (SFP) aims to detect fault-prone software modules, which is beneficial for allocating software testing resources and improving software quality. Recently, ensemble learning(EL)-based SFP methods have attracted much attention. Although many EL algorithms have been applied to SFP, they are still insufficient to generate multiple accurate and diverse base learners. Therefore, this paper presents a multi-modal EL algorithm (called NRSEL) based on neighborhood rough sets. In NRSEL, the technique of neighborhood approximate reduct (NAR) is used to implement the perturbation of attribute space and the bootstrap sampling technique is used to implement the perturbation of sample space. As a novel technique for the perturbation of attribute space, NAR stems from the concept of approximate reduct in rough sets. We also consider the application of NRSEL to SFP, and employ a hybrid scheme (called SMOTE-NRSEL) to handle the problem of imbalanced data in SFP. We compare SMOTE-NRSEL with existing EL algorithms using 20 public datasets. Experimental results indicate that SMOTE-NRSEL is effective for SFP. Compared with the baseline algorithms, on average, SMOTE-NRSEL improves the AUC, F1-score, and MCC by 3.09%, 3.18%, and 7.5%, respectively. Moreover, the results of three statistical tests (including the paired t-test, Friedman test, and Nemenyi test) indicate that SMOTE-NRSEL is significantly better than the baseline algorithms in most cases. This paper shows that NAR is a good choice for the perturbation of attribute space. With the help of NAR and the multi-modal perturbation strategy based on it, SMOTE-NRSEL can generate accurate and diverse base learners. The code is available at https://github.com/jiangfeng0278/NRSEL.

Multi-algorithm for predicting the level accuracy of fault output in software

Software fault output refers to software errors found by testers during the software testing process. Software testing is an overly critical stage in software development; hence, a software testing model is required to systematically classify software errors. For the process of classifying software fault output, accuracy measurements are needed to predict manual fault outputs compared to algorithm-generated fault outputs. Algorithmic methods can be used to measure the accuracy of fault output. The main objective of this research was to compare different algorithms for predicting the accuracy of fault output on a dataset derived from past software testing. The findings indicate that the neural network algorithm outperforms SVM, MLP, RF, and MNB algorithms, achieving 98% accuracy when using 10 software testing variables (function, interface, structure, performance, requirement, documentation, positive, negative, basis path, and times) to predict expected fault outputs.

BLESS Behavior Correctness Proof as ConvincingVerification Artifact

Safety-critical cyber-physical systems require evidence they are indeed safe. In practice, such evidence is results of system tests. Unfortunately, tests can only demonstrate the presence of software errors, not their absence, and can practically cover a tiny fraction of system state space. Valiant efforts to formally verify program correctness have either been excruciatingly difficult (theorem proving) or incomplete (static analysis, model checking, SAT or SMT solving). The BLESS Methodology was created specifically for engineers in industry to formally verify software controlling machines. The BLESS Methodology transforms programs that control machines, annotated with assertions to form proof outlines, into deductive proofs that every possible program execution will conform to its specification. To the extent that cyber-physical system specifications have been validated to express system safety and performance, a deductive proof can be a convincing argument to a person of program correctness. This paper uses a simple safety-critical system to argue that behavior correctness proof under the BLESS Methodology is a convincing verification artifact in addition to customary testing.

Quantification of effects of degraded hardware on the execution of self-healing software using Continuous Time Markov Chain and Reliability

Embedded systems and Systems on Chips (SoC) are widely used in many applications. The tasks are to be executed in a time-bound manner for critical applications. The latency and worst-case delays shall not exceed the acceptable limits. These systems are designed using software/firmware and hardware components that are tightly coupled. The state of hardware and software/firmware plays a major role in determining the performance and dependability of the system. This paper studies the effects of the degraded state of the hardware on the execution of self-healing software and the system. While hardware and software design teams aim to ensure predictability, it is impossible to guarantee it completely due to the randomness of hardware failure events and resident errors in software. Usage of the system for safety critical systems requires these uncertainties to be quantified for design assurance and certification. Therefore, we propose a Continuous Time Markov Chain (CTMC) based approach to characterize the effects of degraded hardware on software execution. Reliability, which is one of the dependability metrics, is also quantified. The proposed approach is validated using data collected from one of the software-intensive complex hardware equipment of a fighter aircraft.

Improving bug triage with the bug personalized tossing relationship

Improving bug triage with the bug personalized tossing relationship

User Interface Bug Classification Model Using ML and NLP Techniques: A Comparative Performance Analysis of ML Models

Analyzing user interface (UI) bugs is an important step taken by testers and developers to assess the usability of the software product. UI bug classification helps in understanding the nature and cause of software failures. Manually classifying thousands of bugs is an inefficient and tedious job for both testers and developers. Objective of this research is to develop a classification model for the User Interface (UI) related bugs using supervised Machine Learning (ML) algorithms and Natural Language Processing (NLP) techniques. Also, to assess the effect of different sampling and feature vectorization techniques on the performance of ML algorithms. Classification is based upon ‘Summary’ feature of the bug report and utilizes six classifiers i.e., Gaussian Naïve Bayes (GNB), Multinomial Naïve Bayes (MNB), Logistic Regression (LR), Support Vector Machines (SVM), Random Forest (RF) and Gradient Boosting (GB). Dataset obtained is vectored using two vectorization techniques of NLP i.e., Bag of Words (BoW) and Term Frequency-Inverse Document Frequency (TF-IDF). ML models are trained after vectorization and data balancing. The models ' hyperparameter tuning (HT) has also been done using the grid search approach to improve their efficacy. This work provides a comparative performance analysis of ML techniques using Accuracy, Precision, Recall and F1 Score. Performance results showed that a UI bug classification model can be built by training a tuned SVM classifier using TF-IDF and SMOTE (Synthetic Minority Oversampling Techniques). SVM classifier provided the highest performance measure with Accuracy: 0.88, Precision: 0.86, Recall: 0.85 and F1: 0.85. Result also inferred that the performance of ML algorithms with TF-IDF is better than BoW in most cases. This work provides classification of bugs that are related to only the user interface. Also, the effect of two different feature extraction techniques and sampling techniques on algorithms were analyzed, adding novelty to the research work.

Understanding the impact of data gaps on long-term offshore wind resource estimates

Abstract. In the context of a wind farm project, the wind resource is assessed to predict the power output and the optimal positioning of wind turbines. This requires taking wind measurements on the site of interest and extrapolating these to the long term using so-called “measure, correlate, and predict” (MCP) methods. Sensor, power supply, and software failures are common phenomena. These disruptions cause gaps in the measured data, which can especially be long in offshore measurement campaigns due to harsh weather conditions causing system failures and preventing servicing and redeployment. The present study investigates the effect of measurement data gaps on long-term offshore wind estimates by analyzing the bias they introduce in the parameters commonly used for wind resource assessment. Furthermore, it aims to show how filling the gaps can mitigate their effect. To achieve this, we perform investigations for three offshore sites in Europe with 2 years of concurrent measurements. We use reanalysis data and various MCP methods to fill gaps in the measured data and extrapolate these data to the long term. Current standards demand high data availability (80 % or 90 %) for wind measurement campaigns, so we expect that the effect of missing data on the uncertainty in long-term extrapolations is of the same order of magnitude as other uncertainty components such as the measurement uncertainty or the inter-annual variability. Nevertheless, our results show that the effects of gaps are considerably smaller than the other uncertainty components. For instance, gaps of 180 d cause an average deviation of the long-term mean wind speed of less than 0.04 m s−1 and a 95th percentile deviation of less than 0.075 m s−1 for all tested sites. Due to the low impact of gaps, gap filling does not have the potential to significantly reduce the uncertainty in the long-term extrapolation.

Prioritization of Software Bugs Using Entropy‐Based Measures

ABSTRACTOpen‐source software is evolved through the active participation of users. In general, a user request for bug fixing, the addition of new features, and feature enhancements. Due to this, the software repositories are increasing day by day at an enormous rate. Additionally, user distinct requests add uncertainty and irregularity to the reported bug data. The performance of machine learning algorithms drastically gets influenced by the inappropriate handling of uncertainty and irregularity in the bug data. Researchers have used machine learning techniques for assigning priority to the bug without considering the uncertainty and irregularity in reported bug data. In order to capture the uncertainty and irregularity in the reported bug data, the summary entropy–based measure in combination with the severity and summary weight is considered in this study to predict the priority of bugs in the open‐source projects. Accordingly, the classifiers are build using these measures for different machine learning techniques, namely, k‐nearest neighbor (KNN), naïve Bayes (NB), J48, random forest (RF), condensed nearest neighbor (CNN), multinomial logistic regression (MLR), decision tree (DT), deep learning (DL), and neural network (NNet) for bug priority prediction This research aims to systematically analyze the summary entropy–based machine learning classifiers from three aspects: type of machine learning technique considered, estimation of various performance measures: Accuracy, Precision, Recall, and F‐measure and through existing model comparison. The experimental analysis is carried out using three open‐source projects, namely, Eclipse, Mozilla, and OpenOffice. Out of 145 cases (29 products X 5 priority levels), the J48, RF, DT, CNN, NNet, DL, MLR, and KNN techniques give the maximum F‐measure for 46, 35, 28, 11, 15, 4, 3, and 1 cases, respectively. The result shows that the proposed summary entropy–based approach using different machine learning techniques performs better than without entropy‐based approach and also entropy‐based approach improves the Accuracy and F‐measure as compared with the existing approaches. It can be concluded that the classifier build using summary entropy measure significantly improves the machine learning algorithms' performance with appropriate handling of uncertainty and irregularity. Moreover, the proposed summary entropy–based classifiers outperform the existing models available in the literature for predicting bug priority.

Accounting for Natural Disasters Using Risk of Information Security

Objective: The harm that can be done to an information technology system is referred to as "information security risk" (IS). Data breaches, administrative actions, financial expenses, and reputational harm are just a few of the many possible dangers that fall under the broad category of information systems risks. IS hazards include hostile assaults, spam, viruses, human error, hardware and software malfunctions, and natural disasters like floods, hurricanes, and fires. Methods: Key security controls for applications are identified, evaluated, and implemented through security risk assessments. Another priority is to avoid security gaps and defects in software. This article analyzes documented historical research on natural disaster accounting. Based on the results of 35 publications on general topics, economic history, and accounting, 11 papers were selected and reviewed. By analyzing the scattered literature, emerging themes, investigated disasters, investigated time periods, and main contributions of published research were identified. Result: To compare the predictive capabilities of the machine learning models and validate the landslide, flood, and wildfire hazard maps, the area under the curve (AUC) was calculated. For the landslide, flood, and wildfire hazard maps, the RF model provided the highest AUC values (0.81, 0.85, and 0.94, respectively).

Software defect density prediction using grey system theory and fuzzy logic

Software defect density prediction using grey system theory and fuzzy logic

A Brief Analysis of the Progress and Trends in Software Defect Prediction Methods

Software defect detection is particularly important for modern society, as it is a crucial step in ensuring the quality and reliability of software systems. With the emergence of artificial intelligence (AI), research in software defect detection has evolved from traditional methods to more complex approaches that utilize deep learning and large language models (LLMs). The advent of LLMs has fundamentally changed the paradigm of software development and defect detection, bringing new challenges and confusion to the field of software defect prediction research. To address these issues, we compare software defect detection methods based on traditional techniques, deep learning approaches, and LLMs through a literature review. We analyze the changes brought about by the introduction of LLMs to software development and propose new insights. Additionally, we examine the progress and trends in software defect prediction to provide inspiration for subsequent research.

An adaptive synthetic sampling and batch generation-oriented hybrid approach for addressing class imbalance problem in software defect prediction

An adaptive synthetic sampling and batch generation-oriented hybrid approach for addressing class imbalance problem in software defect prediction

Multi objective binary Rao feature optimization for software defect prediction using machine learning models

Multi objective binary Rao feature optimization for software defect prediction using machine learning models

Reinforcing defect prediction: a reinforcement learning approach to mitigate class imbalance in software defect prediction

Reinforcing defect prediction: a reinforcement learning approach to mitigate class imbalance in software defect prediction

Enhancing software defect prediction models using metaheuristics with a learning to rank approach

In today’s world, software systems are extensively used across industries like medicine, commerce, education, entertainment, and industry. The development process includes design, development, and testing, where software testing is vital for quality assurance. Manual testing is laborious and resource-heavy, especially in large projects. Researchers are investigating methods to predict software defects by analyzing source code metrics, with Machine Learning techniques showing promise. The Machine learning models typically employs three approaches to predict defects: presence, number, and density. One cutting-edge approach in such models is LTR (Learning To Rank), which considers the order and values of predictions, aiding enterprises in efficiently allocating resources to more defective components. This article aims to enhance the performance of 5 machine learning models by leveraging 5 metaheuristics to optimize their hyperparameters. The study was conducted using 28 common datasets from the literature, with the FPA metric as the evaluation criteria. The results demonstrate that the proposed method achieved a 13% increase compared to state-of-the-art models with Random Forest, achieving an average FPA of 0.84 across the datasets.

Generation of algebraic data type values using evolutionary algorithms

Automatic data generation is a key component of automated software testing. Random generation of test input data can uncover some bugs in software, but its effectiveness decreases when those inputs must satisfy complex properties in order to be meaningful. In this work, we study an evolutionary approach to generate values that can be encoded as algebraic data types plus additional properties. First, the approach is illustrated with the generation of sorted lists. Then, we generalize the technique to arbitrary algebraic data type definitions. Finally, we consider the problem of constrained data types where the data must satisfy some nontrivial property, using the well-known example of red-black trees for our experiments. This example will allow us to introduce the main principles of evolutionary algorithms and how these principles can be applied to obtain valid, nontrivial samples of a given data structure. Our experiments have revealed that this evolutionary approach is able to improve diversity, and increase the size of valid generated values with respect to simple random sampling techniques.

Comparison of Oversampling Techniques on Minority Data Using Imbalance Software Defect Prediction Dataset

Software Defect Prediction Dataset as a component of the Software Defect Prediction model has a very vital role. However, NASA Software Defect Prediction has a problem with imbalance in minority data. This study compares the performance of oversampling techniques in overcoming this. A total of 90 oversampling techniques in the form of SMOTE and its variants were used. The results of this study indicate that there is no oversampling technique that is able to overcome this. The original dataset without oversampling shows good performance at the level of accuracy and f1-score but has low performance on auc-score and g-score. Several oversampling techniques show increased performance on auc-score and g-score, unfortunately at the same time showing a decrease in performance on accuracy and f1-score.

A Systematic Exploration of Mutation‐Based Fault Localization Formulae

ABSTRACTFault localization (FL) aims to automatically find the location of bugs in a software program. In the family of FL approaches, spectrum‐based fault localization (SBFL) is the most widely used and has been extensively studied, which computes the suspicious scores of a code element to be buggy via test coverage information. Mutation‐based fault localization (MBFL) further analyses the relationship between mutants and test results. Although MBFL involves more information, it also faces the following limitations. (1) To precisely evaluate suspicious sources, SBFL approaches proposed dozens of formulae based on coverage, while only a few of them have been adopted by MBFL. (2) The current MBFL approaches are based on the assumption that the capability in localizing bugs of each mutant is the same, so they assign the same weight to the mutants that change the test outputs and consider the mutant from the same code element equivalent. In this paper, we intend to enrich the MBFL family by the following two approaches. First, we collect 25 typical SBFL formulae and transform them into MBFL versions. Second, we propose new assumptions that the mutants should have different weights in computing suspicious sources and propose BLMu, a novel MBFL approach that treats mutants differently. We also propose novel metrics for MBFL by considering the high cost of mutation analysis. We perform large‐scale experiments by evaluating all the MBFL approaches against 395 real‐world Java bugs of the Defects4J benchmark. Our evaluation results reveal that BLMu demonstrates a substantial improvement over both MUSE and Metallaxis, the most popular MBFL approaches, at both the statement level and the method level. Specifically, in terms of Top‐1, BLMu improves by 106% at the statement level and 69% at the method level. When compared with other types of FL approaches, MBFL outperforms typical SBFL approaches while still far behind the state‐of‐the‐art learning‐based FL approaches.

An ensemble model for addressing class imbalance and class overlap in software defect prediction

An ensemble model for addressing class imbalance and class overlap in software defect prediction