Software Fault Research Articles

Method-level Bug Prediction: Problems and Promises

Fixing software bugs can be colossally expensive, especially if they are discovered in the later phases of the software development life cycle. As such, bug prediction has been a classic problem for the research community. As of now, the Google Scholar site generates ∼113,000 hits if searched with the “bug prediction” phrase. Despite this staggering effort by the research community, bug prediction research is criticized for not being decisively adopted in practice. A significant problem of the existing research is the granularity level (i.e., class/file level) at which bug prediction is historically studied. Practitioners find it difficult and time-consuming to locate bugs at the class/file level granularity. Consequently, method-level bug prediction has become popular in the past decade. We ask, are these method-level bug prediction models ready for industry use? Unfortunately, the answer is no . The reported high accuracies of these models dwindle significantly if we evaluate them in different realistic time-sensitive contexts. It may seem hopeless at first, but, encouragingly, we show that future method-level bug prediction can be improved significantly. In general, we show how to reliably evaluate future method-level bug prediction models and how to improve them by focusing on four different improvement avenues: building noise-free bug data, addressing concept drift, selecting similar training projects, and developing a mixture of models. Our findings are based on three publicly available method-level bug datasets and a newly built bug dataset of 774,051 Java methods originating from 49 open-source software projects.

PCG: A joint framework of graph collaborative filtering for bug triaging

AbstractBug triaging is a vital process in software maintenance, involving assigning bug reports to developers in the issue tracking system. Current studies predominantly treat automatic bug triaging as a classification task, categorizing bug reports using developers as labels. However, this approach deviates from the essence of triaging, which is establishing bug–developer correlations. These correlations should be explicitly leveraged, offering a more comprehensive and promising paradigm. Our bug triaging model utilizes graph collaborative filtering (GCF), a method known for handling correlations. However, GCF encounters two challenges in bug triaging: data sparsity in bug fixing records and semantic deficiency in exploiting input data. To address them, we propose PCG, an innovative framework that integrates prototype augmentation and contrastive learning with GCF. With bug triaging modeled as predicting links on the bipartite graph of bug–developer correlations, we introduce prototype clustering‐based augmentation to mitigate data sparsity and devise a semantic contrastive learning task to overcome semantic deficiency. Extensive experiments against competitive baselines validate the superiority of PCG. This work may open new avenues for investigating correlations in bug triaging and related scenarios.

Decision Making on a Software Upgrade or Decommission with Data Mining and Machine Learning Techniques in Information Technology Industry

The Organizations have been investing more in Technology and Infrastructure spends like software upgrades, software renewals, software replacements, platform migrations etc., apart from investment in Business, People, and Processes. In this context, it is not an easy task for stakeholders to decide whether to go for a software upgrade or to replace it with another software. There is no unified approach or solution to consolidate data and relationships of Information Technology Assets, Software Upgrades, Software costs, Software defects, Software Performance Metrics, Security issues, IT system versions, service level objectives etc. Due to this, the decision making of software upgrades and software decommissioning is a tedious process and takes more time and effort. There is a need to build a solution that can integrate and validate the information like software assets, software upgrade success and failure likelihoods, cost benefit analysis of Cloud Computing, software metrics for fault prediction, software maintainability prediction results, Digital Transformation readiness and other related factors. There is an opportunity to apply Machine Learning techniques in defining and deriving the success likelihoods on the following data: Systems and data integration, software assets compatibility, operational service level agreement breaches, quality assurance metrics, security issues, number of open defects, number of defect fixes, number of priority incidents, mean time to resolve critical incidents, expected cost increase in software maintenance, potential cost reduction with the software or hardware replacement etc. This Research Proposal outlines the above mentioned to build a recommendation system aka decision tree namely Software Upgrades or Decommissions Life Cycle.

Advancing modern code review effectiveness through human error mechanisms

Modern code reviews tend to take a lightweight process, in which the accuracy and efficiency of identifying defects rely heavily on code reviewers’ experience. The human errors of developers, as a significant cause of software defects, is a key to identifying defects. However, there is a lack of understanding of the human error mechanisms underlying defects in code. This paper proposes an innovative code review method for identifying defects by pinpointing the scenarios that developers tend to commit errors. The method was validated by two experimental studies that involved 40 participants of about 5 years’ programming experience and modest code review experience. The experiment shows that the proposed method has significantly improved True Positives and Sensitivity by about 400 %, improved Precision by approximately 200 %, and reduced around one-third of False Positives. The effects were consistent across different tasks and different code reviewers.

KCO: Balancing class distribution in just-in-time software defect prediction using kernel crossover oversampling.

The performance of the defect prediction model by using balanced and imbalanced datasets makes a big impact on the discovery of future defects. Current resampling techniques only address the imbalanced datasets without taking into consideration redundancy and noise inherent to the imbalanced datasets. To address the imbalance issue, we propose Kernel Crossover Oversampling (KCO), an oversampling technique based on kernel analysis and crossover interpolation. Specifically, the proposed technique aims to generate balanced datasets by increasing data diversity in order to reduce redundancy and noise. KCO first represents multidimensional features into two-dimensional features by employing Kernel Principal Component Analysis (KPCA). KCO then divides the plotted data distribution by deploying spectral clustering to select the best region for interpolation. Lastly, KCO generates the new defect data by interpolating different data templates within the selected data clusters. According to the prediction evaluation conducted, KCO consistently produced F-scores ranging from 21% to 63% across six datasets, on average. According to the experimental results presented in this study, KCO provides more effective prediction performance than other baseline techniques. The experimental results show that KCO within project and cross project predictions especially consistently achieve higher performance of F-score results.

Optimizing Software Defect Prediction Models: Integrating Hybrid Grey Wolf and Particle Swarm Optimization for Enhanced Feature Selection with Popular Gradient Boosting Algorithm

Software defects, also referred to as software bugs, are anomalies or flaws in computer program that cause software to behave unexpectedly or produce incorrect results. These defects can manifest in various forms, including coding errors, design flaws, and logic mistakes, this defect have the potential to emerge at any stage of the software development lifecycle. Traditional prediction models usually have lower prediction performance. To address this issue, this paper proposes a novel prediction model using Hybrid Grey Wolf Optimizer and Particle Swarm Optimization (HGWOPSO). This research aims to determine whether the Hybrid Grey Wolf and Particle Swarm Optimization model could potentially improve the effectiveness of software defect prediction compared to base PSO and GWO algorithms without hybridization. Furthermore, this study aims to determine the effectiveness of different Gradient Boosting Algorithm classification algorithms when combined with HGWOPSO feature selection in predicting software defects. The study utilizes 13 NASA MDP dataset. These dataset are divided into testing and training data using 10-fold cross-validation. After data is divided, SMOTE technique is employed in training data. This technique generates synthetic samples to balance the dataset, ensuring better performance of the predictive model. Subsequently feature selection is conducted using HGWOPSO Algorithm. Each subset of the NASA MDP dataset will be processed by three boosting classification algorithms namely XGBoost, LightGBM, and CatBoost. Performance evaluation is based on the Area under the ROC Curve (AUC) value. Average AUC values yielded by HGWOPSO XGBoost, HGWOPSO LightGBM, and HGWOPSO CatBoost are 0.891, 0.881, and 0.894, respectively. Results of this study indicated that utilizing the HGWOPSO algorithm improved AUC performance compared to the base GWO and PSO algorithms. Specifically, HGWOPSO CatBoost achieved the highest AUC of 0.894. This represents a 6.5% increase in AUC with a significance value of 0.00552 compared to PSO CatBoost, and a 6.3% AUC increase with a significance value of 0.00148 compared to GWO CatBoost. This study demonstrated that HGWOPSO significantly improves the performance of software defect prediction. The implication of this research is to enhance software defect prediction models by incorporating hybrid optimization techniques and combining them with gradient boosting algorithms, which can potentially identify and address defects more accurately

Hybridization of fuzzy rough feature selection with ANFIS and turbulent flow of water optimization for managing software defect prediction uncertainty

By anticipating system defect-prone units, software-developing businesses aim to increase the quality of software. Despite the development of numerous Data Mining (DM) and Artificial Intelligence (AI) techniques in the Software Defect Prediction (SDP) field, dealing with the uncertainty of datasets persists due to noise, data distribution, class overlapping, proposed model parameters, and old data. This uncertainty issue has a negative impact on the accuracy of software defect prediction. To overcome this limitation, a model-based hybridization of Ant Colony Optimization-inspired Fuzzy Rough Feature Selection (FRAC) followed by adapting the parameters of Adaptive Neuro-Fuzzy Inference System (ANFIS) with a novel algorithm called Turbulent Flow of Water Optimization (TFWO) is recommended. The proposed model (FRAC+TFWANFIS) performed better than contemporary literature and other optimization algorithms in SDP, such as Ant Colony Optimization (ACO), Differential Evolution (DE), ANFIS, Grey Wolf Optimization (GWO), Particle Swarm Optimization (PSO), and Genetic Algorithm (GA). Also, the performance of the proposed model is superior to that of other conventional classification techniques such as Naïve Bayes (NB), Logistic Regression (LR), Multilayer Perceptron (MLP), Support Vector Machine (SVM), Fuzzy Rough Nearest Neighbor (FRNN), Fuzzy Nearest Neighbor (FNN), Bagging, C4.5, Random Forest (RF), and K-Nearest Neighbor (K-NN). Two datasets, PC3 and PC4, with large dimensions from the OPENML platform are used. The experiments are applied with regard to accuracy, Standard Deviation (SD), Root Mean Square Error (RMSE), Mean Square Error (MSE), and other measurement metrics. The uncertainty issue is addressed by the (FRAC+TFWANFIS) model with accuracy 90.8% and 91.1% for PC3 and PC4, respectively.

Predicting the Number of Software Faults using Deep Learning

The software testing phase requires considerable time, effort, and cost, particularly when there are many faults. Thus, developers focus on the evolution of Software Fault Prediction (SFP) to predict faulty units in advance, therefore, improving software quality significantly. Forecasting the number of faults in software units can efficiently direct software testing efforts. Previous studies have employed several machine learning models to determine whether a software unit is faulty. In this study, a new, simple deep neural network approach that can adapt to the type of input data was designed, utilizing Convolutional Neural Networks (CNNs) and Multi-Layer Perceptron (MLP), to predict the number of software faults. Twelve open-source software project datasets from the PROMISE repository were used for testing and validation. As data imbalance can negatively impact prediction accuracy, the new version of synthetic minority over-sampling technique (SMOTEND) was used to resolve data imbalance. In experimental results, a lower error rate was obtained for MLP, compared to CNN, reaching 0.195, indicating the accuracy of this prediction model. The proposed approach proved to be effective when compared with two of the best machine learning models in the field of prediction. The code will be available on GitHub.

An energy‐aware software fault detection system based on hierarchical rule approach for enhancing quality of service in internet of things‐enabled wireless sensor network

AbstractOf late, the Internet of Things (IoT) has progressed in its pervasiveness across the globe for diverse applications. Wireless Sensor Network (WSN) is one of the prominent technologies employed in IoT environments where multiple tiny sensor nodes are distributed to sense real‐time observations about unforeseeable areas for control and managerial purposes. Owing to the presence of sensors in inaccessible regions and their battery restrictions, different types of software faults occur in IoT‐enabled WSNs (IWSNs). These faults create uncertainty in data reading which causes serious damage to the sensor network. Hence, the IWSN necessitates an effective fault‐detection methodology to continue optimal activity despite the existence of software faults. This work proposes a novel Energy‐Aware Hierarchical Rule‐based Software Fault Detection (HRSFD) model to identify various software faults with minimum energy depletion in the IWSN environment. Primarily, the proposed model extracts antecedent attributes from the characteristics of the sensed data. Its abnormal values can be identified based on the obtained antecedent attributes. Subsequently, the category of the software fault is determined by applying a hierarchical rule strategy. Finally, from the simulation results, it is apparent that the fault detection accuracy rate of the proposed HRSFD model attains 99.12% for dense networks. The lifetime of the network is also prolonged by 18% as compared to the existing state‐of‐the‐art models.

Proactive Fault Tolerance in Distributed Cloud Systems: A Review of Predictive and Preventive Techniques

In a cloud computing environment, various hardware and software services are provided to the users across multiple servers and data centers. These servers are communicated to each other to allow greater scalability, flexibility, and reliability. Reliability is a vital factor in cloud computing that ensures that the requested services will be delivered to the users whenever they request them. However, different hardware or software faults may occur in cloud servers or data centers that prevent the users from receiving the service. Fault tolerance is defined as the ability of the system to provide services to the users even with the presence of faults or failures. In this review, we focused on some of the emerging fault tolerance techniques researchers have proposed to tackle the fault issues in cloud computing. We divided these techniques into three main categories: proactive and reactive techniques. Proactive techniques involve protecting the system defects by proposing certain procedures to prevent reaching the defective condition. Reactive techniques refer to the ability of the cloud system to recover the defective server or framework to continue working and providing the service.

Improving transfer learning for software cross-project defect prediction

AbstractSoftware cross-project defect prediction (CPDP) makes use of cross-project (CP) data to overcome the lack of data necessary to train well-performing software defect prediction (SDP) classifiers in the early stage of new software projects. Since the CP data (known as the source) may be different from the new project’s data (known as the target), this makes it difficult for CPDP classifiers to perform well. In particular, it is a mismatch of data distributions between source and target that creates this difficulty. Transfer learning-based CPDP classifiers are designed to minimize these distribution differences. The first Transfer learning-based CPDP classifiers treated these differences equally, thereby degrading prediction performance. To this end, recent research has the Weighted Balanced Distribution Adaptation (W-BDA) method to leverage the importance of both distribution differences to improve classification performance. Although W-BDA has been shown to improve model performance in CPDP and tackle the class imbalance by balancing the class proportion of each domain, research to date has failed to consider model performance in light of increasing target data. We provide the first investigation studying the effects of increasing the target data when leveraging the importance of both distribution differences. We extend the initial W-BDA method and call this extension the W-BDA$$\mathbf {^{+}}$$ + method. To evaluate the effectiveness of W-BDA$$\mathbf {^{+}}$$ + for improving CPDP performance, we conduct eight experiments on 18 projects from four datasets, where data sampling was performed with different sampling methods. Data sampling was only performed on the baseline methods and not on our proposed W-BDA$$\mathbf {^{+}}$$ + and the original W-BDA because data sampling issues do not exist for these two methods. We evaluate our method using four complementary indicators (i.e., Balanced Accuracy, AUC, F-measure and G-Measure). Our findings reveal an average improvement of 6%, 7.5%, 10% and 12% for these four indicators when W-BDA$$\mathbf {^{+}}$$ + is compared to the original W-BDA and five other baseline methods (for all four of the sampling methods used). Also, as the target to source ratio is increased with different sampling methods, we observe a decrease in performance for the original W-BDA, with our W-BDA$$\mathbf {^{+}}$$ + approach outperforming the original W-BDA in most cases. Our results highlight the importance of having an awareness of the effect of the increasing availability of target data in CPDP scenarios when using a method that can handle the class imbalance problem.

An optimized deep learning method for software defect prediction using Whale Optimization Algorithm

An optimized deep learning method for software defect prediction using Whale Optimization Algorithm

Insights of effectivity analysis of learning-based approaches towards software defect prediction

Software defect prediction is one of the essential sets of operation towards mitigating issues of risk management in software development known to contribute towards enhancing the quality of software. There is evolution of various methodologies towards resolving this issue while learning-based methodology is witnessed to be the most dominant contributor. The problem identified is that there are yet many unsolved queries associated with practical viability of such learning-based approach adoption in software quality management. Proposed approaches discussed in this paper contributes towards mitigating this challenge by introducing a simplified, compact, and crisp analysis of effectiveness associated with learning-based schemes. The paper presents its major findings of effectivity analysis of machine learning, deep learning, hybrid, and other miscellaneous approaches deployed for fault prediction followed by highlighting research trend. The major findings infer that feature selection, data imbalance, interpretability, and in adequate involvement of context are prime gaps in existing methods. The paper also contributes towards research gap as well as essential learning outcomes of present review work.

Joint optimization of maintenance and spares inventory policy for a series-parallel system considering dependent failure processes

This paper investigates the joint optimization of condition-based opportunistic maintenance and spare inventory policies for a series-parallel system consisting of software and components. Software failure follows a homogeneous Poisson process. Components are subject to mutually dependent competing failure processes. When the software failure occurs, an opportunistic inspection will be implemented for the components. Besides the maintenance policy, the (s, S) inventory policy is applied for components. The maintenance and order decisions will be made according to the conditions of the components and inventory level at each inspection. The state transition probabilities and expected sojourn times can be obtained by the formulated semi-Markov decision process. The optimal inspection interval, the preventive replacement threshold of the component, and the order point can be derived by minimizing the expected average cost rate. Finally, the effectiveness and superiority of the proposed joint optimization model are verified by an illustrative example.

A Reliable Framework for Detection of Smart Contract Vulnerabilities for Enhancing Operability in Inter-Organizational Systems

Information and communication technology based inter-organizational systems enable companies to integrate information and conduct business electronically across different parts of the organization. For organizations embracing blockchain, smart contracts provide automation and operational efficiency for inter-organizational systems. Initially utilised for financial transactions, smart contract are extended beyond banking and deployed in wide number of organizations. Smart contracts are regarded as self-executing type of contract consisting of agreement’s terms embedded directly into the code which plays a vital role in operability for inter-organizational systems, however, smart contract vulnerabilities can arise due to programming errors, leading to security issues. The effects of smart contract vulnerabilities can be significant, including loss of funds, unauthorized access to sensitive information, manipulation of data, and loss of trust in the application leading to catastrophic financial losses followed by legal implications for an organization based on blockchain technology. The goal of smart contracts exploiting vulnerabilities is to discover and eliminate potential security vulnerabilities in smart contract code prior to it being deployed. Detecting vulnerabilities in a timely manner helps to prevent financial losses, unauthorized access, and data manipulation. In order to provide a robust solution to detect vulnerabilities in smart contracts, the proposed methodology presents a novel approach for rapid detection of vulnerabilities by integrating genetic algorithm with isolation forest. Furthermore, enhancing smart contract vulnerability identification with higher accuracy and false-positive rate provides a reliable gateway for organizations to adopt blockchain.

Enhancing Software Defect Prediction accuracy using Modified Entropy Calculation in Random Forest Algorithm

Imagine you are trying to classify software defect for a large dataset. How will you choose the best algorithm to do that? For the above problem we have various algorithms like Random Forest, Support Vector Machine, Neural Networks, Naive Bayes, K-Nearest Neighbours, Decision Tree, Logistic Regression etc. One of the most used methods is Random Forest algorithm, which uses multiple Decision Trees to make predictions. However, this algorithm relies on a complex calculation called Entropy, which measures the uncertainty in the data. Entropy is a function that uses natural logarithm which may be time consuming calculation. Is there a better way to calculate entropy? In this research, we have explored a different way to calculate the natural logarithm using the Taylor series expression. It is a series consisting of sum of infinite terms that approximates any function by using its derivatives. We further modified the Random Forest algorithm by replacing the natural logarithm with the Taylor series expression in the Entropy formula. We tested our modified algorithm on dataset and compared its performance with the original Entropy formula. We found that our modification in the algorithm has improved the accuracy of the algorithm on software defect prediction

Effect of Data Sampling on Cone Shaped Embedded Normalization in Just in Time Software Defect Prediction

Effect of Data Sampling on Cone Shaped Embedded Normalization in Just in Time Software Defect Prediction

Failures and Repairs: An Examination of Software System Failure

The central theme of the article is to provide a better knowledge of software system failures and how to assure, maintain, and provide the support software systems that are in production. It includes the results of our search study. We conducted a qualitative analysis of thirty cases: fifteen from public incident reports and fifteen from in-depth interviews with engineers. Understanding and classifying failures as well as their identification, investigation, and mitigation were the main goals of our study. Furthermore, we obtained important analytical insights that are pertinent to the condition of practice as it is now and related problems. It is common for engineers to be unaware of the scaling limitations of the systems they support until those limits are exceeded, and failures have the potential to cascade across a system and cause catastrophic outages.We argue that the difficulties we've discovered may lead to changes in how systems are designed and supported.

Software Failure Prediction Based On Program State and First-Error Characteristics

Abstract Evaluating program reliability against software faults involves the work of analyzing program failure against software faults as behavior on failures is fundamental for assessing reliability. Similar failure behavior reflects similar reliability assessment against software faults of same time–space distribution. Since program failure is determined by fault manifestations, working on similarities on fault manifestations helps figuring out failures in similarity as well. In this paper, we propose a novel method to characterize program behavior by defining fine-grained runtime states at assembly-level of code, aiming to capture the very first abnormal manifestation—first-error—after software fault being activated during program run. Failure prediction model and measurement is presented based on similarities reflected by the runtime behaviors obtained. Fault injection experiments are conducted to verify the prediction measurement by utilizing software faults of Orthogonal Defect Classification to inject and MiBench programs to perform. Over 15 000+ times of fault injection by considering type and location of fault were conducted into each program in order to quantify and analyze the sensitivity on first-error and similarity on failure accordingly. The results show that programs having particular structural features (e.g. massive operations with regards to calculations, nested function-calls, case structures and loop structures, etc.) are well characterized toward first-error behaviors by extracting fine-grained states. Similarities on first-error sensitivity can be represented better for these programs as well. Same trend is seen on failure behavior and its prediction measurement.

ROBUST: 221 bugs in the Robot Operating System

As robotic systems such as autonomous cars and delivery drones assume greater roles and responsibilities within society, the likelihood and impact of catastrophic software failure within those systems is increased. To aid researchers in the development of new methods to measure and assure the safety and quality of robotics software, we systematically curated a dataset of 221 bugs across 7 popular and diverse software systems implemented via the Robot Operating System (ROS). We produce historically accurate recreations of each of the 221 defective software versions in the form of Docker images, and use a grounded theory approach to examine and categorize their corresponding faults, failures, and fixes. Finally, we reflect on the implications of our findings and outline future research directions for the community.