Defect Prediction Techniques Research Articles

Data Complexity: A New Perspective for Analyzing the Difficulty of Defect Prediction Tasks

Defect prediction is crucial for software quality assurance and has been extensively researched over recent decades. However, prior studies rarely focus on data complexity in defect prediction tasks, and even less on understanding the difficulties of these tasks from the perspective of data complexity. In this article, we conduct an empirical study to estimate the hardness of over 33,000 instances, employing a set of measures to characterize the inherent difficulty of instances and the characteristics of defect datasets. Our findings indicate that: (1) instance hardness in both classes displays a right-skewed distribution, with the defective class exhibiting a more scattered distribution; (2) class overlap is the primary factor influencing instance hardness and can be characterized through feature, structural, and instance-level overlap; (3) no universal preprocessing technique is applicable to all datasets, and it may not consistently reduce data complexity, fortunately, dataset complexity measures can help identify suitable techniques for specific datasets; (4) integrating data complexity information into the learning process can enhance an algorithm’s learning capacity. In summary, this empirical study highlights the crucial role of data complexity in defect prediction tasks, and provides a novel perspective for advancing research in defect prediction techniques.

Exploring the impact of data preprocessing techniques on composite classifier algorithms in cross-project defect prediction

Success in software projects is now an important challenge. The main focus of the engineering community is to predict software defects based on the history of classes and other code elements. However, these software defect prediction techniques are effective only as long as there is enough data to train the prediction model. To mitigate this problem, cross-project defect prediction is used. The purpose of this research investigation is twofold: first, to replicate the experiments in the original paper proposal, and second, to investigate other settings regarding defect prediction with the aim of providing new insights and results regarding the best approach. In this study, three composite algorithms, namely AvgVoting, MaxVoting and Bagging are used. These algorithms integrate multiple machine classifiers to improve cross-project defect prediction. The experiments use pre-processed methods (normalization and standardization) and also feature selection. The results of the replicated experiments confirm the original findings when using raw data for all three methods. When normalization is applied, better results than in the original paper are obtained. Even better results are obtained when feature selection is used. In the original paper, the MaxVoting approach shows the best performance in terms of the F-measure, and BaggingJ48 shows the best performance in terms of cost-effectiveness. The same results in terms of F-measure were obtained in the current experiments: best MaxVoting, followed by AvgVoting and then by BaggingJ48. Our results emphasize the previously obtained outcome; the original study is confirmed when using raw data. Moreover, we obtained better results when using preprocessing and feature selection.

Just-in-Time crash prediction for mobile apps

Just-In-Time (JIT) defect prediction aims to identify defects early, at commit time. Hence, developers can take precautions to avoid defects when the code changes are still fresh in their minds. However, the utility of JIT defect prediction has not been investigated in relation to crashes of mobile apps. We therefore conducted a multi-case study employing both quantitative and qualitative analysis. In the quantitative analysis, we used machine learning techniques for prediction. We collected 113 reliability-related metrics for about 30,000 commits from 14 Android apps and selected 14 important metrics for prediction. We found that both standard JIT metrics and static analysis warnings are important for JIT prediction of mobile app crashes. We further optimized prediction performance, comparing seven state-of-the-art defect prediction techniques with hyperparameter optimization. Our results showed that Random Forest is the best performing model with an AUC-ROC of 0.83. In our qualitative analysis, we manually analysed a sample of 642 commits and identified different types of changes that are common in crash-inducing commits. We explored whether different aspects of changes can be used as metrics in JIT models to improve prediction performance. We found these metrics improve the prediction performance significantly. Hence, we suggest considering static analysis warnings and Android-specific metrics to adapt standard JIT defect prediction models for a mobile context to predict crashes. Finally, we provide recommendations to bridge the gap between research and practice and point to opportunities for future research.

A novel defect prediction method based on semantic feature enhancement

SummaryAlthough cross‐project defect prediction (CPDP) techniques that use traditional manual features to build defect prediction model have been well‐developed, they usually ignore the semantic and structural information inside the program and fail to capture the hidden features that are critical for program category prediction, resulting in poor defect prediction results. Researchers have proposed using deep learning to automatically extract the semantic features of programs and fuse them with traditional features as training data. However, in practice, it is important to explore the effective representation of the semantic features in the programs and how the fusion of a reasonable ratio between the two types of features can maximize the effectiveness of the model. In this paper, we propose a semantic feature enhancement‐based defect prediction framework (SFE‐DP), which augments the semantic feature set extracted from the program code with data. We also introduce a layer of self‐attentive mechanism and a matching layer to filter low‐efficiency and non‐critical semantic features in the model structure. Finally, we combine the idea of hybrid loss function to iteratively optimize the model parameters. Extensive experiments validate that SFE‐DP can outperform the baseline approaches on 90 pairs of CPDP tasks formed by 10 open‐source projects.

On the relative value of clustering techniques for Unsupervised Effort-Aware Defect Prediction

Unsupervised Effort-Aware Defect Prediction (EADP) uses unlabeled data to construct a model and ranks software modules according to the software feature values. Xu et al. (JSS 2021) conducted an exploration of clustering techniques for unsupervised defect prediction and found that several clustering methods exhibit better performance on the F1@20% effort-aware metric. However, their conclusion may not be convincing, as they did not take into account the impact of the Initial False Alarms (IFA) metric on unsupervised EADP. Furthermore, their study did not compare with the state-of-the-art supervised EADP models. To further investigate clustering techniques for unsupervised EADP more comprehensively, we explore the performance of 22 clustering techniques for unsupervised EADP using three classification metrics and six effort-aware metrics. The experimental results demonstrate that (1) the best clustering technique for unsupervised EADP, K-medoids, can significantly reduce the IFA of the ManualUp method to an acceptable range. In contrast, the clustering techniques recommended by Xu et al. exhibit a high IFA value that cannot be deemed acceptable by testing teams; (2) K-medoids performs better than some supervised EADP methods, especially on metrics such as IFA and PMI@20% (Proportion of Modules Inspected when inspecting the top 20% lines of code); (3) better classification performance of clustering techniques could lead to better effort-aware performance. In summary, we recommend using the K-medoids clustering technique for unsupervised EADP and suggest that future research devote more effort to exploring better-unsupervised clustering techniques. In support of reproducibility and future research, we provide the source code used in our study (https://github.com/Andre-Yang816/Clustering4UEADP).

A Comprehensive Hybrid Model for Language-Independent Defect Prediction in Microservices Architecture

The transformation of software development from monolithic frameworks to microservices-based architectures, focusing on the challenges of creating a unified defect prediction model that spans various programming languages in practice of automating integration of code modification into a single codebase. It proposes a hybrid machine learning approach to enhance defect prediction accuracy by integrating different data sources and algorithms. The goal is to create a language and project-independent model. The hybrid model combines Bi-Directional LSTM (BiD-LSTM) networks and Attention mechanisms, static code metrics, and BERT-based language models. BiLSTM-Attention captures temporal dependencies within Abstract Syntax Trees (ASTs), static code metrics provide insights into software complexity, and BERT interprets textual context for a holistic understanding of code snippets. The research methodology involves quantitative techniques, starting with a literature review to establish the theoretical foundation. An empirical study follows, encompassing data gathering, feature crafting and pre-processing, model building, training and evaluation, validation and analysis and conclusions. The research’s insights aim to improve defect prediction techniques, contributing to software engineering’s pursuit of better quality and reliability.

Deep Learning for Software Defect Prediction: An LSTM-based Approach

Software defect prediction is an important aspect of software development, as it helps developers and organizations to identify and resolve bugs in the software before they become major issues. In this paper, we explore the use of machine learning algorithms for software defect prediction. We discuss the different types of machine learning algorithms that have been used for software defect prediction and their advantages and disadvantages. We also provide a comprehensive review of recent studies that have used machine learning algorithms for software defect prediction. The paper concludes with a discussion of the challenges and opportunities in using machine learning algorithms for software defect prediction and the future directions of research in this field. This paper surveys the existing literature on software defect prediction, focusing specifically on deep learning techniques. Compared to existing surveys on the topic, this paper offers a more in-depth analysis of the strengths and weaknesses of deep learning approaches for software defect prediction. It explores the use of LSTMs for this task, which have not been extensively studied in previous surveys. Additionally, this paper provides a comprehensive review of recent research in the field, highlighting the most promising deep learning models and techniques for software defect prediction. The results of this survey demonstrate that LSTM-based deep learning models can outperform traditional machine learning approaches and achieve state-of-the-art results in software defect prediction. Furthermore, this paper provides insights into the challenges and limitations of deep learning approaches for software defect prediction, highlighting areas for future research and improvement. Overall, this paper offers a valuable resource for researchers and practitioners interested in using deep learning techniques for software defect prediction..

Deep Learning for Software Defect Prediction: An LSTM-based Approach

Software defect prediction is an important aspect of software development, as it helps developers and organizations to identify and resolve bugs in the software before they become major issues. In this paper, we explore the use of machine learning algorithms for software defect prediction. We discuss the different types of machine learning algorithms that have been used for software defect prediction and their advantages and disadvantages. We also provide a comprehensive review of recent studies that have used machine learning algorithms for software defect prediction. The paper concludes with a discussion of the challenges and opportunities in using machine learning algorithms for software defect prediction and the future directions of research in this field. This paper surveys the existing literature on software defect prediction, focusing specifically on deep learning techniques. Compared to existing surveys on the topic, this paper offers a more in-depth analysis of the strengths and weaknesses of deep learning approaches for software defect prediction. It explores the use of LSTMs for this task, which have not been extensively studied in previous surveys. Additionally, this paper provides a comprehensive review of recent research in the field, highlighting the most promising deep learning models and techniques for software defect prediction. The results of this survey demonstrate that LSTM-based deep learning models can outperform traditional machine learning approaches and achieve state-of-the-art results in software defect prediction. Furthermore, this paper provides insights into the challenges and limitations of deep learning approaches for software defect prediction, highlighting areas for future research and improvement. Overall, this paper offers a valuable resource for researchers and practitioners interested in using deep learning techniques for software defect prediction..

Empirical validation of feature selection techniques for cross-project defect prediction

Empirical validation of feature selection techniques for cross-project defect prediction

A research landscape on software defect prediction

AbstractSoftware defect prediction is the process of identifying defective files and modules that need rigorous testing. In the literature, several secondary studies including systematic reviews, mapping studies, and review studies have been reported. However, no research work such as a tertiary study that combines secondary studies has focused on providing a landscape of software defect prediction useful to understand the body of knowledge. Motivated by this, we intend to perform a tertiary study by following a systematic literature review protocol to provide a research landscape of the targeted domain. We synthesize the quality of the secondary studies and investigate the employed techniques and the performance evaluation measures for evaluating the software defect prediction model. Furthermore, this study aims at exploring different datasets employed in the reported experimentation. Moreover, the current study intends at highlighting the research trends, gaps, and opportunities in the targeted research domain. The results indicate that none of the reported defect prediction techniques can be regarded as the best; however, the reported techniques performed better in different testing situations. In addition, machine learning (ML)‐based techniques perform better than traditional statistical techniques mainly due to the potential of discovering the defects and generating generalized results. Moreover, the obtained results highlight the need for further work in the domain of ML‐based techniques. Furthermore, publicly available datasets should be considered for experimentation or replication purposes. The potential future work can focus on data quality, ethical ML, cross‐project defect prediction, early defect prediction process, class imbalance problem, and model overfitting.

Multi‐task deep neural networks for just‐in‐time software defect prediction on mobile apps

SummaryWith the development of smartphones, mobile applications play an irreplaceable role in our daily life, which characteristics often commit code changes to meet new requirements. This characteristic can introduce defects into the software. To provide immediate feedback to developers, previous researchers began to focus on just‐in‐time (JIT) software defect prediction techniques. JIT defect prediction aims to determine whether code commits will introduce defects into the software. It contains two scenarios, within‐project JIT defect prediction and cross‐project JIT defect prediction. Regardless of whether within‐project JIT defect prediction or cross‐project JIT defect prediction all need to have enough labeled data (within‐project JIT defect prediction assumes that have plenty of labeled data from the same project, while cross‐project JIT defect prediction assumes that have sufficient labeled data from source projects). However, in practice, both the source and target projects may only have limited labeled data. We propose the MTL‐DNN method based on multi‐task learning to solve this question. This method contains the data preprocessing layer, input layer, shared layers, task‐specific layers, and output layer. Where the common features of multiple related tasks are learned by sharing layers, and the unique features of each task are learned by the task‐specific layers. For verifying the effectiveness of the MTL‐DNN approach, we evaluate our method on 15 Android mobile apps. The experimental results show that our method significantly outperforms the state‐of‐the‐art single‐task deep learning and classical machine learning methods. This result shows that the MTL‐DNN method can effectively solve the problem of insufficient labeled training data for source and target projects.

A Cloud-Based Software Defect Prediction System Using Data and Decision-Level Machine Learning Fusion

This research contributes an intelligent cloud-based software defect prediction system using data and decision-level machine learning fusion techniques. The proposed system detects the defective modules using a two-step prediction method. In the first step, the prediction is performed using three supervised machine learning techniques, including naïve Bayes, artificial neural network, and decision tree. These classification techniques are iteratively tuned until the maximum accuracy is achieved. In the second step, the final prediction is performed by fusing the accuracy of the used classifiers with a fuzzy logic-based system. The proposed fuzzy logic technique integrates the predictive accuracy of the used classifiers using eight if–then fuzzy rules in order to achieve a higher performance. In the study, to implement the proposed fusion-based defect prediction system, five datasets were fused, which were collected from the NASA repository, including CM1, MW1, PC1, PC3, and PC4. It was observed that the proposed intelligent system achieved a 91.05% accuracy for the fused dataset and outperformed other defect prediction techniques, including base classifiers and state-of-the-art ensemble techniques.

Cross-Project Defect Prediction with Metrics Selection and Balancing Approach

Abstract In software development, defects influence the quality and cost in an undesirable way. Software defect prediction (SDP) is one of the techniques which improves the software quality and testing efficiency by early identification of defects(bug/fault/error). Thus, several experiments have been suggested for defect prediction (DP) techniques. Mainly DP method utilises historical project data for constructing prediction models. SDP performs well within projects until there is an adequate amount of data accessible to train the models. However, if the data are inadequate or limited for the same project, the researchers mainly use Cross-Project Defect Prediction (CPDP). CPDP is a possible alternative option that refers to anticipating defects using prediction models built on historical data from other projects. CPDP is challenging due to its data distribution and domain difference problem. The proposed framework is an effective two-stage approach for CPDP, i.e., model generation and prediction process. In model generation phase, the conglomeration of different pre-processing, including feature selection and class reweights technique, is used to improve the initial data quality. Finally, a fine-tuned efficient bagging and boosting based hybrid ensemble model is developed, which avoids model over -fitting/under-fitting and helps enhance the prediction performance. In the prediction process phase, the generated model predicts the historical data from other projects, which has defects or clean. The framework is evaluated using25 software projects obtained from public repositories. The result analysis shows that the proposed model has achieved a 0.71±0.03 f1-score, which significantly improves the state-of-the-art approaches by 23 % to 60 %.

Hybrid model with optimization tactics for software defect prediction

Defects are frequent in software systems, and they can cause a lot of issues for users. Despite the fact that many studies have been conducted on employing software product metrics to determine defect-prone modules, defect prediction techniques are still worth investigating. Hence, the aim of this work is to provide a unique Software Defect Prediction (SDP) approach that includes four steps like “(a) pre-processing, (b) feature extraction, (c) feature selection and (d) detection.” At first, the input data are given to the pre-processing step, as well as in the feature extraction step; the “statistical features, raw features, higher-order statistical features as well as proposed entropy features” are extracted from the pre-processed data. In addition, the retrieved features are sent into a feature selection step, wherein the appropriate features are selected utilizing a modified chi-square scheme. In the detection step, a hybrid Long Short-Term Memory (LSTM) and Deep Belief Network (DBN) classifiers are used to predict the defects. To provide a more accurate detection, the weights of both DBN and LSTM are optimally tuned via a Self Improved Social Ski-Driver Optimization (SISSDO) algorithm. This proposed SDP model is a beneficial practice for enhancing software quality and reliability. Moreover, the results of the adopted technique are assessed to traditional techniques on the basis of various measures. In particular, the accuracy of the suggested approach for dataset 3 is 5.80%, 6.52%, 5.07%, 7.97%, 5.80%, 9.42%, 9.42%, 10.15%, 2.17%, and 3.62% better than the extant HC[Formula: see text]+[Formula: see text]ALO, HC[Formula: see text]+[Formula: see text]SMO, HC[Formula: see text]+[Formula: see text]CMBO, HC[Formula: see text]+[Formula: see text]SSD, RNN, CNN, NN, Bi-LSTM, HC+SPFCNN, and HC[Formula: see text]+[Formula: see text]CWAR approaches, correspondingly. Moreover, the computation time of the suggested approach is 17.05%, 5.78%, 1.31%, and 50.53% better than the existing HC[Formula: see text]+[Formula: see text]ALO, HC[Formula: see text]+[Formula: see text]SMO, HC[Formula: see text]+[Formula: see text]CMBO, and HC[Formula: see text]+[Formula: see text]SSD approaches, correspondingly.

Deep Learning-Based Software Defect Prediction via Semantic Key Features of Source Code—Systematic Survey

Software defect prediction (SDP) methodology could enhance software’s reliability through predicting any suspicious defects in its source code. However, developing defect prediction models is a difficult task, as has been demonstrated recently. Several research techniques have been proposed over time to predict source code defects. However, most of the previous studies focus on conventional feature extraction and modeling. Such traditional methodologies often fail to find the contextual information of the source code files, which is necessary for building reliable prediction deep learning models. Alternatively, the semantic feature strategies of defect prediction have recently evolved and developed. Such strategies could automatically extract the contextual information from the source code files and use them to directly predict the suspicious defects. In this study, a comprehensive survey is conducted to systematically show recent software defect prediction techniques based on the source code’s key features. The most recent studies on this topic are critically reviewed through analyzing the semantic feature methods based on the source codes, the domain’s critical problems and challenges are described, and the recent and current progress in this domain are discussed. Such a comprehensive survey could enable research communities to identify the current challenges and future research directions. An in-depth literature review of 283 articles on software defect prediction and related work was performed, of which 90 are referenced.

Predicting cost of defects for segmented products and customers using ensemble learning

Due to technological advances, Big Data Analytics (BDA) has become increasingly important over the last few years. This has led companies to evolve BDA capabilities (BDAC) to manage operations and make better decisions. In this study, we propose a model, Clustering Based Classifier Ensemble Method for Cost of Defect Prediction (CBCEM-CoD), incorporating clustering, classification, prediction, and learning techniques of BDA for quality management in the manufacturing industry. CBCEM-CoD (1) is fact-driven, as it is based on a fundamental problem of the manufacturing industry, (2) integrates different BDA techniques in a specific way when an output of one technique is used as an input of another, and (3) extracts insights from real-world big data and directly offers many implications for practice. In the first stage of the CBCEM-CoD, k-means and agglomerative clustering techniques are used comparatively for segmenting customers and products. CoD values of each product and customer segment are predicted using ensemble learning techniques in the second stage. The model is tested using a case data set from the kitchenware industry. As a result, 53 and 720 different types of customers and products in the train data set are segmented in optimal numbers of 4 and 20 clusters. Around 89% accuracy is obtained for CoD predictions in the test data set. These results have substantial business value since they inform managers how to prioritize their focus on specific products and customer types to reduce the cost of a defect. We also highlight the importance of developing BDAC in dynamically changing environments to create a competitive advantage.

Software Defect Prediction and Software Quality Assessment Using Dlr-Lvq and Fuzzy Rules

Recently, Software development has been considerably grown. Fault in the software causes fault and interrupts the output. Characteristics like these make it much challenging to avert software flaws. Spontaneously forecasting the amount of flaws within the software modules is essential and also can assist developers to proficiently allot restricted resources. Recently, numerous Software Defect Prediction (SDP) techniques are developed. But, the accuracy and time consuming challenges still remain to be solved. Also, a few top-notch techniques don't properly classify the software whereas it is a needed metric to ensure quality standards. This work proffers a novel Decaying Learning Rate – Learning vector Quantization (DLR-LVQ) classifier to forecast the software defect. The proposed methods consist of the following steps: redundant data removal, feature extraction (FE), feature oversampling, data normalization, defect prediction (DP), and quality prediction. The proposed DLR-LVQ’s attained outcome is assessed with the existent methodologies. The outcomes exhibit that the methodology proposed attains efficient classification outcomes are examined. Keywords: Software Defect Prediction (SDP), Non defective software quality prediction, BM-SMOTE, Decaying Learning Rate, Learning Vector Quantization, Fuzzy rules, HDFS and Map Reduce.

Towards Design and Feasibility Analysis of DePaaS: AI Based Global Unified Software Defect Prediction Framework

Using artificial intelligence (AI) based software defect prediction (SDP) techniques in the software development process helps isolate defective software modules, count the number of software defects, and identify risky code changes. However, software development teams are unaware of SDP and do not have easy access to relevant models and techniques. The major reason for this problem seems to be the fragmentation of SDP research and SDP practice. To unify SDP research and practice this article introduces a cloud-based, global, unified AI framework for SDP called DePaaS—Defects Prediction as a Service. The article describes the usage context, use cases and detailed architecture of DePaaS and presents the first response of the industry practitioners to DePaaS. In a first of its kind survey, the article captures practitioner’s belief into SDP and ability of DePaaS to solve some of the known challenges of the field of software defect prediction. This article also provides a novel process for SDP, detailed description of the structure and behaviour of DePaaS architecture components, six best SDP models offered by DePaaS, a description of algorithms that recommend SDP models, feature sets and tunable parameters, and a rich set of challenges to build, use and sustain DePaaS. With the contributions of this article, SDP research and practice could be unified enabling building and using more pragmatic defect prediction models leading to increase in the efficiency of software testing.

Cross-Project Software Defect Prediction Based on Class Code Similarity

Software defect prediction techniques can help software developers find software defects as soon as possible. It can also reduce the cost of software development. This technique usually predicts the target project through the entire source project. However, the data distribution difference between the entire source project and the target project is generally large, so the software defect prediction accuracy is not high. we propose a cross-project software defect prediction technique based on class code similarity CCS-CPDP. Firstly, this technique converts the code set extracted by AST(Abstract Syntax Tree) into a vector set through the DTI (Doc2Bow and TF-IDF) strategy; Secondly, the similarity will be calculated between the vector set of target items and training items; Finally, according to the principle of the majority decision subordinate category in KNN, the number of most similar class instances of the training item is determined, and the source item is refined by selecting the class instance, thereby we can realize software defect prediction. This method compares with four traditional classification models (KNN, Random Forest, Naive Bayes, and Logistic Regression) for defect prediction. Experimental results show that CCS-CPDP compared with the baseline, recall and f1 increased by 18.03% and 14.1% respectively. In addition, the refined source projects selected by CCS-CPDP compared with the current source project selection technology, recall and f1 had improved by 37.6% and 12.7%.

Empirical investigation of hyperparameter optimization for software defect count prediction

Prior identification of defects in software modules can help testers to allocate limited resources efficiently. Defect prediction techniques are helpful for this situation because they allow testers to identify and focus on defect prone parts of the software. The regression approach is a machine learning technique used to find the defect count in software segments. These regression techniques are more effective if their hyperparameters are adjusted. However, limited studies are available for hyperparameter optimization of regression techniques. In this paper, we investigated the impact of hyperparameter optimization on defect count prediction. In an empirical analysis on 15 software defect datasets, we find that hyperparameter optimization of learning techniques: (1) improves the prediction performance for MLPR, Lasso, DTR, Hubber, and SVR, by 16.96%, 8.31%, 8.16%, 6.01% and 5.22%, respectively; (2) linear regression is not optimization-sensitive; (3) overall grid search optimization improved the prediction performance by 4.42% while random search optimization by 3.36%.; (4) non-significant classifier have also changed their ranking substantially, and (5) logistic regression obtained the highest ranking concerning hyperparameter optimization. While both random and grid search performed well, but concerning the default parameter, the grid search always obtained better outcomes; however, it may not with random search. This emphasizes the importance of exploring the parameter space when using parameter-sensitive regression techniques.