Software Defect Prediction Techniques Research Articles

Exploring the impact of data preprocessing techniques on composite classifier algorithms in cross-project defect prediction

Success in software projects is now an important challenge. The main focus of the engineering community is to predict software defects based on the history of classes and other code elements. However, these software defect prediction techniques are effective only as long as there is enough data to train the prediction model. To mitigate this problem, cross-project defect prediction is used. The purpose of this research investigation is twofold: first, to replicate the experiments in the original paper proposal, and second, to investigate other settings regarding defect prediction with the aim of providing new insights and results regarding the best approach. In this study, three composite algorithms, namely AvgVoting, MaxVoting and Bagging are used. These algorithms integrate multiple machine classifiers to improve cross-project defect prediction. The experiments use pre-processed methods (normalization and standardization) and also feature selection. The results of the replicated experiments confirm the original findings when using raw data for all three methods. When normalization is applied, better results than in the original paper are obtained. Even better results are obtained when feature selection is used. In the original paper, the MaxVoting approach shows the best performance in terms of the F-measure, and BaggingJ48 shows the best performance in terms of cost-effectiveness. The same results in terms of F-measure were obtained in the current experiments: best MaxVoting, followed by AvgVoting and then by BaggingJ48. Our results emphasize the previously obtained outcome; the original study is confirmed when using raw data. Moreover, we obtained better results when using preprocessing and feature selection.

Multi‐task deep neural networks for just‐in‐time software defect prediction on mobile apps

SummaryWith the development of smartphones, mobile applications play an irreplaceable role in our daily life, which characteristics often commit code changes to meet new requirements. This characteristic can introduce defects into the software. To provide immediate feedback to developers, previous researchers began to focus on just‐in‐time (JIT) software defect prediction techniques. JIT defect prediction aims to determine whether code commits will introduce defects into the software. It contains two scenarios, within‐project JIT defect prediction and cross‐project JIT defect prediction. Regardless of whether within‐project JIT defect prediction or cross‐project JIT defect prediction all need to have enough labeled data (within‐project JIT defect prediction assumes that have plenty of labeled data from the same project, while cross‐project JIT defect prediction assumes that have sufficient labeled data from source projects). However, in practice, both the source and target projects may only have limited labeled data. We propose the MTL‐DNN method based on multi‐task learning to solve this question. This method contains the data preprocessing layer, input layer, shared layers, task‐specific layers, and output layer. Where the common features of multiple related tasks are learned by sharing layers, and the unique features of each task are learned by the task‐specific layers. For verifying the effectiveness of the MTL‐DNN approach, we evaluate our method on 15 Android mobile apps. The experimental results show that our method significantly outperforms the state‐of‐the‐art single‐task deep learning and classical machine learning methods. This result shows that the MTL‐DNN method can effectively solve the problem of insufficient labeled training data for source and target projects.

Deep Learning-Based Software Defect Prediction via Semantic Key Features of Source Code—Systematic Survey

Software defect prediction (SDP) methodology could enhance software’s reliability through predicting any suspicious defects in its source code. However, developing defect prediction models is a difficult task, as has been demonstrated recently. Several research techniques have been proposed over time to predict source code defects. However, most of the previous studies focus on conventional feature extraction and modeling. Such traditional methodologies often fail to find the contextual information of the source code files, which is necessary for building reliable prediction deep learning models. Alternatively, the semantic feature strategies of defect prediction have recently evolved and developed. Such strategies could automatically extract the contextual information from the source code files and use them to directly predict the suspicious defects. In this study, a comprehensive survey is conducted to systematically show recent software defect prediction techniques based on the source code’s key features. The most recent studies on this topic are critically reviewed through analyzing the semantic feature methods based on the source codes, the domain’s critical problems and challenges are described, and the recent and current progress in this domain are discussed. Such a comprehensive survey could enable research communities to identify the current challenges and future research directions. An in-depth literature review of 283 articles on software defect prediction and related work was performed, of which 90 are referenced.

Software Defect Prediction and Software Quality Assessment Using Dlr-Lvq and Fuzzy Rules

Recently, Software development has been considerably grown. Fault in the software causes fault and interrupts the output. Characteristics like these make it much challenging to avert software flaws. Spontaneously forecasting the amount of flaws within the software modules is essential and also can assist developers to proficiently allot restricted resources. Recently, numerous Software Defect Prediction (SDP) techniques are developed. But, the accuracy and time consuming challenges still remain to be solved. Also, a few top-notch techniques don't properly classify the software whereas it is a needed metric to ensure quality standards. This work proffers a novel Decaying Learning Rate – Learning vector Quantization (DLR-LVQ) classifier to forecast the software defect. The proposed methods consist of the following steps: redundant data removal, feature extraction (FE), feature oversampling, data normalization, defect prediction (DP), and quality prediction. The proposed DLR-LVQ’s attained outcome is assessed with the existent methodologies. The outcomes exhibit that the methodology proposed attains efficient classification outcomes are examined. Keywords: Software Defect Prediction (SDP), Non defective software quality prediction, BM-SMOTE, Decaying Learning Rate, Learning Vector Quantization, Fuzzy rules, HDFS and Map Reduce.

Towards Design and Feasibility Analysis of DePaaS: AI Based Global Unified Software Defect Prediction Framework

Using artificial intelligence (AI) based software defect prediction (SDP) techniques in the software development process helps isolate defective software modules, count the number of software defects, and identify risky code changes. However, software development teams are unaware of SDP and do not have easy access to relevant models and techniques. The major reason for this problem seems to be the fragmentation of SDP research and SDP practice. To unify SDP research and practice this article introduces a cloud-based, global, unified AI framework for SDP called DePaaS—Defects Prediction as a Service. The article describes the usage context, use cases and detailed architecture of DePaaS and presents the first response of the industry practitioners to DePaaS. In a first of its kind survey, the article captures practitioner’s belief into SDP and ability of DePaaS to solve some of the known challenges of the field of software defect prediction. This article also provides a novel process for SDP, detailed description of the structure and behaviour of DePaaS architecture components, six best SDP models offered by DePaaS, a description of algorithms that recommend SDP models, feature sets and tunable parameters, and a rich set of challenges to build, use and sustain DePaaS. With the contributions of this article, SDP research and practice could be unified enabling building and using more pragmatic defect prediction models leading to increase in the efficiency of software testing.

Cross-Project Software Defect Prediction Based on Class Code Similarity

Software defect prediction techniques can help software developers find software defects as soon as possible. It can also reduce the cost of software development. This technique usually predicts the target project through the entire source project. However, the data distribution difference between the entire source project and the target project is generally large, so the software defect prediction accuracy is not high. we propose a cross-project software defect prediction technique based on class code similarity CCS-CPDP. Firstly, this technique converts the code set extracted by AST(Abstract Syntax Tree) into a vector set through the DTI (Doc2Bow and TF-IDF) strategy; Secondly, the similarity will be calculated between the vector set of target items and training items; Finally, according to the principle of the majority decision subordinate category in KNN, the number of most similar class instances of the training item is determined, and the source item is refined by selecting the class instance, thereby we can realize software defect prediction. This method compares with four traditional classification models (KNN, Random Forest, Naive Bayes, and Logistic Regression) for defect prediction. Experimental results show that CCS-CPDP compared with the baseline, recall and f1 increased by 18.03% and 14.1% respectively. In addition, the refined source projects selected by CCS-CPDP compared with the current source project selection technology, recall and f1 had improved by 37.6% and 12.7%.

Optimization of association rules using hybrid data mining technique

Software quality has been the important area of interest for decades now in the IT sector and software firms. Defect prediction gives the tester the pointers as to where the bugs will most likely be hidden in the software product. Identifying and reporting the defect probe areas is the main job of software defect prediction techniques. Early detection of software defects during Software Development Life Cycle could lead to a reduction in cost of development, time involved in further testing activities and rework effort post-production and maintenance phase, thus resulting in more reliable software. Software metrics can be used for developing the defect prediction models. Several data mining techniques can be applied on the available open-source software datasets. These datasets are extracted from software programs. Such datasets made publicly available by National Aeronautics and Space Administration for their various softwares have been extensively used in software engineering-related research activities. These datasets contain information on associated Software Metrics at module level. The proposed idea is a novel hybrid data mining technique consisting of Clustering and Modified Apriori Algorithm that results in improved efficiency and reliability of Software Defect Prediction. This technique works by reducing the number of association rules generated. The results are achieved by using interestingness measure called spread. The paper also does a comparative analysis of the results obtained from the novel technique with the existing hybrid technique of Clustering and Apriori.

Survey on software defect prediction techniques

ABSTRACT Recent advancements in technology have emerged the requirements of hardware and software applications. Along with this technical growth, software industries also have faced drastic growth in the demand of software for several applications. For any software industry, developing good quality software and maintaining its eminence for user end is considered as most important task for software industrial growth. In order to achieve this, software engineering plays an important role for software industries. Software applications are developed with the help of computer programming where codes are written for desired task. Generally, these codes contain some faulty instances which may lead to the buggy software development cause due to software defects. In the field of software engineering, software defect prediction is considered as most important task which can be used for maintaining the quality of software. Defect prediction results provide the list of defect-prone source code artefacts so that quality assurance team scan effectively allocate limited resources for validating software products by putting more effort on the defect-prone source code. As the size of software projects becomes larger, defect prediction techniques will play an important role to support developers as well as to speed up time to market with more reliable software products. One of the most exhaustive and pricey part of embedded software development is consider as the process of finding and fixing the defects. Due to complex infrastructure, magnitude, cost and time limitations, monitoring and fulfilling the quality is a big challenge, especially in automotive embedded systems. However, meeting the superior product quality and reliability is mandatory. Hence, higher importance is given to V&V (Verification & Validation). Software testing is an integral part of software V&V, which is focused on promising accurate functionality and long-term reliability of software systems. Simultaneously, software testing requires much effort, cost, infrastructure and expertise as the development. The costs and efforts elevate in safety critical software systems. Therefore, it is essential to have a good testing strategy for any industry with high software development costs. In this work, we are planning to develop an efficient approach for software defect prediction by using soft computing based machine learning techniques which helps to predict optimize the features and efficiently learn the features.

A Survey of Automatic Software Vulnerability Detection, Program Repair, and Defect Prediction Techniques

Open source software has been widely used in various industries due to its openness and flexibility, but it also brings potential software security problems. Together with the large-scale increase in the number of software and the increase in complexity, the traditional manual methods to deal with these security issues are inefficient and cannot meet the current cyberspace security requirements. Therefore, it is an important research topic for researchers in the field of software security to develop more intelligent technologies to apply to potential security issues in software. The development of deep learning technology has brought new opportunities for the study of potential security issues in software, and researchers have successively proposed many automation methods. In this paper, these automation technologies are evaluated and analysed in detail from three aspects: software vulnerability detection, software program repair, and software defect prediction. At the same time, we point out some problems of these research methods, give corresponding solutions, and finally look forward to the application prospect of deep learning technology in automated software vulnerability detection, automated program repair, and automated defect prediction.

Can Defect Prediction Be Useful for Coarse-Level Tasks of Software Testing?

It is popular to use software defect prediction (SDP) techniques to predict bugs in software in the past 20 years. Before conducting software testing (ST), the result of SDP assists on resource allocation for ST. However, DP usually works on fine-level tasks (or white-box testing) instead of coarse-level tasks (or black-box testing). Before ST or without historical execution information, it is difficult to get resource allocated properly. Therefore, a SDP-based approach, named DPAHM, is proposed to assist on arranging resource for coarse-level tasks. The method combines analytic hierarchy process (AHP) and variant incidence matrix. Besides, we apply the proposed DPAHM into a proprietary software, named MC. Besides, we conduct an up-to-down structure, including three layers for MC. Additionally, the performance measure of each layer is calculated based on the SDP result. Therefore, the resource allocation strategy for coarse-level tasks is gained according to the prediction result. The experiment indicates our proposed method is effective for resource allocation of coarse-level tasks before executing ST.

SLDeep: Statement-level software defect prediction using deep-learning model on static code features

Software defect prediction (SDP) seeks to estimate fault-prone areas of the code to focus testing activities on more suspicious portions. Consequently, high-quality software is released with less time and effort. The current SDP techniques however work at coarse-grained units, such as a module or a class, putting some burden on the developers to locate the fault. To address this issue, we propose a new technique called as Statement-Level software defect prediction using Deep-learning model (SLDeep). The significance of SLDeep for intelligent and expert systems is that it demonstrates a novel use of deep-learning models to the solution of a practical problem faced by software developers. To reify our proposal, we defined a suite of 32 statement-level metrics, such as the number of binary and unary operators used in a statement. Then, we applied as learning model, long short-term memory (LSTM). We conducted experiments using 119,989 C/C++ programs within Code4Bench. The programs comprise 2,356,458 lines of code of which 292,064 lines are faulty. The benchmark comprises a diverse set of programs and versions, written by thousands of developers. Therefore, it tends to give a model that can be used for cross-project SDP. In the experiments, our trained model could successfully classify the unseen data (that is, fault-proneness of new statements) with average performance measures 0.979, 0.570, and 0.702 in terms of recall, precision, and accuracy, respectively. These experimental results suggest that SLDeep is effective for statement-level SDP. The impact of this work is twofold. Working at statement-level further alleviates developer's burden in pinpointing the fault locations. Second, cross-project feature of SLDeep helps defect prediction research become more industrially-viable.

Statistical assessment of nonlinear manifold detection-based software defect prediction techniques

Prediction of software defects has immense importance for obtaining desired outcome at minimised cost and so attracted researchers working on this topic applying various techniques, which were not found fully effective. Software datasets comprise of redundant features that hinder effective application of techniques resulting inappropriate defect prediction. Hence, it requires newer application of nonlinear manifold detection techniques (nonlinear MDTs) that has been examined for accurate prediction of defects at lesser time and cost using different classification techniques. In this work, we analysed and tested the effect of nonlinear MDTs to find out accurate and best classification technique for all datasets. Comparison has been made between the results of without or with nonlinear MDTs and paired two-tailed T-test has been performed for statistical testing and verifying the performance of classifiers using nonlinear MDTs on all datasets. Outcome revealed that among all nonlinear MDTs, FastMVU makes most accurate prediction of software defects.

Statistical assessment of nonlinear manifold detection-based software defect prediction techniques

Prediction of software defects has immense importance for obtaining desired outcome at minimised cost and so attracted researchers working on this topic applying various techniques, which were not found fully effective. Software datasets comprise of redundant features that hinder effective application of techniques resulting inappropriate defect prediction. Hence, it requires newer application of nonlinear manifold detection techniques (nonlinear MDTs) that has been examined for accurate prediction of defects at lesser time and cost using different classification techniques. In this work, we analysed and tested the effect of nonlinear MDTs to find out accurate and best classification technique for all datasets. Comparison has been made between the results of without or with nonlinear MDTs and paired two-tailed T-test has been performed for statistical testing and verifying the performance of classifiers using nonlinear MDTs on all datasets. Outcome revealed that among all nonlinear MDTs, FastMVU makes most accurate prediction of software defects.

SOFTWARE DEFECT PREDICTION USING DEEP BELIEF NETWORK WITH L1-REGULARIZATION BASED OPTIMIZATION

Abstract: Software defect prediction is considered as most interesting area for researchers in the field of software engineering. This process of defect prediction, identifies the bug during automated testing process which prevents the development of faulty software module. According to this process, previous archives of software modules are considered for analyzing the quality based on data mining and machine leaning concepts, which identifies the faults in software modules. Several techniques have been presented for software defect prediction using data mining techniques and machine learning techniques but achieving desired accuracy in performance is a challenging task for researchers. To address this issue, in this work we have presented a new approach for software defect prediction by combining genetic algorithm optimization process for feature subspace reduction and deep belief network for pattern learning. Deep belief networks are further improved by applying L1-regularization scheme resulting in better learning process by reducing the overfitting error. This combined model is implemented on SPIE lab software defect database. An extensive experimental study is carried out which shows that proposed approach achieves higher accuracy when compared with state-of-the-art software defect prediction techniques.

Combined classifier for cross-project defect prediction: an extended empirical study

To facilitate developers in effective allocation of their testing and debugging efforts, many software defect prediction techniques have been proposed in the literature. These techniques can be used to predict classes that are more likely to be buggy based on the past history of classes, methods, or certain other code elements. These techniques are effective provided that a sufficient amount of data is available to train a prediction model. However, sufficient training data are rarely available for new software projects. To resolve this problem, cross-project defect prediction, which transfers a prediction model trained using data from one project to another, was proposed and is regarded as a new challenge in the area of defect prediction. Thus far, only a few cross-project defect prediction techniques have been proposed. To advance the state of the art, in this study, we investigated seven composite algorithms that integrate multiple machine learning classifiers to improve cross-project defect prediction. To evaluate the performance of the composite algorithms, we performed experiments on 10 open-source software systems from the PROMISE repository, which contain a total of 5,305 instances labeled as defective or clean. We compared the composite algorithms with the combined defect predictor where logistic regression is used as the meta classification algorithm (CODEP Logistic ), which is the most recent cross-project defect prediction algorithm in terms of two standard evaluation metrics: cost effectiveness and F-measure. Our experimental results show that several algorithms outperform CODEP Logistic : Maximum voting shows the best performance in terms of F-measure and its average F-measure is superior to that of CODEP Logistic by 36.88%. Bootstrap aggregation (BaggingJ48) shows the best performance in terms of cost effectiveness and its average cost effectiveness is superior to that of CODEP Logistic by 15.34%.

Software defect prediction techniques using metrics based on neural network classifier

Software industries strive for software quality improvement by consistent bug prediction, bug removal and prediction of fault-prone module. This area has attracted researchers due to its significant involvement in software industries. Various techniques have been presented for software defect prediction. Recent researches have recommended data-mining using machine learning as an important paradigm for software bug prediction. state-of-art software defect prediction task suffer from various issues such as classification accuracy. However, software defect datasets are imbalanced in nature and known fault prone due to its huge dimension. To address this issue, here we present a combined approach for software defect prediction and prediction of software bugs. Proposed approach delivers a concept of feature reduction and artificial intelligence where feature reduction is carried out by well-known principle component analysis (PCA) scheme which is further improved by incorporating maximum-likelihood estimation for error reduction in PCA data reconstruction. Finally, neural network based classification technique is applied which shows prediction results. A framework is formulated and implemented on NASA software dataset where four datasets i.e., KC1, PC3, PC4 and JM1 are considered for performance analysis using MATLAB simulation tool. An extensive experimental study is performed where confusion, precision, recall, classification accuracy etc. parameters are computed and compared with existing software defect prediction techniques. Experimental study shows that proposed approach can provide better performance for software defect prediction.

Investigating software testing and maintenance reports: Case study

ContextAlthough many papers have been published on software development and defect prediction techniques, problem reports in real projects quite often differ from those described in the literature. Hence, there is still a need for deeper exploration of case studies from industry. ObjectiveThe aim of this study is to present the impact of fine-grained problem reports on improving evaluation of testing and maintenance processes. It is targeted at projects involving several releases and complex schemes of problem handling. This is based on our experience gained while monitoring several commercial projects. MethodExtracting certain features from detailed problem reports, we derive various measures and present analysis models which characterize and visualize the effectiveness of testing and problem resolution processes. The considered reports describe types of problems (e.g. defects), their locations in project versions and software modules, ways of their resolution, etc. The performed analysis is related to eleven projects developed in the same company. This study is an exploratory research with some explanatory features. Moreover, having identified some drawbacks, we present extensions of problem reports and their analysis which have been verified in another industrial case study project. ResultsFine-grained (accurate) problem handling reports provide a wider scope of possible measures to assess the relevant development processes. This is helpful in controlling single projects (local perspective) as well as in managing these processes in the whole company (global perspective). ConclusionDetailed problem handling reports extend the space and quality of statistical analysis, they provide significant enhancement in evaluation and refinement of software development processes as well as in reliability prediction.

A study of subgroup discovery approaches for defect prediction

ContextAlthough many papers have been published on software defect prediction techniques, machine learning approaches have yet to be fully explored. ObjectiveIn this paper we suggest using a descriptive approach for defect prediction rather than the precise classification techniques that are usually adopted. This allows us to characterise defective modules with simple rules that can easily be applied by practitioners and deliver a practical (or engineering) approach rather than a highly accurate result. MethodWe describe two well-known subgroup discovery algorithms, the SD algorithm and the CN2-SD algorithm to obtain rules that identify defect prone modules. The empirical work is performed with publicly available datasets from the Promise repository and object-oriented metrics from an Eclipse repository related to defect prediction. Subgroup discovery algorithms mitigate against characteristics of datasets that hinder the applicability of classification algorithms and so remove the need for preprocessing techniques. ResultsThe results show that the generated rules can be used to guide testing effort in order to improve the quality of software development projects. Such rules can indicate metrics, their threshold values and relationships between metrics of defective modules. ConclusionsThe induced rules are simple to use and easy to understand as they provide a description rather than a complete classification of the whole dataset. Thus this paper represents an engineering approach to defect prediction, i.e., an approach which is useful in practice, easily understandable and can be applied by practitioners.

EMPIRICAL EVALUATION OF CLASSIFIERS FOR SOFTWARE RISK MANAGEMENT

Software development involves plenty of risks, and errors exist in software modules represent a major kind of risk. Software defect prediction techniques and tools that identify software errors play a crucial role in software risk management. Among software defect prediction techniques, classification is a commonly used approach. Various types of classifiers have been applied to software defect prediction in recent years. How to select an adequate classifier (or set of classifiers) to identify error prone software modules is an important task for software development organizations. There are many different measures for classifiers and each measure is intended for assessing different aspect of a classifier. This paper developed a performance metric that combines various measures to evaluate the quality of classifiers for software defect prediction. The performance metric is analyzed experimentally using 13 classifiers on 11 public domain software defect datasets. The results of the experiment indicate that support vector machines (SVM), C4.5 algorithm, and K-nearest-neighbor algorithm ranked the top three classifiers.

EMPIRICAL ASSESSMENT OF MACHINE LEARNING BASED SOFTWARE DEFECT PREDICTION TECHNIQUES

Automated reliability assessment is essential for systems that entail dynamic adaptation based on runtime mission-specific requirements. One approach along this direction is to monitor and assess the system using machine learning-based software defect prediction techniques. Due to the dynamic nature of software data collected, Instance-based learning algorithms are proposed for the above purposes. To evaluate the accuracy of these methods, the paper presents an empirical analysis of four different real-time software defect data sets using different predictor models. The results show that a combination of 1R and Instance-based learning along with Consistency-based subset evaluation technique provides a relatively better consistency in achieving accurate predictions as compared with other models. No direct relationship is observed between the skewness present in the data sets and the prediction accuracy of these models. Principal Component Analysis (PCA) does not show a consistent advantage in improving the accuracy of the predictions. While random reduction of attributes gave poor accuracy results, simple Feature Subset Selection methods performed better than PCA for most prediction models. Based on these results, the paper presents a high-level design of an Intelligent Software Defect Analysis tool (ISDAT) for dynamic monitoring and defect assessment of software modules.