Effort-aware and just-in-time defect prediction with neural network.

Lei Qiao,Yan Wang,Jie Zhang

doi:10.1371/journal.pone.0211359

Abstract

Effort-aware just-in-time (JIT) defect prediction is to rank source code changes based on the likelihood of detects as well as the effort to inspect such changes. Accurate defect prediction algorithms help to find more defects with limited effort. To improve the accuracy of defect prediction, in this paper, we propose a deep learning based approach for effort-aware just-in-time defect prediction. The key idea of the proposed approach is that neural network and deep learning could be exploited to select useful features for defect prediction because they have been proved excellent at selecting useful features for classification and regression. First, we preprocess ten numerical metrics of code changes, and then feed them to a neural network whose output indicates how likely the code change under test contains bugs. Second, we compute the benefit cost ratio for each code change by dividing the likelihood by its size. Finally, we rank code changes according to their benefit cost ratio. Evaluation results on a well-known data set suggest that the proposed approach outperforms the state-of-the-art approaches on each of the subject projects. It improves the average recall and popt by 15.6% and 8.1%, respectively.

Highlights

Software quality assurance activities, such as the defect prediction and source code inspection, have a great influence on producing high quality reliable software [1, 2]
In consistent with previous studies, in our paper, we adopt 10-fold cross-validation technique to evaluate the performance of our defect prediction approach (i.e., NNR) against the state-of-the-art approaches (i.e., effort-aware linear regression (EALR), LT, and CBS) on the data set of each subject project
We carry out 10-fold cross-validation on the data set and compare the proposed approach against the state-of-the-art approaches, i.e., EALR [13], LT [18], and CBS [36]

Summary

Introduction

Software quality assurance activities, such as the defect prediction and source code inspection, have a great influence on producing high quality reliable software [1, 2]. Most defect prediction approaches take defect prediction as a binary classification problem that could be solved by different classification algorithms [22,23,24,25,26], e.g., Support Vector Machine (SVM) [23], Random Forest (RF) [27] and Nearest-Neighbor [28] Such approaches classify source code changes into two categories: buggy or clean. Such approaches do not take into account the inspection effort required to discover the predicted defects, i.e., the effort of code inspection on suspicious code changes As a result such approaches maximize classification performance, e.g., precision, recall, and F-measure, but fail to maximize the number of defects identified with given inspection effort (resource) [29]. To improve software reliability [31] with limited human resource, in this paper we propose an effort-aware just-in-time defect prediction approach based on neural networks and deep learning.

Related work Defect prediction

Evaluation

Results

Conclusion