Which type of metrics are useful to deal with class imbalance in software defect prediction?

Abstract

ContextThere are various ways to cope with class imbalance problem which is one of the main issues of software defect prediction. Sampling algorithms are implemented on both industrial and open-source software defect prediction data sets by practitioners to wipe out imbalanced data points. Sampling algorithms, up-to-date, have been employed either static or process code metrics. ObjectiveIn this study, sampling algorithms including Virtual, SMOTE, and HSDD (hybrid sampling for defect data sets) are explored using static code and quality metrics together. Our goal is not only to lead practitioners to decide the type of the metrics in defect prediction but also provide useful information for developers to design less defective software projects. MethodWe ran sampling experiments with three sampling algorithms on ten data sets (from GitHub). Feature selection is applied on large features of the data sets. Using five classifiers, the performance of the data sets after sampling is compared with initial data sets. Regression analyzes are implemented on quality metrics to find the most influential metrics for detecting defect proneness. ResultsRegardless of the type of the sampling, prediction performances are similar. Quality metrics surpassed static code metrics with respect to training times and prediction accuracies. ConclusionUsing quality metrics yields better prediction results rather than static code metrics in imbalanced data sets. As the count of project cloning increases, the number of defects decreases. Thus, approaches, related to the class imbalance, should be evaluated not only in terms of static code metrics but also for quality metrics.