Boosting multi-objective just-in-time software defect prediction by fusing expert metrics and semantic metrics

Abstract

Just-in-time software defect prediction (JIT-SDP) aims to predict whether a code commit is defect-inducing or defect-clean immediately after developers submit their code commits. In our previous study, we modeled JIT-SDP as a multi-objective optimization problem by designing two potential conflict optimization objectives. By only considering expert metrics for code commits, our proposed multi-objective just-in-time software defect prediction (MOJ-SDP) approach can significantly outperform state-of-the-art supervised and unsupervised baselines. Recent studies have shown that deep learning techniques can be used to automatically extract semantic metrics from code commits and achieved promising performance for JIT-SDP. However, it is unclear how well MOJ-SDP performs when semantic metrics are used, and whether these two types of metrics are complementary and can be boosted by fusing them for MOJ-SDP. We conducted an extensive experiment using 27,319 code commits from 21 real-world open-source projects. Our results show that when using semantic features, the performance of MOJ-SDP can be slightly decreased for Popt, but greatly improved for Recall@20%Effort. However, when these two types of metrics are fused based on the model-level fusion with the maximum rule, the performance can be boosted by a large margin and outperform state-of-the-art JIT-SDP baselines.