Assessing the Significant Impact of Concept Drift in Software Defect Prediction

Abstract

Concept drift is a known phenomenon in software data analytics. It refers to the changes in the data distribution over time. The performance of analytic and prediction models degrades due to the changes in the data over time. To improve prediction performance, most studies propose that the prediction model be updated when concept drift occurs. In this work, we investigate the existence of concept drift and its associated effects on software defect prediction performance. We adopt the strategy of an empirically proven method DDM (Drift Detection Method) and evaluate its statistical significance using the chi-square test with Yates continuity correction. The objective is to empirically determine the concept drift and to calibrate the base model accordingly. The empirical study indicates that the concept drift occurs in software defect datasets, and its existence subsequently degrades the performance of prediction models. Two types of concept drifts (gradual and sudden drifts) were identified using the chi-square test with Yates continuity correction in the software defect datasets studied. We suggest concept drift should be considered by software quality assurance teams when building prediction models.