In Pursuit of the Best Detection of Positive Data Under User’s Concern on False Alarm

Abstract

Just-In-Time Software Defect Predict (JIT-SDP) has been a popular research topic in the literature of software engineering. In many practical scenarios, software engineers would prefer to pursue the best detection of defect-inducing software changes under the concern of a given false alarm tolerance. However, there have been only two related studies in the Machine Learning (ML) community that are capable of tackling this constraint optimization problem. This paper aims to study how can we utilize the existing ML methods for addressing the research problem in JIT-SDP and how well do they perform on it. Considering the fact that the objective and the constraint are not differentiable, a Differential Evolution (DE) algorithm is by nature suitable for tackling this research problem. Thus, this paper also aims to investigate how can we propose a novel DE algorithm to better address the constraint optimization problem in JIT-SDP. With these aims in mind, this paper adapts the ML methods with a spared validation set to facilitate the constraint learning process, and it also proposes an advanced DE algorithm with an adaptive constraint to pursue the best detection of the positive class under a given false alarm. Experimental results with 10 real-world data sets from the domain of software defect prediction demonstrate that our proposed DE based approach can achieve generally better performance on the constraint optimization problem, deriving better classification models in terms of both objective and the constraint.