On cost-effective software defect prediction: Classification or ranking?

Abstract

Software defect prediction aims to identify defect-prone software modules in the purpose of allocating limited software reliability resources in a cost-effective manner. To identify these defect-prone modules, a classification model is usually built to classify software modules into defective or clean. However, a number of false alarms may be generated if the classifier is not so strong or the training modules are insufficient, which may waste many software reliability resources especially when the identified defect-prone modules are more than the available software reliability resources. A straight-forward way to prioritize the defect-prone modules to be checked is to rank the modules based on the predictive confidence generated by the classifier. Nevertheless, due to the ‘point-wise’ learning nature of the classifier, such predictive confidence may not be effective in generating the ordering of defect-proneness. To address this problem, we argue that the defect prediction model should be generated from a ‘pair-wise’ perspective instead of a ‘point-wise’ perspective, to explicitly model the pair-wise relations between software modules and help improve the prediction. We cast the software defect prediction as a learning to rank problem and address it by proposing a novel defect ranking model named DeRanken. Empirical studies on multiple open source software projects indicate that ranking model is able to provide better module prioritization than the classification model for Within-Project, Cross-Version and Cross-Project Defect Prediction, leading to a more cost effective software reliability.