The need for more informative defect prediction: A systematic literature review

Abstract

Context:Software defect prediction is crucial for prioritising quality assurance tasks, however, there are still limitations to the use of defect models. For example, the outputs often do not provide the defect type, severity, or the cause of the defect. Current models are also often complex in implementation (they use low transparency classifiers such as random forest or support vector machines) and primarily output binary predic- tions. They lack directly actionable outputs, that is, outputs that provide additional information (e.g., defect severity or defect type) to aid in fixing the defect. One approach is to utilise tools of explainable AI. Objective:In order to improve current models and plan the direction for explainability in software defect prediction, we need to understand how explainable current models are. Methods:Starting from 861 papers from multiple databases, we inves- tigated a sample of 132 papers in a systematic literature review. We extracted the following information to answer our research questions: (i) information about the outputs (e.g., how informative they were) and ex- plainability methods used, (ii) how explainability and performance is mea- sured and (iii) explainability in future research. Our results were sum- marised by manually labelling the data so that trends could be analysed across selected papers, along with a thematic analysis. Results:We found that 71% of current models used binary outputs, while 68% of models were not yet utilising any explainability techniques. Only 7% of studies considered explainability in their future research sug- gestions. Conclusion:There is still a lack of awareness among researchers for the need for explainability and motivation to invest further research into more explainable and more informative software defect prediction models.