Fault-Prone Module Prediction Using a Prediction Model and Manual Inspection

Abstract

This paper proposes a fault-prone prediction approach that combines a fault-prone prediction model and manual inspection. Manual inspection is conducted by a predefined checklist that consists of questions and scoring procedures. The questions capture the fault signs or indications that are difficult to be captured by source code metrics used as input by prediction models. Our approach consists of two steps. In the first, the modules are prioritized by a fault-prone prediction model. In the second step, an inspector inspects and scores α percent of the prioritized modules. We conducted a case study of source code modules in commercial software that had been maintained and evolved over ten years and compared AUC (Area Under the Curve) values of Alberg Diagram among three prediction models: (A) support vector machines, (B) lines of code, and (C) random predictor with four prioritization orders. Our results indicated that the maximum AUC values under appropriate α and the coefficient of the inspection score were larger than the AUC values of the prediction models without manual inspection in each of the four combinations and the three models in our context. In two combinations, our approach increased the AUC values to 0.860 from 0.774 and 0.724. Our results also indicated that one of the combinations monotonically increased the AUC values with the numbers of manually inspected modules. This might lead to flexible inspection; the number of manually inspected modules has not been preliminary determined, and the inspectors can inspect as many modules as possible, depending on the available effort.