Predicting different levels of the unit testing effort of classes using source code metrics: a multiple case study on open-source software

Abstract

Nowadays, the growth in size and complexity of object-oriented software systems bring new software quality assurance challenges. Applying equally testing (quality assurance) effort to all classes of a large and complex object-oriented software system is cost prohibitive and not realistic in practice. So, predicting early the different levels of the unit testing effort required for testing classes can help managers to: (1) better identify critical classes, which will involve a relatively high-testing effort, on which developers and testers have to focus to ensure software quality, (2) plan testing activities, and (3) optimally allocate resources. In this paper, we investigate empirically the ability of a Quality Assurance Indicator (Qi), a synthetic metric that we proposed in a previous work, to predict different levels of the unit testing effort of classes in object-oriented software systems. The unit testing effort of classes is addressed from the perspective of unit test cases construction. We focused particularly on the effort involved in writing the code of unit test cases. To capture the involved unit testing effort of classes, we used four metrics that quantify different characteristics related to the code of corresponding unit test cases. We used Means and K-Means-based categorizations to group software classes into five categories according to the involved unit testing effort. We performed an empirical analysis using data collected from eight open-source Java software systems from different domains, for which the JUnit test cases were available. To evaluate the ability of the Qi metric to predict different levels of the unit testing effort of classes, we used three modeling techniques: the univariate logistic regression, the univariate linear regression, and the multinomial logistic regression. The performance of the models based on the Qi metric has been compared to the performance of the models based on various well-known object-oriented source code metrics. We used different evaluation criteria to compare the prediction models. Results indicate that the models based on the Qi metric have more promising prediction potential than those based on traditional object-oriented metrics.