To What Extent Can Code Quality be Improved by Eliminating Test Smells?

Abstract

Software testing is a key activity to guarantee software reliability and maintainability. However, developers tend to ignore the maintenance of test code when performing a tradeoff between code quality and release deadlines. Moreover, the lack of research to quantify the relationship between test code and production code quality. As a result, test quality degrades due to the lack of appropriate refactoring plans. This paper fills the gap by evaluating to what extent can code quality be improved by eliminating test smells. First, we detect the presence of test smells in 119 historical releases of 10 open-source projects. Afterward, we evaluate code quality in 2 aspects, i.e., defect- and change-proneness. Finally, we exploit the odds ratio and Mann-Whitney test to quantify the extent of variation for the code quality. Results show that the OR values of the test code and production code are both much greater than 1, which proves that the test smell is indeed a risk factor to increase the defect-proneness of code. Moreover, the change-proneness of the test code and associated production code reduces significantly after their elimination. Experiment also reveals Assertion Roulette is the riskiest smell to degrade production code quality.