Abstract
Mutation testing is well-known for its efficacy in assessing test quality, and starting to be applied in the industry. However, what should a developer do when confronted with a low mutation score? Should the test suite be plainly reinforced to increase the mutation score, or should the production code be improved as well, to make the creation of better tests possible? In this paper, we aim to provide a new perspective to developers that enables them to understand and reason about the mutation score in the light of testability and observability. First, we investigate whether testability and observability metrics are correlated with the mutation score on six open-source Java projects. We observe a correlation between observability metrics and the mutation score, e.g., test directness, which measures the extent to which the production code is tested directly, seems to be an essential factor. Based on our insights from the correlation study, we propose a number of ”mutation score anti-patterns”, enabling software engineers to refactor their existing code or add tests to improve the mutation score. In doing so, we observe that relatively simple refactoring operations enable an improvement or increase in the mutation score.
Highlights
Mutation testing has been a very active research field since the 1970s as a technique to evaluate test suite quality in terms of the fault-revealing capability (Jia and Harman, 2011)
Based on all 16 cases that we analysed, we found that our code observability metrics can lead to simple refactorings that enable to kill mutants that were previously not being killed
This paper aims to bring a new perspective to software developers helping them to understand and reason about the mutation score in the light of testability and observability
Summary
Mutation testing has been a very active research field since the 1970s as a technique to evaluate test suite quality in terms of the fault-revealing capability (Jia and Harman, 2011). In our previous study, we have observed that certain mutants could be killed only after refactoring the production code to increase the observability of state changes In such cases, test deficiency is not the only reason for the survival of mutants. Using structural coverage metrics alone might be misleading because, in many cases, statements might be covered, but their consequences might not be asserted (Inozemtseva and Holmes, 2014) Another factor is that well-developed open-source mutation testing tools (e.g., PIT/PiTest Coles, 2019a and Mull GitHub, 2019) have contributed to mutation testing being applied in the industrial environments (Petrovic et al, 2018; Petrovic and Ivankovic, 2018; Coles, 2019e)
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
