Boolean expression extender — A mutation operator for strengthening and weakening boolean expressions

Abstract

The degree in which a software system is guaranteed to correctly provide its intended functionality, is one of the most relevant properties in the development of quality software, and the concern of software verification. Among the many techniques that approach software verification, testing, i.e., contrasting actual software behavior against expected behavior in a set of specific scenarios called test cases, is without any doubt the most widely used. Performing software testing appropriately demands evaluating the quality of the test cases, and mutation testing is a particularly relevant technique for evaluating the adequacy of test suites, that occupies us in this paper. Mutation testing proposes evaluating how good a set of test cases is by injecting artificial bugs in a program, and checking whether the suite is able to catch these buggy programs, called mutants. Mutation testing's performance is strongly related to the mutation operators considered, i.e., the set of changes that are applied to a program to generate mutants. We present a novel mutation operator whose main focus is boolean expressions, and that operates by strengthening and weakening such expressions. Although operators that perform this task already exist in the context of mutation testing, e.g. the COR operator, these do not provide finer-grained changes in expressions. The rationale behind our new operator is that performing finer-grained mutations in boolean expressions results in more “stubborn” mutants, i.e., more mutants that are difficult to kill. We present our novel mutation operator in detail, discuss its characteristics and rationale, and evaluate the impact of introducing this operator in mutation testing tools, with respect to how it performs in assessing test suite quality.