Ordering Coverage Goals in Model Checker Based Testing

Abstract

Model-based testing techniques select test cases according to test goals, which might be coverage criteria or mutation adequacy. Complex criteria and large models lead to large test suites, and a test case created for one coverage item might cover several other items as well. Therefore, test case generation is optimized in order to avoid unnecessary test cases and minimize the test generation and execution costs. Because of this optimization the order in which test goals are selected is expected to have an impact on both the performance of the test case generation and the size of resulting test suites, although finding the optimal order is not feasible in general. In this paper we report on experiments to determine the effects of the order in which test goals are selected on performance and the size of resulting test suites, and evaluate different heuristics to select test goals such that time and size are minimized. The approach described is based on test case generation with model checkers, and experimentation leads to the conclusion that the order matters less than expected.