Extracting Combinatorial Test parameters and their values using model checking and evolutionary algorithms

Abstract

Combinatorial Testing (CT) is one of the popular testing approaches for generating a minimum test suite to detect defects caused by interactions between subsystems. One of the most practical CT methods is the Covering Array (CA). While generating CA, there are at least two main groups of challenges. The first one is extracting information about parameters, identifying constraints and detecting interactions between subsystems automatically. In most of the existing approaches, this information is fed to the system manually which makes it difficult or even impossible for testing modern software systems. The second one is the speed and the array size. Even though most of the existing approaches are concentrated on this challenge, their results show that there is still room for improvement. In this paper, we propose an idea to cope with both challenges. At first, we represent a method to extract information about the system under test (SUT) from its model using model checking (MC) techniques. MC is a method that scans all possible states of the system for detecting errors. After that, we propose another new approach using genetic algorithm to generate the optimal CA in terms of speed and size. To evaluate the results, we implemented the proposed strategy along with several other metaheuristic algorithms in the GROOVE tool, an open toolset for designing and model checking graph transformation specifications. The results represent that the proposed strategy performs better than others.