Metaheuristic approach for constructing functional test‐suites

Abstract

Today, software systems are complex and have many possible configurations. A deficient software testing process often leads to unfortunate consequences, including data losses, large economic losses, security breaches, and even bodily harm. Thus, the problem of performing effective and economical testing is a key issue. Combinatorial testing is a method that can reduce cost and increase the effectiveness of software testing for many applications. It is based on constructing economical sized test-suites that provide coverage of the most prevalent configurations. Mixed covering arrays (MCAs) are combinatorial structures that can be used to represent these test-suites. MCAs are combinatorial objects represented as matrices having a test case per row. MCAs are small, in comparison to an exhaustive approach, and guarantee a level of interaction coverage among the parameters involved. This study presents a metaheuristic approach based on a simulated annealing (SA) algorithm for constructing MCAs. This algorithm incorporates several distinguishing features, including an efficient heuristic to generate good quality initial solutions, and a compound neighbourhood function that combines two carefully designed neighbourhood functions. The experimental design involved a benchmark reported in the literature and two real cases of software components. The experimental evidence showed that the SA algorithm equals or improves the obtained results by other approaches reported in the literature, and also finds the optimal solution in some of the solved cases.