New bounds for binary covering arrays using simulated annealing

Abstract

Covering arrays (CAs) are combinatorial structures specified as a matrix of N rows and k columns over an alphabet on v symbols such that for each set of t columns (called the strength of the array) every t-tuple of symbols is covered. Recently they have been used to represent interaction test suites for software testing given that they provide economical sized test cases while still preserving important fault detection capabilities. This paper introduces an improved implementation of a simulated annealing algorithm, called ISA, for constructing CAs of strengths three through six over a binary alphabet (i.e., binary CAs). Extensive experimentation is carried out, using 127 well-known benchmark instances, for assessing its performance with respect to an existing simulated annealing implementation, a greedy method, and five state-of-the-art algorithms. The results show that our algorithm attains 104 new bounds and equals the best-known solutions for the other 23 instances consuming reasonable computational time. Furthermore, the implications of using these results as ingredients to recursive constructions are also analyzed.