Empirical study of the effects of different profiles on regression test case reduction

Abstract

Determining how to select a subset of test cases with high-fault detection capability becomes a key issue in code-level regression testing. Cluster analysis has been proposed to deal with this issue. It partitions test cases into clusters based on the similarity of execution profiles. In previous studies, execution profiles were represented as binary or numeric vectors. The vector model only considers the number of times that a function or statement is executed. However, it ignores sequential, the relations and structural information between function calls. Therefore vector-based methods do not always generate satisfying results. In this study, the authors presented cluster analysis of three different types of structural profiles, that is, function execution sequence, function call sequence (FCS) and function call tree. They designed and conducted empirical studies on five medium-sized programs to validate the effects of different profiles on regression test case reduction. Experimental results illustrate that sequential, call relations and structural information can aid to further improve fault detection effectiveness. In view of cost-effectiveness, FCS is regarded as to be the optimal profile. Furthermore, cluster analysis of FCSs is comparable to the additional branch coverage reduction technique with respect to fault detection effectiveness.