Optimal Regression Testing Based on Selective Coverage of Test Requirements

Abstract

Test suite reduction can greatly save the test effort during regression testing. For software development with frequent minor updates, a good policy is to managing selective form regression testing, where the total set of test requirements is divided into two subsets: one is the set of concerned test requirements which ought to be covered by selected regression test cases; the other is the set of irrelevant test requirements which should be avoided to save the test case execution and analysis effort, enable missing parts and stubs, or avoid failure protecting codes. We define the multi-objective test suite reduction to model the selective coverage test suite reduction problem, and put forward the HATS algorithm to solve it. Within the HATS, the weighting factor α is defined to balance the two selecting criteria for each test case: maximizing the number of concerned requirements covered, or minimizing the number of irrelevant requirements involved. Experimental results show that compared with the total-coverage test suite reduction techniques, with proper settings of α, the HATS can further decrease the size of the regression test bucket, and cut down the coverage of irrelevant test requirements, while only slightly compromise the fault detection capability against the set of concerned test requirements.