Automated code-based test selection for software product line regression testing

Abstract

Regression testing for software product lines (SPLs) is challenging and can be expensive because it must ensure that all the products of a product family are correct whenever changes are made. SPL regression testing can be made efficient through a test case selection method that selects only the test cases relevant to the changes. Some approaches for SPL test case selection have been proposed but either they were not efficient by requiring intervention from human experts or they cannot be used if requirements specifications, architecture and/or traceabilities for test cases are not available or partially eroded. To address these limitations, we propose an automated method of source code-based regression test selection for SPLs. Our method reduces the repetition of the selection procedure and minimizes the in-depth analysis effort for source code and test cases based on the commonality and variability of a product family. Evaluation results of our method using six product lines show that our method reduces the overall time to perform regression testing by 14.8% ∼ 49.1% on average compared to an approach of repetitively applying Ekstazi, which is the state-of-the-art regression test selection method for a single product, to each product of a product family.