Evaluating Different Strategies for Testing Software Product Lines

Abstract

The Software Product Line (SPL) approach is associated with many benefits, and with some challenges too. In the SPL testing, a great challenge is the integration of test methods and techniques with the SPL engineering. To do this, some strategies have been proposed, however, they have not been properly evaluated and compared. In this paper a case study is described comparing three different test strategies: the product by product strategy that tests each product individually; an incremental strategy that tests the products reusing test cases from products previously tested; and a strategy that instantiates test data derived in the domain engineering, considering SPL commonalities and variabilities. In the study an SPL from the games domain was used, and the test data were generated from use cases. The results show that SPL oriented strategies are associated to greater percentages of reuse, and consequently, with lower effort to write test cases.