Advances in Testing Software Product Lines

Abstract

Abstract In this chapter, we describe some recent techniques and results in model-based testing of software product lines. Presently, more and more software-based products and services are available in many different variants to choose from. However, this brings about challenges for the software quality assurance processes. Since only few of all possible variants can be tested at the developer's site, several questions arise. How shall the variability be described in order to make sure that all features are being tested? Is it better to test selected variants on a concrete level, or shall the whole software product line be tested abstractly? What is the quality of a test suite for a product line, anyway? If it is impossible to test all possible variants, which products should be selected for testing? Given a certain product, which test cases are appropriate for it, and given a test case, which products can be tested with it? We address these questions from an empirical software engineering point of view. We sketch modeling formalisms for software product lines. Then, we compare domain-centered and application-centered approaches to software product line testing. We define mutation operators for assessing software product line test suites. Subsequently, we analyze methods for selecting product variants on the basis of a given test suite. Finally, we show how model checking can be used to determine whether a certain test case is applicable for a certain product variant. For all our methods we describe supporting tools and algorithms. Currently, we are integrating these in an integrated tool suite supporting several aspects of model-based testing for software product lines.