Model-based testing of software product lines: Mapping study and research roadmap

Abstract

Model-Based Testing (MBT) has been successfully applied to Software Product Lines (SPL). This paper provides a panorama of state-of-the-art on MBT of SPLs. We performed a systematic mapping for answering questions related with domains, approaches, solution types, variability, test case automation, artifacts, and evaluation. We built a roadmap from 44 selected studies. Main obtained results are: Software and Automotive domains are most considered; Black-box testing is widely performed; most studies have fully-automated support; variability is considered in most studies; Finite State Machines is the most used model to test SPLs; Behavioral-based and Scenario-based are the most used models; Case Studies and Experiments are used to evaluate MBT solutions and the majority is performed in industrial environments; traceability is not widely explored for MBT solutions. Furthermore, we provide a roadmap synthesizing studies based on used models, more formal artifacts, supporting tools, variability management, (semi-)automation, and traceability. The roadmap contributes to identify related primary studies based on given artifacts, variability management, tools, automation, and traceability techniques and to identify, from a given primary study, which artifacts, tools, variability management, automation and traceability techniques are related. Therefore, the roadmap serves as a guide to researchers and practitioners on how to model-based test SPLs.