Towards Combined Process & Tool Variability Management in Software Testing

Abstract

Context. Modern software engineering approaches that rely on continuous and automated testing, like Agile Software Engineering and the late DevOps movement, require integrated and fully functional testing tool chain environment, to efficiently identify defects in software artifacts. Such an environment includes the implementation of established testing processes that are utilized by the development teams. However, in practice, different testing tool chains and processes are implemented depending on particular project requirements such as programming language, selected testing tool, or system architecture. This variety of required technologies and processes frequently results in an environment of isolated test automation solutions. Thus, there is a need for a managed and controllable set of testing tool chain variants that consider structured methods to integrate variability. Goal. In this paper, we show ongoing work, as part of a flexible Test Automation Framework (TAF), with focus on requirements for the variability of testing tool chains, established testing processes, and candidate solution approaches. Method. We build on best practices from software and systems testing and variability management to implement variability in the TAF. Results. First results showed that several Test Automation (TA) solutions exist, which support variability in a limited manner and, therefore, increase the need for modeling variability in a flexible TAF. Conclusion. In the context of Software Test Automation, a combination of Variability Modeling (VM) methods for testing architectures, business processes, and a definition of common interface definitions is promising towards a TAF that enables a flexible tool and process integration.