Abstract
Customizing applications through program configuration options has been proved by many open-source and commercial projects as one of the best practices in software engineering. However, traditional performance testing is not in synch with this industrial practice. Traditional performance testing techniques consider program inputs as the only external factor. It ignores the performance influence of configuration options. This study aims to stimulate research interest in performance testing in the context of configurable software systems by answering three research questions. That is, why it is necessary to conduct research in performance testing, what are the state-of-the-art techniques, and how do we conduct performance testing research in configurable software systems. In this study, we examine the unique characteristics and challenges of performance testing research in configurable software systems. We review and discuss research topics on the performance bug study, performance anti-patterns, program analysis, and performance testing. We share the research findings from the empirical study and outline the opening opportunities for new and advanced researchers to contribute to the research community.
Highlights
Software performance is an inseparable part of user experience
We examine the importance of performance testing and the uniqueness of performance testing research in configurable software systems
We provide a research map to help researchers to navigate performance testing research topics in configurable software systems
Summary
How to cite this paper: Han, X. (2021) A Study of Performance Testing in Configurable Software Systems.
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have