Abstract
The detection of performance bugs, like those causing an unexpected execution time, has gained much attention in the last years due to their potential impact in safety-critical and resource-constrained applications. Much effort has been put on trying to understand the nature of performance bugs in different domains as a starting point for the development of effective testing techniques. However, the lack of a widely accepted classification scheme of performance faults and, more importantly, the lack of well-documented and understandable datasets makes it difficult to draw rigorous and verifiable conclusions widely accepted by the community. In this paper, we present TANDEM, a dual contribution related to real-world performance bugs. Firstly, we propose a taxonomy of performance bugs based on a thorough systematic review of the related literature, divided into three main categories: effects, causes and contexts of bugs. Secondly, we provide a complete collection of fully documented real-world performance bugs. Together, these contributions pave the way for the development of stronger and reproducible research results on performance testing.
Highlights
Software testing is a key part of software development aimed to assess whether the program meets its requirements and users’ expectations
We present a taxonomy and a dataset of real-world performance bugs called TANDEM (TAxoNomy and Dataset of pErforMance bugs)
The performance bugs in TANDEM dataset are alphabetically ordered by effect, cause, context, language, year of the publication and, for performance bugs referenced in the same publication, number of the figure or listing in the paper
Summary
Software testing is a key part of software development aimed to assess whether the program meets its requirements and users’ expectations. This work provided a dataset of 133 nonfunctional bug fixes with brief descriptions, collected from open-source projects written in Java and Phyton This dataset can be especially helpful in evaluations of new tools facing the detection of non-functional bugs in those languages. Most of the papers describing real-world performance bugs specialize in issues related to a particular scope Another example of a study focused on specific programming languages is the one presented by Selakovic et al [3], which proposed an approach for automatically finding and fixing performance bugs in JavaScript programs. In our work with TANDEM, we carry out a more wide-ranging review of real performance bugs, without restricting the approach to a particular non-functional property, root cause, kind of system or programming language In this sense, we aim to build a complete picture of performance bugs in real software. We collect those real-world performance bugs that are well documented, contextualized in research papers and include the buggy source code for the sake of understandability and to facilitate their reproducibility
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
