Abstract
When identifying the origin of software bugs, many studies assume that “a bug was introduced by the lines of code that were modified to fix it”. However, this assumption does not always hold and at least in some cases, these modified lines are not responsible for introducing the bug. For example, when the bug was caused by a change in an external API. The lack of empirical evidence makes it impossible to assess how important these cases are and therefore, to which extent the assumption is valid. To advance in this direction, and better understand how bugs “are born”, we propose a model for defining criteria to identify the first snapshot of an evolving software system that exhibits a bug. This model, based on the perfect test idea, decides whether a bug is observed after a change to the software. Furthermore, we studied the model’s criteria by carefully analyzing how 116 bugs were introduced in two different open source software projects. The manual analysis helped classify the root cause of those bugs and created manually curated datasets with bug-introducing changes and with bugs that were not introduced by any change in the source code. Finally, we used these datasets to evaluate the performance of four existing SZZ-based algorithms for detecting bug-introducing changes. We found that SZZ-based algorithms are not very accurate, especially when multiple commits are found; the F-Score varies from 0.44 to 0.77, while the percentage of true positives does not exceed 63%. Our results show empirical evidence that the prevalent assumption, “a bug was introduced by the lines of code that were modified to fix it”, is just one case of how bugs are introduced in a software system. Finding what introduced a bug is not trivial: bugs can be introduced by the developers and be in the code, or be created irrespective of the code. Thus, further research towards a better understanding of the origin of bugs in software projects could help to improve design integration tests and to design other procedures to make software development more robust.
Highlights
During the life of a software product developers often fix bugs1 (Pan et al 2009; MurphyHill et al 2015)
We propose a model of how bugs were introduced, from which the assumption that a bug was introduced by the lines of code that were modified to fix it can be derived as a specific case
Some of these reasons are already known from previous studies and, we do not pretend to do an exhaustive classification of why the previous commits analyzed were not the bug-introducing changes (BICs), we identified some other reasons that have not been taken into account previously and added them to the list of reasons:
Summary
During the life of a software product developers often fix bugs (Pan et al 2009; MurphyHill et al 2015). A defect/fault can be introduced in different phases of a software product life (e.g., planning, coding, deployment) due to many reasons, such as missing or changing requirements, wrong specifications, miscommunication, programming errors, time pressure, poorly documented code, among others (Nakajo and Kume 1991; Jacobs et al 2007; Nuseibeh and Easterbrook 2000). When the software is executed and the system produces wrong results, defects may lead to failures, described as “[e]vents in which a system or system component does not perform a required function within specified limits” Developers, and in many cases researchers too, typically use the term “bug” to refer both to defects/faults (deficiencies) and failures (their manifestation), depending on the context. We will be interested in the first fault (per order of introduction in the source code), in case there are more that one causing a failure
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
