Abstract
Context: There is still little evidence on differences between Test-Driven Development and Test-Last Development, especially for real-world projects, so their impact on code/test quality is an ongoing research trend. An empirical comparison is presented, with 19 participants working on an industrial project developed for an energy market software company, implementing real-world requirements for one of the company's customers. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Objective:</i> Examine the impact of TDD and TLD on quality of the code and the tests. The aim is to evaluate if there is a significant difference in external code quality and test quality between these techniques. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Method:</i> The experiment is based on a randomized within-subjects block design, with participants working for three months on the same requirements using different techniques, changed from week to week, within three different competence blocks: Intermediate, Novice and Mixed. The resulting code was verified for process conformance. The participants developed only business logic and were separated from infrastructural concerns. A separate group of code repositories was used to work without unit tests, to verify that the requirements were not too easy for the participants. Also, it was analysed if there is any difference between the code created by shared efforts of developers with different competences and the code created by participants isolated in the competence blocks. The resulting implementations had LOC order of magnitude of 10k. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Results:</i> Statistically significant advantage of TDD in terms of external code quality (1.8 fewer bugs) and test quality (5 percentage points higher) than TLD. Additionally, TDD narrows the gap in code coverage between developers from different competence blocks. At the same time, TDD proved to have a considerable entry barrier and was hard to follow strictly, especially by Novices. Still, no significant difference w.r.t. code coverage has been observed between the Intermediate and the Novice developers - as opposed to TLD, which was easier to follow. Lastly, isolating the Intermediate developers from the Novices had significant impact on the code quality. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Conclusion:</i> TDD is a recommended technique for software projects with a long horizon or when it is critical to minimize the number of bugs and achieve high code coverage.
Highlights
U NIT testing is the most fundamental approach to software testing [1], in the sense that it verifies the desired part of the implementation directly, by executing it
The general results of various experiments in current research are coherent and indicate mostly that Test-Driven Development (TDD) improves the quality of the output code in some way [6]
In [9], [29], and [10] no statistically significant differences are reported between most important metrics used
Summary
U NIT testing is the most fundamental approach to software testing [1], in the sense that it verifies the desired part of the implementation directly, by executing it. Among many possible approaches to unit testing, two main development techniques are chosen for comparison: Test-Driven Development (TDD) and Test-Last Development (TLD). The studies concerning TDD are still reporting inconsistent results about the merits of TDD [2], [3], [4], [5], [6] so it is important to keep gathering more data on the subject. There are two existing research trends concerning measuring this impact: simulation modelling approach and experimental approach. There are many existing experimental assessments of unit testing techniques, they are usually constrained by their academic or commercial nature (as in [7], [8], [9]). The purpose of this work is to achieve experimental conditions that are very rarely met in the studies on the subject [10]. The company was successfully convinced to perform the same real, commercial project
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
