Test-driven Development Process Research Articles

A family of experiments on test-driven development

Context:Test-driven development (TDD) is an agile software development approach that has been widely claimed to improve software quality. However, the extent to which TDD improves quality appears to be largely dependent upon the characteristics of the study in which it is evaluated (e.g., the research method, participant type, programming environment, etc.). The particularities of each study make the aggregation of results untenable.Objectives:The goal of this paper is to: increase the accuracy and generalizability of the results achieved in isolated experiments on TDD, provide joint conclusions on the performance of TDD across different industrial and academic settings, and assess the extent to which the characteristics of the experiments affect the quality-related performance of TDD.Method:We conduct a family of 12 experiments on TDD in academia and industry. We aggregate their results by means of meta-analysis. We perform exploratory analyses to identify variables impacting the quality-related performance of TDD.Results:TDD novices achieve a slightly higher code quality with iterative test-last development (i.e., ITL, the reverse approach of TDD) than with TDD. The task being developed largely determines quality. The programming environment, the order in which TDD and ITL are applied, or the learning effects from one development approach to another do not appear to affect quality. The quality-related performance of professionals using TDD drops more than for students. We hypothesize that this may be due to their being more resistant to change and potentially less motivated than students.Conclusion:Previous studies seem to provide conflicting results on TDD performance (i.e., positive vs. negative, respectively). We hypothesize that these conflicting results may be due to different study durations, experiment participants being unfamiliar with the TDD process, or case studies comparing the performance achieved by TDD vs. the control approach (e.g., the waterfall model), each applied to develop a different system. Further experiments with TDD experts are needed to validate these hypotheses.

Test-driven development with mutation testing \u2013 an experimental study

Test-driven development (TDD) is a popular design approach used by the developers with testing being the important software development driving factor. On the other hand, mutation testing is considered one of the most effective testing techniques. However, there is not so much research on combining these two techniques together. In this paper, we propose a novel, hybrid approach called TDD+M which combines test-driven development process together with the mutation approach. The aim was to check whether this modified approach allows the developers to write a better quality code. We verify our approach by conducting a controlled experiment and we show that it achieves better results than the sole TDD technique. The experiment involved 22 computer science students split into eight groups. Four groups (TDD+M) were using our approach, the other four (TDD) – a normal TDD process. We performed a cross-experiment by measuring the code coverage and mutation coverage for each combination (code of group X, tests from group Y). The TDD+M tests achieved better coverage on the code from TDD groups than the TDD tests on their own code (53.3% vs. 33.5% statement coverage and 64.9% vs. 37.5% mutation coverage). The TDD+M tests also found more post-release defects in the TDD code than TDD tests in the TDD+M code. The experiment showed that adding mutation into the TDD process allows the developers to provide better, stronger tests and to write a better quality code.

MoFQA: A TDD Process and Tool for Automatic Test Case Generation from MDD Models

Techniques for quality assurance have to deal with the complexity of software systems and the high probabilities of new errors appearing in any stage of the software life cycle. Software testing is a widely used approach but, due to the costs involved in this process, development teams often debate its applicability in their projects. In the endeavor to reduce the complexity of this process, this study presents an approach for software development based in Test-Driven Development (TDD) supported by Model-Based Testing (MBT) tools to allow automatic test-case generation. The approach, called MoFQA (Model-First Quality Assurance), consists of two main aspects: i) a method to drive software development based on testing techniques which defines steps and recom-mended practices; and, ii) a tool-set to allow clients and stakeholders to model system requirements, testers to create models that represent unit and abstract tests, and transformation tools used to generate executable tests. The tools that MoFQA provides are focusing on web applications. In order to validate the MoFQA tools, two validation experiences are presented.

A Dissection of the Test-Driven Development Process: Does It Really Matter to Test-First or to Test-Last?

Background: Test-driven development (TDD) is a technique that repeats short coding cycles interleaved with testing. The developer first writes a unit test for the desired functionality, followed by the necessary production code, and refactors the code. Many empirical studies neglect unique process characteristics related to TDD iterative nature. Aim: We formulate four process characteristic: sequencing, granularity, uniformity, and refactoring effort. We investigate how these characteristics impact quality and productivity in TDD and related variations. Method: We analyzed 82 data points collected from 39 professionals, each capturing the process used while performing a specific development task. We built regression models to assess the impact of process characteristics on quality and productivity. Quality was measured by functional correctness. Result: Quality and productivity improvements were primarily positively associated with the granularity and uniformity. Sequencing, the order in which test and production code are written, had no important influence. Refactoring effort was negatively associated with both outcomes. We explain the unexpected negative correlation with quality by possible prevalence of mixed refactoring. Conclusion: The claimed benefits of TDD may not be due to its distinctive test-first dynamic, but rather due to the fact that TDD-like processes encourage fine-grained, steady steps that improve focus and flow.

Test-Driven Development in scientific software: a survey

Scientific software developers are increasingly employing various software engineering practices. Specifically, scientists are beginning to use Test-Driven Development (TDD). Even with this increasing use of TDD, the effect of TDD on scientific software development is not fully understood. To help scientific developers determine whether TDD is appropriate for their scientific projects, we surveyed scientific developers who use TDD to understand: (1) TDDs effectiveness, (2) the benefits and challenges of using TDD, and (3) the use of refactoring practices (an important part of the TDD process). Some key positive results include: (1) TDD helps scientific developers increase software quality, in particular functionality and reliability; and (2) TDD helps scientific developers reduce the number of problems in the early phase of projects. Conversely, some key challenges include: (1) TDD may not be effective for all types of scientific projects; and (2) Writing a good test is the most difficult task in TDD, particularly in a parallel computing environment. To summarize, TDD generally has a positive effect on the quality of scientific software, but it often requires a large effort investment. The results of this survey indicate the need for additional empirical evaluation of the use of TDD for the development of scientific software to help organizations make better decisions.

USING TEST ORACLES AND FORMAL SPECIFICATIONS WITH TEST-DRIVEN DEVELOPMENT

This paper illustrates how Test Oracles and Formal Specifications, with appropriate tool support, can be used with Test-Driven Development (TDD). In TDD, the test code is a formal documentation of the required behavior of the component or system that is being developed, but this documentation is necessarily incomplete and often over-specific. We describe an alternative approach to TDD that is to develop the specification of the required behavior in a formal notation as a part of the TDD process and to generate test oracles from that specification. We present the results of using this approach to develop programs used in a project at the Faculty of Engineering and Applied Science at Memorial University.

The effect of experience on the test-driven development process

We conducted a quasi-experiment to compare the characteristics of experts' and novices' test-driven development processes. Our novices were 11 computers science students who participated in an Extreme Programming lab course, the expert group consisted of seven professionals who had industrial experience in test-driven development. The novices as well as two of the experts worked in a laboratory environment whereas the remaining five experts worked in their office. The experts complied more to the rules of test-driven development and had shorter test-cycles than the novices. The tests written by the experts were of higher quality in terms of statement and block coverage as well. All reported results are statistically significant on the 5% level. We conclude that the results of studies which evaluate performance of test-driven development using subjects inexperienced in TDD are not easily generalisable.

Measured Test-Driven Development: Using Measures to Monitor and Control the Unit Development

We analyze Test Driven Development (TDD) from cognitive and social perspectives. Based on our analysis, we suggest a technique for controlling and monitoring the TDD process by examining measures that relate to the size and complexity of both code and tests. We call this approach Measured TDD. The motivation for TDD arose from practitioners’ tendency to rush into code production, skipping the required testing needed to manufacture quality products. The motivation for Measured TDD is based on difficulties encountered by practitioners in applying TDD. Specifically, with the need to frequently refactor the unit, after every few test and code steps have been performed. We found that the suggested technique enables developers to gain better control over the development process.

Test-Driven Development in Large Projects

Test-driven development (TDD) is a key practice for agile developers because it involves writing test cases ahead of the code, which can improve design. The TDD process works well for projects in which a collocated team develops a small to medium system, but it can be challenging for large systems, especially those involving geographically distributed teams. The main obstacle is the degree of integration: when the team must integrate many individual classes developed at distributed sites, the coordination and communication grows exponentially with the number of individual developers and sites. This does not mean that TDD is ineffective for large-scale geographically distributed projects, but developers must take care to account for its focus on unit testing and its failure to rigorously address communications issues during system and integration testing. In this article, suggestions to scale up TDD are presented with two large-scale global software development projects at a major corporation and a recent meeting to exchange global software development best practices with a Fortune 500 company