Truth and myth of independent software testing

Abstract

Organizations often outsource software testing activities to optimize software testing and verify developed software by independent experts. Related benefits include expectations for higher software quality and anticipated commercial benefits, but have not been empirically evaluated yet. To close this gap, we hypothesize independent software testing to positively impact software quality and test our hypothesis in a controlled human experiment. We investigate the functional quality of six functionally identical systems developed in software projects by six teams of five students each. Of these six teams, three tested their software themselves, while the others used independent testing teams. On system-testing level, we compare the fulfillment of predefined software functions by applying function point analysis. The results show that independent software testing has a positive impact on software quality. Using t-tests, we show functional completeness of software tested by independent testers to be significantly higher (16%) than the functional completeness of software tested by software developers themselves. Our work provides insights into decision-making on software test outsourcing. While the expected higher quality of the products is corroborated by the experiment, there is a need for further research studies. With our experiment, we raise practitioners' awareness to consciously deal with independent software testing.