Case study: How analysis of customer found defects can be used by system test to improve quality

Abstract

In the context of long-term, large-scale, industrial software development, process improvement and measurement to support process improvement is a necessity. Our software projects face the ongoing challenge, ldquoHow can we reduce the number of customer found defects in a cost-effective manner?rdquo This paper describes how the system test team of a large telecommunications company approached this challenge. Customer found defects were analyzed in a systematic way to provide information that system testers could turn into actionable activities in a test lab. Our findings indicated that most defects found by customers were associated with existing feature usage in various configurations while our test focus had primarily been on new feature operation. We categorized the information from our analysis using a technique called minimum conditions. Then, these data, together with an exploratory test method, were used to supplement existing test processes. The effort resulted in a $750,000 savings over two product releases.