Reproducing failures based on semiformal failure scenario descriptions

Abstract

Due to the increasing size and complexity of software systems, it becomes hard to test these systems exhaustively. As a result, some faults can be left undetected. Undetected faults can lead to failures in deployed systems. Such failures are usually reported by the users from the field or test engineers back to developers. It requires considerable time and effort to analyze and reproduce the reported failures because their descriptions are not always complete, structured and formal. In this paper, we introduce a novel approach for automatically reproducing failures to aid their debugging. Our approach relies on semi-structured failure scenario descriptions that employ a set of keywords. These descriptions are preprocessed and mapped to a set of predefined test case templates with valid input sets. Then, test cases are generated and executed to reproduce the reported failure scenarios. The approach is evaluated with an industrial case study performed in a company from the telecommunications domain. Several failures were successfully reproduced. The approach is also adopted in the quality assurance process of the company. After one-time preparation of reusable test case templates and training of test engineers, 24.9 % of the reported failures (and 40 % of those that were manually reproducible) could be reproduced without any manual effort.