Deriving a Fault Architecture to Guide Testing

Abstract

Defect analysis of software components can be used to guide testing, with the goal of focusing on parts of the software that were fault-prone in earlier releases or earlier life cycle phases, such as development. We replicate a study that adapted a reverse architecting technique using defect reports to derive fault architectures. A fault architecture determines and visualizes components that are fault-prone in their relationships with other components, as well as those that are locally fault-prone. Our case study uses defect data from three releases of a large medical record system to identify relationships among system components, based on whether they are involved in the same defect report. We investigate measures that assess the fault-proneness of components and component relationships. Component relationships are used to derive a fault architecture. The resulting fault architecture indicates what the most fault-prone relationships are in a release. We also apply the technique in a new way. Not only do we derive fault architectures for each release, we derive fault architectures for the development, system test and post release phases within each release. Comparing across releases, makes it possible to see whether some components are repeatedly in fault-prone relationships. Comparing across phases, makes it possible to see whether development fault architectures can be used to identify those parts of the software that need to be tested more. We validate our predictions using system test data from the same release. We also use the development and system test fault architectures to identify fault-prone components after release, and validate our predictions using post release data.