The network of faults: a complex network approach to prioritize test cases for regression testing

Abstract

Regression testing is performed to provide confidence that changes in a part of software do not affect other parts of the software. An execution of all existing test cases is the best way to re-establish this confidence. However, regression testing is an expensive process--there might be insufficient resources (e.g., time, workforce) to allow for the re-execution of all test cases. Regression test prioritization techniques attempt to re-order a regression test suite based on some criteria so that highest priority test cases are executed earlier. In this study, we prioritize test cases for regression testing based on the dependency network of faults. In software testing, it is common that some faults are the consequences of other faults (leading faults). Dependent faults can be removed if and only if the leading faults have been removed. Our goal is to prioritize test cases so that test cases that have exposed the leading faults in the system testing phase, are executed first in regression testing. The leading faults are modeled as the most central faults in the fault dependency network. We present ComReg, a test-case prioritization technique based on the dependency network of faults. We model a fault dependency network as a directed graph and identify leading faults to prioritize test cases for regression testing. We use a centrality aggregation technique which considers six network representative centrality metrics to identify leading faults in the fault dependency network. We also discuss the use of fault communities to select an arbitrary percentage of the test cases from a prioritized regression test suite. We conduct a case study that evaluates the effectiveness and applicability of the proposed method. We obtain a fault dependency network from the development of a vocabulary learning software. We found that the fault network is a small-world graph with distinguishable community structure. The leading faults are common in all centralities and a re-ordering of test cases is feasible for regression testing based on those leading faults. Our method outperforms traditional regression testing prioritization techniques in detecting fault dependencies. Our modeling of the network of faults provides insights into the requirement of recognizing fault dependencies while re-ordering regression test suites for both research and practice. The dependency model needs further evaluation and improvement considering relevant resources (e.g., man-hours).