On the Integration of Test Adequacy, Test Case Prioritization, and Statistical Fault Localization

Abstract

Testing and debugging account for at least 30% of the project effort. Scientific advancements in individual activities or their integration may bring significant impacts to the practice of software development. Fault localization is the foremost debugging sub-activity. Any effective integration between testing and debugging should address how well testing and fault localization can be worked together productively. How likely does a testing technique provide test suites for effective fault localization? To what extent may such a test suite be prioritized so that the test cases having higher priority can be effectively used in a standalone manner to support fault localization? In this paper, we empirically study these two research questions in the context of test data adequacy, test case prioritization and statistical fault localization. Our preliminary postmortem analysis results on 16 test case prioritization techniques and four statistical fault localizations show that branch-adequate test suites on the Siemens suite are unlikely to support effective fault localization. On the other hand, if such a test suite is effective, around 60% of the test cases can be further prioritized to support effective fault localization, which indicates that the potential savings in terms of effort can be significant.