Evaluation and analysis of incorporating Fuzzy Expert System approach into test suite reduction

Abstract

ContextSoftware has become increasingly important in our modern society. However, when new features are developed due to user requests, such requests could make the sizes of test-case pools bigger. Many techniques are proposed to solve this problem, such as test suite reduction. However, the ability to expose faults may be weakened when reducing the sizes of the test suites. In this paper, we propose some methods using fuzzy logic in order to improve existing test-suite reduction techniques. ObjectiveThe main purpose of this research is to use a Fuzzy Expert System approach in order to enhance the effectiveness of fault detection during software testing. MethodIncorporating a Fuzzy Expert System into traditional test suite reduction techniques is presented and studied. More objective criteria are used in order to compare the performance of our proposed and selected test suite reduction methods. Some important measures (and metrics) will also be obtained and discussed. Application of the Fuzzy Expert System approach for test case prioritization is also discussed. ResultsThe experiments in three improved test-suite reduction techniques show that the modified algorithms can reduce the size of test suites, which have improved the fault detection quality. ConclusionDuring software testing, test data are generally classified with Boolean logic. This method can classify data into groups easily. However, there may be ambiguity in classifications due to similar properties for certain data. Ambiguous data can be classified in each group. In this study, Boolean logic will be replaced by fuzzy logic. Incorporating the Fuzzy Expert System approach into three traditional test suite reduction techniques (i.e., HGS, GRE, and Greedy) is presented and evaluated. The experiments, based on nine real subject programs ranging from 173 LOCs to 35,545 LOCs, have demonstrated that our proposed Fuzzy-HGS, Fuzzy-GRE, and Fuzzy-Greedy algorithms can significantly reduce the sizes of test suites while also improving fault detection effectiveness. For instance, Fuzzy-HGS, Fuzzy-GRE, and Fuzzy-Greedy algorithms have almost the same reduction capability of test suite as traditional HGS, GRE, and Greedy algorithms. But in terms of the percentage of fault detection effectiveness loss (FDE loss), both Fuzzy-HGS and Fuzzy-GRE algorithms are averagely decreased by 21% and 5%, respectively. Additionally, Fuzzy-Greedy algorithm still provide the lower FDE loss for large subject programs compared to traditional Greedy algorithm. Based upon the integrated theoretical foundation, the approaches presented in this paper offer an efficient, useful software testing scheme in the testing and debugging phases.