Cost and Effectiveness of Search-Based Techniques for Model-Based Testing: An Empirical Analysis

Abstract

Model-based testing (MBT) seems to be gaining interest in industry and academia due to its provision of systematic, automated and comprehensive testing. The challenge in MBT is to generate optimal test data to execute test cases. Recently, researchers have successfully applied search-based techniques (SBTs) by automating the search for an optimal set of test data at reasonable cost compared to other more expensive techniques. In real complex systems, effectiveness and cost of SBTs for MBT in industrial context are little known. The objective of this study is to empirically evaluate the cost and the effectiveness of SBTs for MBT on industrial case studies. We applied a model-driven approach and SBTs to automatically generate executable feasible test cases. The results show that the model-driven approach generated high number of infeasible test cases with less time while genetic algorithm (GA) and simulating annealing (SA) outperformed significantly random search (RS) with high generation time. We concluded that local SBTs are more appropriate to generate test data when the type of the constraints is simple. Current work on analyzing the cost and effectiveness on SBTs for MBT indicates possible enhancement using the model-driven approach to detect the infeasible paths and SBTs to achieve optimal success rate.