Use Of Search-based Techniques Research Articles

On the Effectiveness of Using Elitist Genetic Algorithm in Mutation Testing

Manual test case generation is an exhaustive and time-consuming process. However, automated test data generation may reduce the efforts and assist in creating an adequate test suite embracing predefined goals. The quality of a test suite depends on its fault-finding behavior. Mutants have been widely accepted for simulating the artificial faults that behave similarly to realistic ones for test data generation. In prior studies, the use of search-based techniques has been extensively reported to enhance the quality of test suites. Symmetry, however, can have a detrimental impact on the dynamics of a search-based algorithm, whose performance strongly depends on breaking the “symmetry” of search space by the evolving population. This study implements an elitist Genetic Algorithm (GA) with an improved fitness function to expose maximum faults while also minimizing the cost of testing by generating less complex and asymmetric test cases. It uses the selective mutation strategy to create low-cost artificial faults that result in a lesser number of redundant and equivalent mutants. For evolution, reproduction operator selection is repeatedly guided by the traces of test execution and mutant detection that decides whether to diversify or intensify the previous population of test cases. An iterative elimination of redundant test cases further minimizes the size of the test suite. This study uses 14 Java programs of significant sizes to validate the efficacy of the proposed approach in comparison to Initial Random tests and a widely used evolutionary framework in academia, namely Evosuite. Empirically, our approach is found to be more stable with significant improvement in the test case efficiency of the optimized test suite.

An extensive analysis of search-based techniques for predicting defective classes

In spite of constant planning, effective documentation and proper implementation of a software during its life cycle, many defects still occur. Various empirical studies have found that prediction models developed using software metrics can be used to predict these defects. Researchers have advocated the use of search-based techniques and their hybridized versions in literature for developing software quality prediction models. This study conducts an extensive comparison of 20 search-based techniques, 16 hybridized techniques and 17 machine-learning techniques amongst each other, to develop software defect prediction models using 17 data sets. The comparison framework used in the study is efficient as it (i) deals with the stochastic nature of the techniques (ii) provides a fair comparison amongst the techniques (iii) promotes repeatability of the study and (iv) statistically validates the results. The results of the study indicate promising ability of search-based techniques and their hybridized versions for predicting defective classes.

Search-based mutant selection for efficient test suite improvement: Evaluation and results

ContextSearch-based techniques have been applied to almost all areas in software engineering, especially to software testing, seeking to solve hard optimization problems. However, the problem of selecting mutants to improve the test suite at a lower cost has not been explored to the same extent as other problems, such as mutant selection for test suite evaluation or test data generation. ObjectiveIn this paper, we apply search-based mutant selection to enhance the quality of test suites efficiently. Namely, we use the technique known as Evolutionary Mutation Testing (EMT), which allows reducing the number of mutants while preserving the power to refine the test suite. Despite reported benefits of its application, the existing empirical results were derived from a limited number of case studies, a particular set of mutation operators and a vague measure, which currently makes it difficult to determine the real performance of this technique. MethodThis paper addresses the shortcomings of previous studies, providing a new methodology to evaluate EMT on the basis of the actual improvement of the test suite achieved by using the evolutionary strategy. We make use of that methodology in new experiments with a carefully selected set of real-world C++ case studies. ResultsEMT shows a good performance for most case studies and levels of demand of test suite improvement (around 45% less mutants than random selection in the best case). The results reveal that even a reduced subset of mutants selected with EMT can serve to increase confidence in the test suite, especially in programs with a large set of mutants. ConclusionsThese results support the use of search-based techniques to solve the problem of mutant selection for a more efficient test suite refinement. Additionally, we identify some aspects that could foreseeably help enhance EMT.

On the application of search-based techniques for software engineering predictive modeling: A systematic review and future directions

Software engineering predictive modeling involves construction of models, with the help of software metrics, for estimating quality attributes. Recently, the use of search-based techniques have gained importance as they help the developers and project-managers in the identification of optimal solutions for developing effective prediction models. In this paper, we perform a systematic review of 78 primary studies from January 1992 to December 2015 which analyze the predictive capability of search-based techniques for ascertaining four predominant software quality attributes, i.e., effort, defect proneness, maintainability and change proneness. The review analyses the effective use and application of search-based techniques by evaluating appropriate specifications of fitness functions, parameter settings, validation methods, accounting for their stochastic natures and the evaluation of developmental models with the use of well-known statistical tests. Furthermore, we compare the effectiveness of different models, developed using the various search-based techniques amongst themselves, and also with the prevalent machine learning techniques used in literature. Although there are very few studies which use search-based techniques for predicting maintainability and change proneness, we found that the results of the application of search-based techniques for effort estimation and defect prediction are encouraging. Hence, this comprehensive study and the associated results will provide guidelines to practitioners and researchers and will enable them to make proper choices for applying the search-based techniques to their specific situations.

Search-Based Test Data Generator for Data-Flow Dependencies Using Dominance Concepts, Branch Distance and Elitism

Software testing is a complex and expensive activity of the software development life cycle. Software testing includes test data generation according to a test adequacy criterion. The use of search-based techniques has been the focus of researchers to automate the process of software test data generation for structural control-flow criteria. Automating test data generation remains a challenging problem for more robust adequacy criterion such as satisfying data-flow dependencies of a program. This study proposes a search-based approach that generates test data for data-flow dependencies of a program using dominance concepts, branch distance, and elitism. Genetic algorithm is used for the proposed approach and Gray encoding is used to encode test data. A set of subject programs is taken from the research literature to evaluate efficiency and effectiveness of the proposed approach. For the proposed approach, the measures considered are the mean number of generations and mean percentage coverage achieved. The performance of the proposed approach is evaluated by comparing the results with those of random search and earlier studies on data-flow testing. Over several experiments, it is shown that the proposed approach performed significantly better than random search and earlier studies with respect to data-flow test data generation and optimization. There is an increasing performance gap for more complex subject programs.

Prediction of faults-slip-through in large software projects: an empirical evaluation

A large percentage of the cost of rework can be avoided by finding more faults earlier in a software test process. Therefore, determination of which software test phases to focus improvement work on has considerable industrial interest. We evaluate a number of prediction techniques for predicting the number of faults slipping through to unit, function, integration, and system test phases of a large industrial project. The objective is to quantify improvement potential in different test phases by striving toward finding the faults in the right phase. The results show that a range of techniques are found to be useful in predicting the number of faults slipping through to the four test phases; however, the group of search-based techniques (genetic programming, gene expression programming, artificial immune recognition system, and particle swarm optimization---based artificial neural network) consistently give better predictions, having a representation at all of the test phases. Human predictions are consistently better at two of the four test phases. We conclude that the human predictions regarding the number of faults slipping through to various test phases can be well supported by the use of search-based techniques. A combination of human and an automated search mechanism (such as any of the search-based techniques) has the potential to provide improved prediction results.