Algorithm For Test Data Generation Research Articles

Evolutionary algorithms for path coverage test data generation and optimization: a review

Software testing is very time consuming, labor-intensive and complex process. It is found that 50% of the resources of the software development are consumed for testing. Testing can be done in two different ways such as manual testing and automatic testing. Automatic testing can overcomes the limitations of manual testing by decreasing the cost and time of testing process. Path testing is the strongest coverage criteria among all white box testing techniques as it can detect about 65% of defects present in a SUT. With the help of path testing, the test cases are created and executed for all possible paths which results in 100% statement coverage and 100% branch coverage .This paper presents a systematic review of test data generation and optimization for path testing using Evolutionary Algorithms (EAs). Different EAs like GA, PSO, ACO, and ABCO based methods has been already proposed for automatic test case generation and optimization to achieve maximum path coverage.

A Hybrid Particle Swarm Optimization and Differential Evolution based Test Data Generation Algorithm for Data-Flow Coverage using Neighbourhood Search Strategy

Meta-heuristic search techniques, mainly Genetic Algorithm (GA), have been widely applied for automated test data generation according to a structural test adequacy criterion . However, it remains a challenging task for more robust adequacy criterion such as data-flow coverage of a program. Now, focus is on the use of other highly-adaptive meta-heuristic search techniques such as Particle Swarm Optimization (PSO) and Differential Evolution (DE). In this paper, a hybrid (adaptive PSO and DE) algorithm is proposed to generate test data for data-flow dependencies of a program with a neighbourhood search strategy to improve the search capability of the hybrid algorithm. The fitness function is based on the concepts of dominance relations and branch distance. The measures considered are mean number of generations and mean percentage coverage. The performance of the hybrid algorithm is compared with that of DE, PSO, GA, and random search. Over several experiments on a set of benchmark programs, it is shown that the hybrid algorithm performed significantly better than DE, PSO, GA and random search in data-flow test data generation with respect to the measures collected.

Using Artificial Bee Colony Algorithm for Test Data Generation and Path Testing Coverage

Software testing is a significant stage in software development lifecycle. There are different sorts of' structural software testing methodologies that may be generally utilized and moved forward through enhancing the traverse of all of the conceivable code software paths. The interest for automating data testing is growing; however, manual testing strategies utilization would be expensive and costly. Heuristic measure is being applied to; detect how better the result might be (solution fitness); result development mechanism; and suitableness criteria with stop search mechanism depending on wither a result is found or not. Testing experience could be exploited for finding a solution to the optimization problem by utilizing Meta heuristic procedures. The presented approach might have been tested for five programs to demonstrate the distinctive tests issues. This paper proposes an automatic test data generation approach that use artificial bee colony algorithm for software structural testing, particularly, path testing. This is brought on moving the centralization of data generation testing, as opposed to the automation of the whole testing operation. It executes artificial bee colony algorithm by creating testing data for the criteria of path coverage testing, and then applying the strategy to a group of test programs.

A reasoning-based approach to dynamic domain reduction in test data generation

In this article, a new criterion, domain coverage, for white box testing is introduced. In search of the biggest subdomains for input variables exercising a given path, the variable domains are broken down into overlapping subregions in such a way that the resultant subregions either exercise or deviate from the path. To this aim, an incremental method is applied to detect the subregions, covering the longest subpaths from the start of the given path. The detected subdomain is further subdivided, and this process is repeated as far as the path is fully covered. If no subdomain is detected, the method is backtracked. The backtrack begins with the subdomain covering the longest subpath at the beginning of the given path, but this time the subdomain is divided into relatively smaller subregions, compared with the last time. A directed acyclic graph representation of each input variable domain is used to ignore re-examination of those subdomains which deviate from the path. Conducted experiments reveal that our proposed domain-oriented algorithm outperforms the existing path-oriented test data generation algorithms.

Evaluating Bees Algorithm for Sequence-based T-way Testing Test Data Generation

T-way testing is a testing technique that is used to detect faults due to parameter interactions or software configurations. In order to perform t-way testing, software testers need to prepare the test data. For a system with many configuration parameters, the test data could lead to combinatorial explosion problem, since all possible parameter combinations need to be considered. Therefore, test data generation for t-way testing need to be optimized. Many t-way test data generation strategies have been proposed in the literature to generate optimized t-way test data. However, very few strategies have been proposed for sequence-based t-way. This paper presents statistical analysis on the performance of Bees Algorithm against the other sequence t-way strategies, in order to generate test cases.

Structural test data generation using a memetic ant colony optimization based on evolution strategies

Abstract Test data generation is one of the key activities that has a significant impact on the efficiency and effectiveness of software testing. Since manual test data generation is quite inefficient and even impractical, automated test data generation has been realized to produce an appropriate subset of input data to carry out effective software testing in reasonable times. This paper presents a memetic ant colony optimization (ACO) algorithm for structural test data generation. The proposed approach incorporates (1+1)-evolution strategies (ES) to improve the search functionality of ants in local moves and enhance search exploitation. Moreover, we have introduced a novel definition of the pheromone functionality in the way that it discourages ants from choosing mostly covered paths of the program to reinforce search exploration. Given that branch coverage is considered as the coverage criterion, two fitness functions are used accordingly for our proposed algorithm. The first fitness function is a Boolean function which is particularly defined to maximize branch coverage. It outputs one if a given solution is successful in traversing at least a yet uncovered branch; otherwise, it returns zero. The second fitness function is formulated according to the complexity of branches covered. The value of the second fitness function is not taken into account for solutions whose Boolean function value equals one. For these solutions, the decision-making process of ants is merely carried out based on the first fitness function. The experimental results indicate the superiority of our memetic ACO algorithm relative to existing test data generation techniques in terms of both branch coverage and convergence speed.

Testing the Trustworthiness of IC Testing: An Oracle-Less Attack on IC Camouflaging

Test of integrated circuits (ICs) is essential to ensure their quality; the test is meant to prevent defective and out-of-spec ICs from entering into the supply chain. The test is conducted by comparing the observed IC output with the expected test responses for a set of test patterns; the test patterns are generated using automatic test pattern generation algorithms. Existing test-pattern generation algorithms aim to achieve higher fault coverage at lower test costs. In an attempt to reduce the size of test data, these algorithms reveal the maximum information about the internal circuit structure. This is realized through sensitizing the internal nets to the outputs as much as possible, unintentionally leaking the secrets embedded in the circuit as well. In this paper, we present HackTest, an attack that extracts secret information generated in the test data, even if the test data do not explicitly contain the secret. HackTest can break the existing intellectual property protection techniques, such as camouflaging, within 2 min for our benchmarks using only the camouflaged layout and the test data. HackTest applies to all existing camouflaged gate-selection techniques and is successful even in the presence of the state-of-the-art test infrastructure, i.e., test data compression circuits. Our attack necessitates that the IC test data generation algorithms can be reinforced with security.

Application of Negative Selection Algorithm (NSA) for test data generation of path testing

Path testing is one of the areas covered in structural testing. In this process, it is a key challenge to search for a set of test data in the whole search space to satisfy path coverage. Thus, finding an efficient method for generating test data automatically is a key issue in software testing. This paper proposed a method based on Negative Selection Algorithm (NSA) for generating test data to satisfy the path coverage criterion. The results show that NSA could reduce the number of test data generated and improve the coverage percentage, as well as enhance the efficiency of the test data generation process. To evaluate the performance of the method, results from the proposed method were compared with random testing and a previous work that used Genetic Algorithm and Ant Colony Optimization. The results demonstrate that NSA outperforms other methods in reducing the number of test data that cover all program paths even the difficult ones.

Evolution or revolution: the critical need in genetic algorithm based testing

Software testing is one of the most inevitable processes in software development. The field of software testing has seen an extensive use of search based techniques in the last decade. Among the search based techniques, it is the metaheuristic techniques such as genetic algorithm that has garnered the major share of attention from researchers. Looking at the large body of work that has happened and is happening in this field, we feel that it is high time someone studied how well genetic algorithm based techniques fare in practical testing process. Method: In this work, we present a roadmap to the future of genetic algorithm based software testing, based on a review of literature. We have mainly reviewed the works which use genetic algorithm for software test data generation. This independent review is designed to direct the attention of future researchers to the deficiencies of genetic algorithm based testing, their possible solutions and the extent to which they are correctable. The observations from the selected primary studies highlight the issues faced when genetic algorithm is applied in software testing. The observations form the review reveal that the type of genetic algorithm used, fitness function design, population initialization and parameter settings does impact the quality of solution obtained in software testing using genetic algorithm. From the review we conclude that, more generalized approaches can make genetic algorithm based software testing one of the strongest methods in practical software testing. We hope that, this review will be a major breakthrough in genetic algorithm based software testing field.

A Novel Fitness function of metaheuristic algorithms for test data generation for simulink models based on mutation analysis

Testing is one of the crucial activities to assure the software quality. The main objective of testing is to generate test data uncovering faults in software modules. There are a variety of testing techniques in which mutation testing is a popular approach to generate test sets and evaluate their fault detection ability. Simulink is an environment widely used in industry to design and simulate critical systems. Testing such a system at the design phase could help to detect faults earlier. This study aims to propose a novel fitness function of metaheuristic algorithms to generate test data based on the mutation technique for the Simulink models. The fitness function is designed by analyzing each mutation operator and the features of blocks in the Simulink environment in order to guide the search process to reach the test data killing mutants more easily. Then, this fitness function is used in the multi-parent crossover genetic algorithm to generate test sets. The obtained results indicated that the mutation score has been significantly improved for all models when using the novel fitness function. In addition, each stubborn mutant was killed with a lower number of test data evaluations in comparison with the work of other authors.

Software Testing Using Genetic Algorithms

This paper presents a set of methods that uses a genetic algorithm for automatic test-data generation in software testing. For several years researchers have proposed several methods for generating test data which had different drawbacks. In this paper, we have presented various Genetic Algorithm (GA) based test methods which will be having different parameters to automate the structural-oriented test data generation on the basis of internal program structure. The factors discovered are used in evaluating the fitness function of Genetic algorithm for selecting the best possible Test method. These methods take the test populations as an input and then evaluate the test cases for that program. This integration will help in improving the overall performance of genetic algorithm in search space exploration and exploitation fields with better convergence rate.

RGA: A lightweight and effective regeneration genetic algorithm for coverage-oriented software test data generation

ContextGenetic algorithm (GA) is an important intelligent method in the area of automatic software test data generation. However, existing GAs tend to get trapped in the local optimal solution, leading to population aging, which can significantly reduce the benefits of GA-based software testing and increase cost and effort. Although much attention has been focused on solving this problem by improving chromosome population, genetic operations, and genetic parameters adjustment, the applicability of most of the algorithms proposed is often narrow because of the complex operations involved and nondeterminism inherited from traditional GAs. ObjectivesThis paper proposes a new algorithm called the regenerate genetic algorithm (RGA), which is based on a new simple, stable, and easy-to-implement regeneration strategy that involves judging the population aging process. MethodsWe propose a new regeneration strategy—called Regenerate Genetic Algorithm (RGA)—that solves these problems easily and effectively. The proposed strategy defines population aging factors and process in order to determine the degree of population aging. Subsequently, when population aging has reached a certain limit, a population regeneration operation is triggered. In contrast to other improved methods, the proposed regeneration strategy for population aging easily achieves a stronger ability to jump out of the local optimal solution, thereby preventing population aging and effectively improving test coverage, without modifying any parameter of the original GA. ResultsThe proposed algorithm is experimentally evaluated by comparing it to the basic GA, Random Testing (RT) and several other methods in terms of both efficiency and effectiveness on the Siemens Suite of test programs and a more complex real program. The results obtained indicate that the proposed algorithm can effectively increase search efficiency, restrain population aging, increase test coverage, and reduce the number of test cases. ConclusionRGA has better optimization ability than the conventional algorithms, especially for large-scale and highly complex programs.

AUSTIN: An open source tool for search based software testing of C programs

ContextDespite the large number of publications on Search-Based Software Testing (SBST), there remain few publicly available tools. This paper introduces AUSTIN, a publicly available open source SBST tool for the C language.1http://code.google.com/p/austin-sbst/.1 The paper is an extension of previous work [1]. It includes a new hill climb algorithm implemented in AUSTIN and an investigation into the effectiveness and efficiency of different pointer handling techniques implemented by AUSTIN’s test data generation algorithms. ObjectiveTo evaluate the different search algorithms implemented within AUSTIN on open source systems with respect to effectiveness and efficiency in achieving branch coverage. Further, to compare AUSTIN against a non-publicly available, state-of-the-art Evolutionary Testing Framework (ETF). MethodFirst, we use example functions from open source benchmarks as well as common data structure implementations to check if the decision procedure for pointer inputs, introduced in this paper, differs in terms of effectiveness and efficiency compared to a simpler alternative that generates random memory graphs. A second empirical study formulates two alternate hypotheses regarding the effectiveness and efficiency of AUSTIN compared to the ETF. These hypotheses are tested using a paired Wilcoxon test. Results and ConclusionThe first study highlights some practical problems with the decision procedure for pointer inputs described in this paper. In particular, if the code under test contains insufficient guard statements to enforce constraints over pointers, then using a constraint solver for pointer inputs may be suboptimal compared to a method that generates random memory graphs. The programs used in the second study do not require any constraint solving for pointer inputs and consist of eight non-trivial, real-world C functions drawn from three embedded automotive software modules. For these functions, AUSTIN is competitive compared to the ETF, achieving an equal or higher branch coverage for six of the functions. In addition, for functions where AUSTIN’s branch coverage is equal or higher, AUSTIN is more efficient than the ETF.

A Parallel Approach to Concolic Testing with Low-cost Synchronization

This paper presents a practical approach to parallelize the test data generation algorithm by which computing resources can be fully used. The test data generation approach that we are using is based on the dynamic symbolic execution (concolic testing). The basic idea of parallelizing the algorithm is to distribute analysis processes of different paths to different computing units. Although a centralized scheduler with several sub processes can directly achieve the goal of parallelism, it may cause global idle time when parallel processes frequently end at same time. In our approach, a runtime deterministic scheduler is introduced to reduce the potential global idle time. Our experiments show some notable results when using a proper scheduling function. Compared with the sequential concolic testing, our approach can save nearly 70% computing time in some cases on a system with eight CPU cores from our experiments.

It really does matter how you normalize the branch distance in search‐based software testing

SUMMARYThe use of search algorithms for test data generation has seen many successful results. For structural criteria like branch coverage, heuristics have been designed to help the search. The most common heuristic is the use of approach level (usually represented with an integer) to reward test cases whose executions get close (in the control flow graph) to the target branch. To solve the constraints of the predicates in the control flow graph, the branch distance is commonly employed. These two measures are linearly combined. Since the approach level is more important, the branch distance is normalized, often in the range [0, 1]. In this paper, different types of normalizing functions are analyzed. The analyses show that the one that is usually employed in the literature has several flaws. The paper presents a different normalizing function that is very simple and does not suffer from these limitations. Empirical and analytical analyses are carried out to compare these two functions. In particular, their effect is studied on commonly used search algorithms, such as Hill Climbing, Simulated Annealing and Genetic Algorithms. Copyright © 2011 John Wiley & Sons, Ltd.

Test Data Generation Algorithms of Combinatorial Testing and Comparison Based on Cross-Entropy and Particle Swarm Optimization Method

The test suite generation is one of key issues of combinatorial testing.This paper uses Cross-Entropy method of statistics and Particle Swarm Optimization from bionics to generate pairwise test data of combinatorial testing.The cross-entropy method used the best selection probability to generate test data and the particle swarm optimization generates test data by searching one with best fitness in a feasible solution.A theoretical method is given for the optimal probability selection of the cross-entropy method and a reduction technique is proposed to reduce the test suite generated by two methods.The empirical results show that the cross-entropy method and the particle swarm optimization have some merits compared with existing greedy algorithms,algebra methods and other heuristic search algorithms.

A multiple-population genetic algorithm for branch coverage test data generation

The software testing phase in the software development process is considered a time-consuming process. In order to reduce the overall development cost, automatic test data generation techniques based on genetic algorithms have been widely applied. This research explores a new approach for using genetic algorithms as test data generators to execute all the branches in a program. In the literature, existing approaches for test data generation using genetic algorithms are mainly focused on maintaining a single-population of candidate tests, where the computation of the fitness function for a particular target branch is based on the closeness of the input execution path to the control dependency condition of that branch. The new approach utilizes acyclic predicate paths of the program's control flow graph containing the target branch as goals of separate search processes using distinct island populations. The advantages of the suggested approach is its ability to explore a greater variety of execution paths, and in certain conditions, increasing the search effectiveness. When applied to a collection of programs with a moderate number of branches, it has been shown experimentally that the proposed multiple-population algorithm outperforms the single-population algorithm significantly in terms of the number of executions, execution time, time improvement, and search effectiveness.

Creating and Visualizing Test Data from Programming Exercises

Automatic assessment of programming exercises is typically based on testing approach. Most automatic assessment frameworks execute tests and evaluate test results automatically, but the test data generation is not automated. No matter that automatic test data generation techniques and tools are available. We have researched how the Java PathFinder software model checker can be adopted to the specific needs of test data generation in automatic assessment. Practical problems considered are: how to derive test data directly from students' programs (i.e., without annotation) and how to visualize and how to abstract test data automatically for students? Interesting outcomes of our research are that with minor refinements generalized symbolic execution with lazy initialization (a test data generation algorithm implemented in PathFinder) can be used to construct test data directly from students' programs without annotation, and that intermediate results of the same algorithm can be used to provide novel visualizations of the test data.

Generating software test data by evolution

This paper discusses the use of genetic algorithms (GAs) for automatic software test data generation. This research extends previous work on dynamic test data generation where the problem of test data generation is reduced to one of minimizing a function. In our work, the function is minimized by using one of two genetic algorithms in place of the local minimization techniques used in earlier research. We describe the implementation of our GA-based system and examine the effectiveness of this approach on a number of programs, one of which is significantly larger than those for which results have previously been reported in the literature. We also examine the effect of program complexity on the test data generation problem by executing our system on a number of synthetic programs that have varying complexities.

Test-data generation using genetic algorithms

This paper presents a technique that uses a genetic algorithm for automatic test-data generation. A genetic algorithm is a heuristic that mimics the evolution of natural species in searching for the optimal solution to a problem. In the test-data generation application, the solution sought by the genetic algorithm is test data that causes execution of a given statement, branch, path, or definition–use pair in the program under test. The test-data-generation technique was implemented in a tool called TGen, in which parallel processing was used to improve the performance of the search. To experiment with TGen, a random test-data generator called Random was also implemented. Both Tgen and Random were used to experiment with the generation of test-data for statement and branch coverage of six programs. Copyright © 1999 John Wiley & Sons, Ltd.