Test Data Generation Approach Research Articles

Automatic Generation of Unit Test Data for Dynamically Typed Languages

Testing is the major means of verifying and validating software. It is a repetitive and time-consuming activity. Testing is neglected because of its high cost and the fact that it does not add functionality to the system. As a result, many programmers don't write tests. To remedy this, some researcher proposed automatic test generation. Test generation is a solution that reduces workload and increases productivity. In this paper, we propose a test data generation approach for unit tests in dynamically typed languages. Our approach is based on the analysis and decomposition of the AST (Abstract Syntax Tree) obtained when compiling the source code of the method under test. We validate this approach in Pharo a real system. The results on three systems show the effectiveness of the approach.

Bio‐inspired optimization to support the test data generation of concurrent software

SummaryConcurrent programming is increasingly present in modern applications. Although it provides higher performance and better use of available resources, the mechanisms of interaction between processes/threads result in a greater challenge for software testing activity. The nondeterminism present in those applications is one of the main issues during the test activity since the same test input can produce different possible execution paths, which may or not contain defects. The test data automatic generation can alleviate this problem, ensuring higher speed and reliability in software testing activity. This paper explores the automatic test data generation for concurrent programs through Genetic Algorithm, a bioinspired optimization technique, and proposes a test data generation approach for concurrent programs, called BioConcST, and a new operator for the selection of test subjects, called FuzzyST, which uses fuzzy logic. The approaches were evaluated in an experimental study towards their validation. The results showed that BioConcST is more promising than the other approaches at all analyzed levels. FuzzyST, together with Elitism and Tournament operators, provided the best results; however, it proved more suitable for concurrent programs of higher complexity.

Seeding Grammars in Grammatical Evolution to Improve Search-Based Software Testing

Heuristic-based optimization techniques have been increasingly used to automate different types of code coverage analysis. Several studies suggest that interdependencies (in the form of comparisons) may exist between the condition constructs, of variables and constant values, in the branching conditions of real-world programs, e.g. (i le 100) or (i==j), etc. In this work, by interdependencies we refer to the situations where, to satisfy a branching condition, there must be a certain relationship between the values of some specific condition constructs (which may or may not be a part of the respective condition predicates). For example, the values of variables i and j must be equal to satisfy the condition of (i==j), and the value of variable k must be equal to 100 for the satisfaction of the condition of (k==100). To date, only the Ariadne, a Grammatical Evolution (GE)-based system, exploits these interdependencies between input variables (e.g. of the form (i le j) or (i==j), etc.) to efficiently generate test data. Ariadne employs a simple attribute grammar to exploit these dependencies, which enables it to evolve complex test data, and has been compared favourably to other well-known techniques in the literature. However, Ariadne does not benefit from interdependencies involving constants, e.g. (i le 100) or (j==500), etc., due to the difficulty in evolving precise values, and these are equally important constructs of condition predicates. Furthermore, constant creation in GE can be difficult, particularly with high precision. We propose to seed the grammar with constants extracted from the source code of the program under test to enhance and extend Ariadne’s capability to exploit richer types of dependencies (involving all combinations of both variables and constant values). We compared our results with the original system of Ariadne against a large set of benchmark problems which include 10 numeric programs in addition to the ones originally used for Ariadne. Our results demonstrate that the seeding strategy not only dramatically improves the generality of the system, as it improves the code coverage (effectiveness) by impressive margins, but it also reduces the search budgets (efficiency) often up to an order of magnitude. Moreover, we also performed a rigorous analysis to investigate the scalability of our improved Ariadne, showing that it stays highly scalable when compared to both the original system of Ariadne and GA-based test data generation approach.

Augmenting ant colony optimization with adaptive random testing to cover prime paths

Test data generation has a notable impact on the performance of software testing. A well-known approach to automate this activity is search-based test data generation. Most studies in this area use branch coverage as the test criterion. Since the prime path coverage criterion includes branch coverage, it has higher probability to detect software failures than the branch coverage criterion. This paper customizes and improves ant colony optimization (ACO) to provide a test data generation approach for covering prime paths. The proposed approach incorporates the notion of input space partitioning to maintain pheromone values in the search space. In addition, it employs the idea of adaptive random testing in the local search. At last, it uses the information of program predicates in order to make a relation between the logic of the program and pheromone values in the search space. The experimental results confirm the positive effects of the mentioned contributions, especially for programs with complex predicates. Furthermore, they represent that, on average, test suites generated by the proposed approach has 9% better mutation score in comparison to test suites produced by EvoSuite, a well-known test data generation tool.

Unit Test Data Generation for C Using Rule-Directed Symbolic Execution

Unit testing is widely used in software development. One important activity in unit testing is automatic test data generation. Constraint-based test data generation is a technique for automatic generation of test data, which uses symbolic execution to generate constraints. Unit testing only tests functions instead of the whole program, where individual functions typically have preconditions imposed on their inputs. Conventional symbolic execution cannot detect these preconditions, let alone converting these preconditions into constraints. To overcome these limitations, we propose a novel unit test data generation approach using rule-directed symbolic execution for dealing with functions with missing input preconditions. Rule-directed symbolic execution uses predefined rules to detect preconditions in the individual function, and generates constraints for inputs based on preconditions. We introduce implicit constraints to represent preconditions, and unify implicit constraints and program constraints into integrated constraints. Test data generated based on integrated constraints can explore previously unreachable code and help developers find more functional faults and logical faults. We have implemented our approach in a tool called CTS-IC, and applied it to real-world projects. The experimental results show that rule-directed symbolic execution can find preconditions (implicit constraints) automatically from an individual function. Moreover, the unit test data generated by our approach achieves higher coverage than similar tools and efficiently mitigates missing input preconditions problems in unit testing for individual functions.

Comparative Study Between Flower Pollination Algorithm and Cuckoo Search Algorithm for t -Way Test Data Generation

T-way testing is a sampling approach for test data generation. Recently, adapting meta-heuristic algorithms for t-way testing is very attractive in order to find a minimum subset of test data that can test a system overall. As a consequence, several meta-heuristic algorithms have been used as the basis of t-way strategies. In order to guide software tester (and engineers in general) to select the best algorithm for the problem at hand, there is a need to evaluate and benchmark the performance of each strategy against common case studies. This paper presents a comparative study between two meta-heuristic strategies for t-way test data generation: Flower Pollination Algorithm (FPA) and Cuckoo Search (CS). Our experiments have performed on a real-world case study. Experimental results demonstrate that FPA appears to produce better results in most of the test cases in term of test suite size and convergence rate owing to its ability for controlling local and global search.

Using Artificial Bee Colony Algorithm for Test Data Generation and Path Testing Coverage

Software testing is a significant stage in software development lifecycle. There are different sorts of' structural software testing methodologies that may be generally utilized and moved forward through enhancing the traverse of all of the conceivable code software paths. The interest for automating data testing is growing; however, manual testing strategies utilization would be expensive and costly. Heuristic measure is being applied to; detect how better the result might be (solution fitness); result development mechanism; and suitableness criteria with stop search mechanism depending on wither a result is found or not. Testing experience could be exploited for finding a solution to the optimization problem by utilizing Meta heuristic procedures. The presented approach might have been tested for five programs to demonstrate the distinctive tests issues. This paper proposes an automatic test data generation approach that use artificial bee colony algorithm for software structural testing, particularly, path testing. This is brought on moving the centralization of data generation testing, as opposed to the automation of the whole testing operation. It executes artificial bee colony algorithm by creating testing data for the criteria of path coverage testing, and then applying the strategy to a group of test programs.

Test Data Generation Approaches for Structural Testing and Automatic Programming Assessment: A Systematic Literature Review

Test Data Generation Approaches for Structural Testing and Automatic Programming Assessment: A Systematic Literature Review

A multi-objective test data generation approach for mutation testing of feature models

Mutation approaches have been recently applied for feature testing of Software Product Lines (SPLs). The idea is to select products, associated to mutation operators that describe possible faults in the Feature Model (FM). In this way, the operators and mutation score can be used to evaluate and generate a test set, that is a set of SPL products to be tested. However, the generation of test sets to kill all the mutants with a reduced, possible minimum, number of products is a complex task. To help in this task, in a previous work, we introduced a multi-objective approach that includes a representation to the problem, search operators, and two objectives related to the number of test cases and dead mutants. The proposed approach was implemented and evaluated with three representative multi-objective and evolutionary algorithms: NSGA-II, SPEA2 and IBEA, and obtained promising results. Now in the present paper we extend such an approach to include a third objective: the pairwise coverage. The goal 4 is to reveal other kind of faults not revealed by mutation testing and to improve the efficacy of the generated test sets. Results of new studies are reported, showing that both criteria can be satisfied with a reduced number of products. The approach produces diverse good solutions and different sets of impacting factors can be considered. At the end, the tester can either prioritize one objective, by choosing solutions in the extreme points of the fronts or choose solutions with smaller ED values, according to the testing goals and resources.

An Efficient Test Data Generation Approach for Unit Testing

To ensure the delivery of high-quality software, software testing plays the vital role. One of the major time-consuming and expensive activities in software testing is the generation of test data. Test data generation activity has a strong impact on the effectiveness and efficiency of the whole testing process. In order to reduce the cost and time involved in the process of test data generation, researchers and practitioners have tried to automate it. In literature, many such techniques have been developed and the most commonly used are; Random testing, Symbolic execution and evolutionary testing. In this work, an enhanced and efficient Random test data generation approach is proposed and investigated on a suite of programs and its efficiency is compared with the Genetic algorithm which is an evolutionary approach. The inconsistency of random approach is that it is not capable of generating a specific set or combination of test cases for the program input variables and in search of these effective test cases multiple populations needs to be created that will increase the burden of size . So, in order to remove these inconsistencies from the test suite, it is seeded with a more effective set of test cases through our proposed approach in order to make test suite more granular and limit its size by not generating more populations in search of these effective test cases. In addition to the proposed approach, the classification of test adequacy criteria and issues with random, symbolic execution and genetic algorithm based test data

Instance Generator and Problem Representation to Improve Object Oriented Code Coverage

Search-based approaches have been extensively applied to solve the problem of software test-data generation. Yet, test-data generation for object-oriented programming (OOP) is challenging due to the features of OOP, e.g., abstraction, encapsulation, and visibility that prevent direct access to some parts of the source code. To address this problem we present a new automated search-based software test-data generation approach that achieves high code coverage for unit-class testing. We first describe how we structure the test-data generation problem for unit-class testing to generate relevant sequences of method calls. Through a static analysis, we consider only methods or constructors changing the state of the class-under-test or that may reach a test target. Then we introduce a generator of instances of classes that is based on a family of means-of-instantiation including subclasses and external factory methods. It also uses a seeding strategy and a diversification strategy to increase the likelihood to reach a test target. Using a search heuristic to reach all test targets at the same time, we implement our approach in a tool, JTExpert, that we evaluate on more than a hundred Java classes from different open-source libraries. JTExpert gives better results in terms of search time and code coverage than the state of the art, EvoSuite, which uses traditional techniques.

Automatic test data generation based on reduced adaptive particle swarm optimization algorithm

Software testing aims to search a set of test data in the entire search space to satisfy a certain standard of coverage. Therefore, finding an effective approach for automatic test data generation is a key issue of software testing. This paper proposes a new approach of reduced adaptive particle swarm optimization for generating the test data automatically. First, the approach reduces the particle swarm evolution equations and gets an evolution equation without velocity. Then, the approach makes an adaptive adjustment scheme based on inertia weight for the reduced evolution equation, which is different from the methods that directly act on the particle velocity in the past. The approach directly impacts on the particle position, namely actual problem solution. Next, according to the particle fitness and the particle aggregation degree, the population will be divided into three parts and inertia weight of each part will be designed accordingly. This can balance the search capabilities of algorithm between global and local. Finally, the approach is applied to automatic test data generation. The experiments results show that our approach can enhance convergence speed of algorithm and solve the problems that particle swarm algorithm easily falls into the local optimal solution and has low search accuracy. The experiments results also turn out that our approach can improve the efficiency of generating test data automatically.

An approach for test data generation using program slicing and particle swarm optimization

Heuristic search-based test data generation has a potential higher efficiency in software testing with path covering. However, these approaches are suffered in covering the long and complex path. In this paper, we propose a method for generating test data based on program slicing and particle swarm optimization. With the interest points selected from a target path, we perform a program slicing to remove the statements which are irrelevant to the interest points. Our method simplifies the target path and the actual path to get a better fitness value. After program slices obtained, the population is evolved using particle swarm optimization to improve the efficiency of test data generation.

Heuristic search-based approach for automated test data generation: a survey

The complexity of software has been increasing in the past few years, and software testing as a most intensive factor is becoming more and more expensive. Testing costs often account for up to 50% of the total expense of software project development; hence any techniques leading to the automatic test data generation will have great potential to substantially reduce these costs. Existing approaches of automatic test data generation have achieved some success by using heuristic search-based approach, but they are not summarised. In this paper we presented a survey on heuristic search-based approach, i.e., genetic algorithm for automated test data generation. We summarise the work done by researchers those who have applied the concept of heuristic search-based approach for test data generation. The main objective of this paper is to acquire the concepts related to heuristic search-based approach for automated test data generation and moreover the use of heuristic search-based approach fascinated by the fact that many of the testing problems can be formulated as search problem. We also provide constructive guidelines for future research for automated test data generation.

C 프로그램을 테스팅하기 위한 분기 커버리지에 기반을 둔 자동 테스트 데이터 생성

It is well known that software testing amounts for a significant portion of software development cost. In order to reduce the cost of software testing. a lot of researches on automated test data generation have been performed. Sophisticated tools for performing symbolic execution or solving a system of path constraints are required to support automated test data generation. Developing or purchasing those tools leads to another factor of increasing the cost involving software testing. In this paper, we propose a dynamic test data generation approach that does not depend on symbolic execution or constraint solving at all. The proposed approach extends Korel`s path-oriented method to satisfy the branch coverage criterion effectively. We conducted an experiment to evaluate the effectiveness of the proposed technique with a triangle classification program to show that branch coverage can be easily achieved.

Applying genetic algorithm to increase the efficiency of a data flow-based test data generation approach

The success or failure of the entire software development process relies on the software testing component which is responsible for ensuring that the software that is released is free from bugs. One of the major labor intensive activities of software testing is the generation of the test data for the purpose of applying the testing methodologies. Many approaches have been tried and tested for automating the process of generating the test data. Meta-heuristics have been applied extensively for improving the efficiency of the process. This paper analyses the effectiveness of applying genetic algorithms for generating test data automatically using data flow testing approach. An incremental coverage measurement method is used to improve the convergence.

Test data generation approach for basis path coverage

On the basis of determining the feasibility of paths, this paper proposes an evolutionary approach to generating test data for feasible basis path coverage. First, the structure of the program under test is expressed by a control flow graph, and the target paths are encoded into the form of hybrid-coding that efficiently combines the statement label with the outcome of a conditional statement (i.e. T or F). Then, the genetic algorithm is employed to generate test data for multiple paths coverage, and the fitness function of an input data (an individual) takes into account the degree of the execution track matching the target paths. Finally, the proposed approach is applied in several benchmark programs. The experimental results show that the proposed approach cannot only avoid redundant test but also improve the efficiency of test data generation effectively

Comparative Evaluation of A Maximization And Minimization Approach for Test Data Generation with Genetic Algorithm and Binary Particle Swarm Optimization

In search based test data generation, the problem of test data generation is reduced to that of function minimization or maximization.Traditionally, for branch testing, the problem of test data generation has been formulated as a minimization problem. In this paper we define an alternate maximization formulation and experimentally compare it with the minimization formulation. We use genetic algorithm and binary particle swarm optimization as the search technique and in addition to the usual operators we also employ a branch ordering strategy, memory and elitism. Results indicate that there is no significant difference in the performance or the coverage obtained through the two approaches and either could be used in test data generation when coupled with the branch ordering strategy, memory and elitism.

Automated Test Data Generation Using Cuckoo Search and Tabu Search (CSTS) Algorithm

Software testing is a very important phase in the development of software. Testing includes the generation of test cases which, if done manually, is time consuming. To automate this process and generate optimal test cases, several meta-heuristic techniques have been developed. These approaches include genetic algorithm, cuckoo search, tabu search, intelligent water drop, etc. This paper presents an effective approach for test data generation using the cuckoo search and tabu search algorithms (CSTS). It combines the cuckoo algorithm’s strength of converging to the solution in minimal time along with the tabu mechanism of backtracking from local optima by Levy flight. The experimental results show that the algorithm is effective in generating test cases optimally and its performance is better than various earlier proposed approaches.

FlagRemover

Search-Based Testing is a widely studied technique for automatically generating test inputs, with the aim of reducing the cost of software engineering activities that rely upon testing. However, search-based approaches degenerate to random testing in the presence of flag variables, because flags create spikes and plateaux in the fitness landscape. Both these features are known to denote hard optimization problems for all search-based optimization techniques. Several authors have studied flag removal transformations and fitness function refinements to address the issue of flags, but the problem of loop-assigned flags remains unsolved. This article introduces a testability transformation along with a tool that transforms programs with loop-assigned flags into flag-free equivalents, so that existing search-based test data generation approaches can successfully be applied. The article presents the results of an empirical study that demonstrates the effectiveness and efficiency of the testability transformation on programs including those made up of open source and industrial production code, as well as test data generation problems specifically created to denote hard optimization problems.