Test Data Generation Techniques Research Articles

Test data regeneration: generating new test data from existing test data

SUMMARYExisting automated test data generation techniques tend to start from scratch, implicitly assuming that no pre‐existing test data are available. However, this assumption may not always hold, and where it does not, there may be a missed opportunity; perhaps the pre‐existing test cases could be used to assist the automated generation of additional test cases. This paper introduces search‐based test data regeneration, a technique that can generate additional test data from existing test data using a meta‐heuristic search algorithm. The proposed technique is compared to a widely studied test data generation approach in terms of both efficiency and effectiveness. The empirical evaluation shows that test data regeneration can be up to 2 orders of magnitude more efficient than existing test data generation techniques, while achieving comparable effectiveness in terms of structural coverage and mutation score. Copyright © 2010 John Wiley & Sons, Ltd.

Using Dominance for Aiding the Search-Based Testing to Overcome the Limitations of the Control-Dependencies

Search-based optimization techniques have been utilized for a number of software engineering activities. The representation of the problem and the definition of the fitness function ( FF ) are two key ingredients for the application of search-based optimization to software engineering problems. Therefore, a well-defined fitness function is essential to the effectiveness and efficiency of the search-based testing ( SBT ). Several search based test-data generation techniques have been developed. A wide range of these techniques utilized the control dependencies ( CD ) in the control-flow graph of the program under test for guiding the search in the direction of finding test data. To direct the SBT to generate test data, Ghiduk et al. have presented a search-based technique that utilizes the dominances ( Dom ) between the nodes of the control-flow graph ( CFG ) of the program under test. In this paper, we investigate the efficiency and effectiveness of dominances in a control-flow graph against the control dependencies in guiding the SBT for generating test data. The paper provides a number of structures programming which challenge the SBT that is guided by the control dependencies to find test data. The paper introduces two schemes for overcoming these problems. The first scheme improves the definitions of the fitness functions of the previous work to overcome the control-dependencies problems. The second scheme presents a general form for a fitness function in terms of dominances and postdominances nodes. This function enhances significantly the efficiency of the SBT ; consequently the SBT overcomes the control-dependencies problems. In addition, the paper compares between the efficiency of dominances and control dependencies in guiding SBT with proper examples from the literatures.

An Empirical Evaluation of Test Data Generation Techniques

Software testing cost can be reduced if the process of testing is automated. However, the test data generation task is still performed mostly by hand although numerous theoretical works have been proposed to automate the process of generating test data and even commercial test data generators appeared on the market. Despite prolific research reports, few attempts have been made to evaluate and characterize those techniques. Therefore, a lot of works have been proposed to automate the process of generating test data. However, there is no overall evaluation and comparison of these techniques. Evaluation and comparison of existing techniques are useful for choosing appropriate approaches for particular applications, and also provide insights into the strengths and weaknesses of current methods. This paper conducts experiments on four representative test data generation techniques and discusses the experimental results. The results of the experiments show that the genetic algorithm (GA)-based test data generation performs the best. However, there are still some weaknesses in the GA-based method. Therefore, we modify the standard GA-based method to cope with these weaknesses. The experiments are carried out to compare the standard GA-based method and two modified versions of the GA-based method.

Alloy 명세 기반 자동 테스트 데이터 생성 기법

일반적으로 테스트 데이터 생성 방법들은 테스트 데이터를 자동으로 생성하기 위해서 완전한 프로그램 경로를 기술한 것을 요구한다. 이 논문에서는 프로그램 경로를 완전하게 명시하지 않아도 테스트 데이터를 자동으로 생성하는 새로운 방법을 제안한다. 이를 위해 이 논문에서는 테스트 대상 프로그램을 1차 관계 논리 언어인 Alloy로 변환하고 Alloy 분석기를 통하여 테스트 데이터를 생성하는 방법을 제안한다. 제안된 방법은 사용자로 하여금 프로그램 경로를 선택하도록 하는 부담을 덜어줄 뿐만 아니라 다양한 테스트 적합성 기준에 따라 테스트 데이터를 생성하는 일을 용이하게 한다. 간단하지만 설명에 도움이 될 수 있는 예들을 통하여 제안한 방법에 대해 설명한다. In general, test data generation techniques require the specification of an entire program path for automated test data generation. This paper presents a new way for generating test data automatically een without specifying a program path completely. For the ends, this paper presents a technique for transforming a program under test into Alloy which is the first order relational logic and then producing test data via Alloy analyzer. The proposed method reduces the burden of selecting a program path and also makes it easy to generate test data according to various test adequacy criteria. This paper illustrates the proposed method through simple, but illustrative examples.

Creating and Visualizing Test Data from Programming Exercises

Automatic assessment of programming exercises is typically based on testing approach. Most automatic assessment frameworks execute tests and evaluate test results automatically, but the test data generation is not automated. No matter that automatic test data generation techniques and tools are available. We have researched how the Java PathFinder software model checker can be adopted to the specific needs of test data generation in automatic assessment. Practical problems considered are: how to derive test data directly from students' programs (i.e., without annotation) and how to visualize and how to abstract test data automatically for students? Interesting outcomes of our research are that with minor refinements generalized symbolic execution with lazy initialization (a test data generation algorithm implemented in PathFinder) can be used to construct test data directly from students' programs without annotation, and that intermediate results of the same algorithm can be used to provide novel visualizations of the test data.

Empirical evaluation of optimization algorithms when used in goal-oriented automated test data generation techniques

Software testing is an essential process in software development. Software testing is very costly, often consuming half the financial resources assigned to a project. The most laborious part of software testing is the generation of test-data. Currently, this process is principally a manual process. Hence, the automation of test-data generation can significantly cut the total cost of software testing and the software development cycle in general. A number of automated test-data generation approaches have already been explored. This paper highlights the goal-oriented approach as a promising approach to devise automated test-data generators. A range of optimization techniques can be used within these goal-oriented test-data generators, and their respective characteristics, when applied to these situations remain relatively unexplored. Therefore, in this paper, a comparative study about the effectiveness of the most commonly used optimization techniques is conducted.

Evolutionary testing in the presence of loop-assigned flags

Evolutionary testing is an effective technique for automatically generating good quality test data. However, for structural testing, the technique degenerates to random testing in the presence of flag variables, which also present problems for other automated test data generation techniques. Previous work on the flag problem does not address flags assigned in loops.This paper introduces a testability transformation that transforms programs with loop--assigned flags so that existing genetic approaches can be successfully applied. It then presents empirical data demonstrating the effectiveness of the transformation. Untransformed, the genetic algorithm flounders and is unable to find a solution. Two transformations are considered. The first allows the search to find a solution. The second reduces the time taken by an order of magnitude and, more importantly, reduces the slope of the cost increase; thus, greatly increasing the complexity of the problem to which the genetic algorithm can be applied. The paper also presents a second empirical study showing that loop--assigned flags are prevalent in real world code. They account for just under 11% of all flags.

Ordered mutation testing

Mutation Analysis is an established technique for test data generation and source code testing. Developed systems have concentrated on applying some or all of the possible mutant operators (perturbations) to the submitted code. Recent work tackled the scheduling of mutant program execution on vector processors or a Hypercube as a cost reduction scheme. This paper discusses a logical mechanism of impact driven testing in order to acheive full mutation testing with reduced overheads regardless of implementation. Guiding the mutant generation by block impact together with mutant operator heirarchies and a test data coverage scheme enable an efficient tool to be constructed.