Test Data Generation Techniques Research Articles

CHOMK: Concurrent Higher-Order Mutants Killing Using Genetic Algorithm

Higher-order subtle mutants are faults that are hard to detect or kill by the existing test set used for killing all the first-order mutants of the given program. Recently, some techniques have been proposed to construct higher-order subtle concurrency mutants that are not represented by the first-order mutants. To the best of our knowledge, there is no test-input generation technique proposed to kill this type of mutants. This paper proposes a search-based technique for generating a set of test inputs to kill higher-order subtle concurrency mutants. The proposed technique utilizes genetic algorithms in generating the set of test inputs. The performance of the proposed technique is evaluated and compared with that of the random-based test-data generation technique. The obtained results demonstrate the effectiveness of the proposed technique as it outperforms the random technique in terms of the killing ratio for the generated set of subtle concurrency mutants and the size of test suite. In the range of tested set of subtle concurrency mutants, the proposed technique approximately killed 91.4% of all mutants using 79 test cases compared to 82.8% using 128 test cases for the random technique.

Structural test data generation using a memetic ant colony optimization based on evolution strategies

Abstract Test data generation is one of the key activities that has a significant impact on the efficiency and effectiveness of software testing. Since manual test data generation is quite inefficient and even impractical, automated test data generation has been realized to produce an appropriate subset of input data to carry out effective software testing in reasonable times. This paper presents a memetic ant colony optimization (ACO) algorithm for structural test data generation. The proposed approach incorporates (1+1)-evolution strategies (ES) to improve the search functionality of ants in local moves and enhance search exploitation. Moreover, we have introduced a novel definition of the pheromone functionality in the way that it discourages ants from choosing mostly covered paths of the program to reinforce search exploration. Given that branch coverage is considered as the coverage criterion, two fitness functions are used accordingly for our proposed algorithm. The first fitness function is a Boolean function which is particularly defined to maximize branch coverage. It outputs one if a given solution is successful in traversing at least a yet uncovered branch; otherwise, it returns zero. The second fitness function is formulated according to the complexity of branches covered. The value of the second fitness function is not taken into account for solutions whose Boolean function value equals one. For these solutions, the decision-making process of ants is merely carried out based on the first fitness function. The experimental results indicate the superiority of our memetic ACO algorithm relative to existing test data generation techniques in terms of both branch coverage and convergence speed.

Search-Based Junit Test Case Generation of Code Using Object Instances and Genetic Algorithm

To ensure software quality software testing is done .Test data generation is one of the more expensive parts of software testing. So, for reducing software cost and development time automation of test data generation is required. The object-oriented paradigm can be challenging for generating test data, due to some aspects of its features like abstract class, encapsulation, inheritance and visibility. To reach high code coverage of object oriented code search based test data generation technique is used. Proposed approach show that how efficiency and effectiveness of search based test data generation using static analysis. System takes input as different java class files then instances are generated for that classes then generate sequence of method call for whole code coverage then using genetic algorithm which is very useful and work efficiently when there is large search space it use to reach test target and finally generate test cases in Junit format which are helpful for all developers to perform unit testing.

메트릭에 따른 탐색 기반 테스팅 알고리즘 비교

Search-Based Software Testing (SBST) is an effective technique for test data generation on large domain size. Although the performance of SBST seems to be affected by the structural characteristics of Software Under Test (SUT), studies for the comparison of SBST techniques considering structural characteristics are rare. In addition to the comparison study for SBST, we analyzed the best algorithm with different structural characteristics of SUT. For the generalization of experimental results, we automatically generated 19,800 SUTs by combining four metrics, which are expected to affect the performance of SBST. According to the experiment results, Genetic algorithm showed the best performance for SUTs with high complexity and test data evaluation with count . On the other hand, the genetic simulated annealing and the simulated annealing showed relatively better performance for SUTs with high complexity and test data evaluation with count . Genetic simulated annealing, simulated annealing and hill climbing showed better performance for SUTs with low complexity.

A Comparative Study on Test Case Generation of Concurrent Programs

This paper deals with a comparative study on testing of concurrent programs based on different techniques. The various challenges in testing concurrent programming are: defining test coverage criteria based on control flow, generating control flow graph of nondeterministic programs, investigating the applicability of sequential testing criteria to parallel program testing etc. For solving these issues, some existing techniques are discussed in this study. Various researchers use an intermediate graph called Event Inter Actions Graph (EIAG) to solve the problem of generating the control flow graph of nondeterministic programs. Some researches propose an intermediate graph called Interaction Sequence Testing Criteria (ISTC) approach based on sequence of interactions to solve the problem of test coverage criteria based on control and data flow. Another method to solve the problem of generating test coverage based on control flow graph of nondeterministic programs is constraint based approach. It needs constrained elements to generate test case which includes structural element and constraint. The selection of good test cases has been addressed by test data generation technique. The technique of concurrent path analysis approach is used to solve the problem of applicability of sequential testing criteria to parallel program testing. It reduces the number of combined concurrent test paths. The sequential test paths are combined to form concurrent test path. The Integration and System Test Automation (ISTA) approach is used to solve the problem of applicability of sequential testing criteria to parallel program testing. It is used for automated test case generation and execution by using high-level Petri net is a finite state test model.

Automated Expected Output Generation

What is the importance of automated test data generation? This question is of concern for the entire software community. We want to know the meaning and usefulness of automated test data generation. Is it related to design of test cases or execution of already designed test cases or both? How can we generate expected output using automated test data generation technique? This is a weak area and limits the applicability of automated testing. The test case creation, execution and evaluation are three distinct activities. The execution and evaluation of test cases can be easily automated, but creation needs expected output that is difficult to automate due to its dependency on functionality of the program. This paper addresses the significance of techniques and tools available for automated test data generation. In this work, we analyze and assess the existing techniques for automated expected output generation and discuss the practical significance of these techniques.

Introduction to the special issue on Mutation Testing

Introduction to the special issue on Mutation Testing

Artificial life and cellular automata based automated test case generator

Manual test data generation is carried out by using the ability of neurons to recognize patterns. The nervous system and the brain coordinate to generate test cases, which are capable of finding potential faults. Automated test data generators lack the ability to produce efficient test cases because they do not imitate natural processes. This paper proposes using Artificial Life based systems for generating test cases. Cellular Automata and Langton's loop have been used to accomplish the above task. Cellular Automata are parallel distributed systems capable of reproducing using self generated patterns. These fascinating techniques have been amalgamated with standard test data generation techniques to give rise to a methodology, which generates test cases for white box testing. Langton's Loops have been used to generate test cases for Black Box Testing. The approach has been verified on a set of 20 programs. The programs have been selected on the basis of their Lines of Code and utility. The results obtained have been verified using Average Probability of Fault Detection. This paper also proposes a new framework capable of crafting test cases taking into account the oracle cost.

동적 분기 커버리지를 위한 테스트 데이터 자동 생성

높은 테스트 커버리지 달성을 위해 심볼릭 실행, 자료 흐름 분석 및 제약 해결 기법 등을 이용하여 테스트 데이터 생성을 하는 것이 일반적이다. 최근에 그와 같은 정교한 수단 없이도 높은 커버리지를 효과적으로 달성 할 수 있는 방법이 제안되었다. 그러나 이 방법도 실행 중에 다른 메모리 로케이션들이 바인딩되는 분기 조건을 갖는 프로그램에 대해서는 높은 커버리지를 가져오는 테스트 데이터 생성이 어려웠다. 특히 플래그 조건을 가지는 프로그램에 대해서는 특정 분기들이 실행되지 않아 높은 커버리지를 달성하지 못하는 경우가 발생한다. 이 논문에서는 이 문제를 다루기 위하여 기존 커버리지 기준을 개선한 동적 분기 커버리지 기준들과 동적 분기에 기반 한 테스트 데이터 생성 전략을 제안한다. 실험을 통하여 플래그 조건이 있는 프로그램들에 대해서도 제안된 방법이 기존의 방법에 비해 효과적으로 커버리지를 달성함을 보인다. 요약은 무슨 연구를 어떻게 수행하였는지, 주된 연구결과와 그 중요성에 관해 간결하게 기술하여야 한다. In order to achieve high test coverage, it is usual to generate test data using various techniques including symbolic execution, data flow analysis or constraints solving. Recently, a technique for automated test data generation that fulfills high coverage effectively without those sophisticated means has been proposed. However, the technique shows its weakness in the generation of test data that leads to high coverage for programs having branch conditions where different memory locations are binded during execution. For certain programs with flag conditions, in particular, high coverage can not be achieved because specific branches are not executed. To address the problem, this paper presents dynamic branch coverage criteria and a test data generation technique based on the notion of dynamic branch. It is shown that the proposed technique compared to the previous approach is more effective by conducting experiments involving programs with flag conditions.

Cellular automata based test data generation

Manual Test Data Generation is an expensive, error prone and tedious task. Therefore, there is an immediate need to make the automation of this process as efficient and effective as possible. The work presented intends to automate the process of Test Data Generation with a goal of attaining maximum coverage. A Cellular Automata system is discrete in space and time. Cellular Automata have been applied to things like designing water distribution systems and studying the patterns of migration. This fascinating technique has been amalgamated with standard test data generation techniques to give rise to a technique which generates better test cases than the existing techniques. The approach has been verified on programs selected in accordance with their Lines of Code and utility. The results obtained have been verified. The proposed work is a part of a larger system being developed, which takes into account both black box and white box testing.

Searching and generating test inputs for mutation testing

Mutation testing is usually regarded as an important method towards fault revealing. Despite this advantage, it has proved to be impractical for industrial use because of its expenses. To this extend, automated techniques are needed in order to apply and reduce the method’s demands. Whilst there is much evidence that automated test data generation techniques can effectively automate the testing process, there has been little work on applying them in the context of mutation testing. In this paper, search-based testing is used in order to effectively generate test inputs capable of revealing mutants. To this end, a dynamic execution scheme capable of introducing and guiding the search towards the sought mutants is proposed. Experimentation with the proposed approach reveals its superiority from the previously proposed methods. Additionally, the framework’s feasibility and practicality of producing mutation based test cases are also demonstrated.

Functional test data generation for Simulink-like models

Abstract Embedded systems are increasingly present in many electronic devices and is often related to critical applications. Therefore, the need for a well planned and executed testing procedure is even higher. We intend to contribute in this area by presenting an experimental evaluation of the pairwise combinatorial approach as a technique for test data generation applied specifically to Simulink-like models. In particular, we have applied our strategy to the generated source code of several models. Furthermore, a testing tool was developed to assist in the test data generation process. We show that there is no statistical significant advantages of the proposed approach over random generation of test data, but when used together they yield better results. The feasibility of the experimental results indicate that efforts can be employed in order to obtain a testing strategy integrated within a testing environment.

Empirical Comparison of Test Data Generation Techniques using Activity Diagrams

Unified Modeling Language) is now a leading standard for defining software processes. Test data generation is advantageous in early phases of software development. Activity diagrams are user and developers' friendly because of the ease in their understanding. Many papers have presented techniques for test data generation using activity diagrams. These techniques have their own specific benefits considering required test data to be generated. On application of these techniques on same input i.e. activity diagrams, differences and similarities emerge evidently. These outcomes can provide clarity among testers, so as to decide upon the technique for test data generation depending upon the phase and type of test data required. In this paper we performed a comparative study of the five techniques of test data generation based on activity diagrams using ten examples.

실행가능 목적 코드를 기반으로 하는 자동 테스트 데이터 생성

It is usual for test data generation to be performed using either high-level specifications or source codes written in high-level programming languages. In certain circumstances, however, such information is not always available. This paper presents a technique that generates test data based on executable object codes. The proposed technique makes use of a very simple function minimization technique without sophisticated object code analysis and produces test data dynamically. We have conducted a simple experiment to evaluate the effectiveness of the proposed test data generation technique with a triangle classification program to show that branch coverage can be easily achieved.

목적 지향 콘콜릭 테스팅을 이용한 플래그 변수가 있는 프로그램에 대한 테스트 데이터 생성

This paper presents a goal-oriented test data generation technique that specializes concolic testing. The proposed technique, referred to as GCT (Goal-oriented Concolic Testing) produces test inputs which execute a specific target. Concolic testing can be seen as the brute force approach to search the space of all possible paths until a required test input is found. In contrast, GCT restricts the number of program paths that are explored by using data flow information to identify statements that should be executed beforehand in order for the target to be executed. We conducted experiments to evaluate the performance of GCT with programs with flag variables to show its effectiveness.

Effective Fault Localization Approach Using Feedback

At the stage of software debugging, the effective interaction between software debugging engineers and fault localization techniques can greatly improve fault localization performance. However, most fault localization approaches usually ignore this interaction and merely utilize the information from testing. Due to different goals of testing and fault localization, the lack of interaction may lead to the issue of information inadequacy, which can substantially degrade fault localization performance. In addition, human work is costly and error-prone. It is vital to study and simulate the pattern of debugging engineers as they apply their knowledge and experience to this interaction to promote fault localization effectiveness and reduce their workload. Thus this paper proposes an effective fault localization approach to simulate this interaction via feedback. Based on results obtained from fault localization techniques, this approach utilizes test data generation techniques to automatically produce feedback for interacting with these fault localization techniques, and then iterate this process to improve fault localization performance until a specific stopping condition is satisfied. Experiments on two standard benchmarks demonstrate the significant improvement of our approach over a promising fault localization technique, namely the spectrum-based fault localization technique.

An Adequacy Based Test Data Generation Technique Using Genetic Algorithms

As the complexity of software is increasing, generating an effective test data has become a necessity. This necessity has increased the demand for techniques that can generate test data effectively. This paper proposes a test data generation technique based on adequacy based testing criteria. Adequacy based testing criteria uses the concept of mutation analysis to check the adequacy of test data. In general, mutation analysis is applied after the test data is generated. But, in this work, we propose a technique that applies mutation analysis at the time of test data generation only, rather than applying it after the test data has been generated. This saves significant amount of time (required to generate adequate test cases) as compared to the latter case as the total time in the latter case is the sum of the time to generate test data and the time to apply mutation analysis to the generated test data. We also use genetic algorithms that explore the complete domain of the program to provide near-global optimum solution. In this paper, we first define and explain the proposed technique. Then we validate the proposed technique using ten real time programs. The proposed technique is compared with path testing technique (that use reliability based testing criteria) for these ten programs. The results show that the adequacy based proposed technique is better than the reliability based path testing technique and there is a significant reduce in number of generated test cases and time taken to generate test cases.

Effective Estimation Software cost using Test Generations

In software engineering, we have many problems particularly with software cost estimation and the assessment of the evaluation model. Software cost estimation mainly depends on two factors which are the estimation consistency and estimation accuracy. These are very much important to the perfect development of the software without any drawbacks to it. In the past there have been many models and techniques that have failed to give the correct software estimation that really challenges the all factors of perfect software in this paper we use software requirement specification but we were not getting the accurate results that are helpful for developing of perfect software that is having perfect cost estimation. So we introduce the Automated Test-Data Generation Techniques method that gives us the desired results which will be helping us to know the software cost estimation very much accurately and also estimate consistently.

Diversity oriented test data generation using metaheuristic search techniques

We present a new test data generation technique which uses the concept of diversity of test sets as a basis for the diversity oriented test data generation – DOTG. Using DOTG we translate into an automatic test data generation technique the intuitive belief that increasing the variety, or diversity, of the test data used to test a program can lead to an improvement on the completeness, or quality, of the testing performed. We define the input domain perspective for diversity (DOTG-ID), which considers the distances among the test data in the program input domain to compute a diversity value for test sets. We describe metaheuristics which can be used to automate the generation of test sets for the DOTG-ID testing technique: simulated annealing; a genetic algorithm; and a proposed metaheuristic named simulated repulsion. The effectiveness of DOTG-ID was evaluated by using a Monte Carlo simulation, and also by applying the technique to test simple programs and measuring the data-flow coverage and mutation scores achieved. The standard random testing technique was used as a baseline for these evaluations. Results provide an understanding of the potential gains in terms of testing effectiveness of DOTG-ID over random testing and also reveal testing factors which can make DOTG-ID less effective.

A multiple-population genetic algorithm for branch coverage test data generation

The software testing phase in the software development process is considered a time-consuming process. In order to reduce the overall development cost, automatic test data generation techniques based on genetic algorithms have been widely applied. This research explores a new approach for using genetic algorithms as test data generators to execute all the branches in a program. In the literature, existing approaches for test data generation using genetic algorithms are mainly focused on maintaining a single-population of candidate tests, where the computation of the fitness function for a particular target branch is based on the closeness of the input execution path to the control dependency condition of that branch. The new approach utilizes acyclic predicate paths of the program's control flow graph containing the target branch as goals of separate search processes using distinct island populations. The advantages of the suggested approach is its ability to explore a greater variety of execution paths, and in certain conditions, increasing the search effectiveness. When applied to a collection of programs with a moderate number of branches, it has been shown experimentally that the proposed multiple-population algorithm outperforms the single-population algorithm significantly in terms of the number of executions, execution time, time improvement, and search effectiveness.