Test Data Generation Research Articles

Comparative Study Between Flower Pollination Algorithm and Cuckoo Search Algorithm for t -Way Test Data Generation

T-way testing is a sampling approach for test data generation. Recently, adapting meta-heuristic algorithms for t-way testing is very attractive in order to find a minimum subset of test data that can test a system overall. As a consequence, several meta-heuristic algorithms have been used as the basis of t-way strategies. In order to guide software tester (and engineers in general) to select the best algorithm for the problem at hand, there is a need to evaluate and benchmark the performance of each strategy against common case studies. This paper presents a comparative study between two meta-heuristic strategies for t-way test data generation: Flower Pollination Algorithm (FPA) and Cuckoo Search (CS). Our experiments have performed on a real-world case study. Experimental results demonstrate that FPA appears to produce better results in most of the test cases in term of test suite size and convergence rate owing to its ability for controlling local and global search.

A Hybrid Particle Swarm Optimization and Differential Evolution based Test Data Generation Algorithm for Data-Flow Coverage using Neighbourhood Search Strategy

Meta-heuristic search techniques, mainly Genetic Algorithm (GA), have been widely applied for automated test data generation according to a structural test adequacy criterion . However, it remains a challenging task for more robust adequacy criterion such as data-flow coverage of a program. Now, focus is on the use of other highly-adaptive meta-heuristic search techniques such as Particle Swarm Optimization (PSO) and Differential Evolution (DE). In this paper, a hybrid (adaptive PSO and DE) algorithm is proposed to generate test data for data-flow dependencies of a program with a neighbourhood search strategy to improve the search capability of the hybrid algorithm. The fitness function is based on the concepts of dominance relations and branch distance. The measures considered are mean number of generations and mean percentage coverage. The performance of the hybrid algorithm is compared with that of DE, PSO, GA, and random search. Over several experiments on a set of benchmark programs, it is shown that the hybrid algorithm performed significantly better than DE, PSO, GA and random search in data-flow test data generation with respect to the measures collected.

A generator for test instances of scheduling problems concerning cranes in transshipment terminals

We present a test data generator that can be used for simulating processes of cranes handling containers. The concepts originate from container storage areas at seaports, but the generator can also be used for other applications, particularly for train terminals. A key aspect is that one or multiple cranes handle containers, that is, they store containers, receiving the containers in a designated handover area; retrieve containers, handing the containers over in the handover area; or reshuffle containers. We present a generic model and outline what is captured by the test data itself and what is left to be estimated by the user. Furthermore, we detail how data are generated to capture the considerable variety of container characteristics, which can be found in major terminals. Finally, we present examples to illustrate the variety of research projects supported by our test data generator.

Evolutionary mutation testing for IoT with recorded and generated events

SummaryMutation testing is a testing technique that has been applied successfully to several programming languages. Despite its benefits for software testing, the high computational cost of mutation testing has kept it from being widely used. Several refinements have been proposed to reduce its cost by reducing the number of generated mutants; one of those is evolutionary mutation testing (EMT). Evolutionary mutation testing aims at generating a reduced set of mutants with an evolutionary algorithm, which searches for potentially equivalent and difficult to kill mutants that help improve the test suite. Evolutionary mutation testing has been evaluated in two contexts so far, ie, web service compositions and object‐oriented C++ programmes. This study explores its performance when applied to event processing language queries of various domains. This study also considers the impact of the test data, since a lack of events or the need to have specific values in them can hinder testing. The effectiveness of evolutionary mutation testing with the original test data generators and the new internet of things test event generator tool is compared in multiple case studies.

A Sound Assessment of Test Suite Minimization Techniques

During software development, testing and re-testing occurs frequently to ensure that the software is working correctly before and after modifications. To carry out an effective testing process a test suite is created and executed to detect the faults in the existing code as well as in the modified code. The manual approach of test suite creation and execution is time consuming and labour intensive task as compared to automatically generated test data or test suite. The automatic test data generation is supposed to be an effective way, but a lot of redundant test cases are generated that increase the time, effort and cost of testing. Therefore, test suite minimization techniques are used to further minimize or reduce the number of test cases by selecting a subset from an initially random and large test suite to test the code before as well as after modification. In this study, a comprehensive analysis of the different test suite minimization techniques is presented in order to extend the existing studies and to propose new ideas in this direction.

Evolutionary approach to generating test data for data flow test

Software testing consumes a significant portion of software effort. Program entities such as branch or definition-use pairs (DUPs) are used in diverse software development tasks. In this study, the authors present a novel evolution-based approach to generating test data for all definition-use coverage. First, the subset of DUPs, which can ensure the coverage adequacy, is computed by a reduction algorithm for the whole DUPs. Then they apply a genetic algorithm to generate test data for the subset of DUPs. Furthermore, the fitness of an individual depends on the matching degree between the traversed path and the definition-clear path of each target DUP. They also investigate the coverage and the size of test cases of test data generation by applying the authors' approach on 15 widely used subject programs. The experimental results show that their approach can reduce the size of test cases that generated without affecting the coverage rate.

Search-based mutant selection for efficient test suite improvement: Evaluation and results

ContextSearch-based techniques have been applied to almost all areas in software engineering, especially to software testing, seeking to solve hard optimization problems. However, the problem of selecting mutants to improve the test suite at a lower cost has not been explored to the same extent as other problems, such as mutant selection for test suite evaluation or test data generation. ObjectiveIn this paper, we apply search-based mutant selection to enhance the quality of test suites efficiently. Namely, we use the technique known as Evolutionary Mutation Testing (EMT), which allows reducing the number of mutants while preserving the power to refine the test suite. Despite reported benefits of its application, the existing empirical results were derived from a limited number of case studies, a particular set of mutation operators and a vague measure, which currently makes it difficult to determine the real performance of this technique. MethodThis paper addresses the shortcomings of previous studies, providing a new methodology to evaluate EMT on the basis of the actual improvement of the test suite achieved by using the evolutionary strategy. We make use of that methodology in new experiments with a carefully selected set of real-world C++ case studies. ResultsEMT shows a good performance for most case studies and levels of demand of test suite improvement (around 45% less mutants than random selection in the best case). The results reveal that even a reduced subset of mutants selected with EMT can serve to increase confidence in the test suite, especially in programs with a large set of mutants. ConclusionsThese results support the use of search-based techniques to solve the problem of mutant selection for a more efficient test suite refinement. Additionally, we identify some aspects that could foreseeably help enhance EMT.

Optimizing Cost Function in Imperialist Competitive Algorithm for Path Coverage Problem in Software Testing

Search-based optimization methods have been used for software engineering activities such as software testing. In the field of software testing, search-based test data generation refers to application of meta-heuristic optimization methods to generate test data that cover the code space of a program. Automatic test data generation that can cover all the paths of software is known as a major challenge. The paper establishes a new cost function for automatic test data generation, which can traverse the non-iterative paths of software control flow graphs. This function is later compared with similar cost functions proposed in other articles. The results indicate the superior performance of the proposed function. Still another innovation in this paper is the application of the Imperialist Competitive Algorithm in automatic test data generation along with the proposed cost function. Automatic test data generation is implemented through the Imperialist Competitive Algorithm as well as the Genetic and Particle Swarm Optimization Algorithms for three software programs with different search space sizes. The algorithms are compared with each other in terms of convergence speed, computational time, and local search. Test data generated by the proposed method has achieved better results than other algorithms in finding the number of non-iterative paths, the convergence speed and computational time with growing the searching space of the software's control flow graph.

Using Artificial Bee Colony Algorithm for Test Data Generation and Path Testing Coverage

Software testing is a significant stage in software development lifecycle. There are different sorts of' structural software testing methodologies that may be generally utilized and moved forward through enhancing the traverse of all of the conceivable code software paths. The interest for automating data testing is growing; however, manual testing strategies utilization would be expensive and costly. Heuristic measure is being applied to; detect how better the result might be (solution fitness); result development mechanism; and suitableness criteria with stop search mechanism depending on wither a result is found or not. Testing experience could be exploited for finding a solution to the optimization problem by utilizing Meta heuristic procedures. The presented approach might have been tested for five programs to demonstrate the distinctive tests issues. This paper proposes an automatic test data generation approach that use artificial bee colony algorithm for software structural testing, particularly, path testing. This is brought on moving the centralization of data generation testing, as opposed to the automation of the whole testing operation. It executes artificial bee colony algorithm by creating testing data for the criteria of path coverage testing, and then applying the strategy to a group of test programs.

Computer based sport talent identification

This paper presents the concept of the computer decision support system for talent identification in sport. In this concept the use of two methods was assumed: pattern recognition based on multicriteria optimization and machine learning supervised classification algorithm: decision forest. The data for sport dyscyplin patterns has obtained from publication (Santos, Dawson, Matias et al. 2014). This data also has been used to generate test data sets to research purposes. The researches were carried out in author’s application and in the cloud environment Microsoft Azure Machine Learning Studio. The results show that both methods can be used with success to talent identification in sport.

Bayes‐TDG: effective test data generation using Bayesian belief network: toward failure‐detection effectiveness and maximum coverage

This study presents a novel test data generation method called Bayes-TDG. It is based on principles of Bayesian networks and provides the possibility of making inference from probabilistic data in the model to increase the prime path-coverage ratio for a given programme under test (PUT). In this regard, a new programme structure-based probabilistic network, TDG-NET, is proposed that is capable of modelling the conditional dependencies among the programme basic blocks (BBs) on one hand and conditional dependencies of the transitions between its BBs and input parameters on the other hand. To achieve failure-detection effectiveness, the authors propose a path selection strategy that works based on the predicted outcome of generated test cases. So, they mitigate the need for a human oracle, and the generated test suite could be directly used in fault localisation. Several experiments are conducted to evaluate the performance of Bayes-TDG. The results reveal that the method is promising and the generated test suite could be quite effective.

Using Genetic Algorithms in Test Data Generation

Software testing activities account for a considerable portion of systems development cost and, for this reason, many studies have sought to automate these activities. Test data generation has a high cost reduction potential (especially for complex domain systems), since it can decrease human effort. Although several studies have been published about this subject, articles of reviews covering this topic usually focus only on specific domains. This article presents a systematic mapping aiming at providing a broad, albeit critical, overview of the literature in the topic of test data generation using genetic algorithms. The selected studies were categorized by software testing technique (structural, functional, or mutation testing) for which test data were generated and according to the most significantly adapted genetic algorithms aspects. The most used evaluation metrics and software testing techniques were identified. The results showed that genetic algorithms have been successfully applied to simple test data generation, but are rarely used to generate complex test data such as images, videos, sounds, and 3D (three-dimensional) models. From these results, we discuss some challenges and opportunities for research in this area.

A reasoning-based approach to dynamic domain reduction in test data generation

In this article, a new criterion, domain coverage, for white box testing is introduced. In search of the biggest subdomains for input variables exercising a given path, the variable domains are broken down into overlapping subregions in such a way that the resultant subregions either exercise or deviate from the path. To this aim, an incremental method is applied to detect the subregions, covering the longest subpaths from the start of the given path. The detected subdomain is further subdivided, and this process is repeated as far as the path is fully covered. If no subdomain is detected, the method is backtracked. The backtrack begins with the subdomain covering the longest subpath at the beginning of the given path, but this time the subdomain is divided into relatively smaller subregions, compared with the last time. A directed acyclic graph representation of each input variable domain is used to ignore re-examination of those subdomains which deviate from the path. Conducted experiments reveal that our proposed domain-oriented algorithm outperforms the existing path-oriented test data generation algorithms.

PEMBANGKITAN DATA UJI BERBASIS DIAGRAM AKTIVITAS DAN DIAGRAM STATECHART

Testing is the stage of software development used to determine whether a software is ready for release or not. In making test cases using reference activity diagrams and statechart diagrams, a help representation was made, ie State-Activity-Diagram (SAD). The generation of test cases using a reference between the statechart diagram and the status diagram is still inadequate because in the case of the test produced there is no test data. The selection of test data for many test cases will be tedious and time consuming. In this paper, it is proposed to generate test data automatically based on existing test cases. Test data created based on class diagrams, and data dictionaries. The test case data consists of inputs and results. First enter information about the functions involved in the test case into the SAD node so that the SAD-S Diagram is obtained. Second, after the process of making the test case is completed, the test data is made by looking at the data dictionary function so that the test data is formed.

Polynomial regression with reduced over-fitting—The PALS technique

In order to reduce over-fitting, a new polynomial least-squares technique, named polynomial area least-squares (PALS), was developed. The technique is based on the minimization of the sum of the squared areas defined between the polynomial fitting function and the line segments connecting every two consecutive data points. A relatively simple system of linear equations was developed to compute the free parameters of the polynomial using the available training data. The ability of the PALS technique to generalize was evaluated through the generation of test data and statistical simulations. The PALS technique produces polynomials with reduced over-fitting, with little dependency on the order of the polynomial, especially when the polynomial order is high, so that, in some cases, increasing the order of the polynomial may even contribute to further over-fitting reduction. The major advantage of PALS is that it does not depend on any case-dependent parameter necessary for the data fitting procedure; the major limitation is that it can only be applied to polynomial regression with a single independent variable.

CHOMK: Concurrent Higher-Order Mutants Killing Using Genetic Algorithm

Higher-order subtle mutants are faults that are hard to detect or kill by the existing test set used for killing all the first-order mutants of the given program. Recently, some techniques have been proposed to construct higher-order subtle concurrency mutants that are not represented by the first-order mutants. To the best of our knowledge, there is no test-input generation technique proposed to kill this type of mutants. This paper proposes a search-based technique for generating a set of test inputs to kill higher-order subtle concurrency mutants. The proposed technique utilizes genetic algorithms in generating the set of test inputs. The performance of the proposed technique is evaluated and compared with that of the random-based test-data generation technique. The obtained results demonstrate the effectiveness of the proposed technique as it outperforms the random technique in terms of the killing ratio for the generated set of subtle concurrency mutants and the size of test suite. In the range of tested set of subtle concurrency mutants, the proposed technique approximately killed 91.4% of all mutants using 79 test cases compared to 82.8% using 128 test cases for the random technique.

Design and development of fault classification algorithm based on relevance vector machine for power transformer

Identification of faults within power transformers is the means of ensuring unit transformer protection. Existing relay maloperates during abnormalities such as magnetising inrush, CT saturation and high resistance internal fault condition. Therefore, it is essential to categorise the internal fault and external abnormality/fault in case of transformer protection. This study presents a new scheme, based on relevance vector machine (RVM) as a fault classifier. The developed algorithm is assessed by simulating various disorders on 345 MVA, 400/220 kV transformer in PSCAD/EMTDC software and also on prototype model with 2 kVA, 230/110 V multi-tapping transformer. One cycle post fault current signals are captured from primary and secondary to form feature vectors. These feature vectors are used as an input to RVM for classification of various test cases. Wide variation in system parameters and fault conditions are considered for test data generation and validation. The proposed scheme is compared with the support vector machine (SVM) and probabilistic neural network (PNN)-based techniques. The proposed scheme successfully discriminates various types of internal faults and external abnormalities in power transformer within a short time. The fault classification accuracy obtained by proposed RVM technique is more than 99% in comparison to SVM and PNN-based schemes.

An Effective Test Data Generation Method Based on Improved Particle Swarm Optimization Algorithm

A suitable set of test data can improve test efficiency and automatic level in automated testing. This paper presents an automatic generation method of test data based on improved particle swarm optimization algorithm, which can significantly improve the efficiency of automated testing.

Performance metamorphic testing: A Proof of concept

ContextPerformance testing is a challenging task mainly due to the lack of test oracles, i.e. mechanisms to decide whether the performance of a program is acceptable or not because of a bug. Metamorphic testing enables the generation of test cases in the absence of an oracle by exploiting the so–called metamorphic relations between the inputs and outputs of multiple executions of the program under test. In the last two decades, metamorphic testing has been successfully used to detect functional faults in different domains. However, its applicability to performance testing remains unexplored. ObjectiveWe propose the application of metamorphic testing to reveal performance failures. MethodWe define Performance Metamorphic Relations (PMRs) as expected relations between performance measurements of multiple executions of the program under test. These relations can be turned into assertions for the automated detection of performance bugs, removing the need for complex benchmarks and domain experts guidance. As a further benefit, PMRs can be turned into fitness functions to guide search–based techniques on the generation of test data. ResultsThe feasibility of the approach is illustrated through an experimental proof of concept in the context of the automated analysis of feature models. ConclusionThe results confirm the potential of metamorphic testing, in combination with search-based techniques, to automate the detection of performance bugs.

Evaluating Bees Algorithm for Sequence-based T-way Testing Test Data Generation

T-way testing is a testing technique that is used to detect faults due to parameter interactions or software configurations. In order to perform t-way testing, software testers need to prepare the test data. For a system with many configuration parameters, the test data could lead to combinatorial explosion problem, since all possible parameter combinations need to be considered. Therefore, test data generation for t-way testing need to be optimized. Many t-way test data generation strategies have been proposed in the literature to generate optimized t-way test data. However, very few strategies have been proposed for sequence-based t-way. This paper presents statistical analysis on the performance of Bees Algorithm against the other sequence t-way strategies, in order to generate test cases.