Test Suite Optimization Research Articles

An Optimal Solution for Test Case Generation Using ROBDD Graph and PSO Algorithm

Software testing is one of the most important techniques to examine the behavior of the software products to assure their quality. An effective and efficient testing approach must balance two important but conflicting requirements. One of them is the accuracy that needs a large number of test cases for testing, and the second one is reducing the time and cost, which requires a few test cases. Even for small software, the number of possible test cases is typically very large, and exhaustive testing is impractical. Hence, selecting appropriate test suite is necessary. Cause–effect graph testing is a common black‐box testing technique, which is equivalently representing Boolean relations between input parameters. However, the other traditional black‐box strategies cannot identify the relations that it may result in loss of some of the important test cases. Although the cause–effect graph is regarded very promising in specification testing, it is observed that most of the proposed approaches using the graph are complex or generate impossible and a large number of test cases. This observation has motivated our research to propose an efficient strategy to generate minimal test suite that simultaneously achieves high coverage of input parameters. To do so, at first, we identify major effects from the cause–effect graph using reduced ordered binary decision diagram (ROBDD). ROBDD makes the related Boolean expression of the graph concise and obtains a unique representation of the expression. Using the ROBDD, it is possible to reduce the size of the generated test suite and to perform testing faster. After that, our proposed method utilizes particle swarm optimization (PSO) algorithm to select the optimal test suite, which covers all pairwise combinations of input parameters. The experimental results show that our approach simultaneously achieves high efficacy and reduces cost of testing by selecting appropriate test cases, respectively, to both test size and coverage size. Also, it outperforms some existing state‐of‐the‐art strategies in the black‐box testing. Copyright © 2015 John Wiley & Sons, Ltd.

Hybrid test language processing based framework for test case optimization

One of the major problems of software industry is its inability to develop error free software. Approximately 40–50 % of SDLC cost is spent on software testing. When applications need to be developed rapidly with changing user demands, this expenditure takes a significant value. Software testing is very important but is an effort consuming activity. Large numbers of test cases are possible and some of them may make software fail. Therefore there is a need to automate the existing manual approach to minimize the test case execution time and further optimize test cases. In this paper a keyword oriented data driven hybrid framework—HTLP (hybrid test language processing) been proposed for automatic test case execution and optimization. The HTLP is applied to an open source software based application-Vtiger CRM5. The proposed framework is compared with manual testing approach for test case execution and optimization. Based on experimental results it is concluded that (1) with iterative regression cycles the HTLP testing time is significantly very low as compared to manual testing approach and (2) a high level of test suite optimization (83.78 %) is achieved using the proposed framework.

The Rolling Tide Evolutionary Algorithm: A Multiobjective Optimizer for Noisy Optimization Problems

As the methods for evolutionary multiobjective optimization (EMO) mature and are applied to a greater number of real-world problems, there has been gathering interest in the effect of uncertainty and noise on multiobjective optimization, specifically how algorithms are affected by it, how to mitigate its effects, and whether some optimizers are better suited to dealing with it than others. Here we address the problem of uncertain evaluation, in which the uncertainty can be modeled as an additive noise in objective space. We develop a novel algorithm, the rolling tide evolutionary algorithm (RTEA), which progressively improves the accuracy of its estimated Pareto set, while simultaneously driving the front toward the true Pareto front. It can cope with noise whose characteristics change as a function of location (both design and objective), or which alter during the course of an optimization. Four state-of-the-art noise-tolerant EMO algorithms, as well as four widely used standard EMO algorithms, are compared to RTEA on 70 instances of ten continuous space test problems from the CEC’09 multiobjective optimization test suite. Different instances of these problems are generated by modifying them to exhibit different types and intensities of noise. RTEA seems to provide competitive performance across both the range of test problems used and noise types.

A Distinctive Genetic Approach for Test-Suite Optimization

Software Testing is one of the expensive activities in Software development lifecycle but it used in to assure the quality of the Software. One of the important focuses in software engineering process is lessen the effort of software testing and cut down the cost & time of testing. This paper presents a unified method for reducing the test cases by adopting the hypothesis of Genetic algorithm such as crossover, mutation and crowding distance to elucidate the given problem and results in the minimum set of test cases with the ability to cover all the faults in minimum time.

Generating combinatorial test cases using Simplified Swarm Optimization (SSO) algorithm for automated GUI functional testing

Graphical User Interface (GUI) is the outer skin of programs that facilitate the interaction between the user and different type of computing devices. It is been used in different aspects ranging from normal computers, mobile device, to even very small device nowadays like watches. This interaction uses different tools and programming objects like images, text, buttons, checkboxes, etc. With this emergence of different types of GUIs, they become an essential component to be tested (if available in the software) to ensure that the software meets the required quality by the user. In contrast to non-functional testing, function testing of GUI insures a proper interaction between the user and the application interface without dealing with the coding internals. In this paper, a strategy for GUI functional testing using Simplified Swarm Optimization (SSO) is proposed. The SSO is used to generate an optimized test suite with the help of Event-Interaction Graph (EIG). The proposed strategy also manages and repairs the test suites by deleting the unnecessary event sequences that are not applicable. The proposed generation algorithm based on SSO has proved its effectiveness by evaluating it against other algorithms. In addition, the strategy is applied on a standard case study and proved its applicability in reality.

English

Test driven design (TDD) and design driven testing (DDT) are used for test case generation. TDD generates so many duplicated test cases at the end of the project. DDT is novel approach to generate test cases based on design model of application. Comparative study indicates batter result using DDT. White-box traditional regression testing depends on analysis of impact of changes in source code. This practice minimizes the amount of testing required to validate code changes but they do not influence on requirement specification. Black-box testing supports ability to test from higher level design and requirement. Test optimal is a tool to generate test cases based on model driven environment using UML activity diagrams. Traditional approach generates many numbers of large and duplicate test cases. Proposed approach will minimize generated test cases and generate optimal test suite using model driven testing. Changes in design of project, modified activity diagram element will identify common and uncommon test cases. Where uncommon test cases are focused for further testing. Filtering will lead to better available resource utilization and will improve software project management.

Comparison of Search based Techniques for Automated Test Data Generation

One of the essential parts of the software development process is software testing as it ensures the delivery of a good quality and reliable software. Various techniques and algorithms have been developed to carry out the testing process. This paper deals with utility of the nature based algorithms namely Genetic Algorithm, Ant Colony Optimization algorithm and Artificial Bee Colony algorithm in automatic generation of optimized test suite for a given set of programs. The performance of algorithms is evaluated using various factors such as number of paths covered, number of iterations, number of test cases produced and time taken for generation of test suite. The results of performance analysis concluded that Artificial Bee Colony algorithm is more efficient as compared to other mentioned algorithms and can be employed for optimized test suite generation for various complex programs or software. General Terms Measurement, Performance, Experimentation

Exploration and analysis of regression test suite optimization

Regression Test Suite Optimization (RTO) is an active research area. A Regression Test Suite is always growing due to changes in software, which increases testing time. To save time and resources optimization of regression test suites is mandatory. Researchers have optimized test suites using conventional and Computational Intelligence based approaches and achieve optimization of regression test suites through selection techniques, minimization or reduction techniques and ranking or prioritization techniques. This paper surveys existing techniques for regression test suite optimization, various tools and mathematical models being used for RTO. During this survey we found many interesting facts about regression test suite optimization that will be shared in the conclusion.

Low-Cost Analog/RF IC Testing through Combined Intra- and Inter-Die Correlation Models

This paper presents statistical methods to identify and take advantage of correlations in the test methods and wafer-level spatial correlations among devices. The result, shown for industrial designs, is a far more optimized test suite.

Multiple objective test suite optimization: A fuzzy logic based approach

Multiple objective test suite optimization: A fuzzy logic based approach

Optimization of Software Testing for Discrete Testsuite using Genetic Algorithm and Sampling Technique

Though Software Testing plays a vital role to produce better quality products, but it is time consuming and incurs expenditure. The more test will be conducted, products will be finer. Hence, testing is must for software development. Another side of testing is spending much money on it because people will work rigorously to generate the test suite and executing it. As we know that, no software is bug free software; we cannot assure that the testing which has been done for particular software is sufficient. To get a cost effective testing strategy, one should go for optimization of testsuite. This paper uses heuristic algorithm with sampling techniques used to optimize the test suite. Genetic algorithm may play a major role to have a sound weight on optimization of testsuite. If we go for sampling techniques then it usually gives more optimum result.

Basis Path Analysis for Testing Complex System of Systems

In Systems of Systems (SoS), a major challenge is to determine how to design a test suite that will check that the complete SoS mission and objectives are achieved. Combinatorial strategies that test for every interface in the SoS are not always optimal due to their exponential nature, and given the mission of the SoS, not all the constituent systems might have to actually interact with each other. To overcome these problems, this paper proposes the use of Basis Path Testing, which is a white-box software testing technique that makes use of Graph Theory to analyse the complexity of a structured system by creating a control flow graph from each of the program's functions to design an optimal test suite. This test suite is a set of paths that traverse through the functions, which are assumed linearly independent and that can be used to create a test strategy that will exercise all of the program's functions at least once to verify and validate their functionality. By applying Basis Path Testing analysis to the constituent systems in a SoS, the tester can develop an optimal test suite that will guarantee that all possible independent paths, all possible logical decisions, and all their interfaces are executed at least once. This paper presents a SoS sample architecture and show how to generate a test suite using Basis Path Testing analysis.

Automated Generation of Independent Paths and Test Suite Optimization Using Artificial Bee Colony

Software test suite optimization is one of the most important problems in software engineering research. This paper deals with Automatic Generation of Feasible Independent Paths and Software Test Suite Optimization using Artificial Bee Colony (ABC) based novel search technique. In this approach, ABC combines both global search methods done by scout bees and local search method done by employed bees and onlooker bees. The parallel behavior of these three bees makes generation of feasible independent paths and software test suite optimization faster. Test Cases are generated using Test Path Sequence Comparison Method as the fitness value objective function. This paper also presents an approach for the automated generation of feasible independent test path based on the priority of all edge coverage criteria. Finally, this paper compares the efficiency of ABC based approach with various approaches.

Multi Agent Based Framework for Structural and Model Based Test Case Generation

Software testing is done to find errors and fix them to improve quality. Software testing is a laborious and time–consuming work which spends almost 50% of the software system development resources. The number of test cases required to develop error-free software is high and when done manually the process of test case generation is time consuming and error prone. As systems are increasing in complexity, more systems perform mission-critical functions and the dependability requirements such as safety, reliability and security are vital to the users of the system. The competitive market place forces companies to define new approaches to reduce time-to-market as well as development cost of the systems. To increase the effectiveness and efficiency of the testing process and to reduce the overall development cost for software system, new approaches are required for test automation. The automation of test case generation is the most important aspect of test automation. No powerful test data generation tools combining structural and functional testing are commercially available today. The objective of the proposed system is to automate the process of test case generation for both structural and model based testing by deployment of an agent based framework. This framework automatically generates an optimized Test Suite for system under test that can assist the testers and developers and enable them to correct the errors in earlier stages of software development. The deployment of agent based framework reduces considerable amount of execution time when compared with the system without agents.

A Strategy for Automatic Quality Signing and Verification Processes for Hardware and Software Testing

We propose a novel strategy to optimize the test suite required for testing both hardware and software in a production line. Here, the strategy is based on two processes: Quality Signing Process and Quality Verification Process, respectively. Unlike earlier work, the proposed strategy is based on integration of black box and white box techniques in order to derive an optimum test suite during the Quality Signing Process. In this case, the generated optimal test suite significantly improves the Quality Verification Process. Considering both processes, the novelty of the proposed strategy is the fact that the optimization and reduction of test suite is performed by selecting only mutant killing test cases from cumulating t-way test cases. As such, the proposed strategy can potentially enhance the quality of product with minimal cost in terms of overall resource usage and time execution. As a case study, this paper describes the step-by-step application of the strategy for testing a 4-bit Magnitude Comparator Integrated Circuits in a production line. Comparatively, our result demonstrates that the proposed strategy outperforms the traditional block partitioning strategy with the mutant score of 100% to 90%, respectively, with the same number of test cases.

Bilevel Adaptive Weighted Sum Method for Multidisciplinary Multi-Objective Optimization

The primary goal of this research is to develop a framework for dealing with multi-objective, multidisciplinary optimization problems with a large number of variables. The proposed method is expected to provide a relatively uniformly spaced, widely distributed Pareto front. To achieve this end, a novel integration of the adaptive weighted sum method within a concurrent subspace optimization framework is presented. In the bilevel framework of concurrent subspace optimization, the adaptive weighted sum is used to make tradeoffs among multiple, conflicting objectives. To obtain better distributed solutions, two modifications are made. First, an additional equality constraint in suboptimization for each expected solution is relaxed because it causes slow convergence within the bilevel optimization framework. The probability of entrapment in local minima can also be reduced. Second, the mesh of the Pareto front patches is modified due to the low efficiency of the original scheme. The proposed method is demonstrated with three multidisciplinary design optimization problems: 1) a numerical multidisciplinary design optimization test problem with a convex Pareto front, available within the NASA multidisciplinary design optimization Test Suite; 2) a test problem with a nonconvex Pareto front, which is not easily solved; and 3) a conceptual design of a subsonic passenger aircraft, which consists of two objectives, four design variables, five coupling behavior variables, seven constraints in aerodynamics, and weight discipline. The primary results show that the proposed method is promising with regard to obtaining a uniformly spaced, widely distributed, and smooth Pareto front and is applicable in the design of large-scale, complex engineering systems such as aircraft.

The test suite generation problem: Optimal instances and their implications

In the test suite generation (TSG) problem for software systems, I is a set of n input parameters where each I ∈ I has κ ( I ) data values, and O is a collection of subsets of I where the interactions of the parameters in each O ∈ O are thought to affect the outcome of the system. A test case for ( I , O , κ ) is an n-tuple ( t 1 , t 2 , … , t n ) that specifies the value of each input parameter in I . The goal is to generate a smallest-sized test suite (i.e., a set of test cases) that covers all combinations of each O ∈ O . The decision version of TSG is known to be NP-complete. In this paper, we present new families of ( I , O , κ ) for which optimal test suites can be constructed efficiently. They differ from the ones already known by the way we characterize ( I , O ) and κ . We then use these instances to generate test suites for arbitrary software systems. When each O ∈ O has | O | = 2 , the sizes of the test suite are guaranteed to be at most ⌈ log 2 n ⌉ × OPT , matching the current best bound for this problem. Our constructions utilize the structure of ( I , O ) and κ ; consequently, the less “complex” ( I , O ) and κ are, the better are the bounds on the sizes of the test suites.

An Approach for Optimizing Test Suite Based on Testing Requirement Reduction

Test suite optimization aims at satisfying all testing objectives with the least number of test cases. According to the given testing objectives, the reduced testing requirement set can improve the effectiveness and efficiency of test suite optimization. This paper proposes a testing requirement reduction model that can describe the interrelations among the testing requirements in detail. Based on the model, this paper presents a testing requirement reduction method to generate the reduced testing requirement set, which is the basis of test suite generation, reduction and optimization. The experimental results show that the method is helpful to generate the smaller test suite and it contributes to the systematic, reasonable and effective testing.

An Approach for Optimizing Test Suite Based on Testing Requirement Reduction

Test suite optimization aims at satisfying all testing objectives with the least number of test cases. According to the given testing objectives, the reduced testing requirement set can improve the effectiveness and efficiency of test suite optimization. This paper proposes a testing requirement reduction model that can describe the interrelations among the testing requirements in detail. Based on the model, this paper presents a testing requirement reduction method to generate the reduced testing requirement set, which is the basis of test suite generation, reduction and optimization. The experimental results show that the method is helpful to generate the smaller test suite and it contributes to the systematic, reasonable and effective testing.

A method for assigning a value to a communication protocol test case

One of the main problems in industrial testing is the enormous number of test cases derived from any complex communication protocol. Due to budget constraints and tight schedule, the number of test cases has to be within a certain limit. However, by having a limit on the number of test cases, it raises some issues. For instances, what criteria should be used for selecting the test cases? How can we ensure that important test cases have not been excluded? We are proposing that assigning a value to each of the test cases of a test suite can provide a solution. By doing so, the relative importance of each of the test cases can be ranked and an optimal test suite can then be designed. The value of a test case is to be measured in economic terms, which could be based on the probability that a particular case will occur, and the probability that an error is likely to be uncovered. This paper presents a method for assigning a value to a test case of a communication protocol; it is based on sensitivity analysis, which involves execution, infection and propagation probabilities. To illustrate the method, the results of applying it to the INRES protocol are presented.