Existing Test Cases Research Articles

A Multi-Objective Technique to Prioritize Test Cases

While performing regression testing, an appropriate choice for test case ordering allows the tester to early discover faults in source code. To this end, test case prioritization techniques can be used. Several existing test case prioritization techniques leave out the execution cost of test cases and exploit a single objective function (e.g., code or requirements coverage). In this paper, we present a multi-objective test case prioritization technique that determines the ordering of test cases that maximize the number of discovered faults that are both technical and business critical. In other words, our new technique aims at both early discovering faults and reducing the execution cost of test cases. To this end, we automatically recover links among software artifacts (i.e., requirements specifications, test cases, and source code) and apply a metric-based approach to automatically identify critical and fault-prone portions of software artifacts, thus becoming able to give them more importance during test case prioritization. We experimentally evaluated our technique on 21 Java applications. The obtained results support our hypotheses on efficiency and effectiveness of our new technique and on the use of automatic artifacts analysis and weighting in test case prioritization.

Test Case Reduction Using Data Mining Technique

Software testing is a process of ratifying the functionality of software. It is a crucial area which consumes a great deal of time and cost. The time spent on testing is mainly concerned with testing large numbers of unreliable test cases. The authors' goal is to reduce the numbers and offer more reliable test cases, which can be achieved using certain selection techniques to choose a subset of existing test cases. The main goal of test case selection is to identify a subset of the test cases that are capable of satisfying the requirements as well as exposing most of the existing faults. The state of practice among test case selection heuristics is cyclomatic complexity and code coverage. The authors used clustering algorithm which is a data mining approach to reduce the number of test cases. Their approach was able to obtain 93 unique effective test cases out a total of 504.

Prioritizing manual test cases in rapid release environments

SummaryTest case prioritization is an important testing activity, in practice, specially for large scale systems. The goal is to rank the existing test cases in a way that they detect faults as soon as possible, so that any partial execution of the test suite detects the maximum number of defects for the given budget. Test prioritization becomes even more important when the test execution is time consuming, for example, manual system tests versus automated unit tests. Most existing test case prioritization techniques are based on code coverage, which requires access to source code. However, manual testing is mainly performed in a black‐box manner (manual testers do not have access to the source code). Therefore, in this paper, the existing test case prioritization techniques (e.g. diversity‐based and history‐based techniques) are examined and modified to be applicable on manual black‐box system testing. An empirical study on four older releases of desktop Firefox showed that none of the techniques were strongly dominating the others in all releases. However, when nine more recent releases of desktop Firefox, where the development has been moved from a traditional to a more agile and rapid release environment, were studied, a very significant difference between the history‐based approach and its alternatives was observed. The higher effectiveness of the history‐based approach compared with alternatives also held on 28 additional rapid releases of other Firefox projects – mobile Firefox and tablet Firefox. The conclusion of the paper is that test cases in rapid release environments can be very effectively prioritized for execution, based on their historical failure knowledge. In particular, it is the recency of historical knowledge that explains its effectiveness in rapid release environments rather than other changes in the process. Copyright © 2016 John Wiley & Sons, Ltd.

Regression test selection using class, sequence and activity diagrams

Regression test selection is a process to select a subset of existing test cases, which are then used with some new test cases for regression testing. Regression testing ensures that the changes made to the system have not affected the existing functionality. To date, there is no adequate technique which can do regression test selection by considering changes in semantics of operations (along with the other syntax and semantics changes) using UML diagrams. Change in semantics of an operation refers to the change in conditional statements, change in independent paths/unique paths, change in control flow and addition or deletion of any content from the existing functionality. In this study, a novel approach is presented which can do this using class, sequence and activity diagrams. The tool compared old and new versions of UML diagrams to categorise test cases into reusable, retestable, obsolete and newly generated category. Activity diagrams are specifically used to test the semantics of operations. The changed operations corresponding to these activity diagrams are also searched in class and sequence diagrams for regression test selection. This study has been validated by comparison with previous study. It is found that the authors’ work provides significant increase in accuracy.

Test Case Prioritization for Optimizing a Regression Test

Regression testing makes sure that upgradation of software in terms of adding new features or for bug fixing purposes should not hamper previously working functionalities. Whenever a software is upgraded or modified, a set of test cases are run on each of its functions to assure that the change to that function is not affecting other parts of the software that were previously running flawlessly. For achieving this, all existing test cases need to run as well as new test cases might be required to be created. It is not feasible to reexecute every test case for all the functions of a given software, because if there is a large number of test cases to be run, then a lot of time and effort would be required. This problem can be addressed by prioritizing test cases. Test case prioritization technique reorders the priority in which test cases are implemented, in an attempt to ensure that maximum faults are uncovered early on by the high priority test cases implemented first. In this paper we propose an optimized test case prioritization technique using Ant Colony Optimization (ACO) to reduce the cost, effort and time taken to perform regression testing and also uncover maximum faults. Comparison of different techniques such as Retest All, Test Case Minimization, Test Case Prioritization, Random Test Case Selection and Test Case Prioritization using ACO is also depicted.

Static Program Slicing- An Efficient Approach for Prioritization of Test Cases for Regression Testing

Testing is performed after modification of the program or software; it classified the existing test cases into re-used test cases and affected test cases after modification of the code. Test case prioritization is an approach of arranging the existing test cases in manner that most affected test cases (that generated maximum number of faults) test first after the other one. There are many techniques used for prioritization test cases at the time of regression testing. This paper present a new approach for prioritization of test cases using static executable program slices for regression testing. Program slicing is a process to classify the program into number of parts based on various types of dependencies between program statements. This paper presents an overview of basic concept of generating static program slices and on the basis of these programs slices prioritization of test cases at the time of regression testing Keywordstesting, Program dependency, static program slices, test cases prioritization and execution history

Optimized Regression Test Using Test Case Prioritization

Regression testing ensures that changes made in the fixes or any enhancement changes do not impact the previously working functionality. Whenever software is modified, a set of test cases are run to assure that changes don’t affect the other parts of the software. Hence all existing test cases need to be tested as well as new test cases need to be created. It is nonviable to re-execute every test case for a given software, because if there are more number of test cases to be tested, the more effort and time is required. This problem can be solved by prioritizing test cases. Test case prioritization techniques reorder the priority of a test case in an attempt to ensure that maximum faults are uncovered by the high prioritized test cases. In this paper we propose an optimized test case prioritization technique using Ant Colony Optimization (ACO) to reduce the cost, effort and time taken to perform regression testing and also uncover maximum faults.

A generalized global optimization formulation of the pooling problem with processing facilities and composite quality constraints

We present a generalized formulation of the pooling problem. Our formulation is different from the standard formulations in explicitly modeling component flows. Modeling the physical components directly, allows easy inclusion of processing facilities that may alter the flow composition. It also allows adding composite quality constraints that cannot be added directly as quality parameters as they do not blend linearly. We provide new test instances motivated by natural gas transport problems at the Norwegian Continental Shelf and give computational results. We show examples of nonlinear composite constraints on quality attributes and give computational results on the effect of adding such constraints to the new set of test instances. The increased flexibility of our formulation comes at the cost of less tight constraints and a performance penalty on existing test cases in literature. The advantage is that it can solve the more general test cases described above. We use GAMS implementations of the various formulations and solve the problems with BARON. We compare the performance of our bilinear formulation in BARON with discretizations solved as a Mixed Integer Linear Program using CPLEX. The discretized versions do not generally perform as well as the continuous model; however, they have better worst-case behavior on the new test cases.

Preemptive Regression Testingof Workflow-Based Web Services

An external web service may evolve without prior notification. In the course of the regression testing of a workflow-based web service, existing test case prioritization techniques may only verify the latest service composition using the not-yet-executed test cases, overlooking high-priority test cases that have already been applied to the service composition before the evolution. In this paper, we propose Preemptive Regression Testing ( PRT ), an adaptive testing approach to addressing this challenge. Whenever a change in the coverage of any service artifact is detected, PRT recursively preempts the current session of regression test and creates a sub-session of the current test session to assure such lately identified changes in coverage by adjusting the execution priority of the test cases in the test suite. Then, the sub-session will resume the execution from the suspended position. PRT terminates only when each test case in the test suite has been executed at least once without any preemption activated in between any test case executions. The experimental result confirms that testing workflow-based web service in the face of such changes is very challenging; and one of the PRT-enriched techniques shows its potential to overcome the challenge.

The network of faults: a complex network approach to prioritize test cases for regression testing

Regression testing is performed to provide confidence that changes in a part of software do not affect other parts of the software. An execution of all existing test cases is the best way to re-establish this confidence. However, regression testing is an expensive process--there might be insufficient resources (e.g., time, workforce) to allow for the re-execution of all test cases. Regression test prioritization techniques attempt to re-order a regression test suite based on some criteria so that highest priority test cases are executed earlier. In this study, we prioritize test cases for regression testing based on the dependency network of faults. In software testing, it is common that some faults are the consequences of other faults (leading faults). Dependent faults can be removed if and only if the leading faults have been removed. Our goal is to prioritize test cases so that test cases that have exposed the leading faults in the system testing phase, are executed first in regression testing. The leading faults are modeled as the most central faults in the fault dependency network. We present ComReg, a test-case prioritization technique based on the dependency network of faults. We model a fault dependency network as a directed graph and identify leading faults to prioritize test cases for regression testing. We use a centrality aggregation technique which considers six network representative centrality metrics to identify leading faults in the fault dependency network. We also discuss the use of fault communities to select an arbitrary percentage of the test cases from a prioritized regression test suite. We conduct a case study that evaluates the effectiveness and applicability of the proposed method. We obtain a fault dependency network from the development of a vocabulary learning software. We found that the fault network is a small-world graph with distinguishable community structure. The leading faults are common in all centralities and a re-ordering of test cases is feasible for regression testing based on those leading faults. Our method outperforms traditional regression testing prioritization techniques in detecting fault dependencies. Our modeling of the network of faults provides insights into the requirement of recognizing fault dependencies while re-ordering regression test suites for both research and practice. The dependency model needs further evaluation and improvement considering relevant resources (e.g., man-hours).

Input-based adaptive randomized test case prioritization: A local beam search approach

Test case prioritization assigns the execution priorities of the test cases in a given test suite. Many existing test case prioritization techniques assume the full-fledged availability of code coverage data, fault history, or test specification, which are seldom well-maintained in real-world software development projects. This paper proposes a novel family of input-based local-beam-search adaptive-randomized techniques. They make adaptive tree-based randomized explorations with a randomized candidate test set strategy to even out the search space explorations among the branches of the exploration trees constructed by the test inputs in the test suite. We report a validation experiment on a suite of four medium-size benchmarks. The results show that our techniques achieve either higher APFD values than or the same mean APFD values as the existing code-coverage-based greedy or search-based prioritization techniques, including Genetic, Greedy and ART, in both our controlled experiment and case study. Our techniques are also significantly more efficient than the Genetic and Greedy, but are less efficient than ART.

Energy and wear optimisation of train longitudinal dynamics and of traction and braking systems

Traction and braking systems deeply affect longitudinal train dynamics, especially when an extensive blending phase among different pneumatic, electric and magnetic devices is required. The energy and wear optimisation of longitudinal vehicle dynamics has a crucial economic impact and involves several engineering problems such as wear of braking friction components, energy efficiency, thermal load on components, level of safety under degraded or adhesion conditions (often constrained by the current regulation in force on signalling or other safety-related subsystem). In fact, the application of energy storage systems can lead to an efficiency improvement of at least 10% while, as regards the wear reduction, the improvement due to distributed traction systems and to optimised traction devices can be quantified in about 50%. In this work, an innovative integrated procedure is proposed by the authors to optimise longitudinal train dynamics and traction and braking manoeuvres in terms of both energy and wear. The new approach has been applied to existing test cases and validated with experimental data provided by Breda and, for some components and their homologation process, the results of experimental activities derive from cooperation performed with relevant industrial partners such as Trenitalia and Italcertifer. In particular, simulation results are referred to the simulation tests performed on a high-speed train (Ansaldo Breda Emu V250) and on a tram (Ansaldo Breda Sirio Tram). The proposed approach is based on a modular simulation platform in which the sub-models corresponding to different subsystems can be easily customised, depending on the considered application, on the availability of technical data and on the homologation process of different components.

Efficient testing of software product lines via centralization (short paper)

Software product line~(SPL) engineering manages families of software products that share common features. However, cost-effective test case generation for an SPL is challenging. Applying existing test case generation techniques to each product variant separately may test common code in a redundant way. Moreover, it is difficult to share the test results among multiple product variants. In this paper, we propose the use of centralization, which combines multiple product variants from the same SPL and generates test cases for the entire system. By taking into account all variants, our technique generally avoids generating redundant test cases for common software components. Our case study on three SPLs shows that compared with testing each variant independently, our technique is more efficient and achieves higher test coverage.

Improvement of Component Integration Testing Technique

Component-based technology can increase reuse and productivity, but high-quality component- based systems are often difficult to implement. Component developers do not know the systems where the components will be used, while software engineers must develop new systems with limited knowledge on available components. We propose a new testing technique that generates, at the time of component development, integration test cases from the specification of the behavior expected from other components of the system. The technique presented in this paper supports both the component developer, who can early test the integration of the components with the system, and the software engineers, who can test concrete components at deployment time, simply re- using existing test cases. The technique presented in this paper supports both the component developer, who can early test the integration of the components with the system, and the software engineers, who can test components at deployment time. We used questionnaires to validate the proposed solution.

Structurized grammar‐based fuzz testing for programs with highly structured inputs

ABSTRACTFuzz testing, also known as fuzzing, has long been recognized as an effective technique to detect software vulnerabilities. Unfortunately, this approach is demonstrated noneffective when applied to test programs with highly structured inputs, such as interpreters and compilers. These programs usually process inputs in several stages as lexing and parsing, where the test input will be rejected if its structure does not obey the grammar. In this paper, we present a novel approach for fuzzing highly structured input programs. By disassembling the existing test cases into multiple grammatical fragments and inferring their grammar structures, we build a new series of test cases that can pass the validation and reach the previously unexplored places in the target program. We have implemented this approach in our general fuzzing framework BlendFuzz. Experiments have shown that BlendFuzz achieves higher code coverage compared with other blackbox fuzzing tools. BlendFuzz has also detected over two dozens of previously unreported vulnerabilities in real‐world applications, with seven of them being considered high risky. Copyright © 2013 John Wiley & Sons, Ltd.

Using Dependency Structures for Prioritization of Functional Test Suites

Test case prioritization is the process of ordering the execution of test cases to achieve a certain goal, such as increasing the rate of fault detection. Increasing the rate of fault detection can provide earlier feedback to system developers, improving fault fixing activity and, ultimately, software delivery. Many existing test case prioritization techniques consider that tests can be run in any order. However, due to functional dependencies that may exist between some test cases-that is, one test case must be executed before another-this is often not the case. In this paper, we present a family of test case prioritization techniques that use the dependency information from a test suite to prioritize that test suite. The nature of the techniques preserves the dependencies in the test ordering. The hypothesis of this work is that dependencies between tests are representative of interactions in the system under test, and executing complex interactions earlier is likely to increase the fault detection rate, compared to arbitrary test orderings. Empirical evaluations on six systems built toward industry use demonstrate that these techniques increase the rate of fault detection compared to the rates achieved by the untreated order, random orders, and test suites ordered using existing coarse-grained” techniques based on function coverage.

Efficient Path Selection Strategy Based on Static Analysis for Regression Testing

the changes made to the software system for enhancement. The most common approach to this problem is simply execute the existing test cases in the test suite; i.e. retest all approach, a systematically selected subset is chosen to be reuse, then substantial resources will be saved, due to the limited size of the selected test data (5), (6). There are distinctions between classes of techniques of regression testing: Test suites minimization techniques can be used to reduce test suite by eliminating redundant test suites. In safe approaches, every test case is selected that exercised any program element which could be affected by a program modification. In coverage approaches, test cases are selected which met some structural criteria. In this approach, select a single test case satisfying each coverage requirement introduced by the criterion, minimization techniques work like coverage approaches (7).

Test Pair Selection for Test Case Prioritization in Regression Testing for WS-BPEL Programs

Many web services not only communicate through XML-based messages, but also may dynamically modify their behaviors by applying different interpretations on XML messages through updating the associated XML Schemas or XML-based interface specifications. Such artifacts are usually complex, allowing XML-based messages conforming to these specifications structurally complex. Testing should cost-effectively cover all scenarios. Test case prioritization is a dimension of regression testing that assures a program from unintended modifications by reordering the test cases within a test suite. However, many existing test case prioritization techniques for regression testing treat test cases of different complexity generically. In this paper, the authors exploit the insights on the structural similarity of XML-based artifacts between test cases in both static and dynamic dimensions, and propose a family of test case prioritization techniques that selects pairs of test case without replacement in turn. To the best of their knowledge, it is the first test case prioritization proposal that selects test case pairs for prioritization. The authors validate their techniques by a suite of benchmarks. The empirical results show that when incorporating all dimensions, some members of our technique family can be more effective than conventional coverage-based techniques.

Size-Constrained Regression Test Case Selection Using Multicriteria Optimization

To ensure that a modified software system has not regressed, one approach is to rerun existing test cases. However, this is a potentially costly task. To mitigate the costs, the testing effort can be optimized by executing only a selected subset of the test cases that are believed to have a better chance of revealing faults. This paper proposes a novel approach for selecting and ordering a predetermined number of test cases from an existing test suite. Our approach forms an Integer Linear Programming problem using two different coverage-based criteria, and uses constraint relaxation to find many close-to-optimal solution points. These points are then combined to obtain a final solution using a voting mechanism. The selected subset of test cases is then prioritized using a greedy algorithm that maximizes minimum coverage in an iterative manner. The proposed approach has been empirically evaluated and the results show significant improvements over existing approaches for some cases and comparable results for the rest. Moreover, our approach provides more consistency compared to existing approaches.

Test Case Generation and Reusing Test Cases for GUI Designed with HTML

Graphical User Interface (GUI) is pervasive to the extent that half of the code of the software systems written today is to produce the required GUIs. Test case generation for GUI based software systems is complex as it is necessary to include all possible sequences of events that may be exercised by the widget or end-user. The major issue with GUI based systems is that even a single change in the GUI may make the entire suite of existing test cases unusable. Hence a solution to analyze the existing test cases (i.e. the test cases that were already generated before modification of GUI) and identify the test cases that have become unusable and that are reusable in the context of the modified GUI is provided here. Test cases that are reusable are retained and the test cases that have become unusable are subjected to transformations. These transformations make the unusable test cases reusable through the construction of Event Flow Graph (EFG) generated and parsing EFG using Depth First Search (DFS) to identify reusable and unusable test cases.