Test Case Prioritization Algorithm Research Articles

Test case prioritization based on fault sensitivity analysis using ranked NSGA-2

This paper discusses regression testing in software maintenance, focusing on test case prioritization to verify modifications to software functionality efficiently. The primary goal is to rank test cases, prioritizing those covering more code or faults with minimal execution time. The challenge lies in prioritizing numerous test cases generated during development and maintenance. Various algorithms, including greedy approaches and meta-heuristic techniques, address this challenge. The paper introduces a ranking-based non-dominated sorting genetic algorithm (NSGA-2) for test case prioritization, emphasizing cases sensitive to faults caused by modifications or new functionality. Historical data is prioritized, with key objectives including the sensitive index, execution cost, and average percentage of fault detection (APFD). The proposed model is tested on handcrafted and benchmark Java-based applications, comparing its performance to state-of-the-art algorithms in test case prioritization.

Directed Search Based on Improved Whale Optimization Algorithm for Test Case Prioritization

With the advent of the information age, the iterative speed of software update is gradually accelerating which makes software development severely limited by software testing. Test case prioritization is an effective way to accelerate software testing progress. With the introduction of heuristic algorithm to this task, the processing efficiency of test cases has been greatly improved. However, to overcome the shortcomings of slow convergence speed and easy fall into local optimum, the improved whale optimization algorithm is proposed for test case prioritization. Firstly, a model called n-dimensional directed search space is established for the swarm intelligence algorithm. Secondly, the enhanced whale optimization algorithm is applied to test case prioritization while the backtracking behavior is conducted for individuals when hitting the wall. In addition, a separate storage space for Pareto second optimization is also designed to filter the optimal solutions of the multi-objective tasks. Finally, both single-objective and multi-objective optimization experiments are carried out for open source projects and real-world projects, respectively. The results show that the improved whale optimization algorithm using n-dimensional directed search space is more conducive to the decisions of test case prioritization with fast convergence speed.

A Comparative Analysis of Bat and Genetic Algorithms for Test Case Prioritization in Regression Testing

Regression testing is carried out to ensure that software modifications do not introduce new potential bugs to the existing software. Existing test cases are applied in the testing, such test cases can run into thousands, and there is not much time to execute all of them. Test Case Prioritization (TCP) is a technique to order test cases so that the test cases potentially revealing more faults are performed first. With TCP being deemed an optimization problem, several metaheuristic nature-inspired algorithms such as Bat, Genetic, Ant colony, and Firefly algorithms have been proposed for TCP. These algorithms have been compared theoretically or based on a single metric. This study employed an experimental design to offer an in-depth comparison of bat and genetic algorithms for TCP. Unprioritized test cases and a brute-force approach were used for comparison. Average Percentage Fault Detection (APFD)- a popular metric, execution time and memory usage were used to evaluate the algorithms’ performance. The study underscored the importance of test case prioritization and established the superiority of the Genetic algorithm over the bat algorithm for TCP in APFD. No stark differences were recorded regarding memory usage and execution time for the two algorithms. Both algorithms seemed to scale well with the growth of test cases.

Comparative Analysis of Efficacious Metaheuristic Technique with Genetically Modified- Flower Pollination Algorithm (GM-FPA) for Test Case Prioritization in Regression Testing

Abstract: Regression Testing is most imperative activity of software development life cycle. Test case prioritization being one of the most adopted branch for regression testing and with the invent of nature inspired metaheuristic techniques in optimization, this study makes an attempt to augments the features of test case prioritization with nature inspired metaheuristic techniques to determine the most efficacious metaheuristic techniques from Cuckoo Searh (CS) algorithm, Genetic Algorithm (GA) and Flower Pollination Algorithm (FPA) for three different case studies. APFD metrics is used to compare the algorithms. Further the study compares the most efficacious technique with Genetically Modified- Flower Pollination Algorithm (GM-FPA) to identify the most efficient technique for regression test case prioritization.

Improved novel bat algorithm for test case prioritization and minimization

Improved novel bat algorithm for test case prioritization and minimization

Discrete cuckoo search algorithms for test case prioritization

Regression testing is an essential aspect of the software development lifecycle. As the software evolves, the test suite grows, hence the cost and effort to retest the software. Test case prioritization is one of the mitigation techniques for regression testing. It ranks the test cases to maximize the desired properties, e.g., detecting faults early. The efficiency and effectiveness of test case prioritization techniques can be enhanced using optimization algorithms. Nature-inspired algorithms are gaining more attention due to their easy implementation and quality of the solutions. This paper proposes the discrete cuckoo search algorithm for test case prioritization. The prioritization problem deals with ordering the test cases. Therefore, a new adaptation strategy using asexual genetic reproduction is introduced to convert real numbers into permutation sequences. Furthermore, the cuckoo search algorithm’s effectiveness is extended with the genetic algorithm’s mutation operator to balance the trade-off between exploration and exploitation. An in-depth comparative study on four case studies is conducted between the proposed algorithms, existing state-of-the-art algorithms and baseline approach. Statistical investigation confirms that the proposed hybrid cuckoo search algorithm outperforms the genetic algorithm, particle swarm optimization, ant colony optimization, tree seed algorithm and random search by 4.29%, 5.52%, 8.38%, 2.74% and 10.80%, respectively.

Discrete and combinatorial gravitational search algorithms for test case prioritization and minimization

Regression testing is an essential but expensive activity to re-execute all the test cases every time the software updates. Test case prioritization and minimization reduces the cost and efforts required for retesting by prioritizing the test cases based on their importance and minimizing the redundancy. Optimization approaches further enhance the effectiveness of these techniques. In this paper, a discrete and combinatorial gravitational search algorithm is proposed to solve the test case prioritization and minimization problems. Furthermore, an improved version is developed using the chaotic map to update the gravitational constant. The proposed algorithms are compared with the most commonly used algorithm, i.e., genetic algorithm. Three subject programs of varying sizes are used for evaluation. Simulation results prove that the proposed algorithms are more efficient and effective than the genetic algorithm for test case prioritization and minimization. Statistical representation via boxplots of APFD and interval plots of minimized suite size performance metrics, confirms that the improved gravitational search algorithm with chaotic gravitational constant has a more squeezed distribution than the standard gravitational search algorithm.

AGA: An Accelerated Greedy Additional Algorithm for Test Case Prioritization

In recent years, many test case prioritization (TCP) techniques have been proposed to speed up the process of fault detection. However, little work has taken the efficiency problem of these techniques into account. In this paper, we target the Greedy Additional (GA) algorithm, which has been widely recognized to be effective but less efficient, and try to improve its efficiency while preserving effectiveness. In our Accelerated GA (AGA) algorithm, we use some extra data structures to reduce redundant data accesses in the GA algorithm and thus the time complexity is reduced from $\mathcal{O}(m^2n)$ to $\mathcal{O}(kmn)$ when $n > m$, where $m$ is the number of test cases, $n$ is the number of program elements, and $k$ is the iteration number. Moreover, we observe the impact of iteration numbers on prioritization efficiency on our dataset and propose to use a specific iteration number in the AGA algorithm to further improve the efficiency. We conducted experiments on 55 open-source subjects. In particular, we implemented each TCP algorithm with two kinds of widely-used input formats, adjacency matrix and adjacency list. Since a TCP algorithm with adjacency matrix is less efficient than the algorithm with adjacency list, the result analysis is mainly conducted based on TCP algorithms with adjacency list. The results show that AGA achieves 5.95X speedup ratio over GA on average, while it achieves the same average effectiveness as GA in terms of Average Percentage of Fault Detected (APFD). Moreover, we conducted an industrial case study on 22 subjects, collected from Baidu, and find that the average speedup ratio of AGA over GA is 44.27X, which indicates the practical usage of AGA in real-world scenarios.

Cuckoo Search Algorithm for Test Case Prioritization in Regression Testing

There are countless optimization problems that have been accelerated by Nature Inspired Metaheuristic Optimization Algorithms (NIMOA) in the earlier decades. NIMOA have gained huge popularity owing to their effective results. In this study NIMOA namely, Cuckoo Search Algorithm (CSA) is used to prioritize (order) the test cases for Regression Testing (RT). Prioritizations aids in the execution of higher priority test cases to give early fault detection. This research adopts the aggressive approach of reproduction made by Cuckoos to prioritize the test cases for RT. Average Percentage of Fault Detected (APFD) metrics is used in this paper for validations of results. APFD metrics is used to compare the performance of CSA with Flower Pollination Algorithm (FPA) and traditional approaches for Test Case Prioritization (TCP). Two java applications are used for the study. CSA is implemented in Java on eclipse platform. It is learnt from the study that APFD results of CSA outperformed the FPA for both the applications namely Puzzle Game and AreaandPerimeter. It is inferred from the results that prioritized set of test cases given by NIMOA outperformed the APFD results of traditional approaches and also CSA performed better than the FPA for TCP.

Hybridization of Cuckoo-ACO algorithm for test case prioritization

Regression testing is one of the types of maintenance testing and it is performed in case of bug-fixing or whenever there is any new functionality incorporate in the dynamic environment of software development process. Due to which the cost of development process increases because it directly affects the validation process. The addition of the new requirements with limited resources of development process in a time constrained environment definitely increase the cost. Efficient and effective test case selection from the available test suite becomes very critical problem in this scenario and this situation makes it important to applying of some techniques which are prioritization of test cases and selection of test cases for re-arrangement of test cases in a particular schedule and these test cases are selected in a particular sequence order to fulfillment of some selected criteria. This paper suggests Cuckoo Search (CS) algorithm followed by Modified Ant Colony Optimization (M-ACO) algorithm to conclude the test cases in an optimized order in time constrained environment. CS algorithm is inspired by some species of cuckoos having constrained brood lethargy and laying of their eggs in other host bird’s nest. Due to dependency of this algorithm on one single parameter, cuckoo search unlike other optimization algorithms, is more efficient and very easy to implement. But the hybrid Cuckoo-ACO optimization technique is more appropriate for the test case selection and prioritization according to the proposed empirical study.

Using reliability risk analysis to prioritize test cases

In this paper, we present a risk-based test case prioritization (Ri-TCP) algorithm based on the transmission of information flows among software components. Most of the existing approaches rely on the historical code changes or test case execution data, few of them effectively use the system topology information covered by test cases when scheduling the execution of test cases. From the perspective of code structure, the proposed algorithm firstly maps software into an information flow-based directed network model. Then, functional paths covered by each test case are represented by a set of barbell motifs. Finally, combining with probabilistic risk analysis (PRA) and fault tree model, we assign a priority to each test case by calculating the sum of risk indexes of all the barbells covered by it. Experimental results demonstrate that Ri-TCP technique has a higher detection rate of faults with serious risk indicators and performs stably in different systems, compared with the other state-of-the-art algorithms.

Value based PSO Test Case Prioritization Algorithm

Regression testing is performed to see if any changes introduced in software will not affect the rest of functional software parts. It is inefficient to re-execute all test cases every time the changes are made. In this regard test cases are prioritized by following some criteria to perform efficient testing while meeting limited testing resources. In our research we have proposed value based particle swarm intelligence algorithm for test case prioritization. The aim of our research is to detect maximum faults earlier in testing life cycle. We have introduced the combination of six prioritization factors for prioritization. These factors are customer priority, Requirement volatility, implementation complexity, requirement traceability, execution time and fault impact of requirement. This combination of factors has not been used before for prioritization. A controlled experiment has been performed on three medium size projects and compared results with random prioritization technique. Results are analyzed with the help of average percentage of fault detection (APFD) metric. The obtained results showed our proposed algorithm as more efficient and robust for earlier rate of fault detection. Results are also revalidated by proposing our new validation equation and showed consistent improvement in our proposed algorithm.

Improved Annealing-Genetic Algorithm for Test Case Prioritization

Regression testing, which can improve the quality of software systems, is a useful but time consuming method. Many techniques have been introduced to reduce the time cost of regression testing. Among these techniques, test case prioritization is an effective technique which can reduce the time cost by processing relatively more important test cases at an earlier stage. Previous works have demonstrated that some greedy algorithms are effective for regression test case prioritization. Those algorithms, however, have lower stability and scalability. For this reason, this paper proposes a new regression test case prioritization approach based on the improved Annealing-Genetic algorithm which incorporates Simulated Annealing algorithm and Genetic algorithm to explore a bigger potential solution space for the global optimum. Three Java programs and five C programs were employed to evaluate the performance of the new approach with five former approaches such as Greedy, Additional Greedy, GA, etc. The experimental results showed that the proposed approach has relatively better performance as well as higher stability and scalability than those former approaches.

Empirical Study of the Effects of Different Similarity Measures on Test Case Prioritization

Similarity-based test case prioritization algorithms have been applied to regression testing. The common characteristic of these algorithms is to reschedule the execution order of test cases according to the distances between pair-wise test cases. The distance information can be calculated by different similarity measures. Since the topologies vary with similarity measures, the distances between pair-wise test cases calculated by different similarity measures are different. Similarity measures could significantly influence the effectiveness of test case prioritization. Therefore, we empirically evaluate the effects of six similarity measures on two similarity-based test case prioritization algorithms. The obtained results are statistically analyzed to recommend the best combination of similarity-based prioritization algorithms and similarity measures. The experimental results, confirmed by a statistical analysis, indicate that Euclidean distance is more efficient in finding defects than other similarity measures. The combination of the global similarity-based prioritization algorithm and Euclidean distance could be a better choice. It generates not only higher fault detection effectiveness but also smaller standard deviation. The goal of this study is to provide practical guides for picking the appropriate combination of similarity-based test case prioritization techniques and similarity measures.

Combined Genetic and Simulated Annealing Approach for Test Case Prioritization

Background: The test case prioritization of regression Testing is described. Methods: A new test case prioritization algorithm is proposed to get better the rate of fault detection and cost reduction. Heuristic Techniques like Genetic Algorithm (GA) and Simulated Annealing (SA) are employed. It prioritizes the test cases depends on fault detection ability and execution time taken. Findings: The implementation of proposed algorithm in JAVA is found to produce optimal or near optimal results. The efficiency of proposed regression testing technique is proved by comparing it with GA and SA methods individually. Applications: A complete automation tool for the complete usage of the algorithm is being developed. It will also be analyzed on larger projects with large number of test cases and faults.

Implementation of Regression Testing of Test Case Prioritization

Background: In this paper, we described the regression testing of test case prioritization. Statistical analysis: To categorize the cruel mistakes and get better the rate of fault detection, test case prioritization algorithm is proposed. Result: It prioritizes the test cases depends on few sets of realistic load aspect such as: changes in requirements, traceability, completeness, developer observed code execution complexity, fault impact and customer allotted priority. Conclusion: The implementation of proposed regression testing method is attained by comparing it with other methods.

A Multi-Factor Coverage based Test Case Prioritization Technique for Object Oriented Software

Test case prioritization technique creates the sequence of test cases for execution in such a way that the test cases with higher rate of fault detection are executed earlier than those test cases which have lower rate of fault detection. In this paper a new algorithm is proposed to prioritize the test cases based on coverage of object oriented programming factors. The factors are considered on the basis of complexity and probability of errors introduced by them. For the experimental validation and analysis the proposed test case prioritization algorithm is applied on two case studies. The analyzed case studies are implemented in C++ language. By using the presented algorithm it helps to reduce the cost and time for testing the software.

A New Effective Test Case Prioritization for Regression Testing based on Prioritization Algorithm

Regression Testing is the process of executing the set of test cases which have passed on the previous build or release of the application under test in order to validate that the original features and functions are still working as they were previously. It is impracticable and in-sufficient resources to re-execute every test case for a program if changes occur. This problem of regression testing can be solved by prioritizing test cases. A regression test case prioritization technique involves re-ordering the execution of test suite to increase the rate of fault detection in earlier stages of testing process. In this paper, test case prioritization algorithm is proposed to identify the severe faults and improve the rate of fault detection. This proposed test case prioritization algorithm prioritizes the test cases based on four groups of practical weight factor such as: customer allotted priority, developer observed code execution complexity, changes in requirements, fault impact, completeness and traceability. The proposed prioritization technique is validated with three different validation metrics and is experimented using two projects. The effectiveness of proposed technique is achieved by comparing it with unprioritized ones and by validation metrics.

A New Code Based Test Case Prioritization Technique

Test Cases are treated as one of the most important part of software testing activity. They are responsible for the validation of the software under inspection. Test suites are also used to test changes in the source code during regression testing. In number of cases, the test suites are so big that executing all tests for every source code change is absurd. Developers/testers need to prioritize the test suite so that most beneficial test cases are executed first. This can result in increasing the effectiveness of testing and saving a lot of time and cost. In this paper, we introduce a new algorithm for test case prioritization that is based on the code coverage of the test cases. The test cases that cover maximum part of the code are considered more valuable. We applied this new algorithm to a case study. The results are very effective and promising.

Prioritizing Test Case Generation for Software Testing in High Level Programming Development Environment

Abstract—Software testing is very tedious process and during the software development, testing needs time, efforts and money. Testing and retesting is part of development process and lots of efforts are needed for doing this. Detecting faults and errors in the early stages of development is the main task of any testing team. The entire test suits are written for the same target and the test suits grows as the software evolves over the period of time. Resources are very limited and due to resource constraints like cost, time and money, it is advised to prioritize the execution of test cases so that it can increase chances of early detection of faults in the software development process [1]. In this paper, high level language programming paradigm is considered for the development environment and algorithmic approach of design is considered. In this paper we present a new approach to prioritize test cases of particular software based on the requirements given by the client using high level functional programming language. Running all test cases in a normal Test suite, however, can consume an inordinate amount of time so, its main purpose is to improve rate of fault detection by prioritizing the test cases in a very short span of time and release the updated software to the customer [2]. In this paper a new test case prioritization algorithm, which calculates using data mining technique K-Nearest neighbor, which in turn uses Euclidean distance method approach to prioritize the test cases is proposed. General Terms Software Systems, Software Engineering and Software Testing