Regression Test Case Prioritization Research Articles

Multi-factor based Regression Test Case Prioritization using Fuzzy Logic

The maintenance level activity generally done after the modification in the software to check whether it is functioning right or not is termed as regression testing. Test case prioritization, a key practice, involves strategically ordering test cases based on specific criteria to enhance the efficiency of fault detection within a condensed time frame. The fuzzy rule base serves as an alternative to the conventional crisp value set, offering a nuanced approach beyond binary outcomes (Yes or No). The primary objective of this research is to address critical factors often overlooked in existing literature on prioritization. Notably, prevalent approaches focus on singular factors during test case prioritization, highlighting the need for a comprehensive technique. To enhance the prioritization of test cases, there is a demand for a method that considers multi-factors or combinations thereof, ultimately increasing effectiveness. This paper introduces an innovative approach a multi-factors regression test-case prioritization technique utilizing fuzzy rules. The methodology aims to optimize the prioritization of test cases, striking a balance between effectiveness and time efficiency. Fuzzy rules are formulated to assess the effectiveness of a prioritized set of test cases in developing the proposed approach. A user-friendly tool has been developed to facilitate the application of this technique, allowing users to input relevant factors and subsequently prioritize test cases accordingly. Through extensive experiments using the developed tool, the effectiveness of the proposed approach has been validated. The results demonstrate that the priority lists of test cases generated for different projects, considering multi-factors, show greater promise compared to techniques relying solely on a single factor for prioritization.

Fault-Based Regression Test Case Prioritization

We propose a set of four novel fault-based regression test case prioritization (TCP) techniques for object-oriented programs. We seed bugs into a program to create large number of mutants. We execute each mutant with the originally designed test suite. From this, we record the number of mutants for which a test case fails. Based on this, we prioritize the test cases using four base fault-based prioritization techniques that we have proposed. Finally, we combine the results of our four base prioritizers using three ensemble methods. We have conducted experimental studies to determine the effectiveness of our proposed approaches. Our experimental results show that our proposed TCP techniques exhibit superior performance over related techniques.

A review paper: optimal test cases for regression testing using artificial intelligent techniques

<span lang="EN-US">The goal of the testing process is to find errors and defects in the software being developed so that they can be fixed and corrected before they are delivered to the customer. Regression testing is an essential quality testing technique during the maintenance phase of the program as it is performed to ensure the integrity of the program after modifications have been made. With the development of the software, the test suite becomes too large to be fully implemented within the given test cost in terms of budget and time. Therefore, the cost of regression testing using different techniques should be reduced, here we dealt many methods such as retest all technique, regression test selection technique (RTS) and test case prioritization technique (TCP). The efficiency of these techniques is evaluated through the use of many metrics such as average percentage of fault detected (APFD), average percentage block coverage (APBC) and average percentage decision coverage (APDC). In this paper we dealt with these different techniques used in test case selection and test case prioritization and the metrics used to evaluate their efficiency by using different techniques of artificial intelligent and describe the best of all.</span>

Application of the Law of Minimum and Dissimilarity Analysis to Regression Test Case Prioritization

Regression testing is one of the most expensive processes in testing. Prioritizing test cases in regression testing is critical for the goal of detecting the faults sooner within a large set of test cases. We propose a test case prioritization (TCP) technique for regression testing called <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">LoM-Score</i> inspired by the Law of Minimum (LoM) from biology. This technique calculates the impact probabilities of methods calculated by change impact analysis with forward slicing and orders test cases according to LoM. However, this ordering doesn’t consider the possibility that consecutive test cases may be covering the same methods repeatedly. Thereby, such ordering can delay the time of revealing faults that exist in other methods. To solve this problem, we enhance the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">LoM-Score</i> TCP technique with an adaptive approach, namely with a dissimilarity-based coordinate analysis approach. The dissimilarity-based coordinate analysis uses Jaccard Similarity for calculating the similarity coefficients between test cases in terms of covered methods and the enhanced technique called Dissimilarity-LoM-Score ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Dis-LoM-Score</i> ) applies a penalty with respective on the ordered test cases. We performed our case study on 10 open-source Java projects from Defects4J, which is a dataset of real bugs and an infrastructure for controlled experiments provided for software engineering researchers. Then, we hand-seeded multiple mutants generated by Major, which is a mutation testing tool. Then we compared our TCP techniques <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">LoM-Score</i> and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Dis-LoM-Score</i> with the four traditional TCP techniques based on their Average Percentage of Faults Detected (APFD) results.

Comparative Analysis of Efficacious Metaheuristic Technique with Genetically Modified- Flower Pollination Algorithm (GM-FPA) for Test Case Prioritization in Regression Testing

Abstract: Regression Testing is most imperative activity of software development life cycle. Test case prioritization being one of the most adopted branch for regression testing and with the invent of nature inspired metaheuristic techniques in optimization, this study makes an attempt to augments the features of test case prioritization with nature inspired metaheuristic techniques to determine the most efficacious metaheuristic techniques from Cuckoo Searh (CS) algorithm, Genetic Algorithm (GA) and Flower Pollination Algorithm (FPA) for three different case studies. APFD metrics is used to compare the algorithms. Further the study compares the most efficacious technique with Genetically Modified- Flower Pollination Algorithm (GM-FPA) to identify the most efficient technique for regression test case prioritization.

Analytic hierarchy process-based regression test case prioritization technique enhancing the fault detection rate

Analytic hierarchy process-based regression test case prioritization technique enhancing the fault detection rate

Validating Switch Variable of Flower Pollination Algorithm in Regression Test Case Prioritization

This study aims at analysing and validating the impact of switch variable of Flower Pollination Algorithm (FPA) in regression test case prioritization. The value of switch variable decides whether local or global optimization will take place. The present study makes an attempt to analyse the impact of switch variable with values ranging from 0.8 to 0.1 on test case prioritization in regression testing. The present study is an extension of the prior work done to implement FPA for test case prioritization in regression testing. To validate the study two case studies written in Java programming were used. The results were validated using the APFD metrics. From the empirical results it was deduced that values ranging between 0.8 and 0.5 of switch variable gave promising results for test case prioritization.

A regression test case prioritization technique targeting ‘hard to detect’ faults

A regression test case prioritization technique targeting ‘hard to detect’ faults

Target-based test path prioritization for UML activity diagram using weight assignment methods

The benefit of exploratory testing and ad hoc testing by tester’s experience is that crucial bugs are found quickly. Regression testing and test case prioritization are important processes of software testing when software functions have been changed. We propose a test path prioritization method to generate a sequence of test paths that would match the testers’ interests and focuses on the target area of interest or on the changed area. We generate test paths form the activity diagrams and survey the test path prioritization from testers. We define node and edge weight to the symbols of activity diagrams by applying Time management, Pareto, Buffett, Binary, and Bipolar method. Then we propose a test path score equation to prioritize test paths. We also propose evaluation methods i.e., the difference and the similarity of test path prioritization to testers’ interests. Our proposed method had the least average of the difference and the most average of the similarity compare with the tester’s prioritization of test paths. The Bipolar method was the most suitable for assigning weights to match test path rank by the tester. Our proposed method also has given the affected path by changing area higher priority than the other test path.

A Learn-to-Rank Method for Model-Based Regression Test Case Prioritization

Regression testing plays an indispensable role in software maintenance, which refers to retest the software following modifications to determine whether the changes have introduced new faults. However, regression testing requires massive amounts of effort to achieve a high fault detection rate. To address this issue, the test case prioritization technique is used to improve the fault detection rate by adjusting the execution order of test cases. For model-based regression test case prioritization, existing approaches have been developed using the single aspect of model-related information extracted from the previous executed test cases. In this paper, a novel learn-to-rank technique is proposed to prioritize test cases by combining the multidimensional features of Extended Finite State Machine (EFSM) under test to improve fault detection rate. Specifically, our method utilizes the random forest algorithm to combine multiple existing heuristic prioritization methods. Detailed experiments are conducted to evaluate the proposed method’s performance in terms of Average Percentage Fault Detected (APFD). The experimental results show that the mean APFD value of our method reaches 0.884 for five subject EFSMs, which is 33.9% higher than the compared methods.

Regression test case prioritization by code combinations coverage

Regression test case prioritization (RTCP) aims to improve the rate of fault detection by executing more important test cases as early as possible. Various RTCP techniques have been proposed based on different coverage criteria. Among them, a majority of techniques leverage code coverage information to guide the prioritization process, with code units being considered individually, and in isolation. In this paper, we propose a new coverage criterion, code combinations coverage, that combines the concepts of code coverage and combination coverage. We apply this coverage criterion to RTCP, as a new prioritization technique, code combinations coverage based prioritization (CCCP). We report on empirical studies conducted to compare the testing effectiveness and efficiency of CCCP with four popular RTCP techniques: total, additional, adaptive random, and search-based test prioritization. The experimental results show that even when the lowest combination strength is assigned, overall, the CCCP fault detection rates are greater than those of the other four prioritization techniques. The CCCP prioritization costs are also found to be comparable to the additional test prioritization technique. Moreover, our results also show that when the combination strength is increased, CCCP provides higher fault detection rates than the state-of-the-art, regardless of the levels of code coverage.

A Comprehensive Review on Regression Test Case Prioritization Techniques for Web Services

Test Case Prioritization (TCP) involves the rearrangement of test cases on a prioritized basis for various services. This research work focuses on TCP in web services, as it has been a growing challenge for researchers. Web services continuously evolve and hence require reforming and re-execution of test cases to ensure the accurate working of web services. This study aims to investigate gaps, issues, and existing solutions related to test case prioritization. This study examines research publications within popular selected databases. We perform a meticulous screening of research publications and selected 65 papers through which to answer the proposed research questions. The results show that criteria-based test case prioritization techniques are reported mainly in 41 primary studies. Test case prioritization models, frameworks, and related algorithms are also reported in primary studies. In addition, there are eight issues related to TCP techniques. Among these eight issues, optimization and high effectiveness are most discussed within primary studies. This systematic review has identified that a significant proportion of primary studies are not involved in the use of statistical methods in measuring or comparing the effectiveness of TCP techniques. However, a large number of primary studies use ‘Average Percentage of Faults Detected’ (APFD) or extended APFD metrics to compute the performance of techniques for web services.

Regression Test Case Prioritization Based on Fixed Size Candidate Set ART Algorithm

Regression testing is a very time-consuming and expensive testing activity. Many test case prioritization techniques have been proposed to speed up regression testing. Previous studies show that no one technique is always best. Random strategy, as the simplest strategy, is not always so bad. Particularly, when a test suite has higher fault detection capability, the strategy can generate a better result. Nevertheless, due to the randomness, the strategy is not always as satisfactory as expected. In this context, we present a test case prioritization approach using fixed size candidate set adaptive random testing algorithm to reduce the effect of randomness and improve fault detection effectiveness. The distance between pair-wise test cases is assessed by exclusive OR. We designed and conducted empirical studies on eight C programs to validate the effectiveness of the proposed approach. The experimental results, confirmed by a statistical analysis, indicate that the approach we proposed is more effective than random and the total greedy prioritization techniques in terms of fault detection effectiveness. Although the presented approach has comparable fault detection effectiveness to ART-based and the additional greedy techniques, the time cost is much lower. Consequently, the proposed approach is much more cost-effective.

Test Case Prioritization for Service-Oriented Workflow Applications: A Perspective of Modification Impact Analysis

Test case prioritization is a method to prioritize test cases to improve the testing efficiency of service-oriented workflow applications. Existing prioritization methods prioritize test cases in different application environments according to different metrics (for example, statement coverage, and path coverage). Web services are orchestrated by service-oriented workflow applications to provide different functions, especially in cloud-based mobile systems. As a result, those applications need more precise scheduling to run test cases that can detect faults earlier. Unfortunately, most regression test case prioritization studies in service-oriented workflow applications neglect the use of activity dependency, which is an important factor that affects test case prioritization. By analyzing the dependences between activities, the modification effects of activities on the modified version of service-oriented workflow applications are calculated. On this basis, this paper proposes a new prioritization method for regression test cases. Experimental results show that our method is more effective than the traditional coverage-based technique in testing case priority.

Regression Test Case Prioritization Frameworks: Challenges and Future Directions

Regression testing is a necessary maintenance activity in the software industry where modified software programs are revalidated to make sure that changes do not adversely affect their behavior. Test case prioritization (TCP) is one of the most effective methods in regression testing whereby test cases are rescheduled in an appropriate order for execution to increase test effectiveness in meeting some performance goals such as increasing the rate of fault detection. This paper explores efforts that have been carried out in relation to TCP frameworks. Through the review of related literature, ten existing frameworks were identified, classified and reviewed whereby two are Bayesian network-based, five are multi-objective, while the rest are varied in terms of aspects and purposes. Accordingly, this study analyzes those frameworks based on their proposed year, TCP factors, number of test cases used, evaluations metric and criteria as well as experimental subjects. The results showed that the stated frameworks are not integrated with nature-inspired algorithms as enhancing optimization techniques while several others were insufficiently evaluated according to stated evaluation criteria and metrics for the effective and practical testing process. There is also a scarcity of frameworks that focus on regression test efficiency. This study indicates the need for further research into the topic to enhance TCP frameworks that focus on several directions for practical considerations in this field such as evaluation issues, specific knowledge dependency, and objective deviation. At the end of this study, several future directions such as nature-inspired algorithms assistance are proposed, and a number of limitations are identified and highlighted.

Evolutionary Cost-cognizant Test Case Selection and Prioritization for Object-Oriented Programs

Test case prioritization (TCP) is a software testing technique that finds an ideal ordering of test cases for regression testing, so that testers can obtain the maximum benefit of their test suite, even if the testing process is stop at some arbitrary point. The recent trend of software development uses OO paradigm. This paper proposed a cost-cognizant TCP approach for object-oriented software that uses path-based integration testing. Path-based integration testing will identify the possible execution path and extract these paths from the Java System Dependence Graph (JSDG) model of the source code using forward slicing technique. Afterward evolutionary algorithm (EA) was employed to prioritize test cases based on the severity detection per unit cost for each of the dependent faults. The proposed technique was known as Evolutionary Cost-Cognizant Regression Test Case Prioritization (ECRTP) and being implemented as regression testing approach for experiment.

Novel Fuzzy Clustering Methods for Test Case Prioritization in Software Projects

Systematic Regression Testing is essential for maintaining software quality, but the cost of regression testing is high. Test case prioritization (TCP) is a widely used approach to reduce this cost. Many researchers have proposed regression test case prioritization techniques, and clustering is one of the popular methods for prioritization. The task of selecting appropriate test cases and identifying faulty functions involves ambiguities and uncertainties. To alleviate the issue, in this paper, two fuzzy-based clustering techniques are proposed for TCP using newly derived similarity coefficient and dominancy measure. Proposed techniques adopt grouping technology for clustering and the Weighted Arithmetic Sum Product Assessment (WASPAS) method for ranking. Initially, test cases are clustered using similarity//dominancy measures, which are later prioritized using the WASPAS method under both inter- and intra-perspectives. The proposed algorithms are evaluated using real-time data obtained from Software-artifact Infrastructure Repository (SIR). On evaluation, it is inferred that the proposed algorithms increase the likelihood of selecting more relevant test cases when compared to the recent state-of-the-art techniques. Finally, the strengths of the proposed algorithms are discussed in comparison with state-of-the-art techniques.

Efficiency Enhancement in Regression Test Case Prioritization Technique

Regression testing is an important, but expensive, process that has a powerful impact on software quality. Unfortunately all the test cases, existing and newly added, cannot be re-executed due to insufficient resources. In this scenario, prioritization of test case helps in improving the efficacy of regression testing by arranging the test cases in such a way that the most beneficial (that has the potential to detect the more number of faults) are executed first. Previous work and existing prioritization techniques, though detects faults, but there is a need of improved techniques to enhance the process of regression testing by improving the fault detection rate. The new technique, proposed in this paper, gives improved result than the existing ones. The comparison of the effectiveness of the proposed approach is done with other prioritization and non-prioritization orderings. The result of the proposed approach shows higher average percentage of faults detected (APFD) values. Also, the performance is evaluated and it is observed that the capability of the proposed method outperforms other algorithms by enhancing the fault detection rate.

Comparative Analysis of Regression Test Case Prioritization Techniques

Comparative Analysis of Regression Test Case Prioritization Techniques

A systematic literature study of regression test case prioritization approaches

Regression testing is about running the entire test ensemble again to ensure that amendments do not negatively affect the system. A popular approach in regression testing is test case prioritization which reorders test cases in such a way that those with higher priorities are run earlier than those with lower priorities based on some criterion. Numerous researchers have worked on different aspects of prioritization approaches. This paper presents the result of the study conducted on different prioritization approaches showing mostly stressed areas by researchers and the areas where there is a future scope. For it, studies related to test case prioritization in regression testing from the year 2004 to 2018 are analyzed by dividing this time period into three slots of five years each. 36 studies were selected from 948 studies to answer the research questions framed for this study. The trends followed in TCP along with the approaches evolved are thus documented to find the current trends and future scope for the researchers to work upon.