Regression Test Cases Research Articles

UART IP core Design and Verification using WISHBONE Interface

The organization of communication between the devices is greatly aided by the communication protocol. These protocols have a clear set of guidelines that have been agreed upon by the tools needed for effective communication. A popular protocol for serial communication is UART. The hardware description language for the SV/Verilog UART functional module is designed in this paper. The serial connection capabilities offered by this UART IP CORE enable interaction with modems and other external devices. The Universal Verification Technique is used to undertake functional verification of the UART. By comparing the collected response to the intended response via the scoreboard mechanism, the reusable UVM test bench architecture is meant to push the randomized stimuli to the unit under test to assess the functional correctness. This core is made to be as compatible as possible with commercially accepted designs. The WISHBONE INTERFACE WITH 8-BIT OR 32-BIT SELECTABLE DATA BUS MODES is one of this design's primary characteristics. Interface debugging in 32-bit data bus mode. Functionality and register-level compatibility. FIFO procedure. The UVM methodology is used to validate the design. For optimal functional coverage, the test bench is written with regression test cases. The overall execution time will be shortened if the stimulus may be reused.

Multi-factor based Regression Test Case Prioritization using Fuzzy Logic

The maintenance level activity generally done after the modification in the software to check whether it is functioning right or not is termed as regression testing. Test case prioritization, a key practice, involves strategically ordering test cases based on specific criteria to enhance the efficiency of fault detection within a condensed time frame. The fuzzy rule base serves as an alternative to the conventional crisp value set, offering a nuanced approach beyond binary outcomes (Yes or No). The primary objective of this research is to address critical factors often overlooked in existing literature on prioritization. Notably, prevalent approaches focus on singular factors during test case prioritization, highlighting the need for a comprehensive technique. To enhance the prioritization of test cases, there is a demand for a method that considers multi-factors or combinations thereof, ultimately increasing effectiveness. This paper introduces an innovative approach a multi-factors regression test-case prioritization technique utilizing fuzzy rules. The methodology aims to optimize the prioritization of test cases, striking a balance between effectiveness and time efficiency. Fuzzy rules are formulated to assess the effectiveness of a prioritized set of test cases in developing the proposed approach. A user-friendly tool has been developed to facilitate the application of this technique, allowing users to input relevant factors and subsequently prioritize test cases accordingly. Through extensive experiments using the developed tool, the effectiveness of the proposed approach has been validated. The results demonstrate that the priority lists of test cases generated for different projects, considering multi-factors, show greater promise compared to techniques relying solely on a single factor for prioritization.

Fault-Based Regression Test Case Prioritization

We propose a set of four novel fault-based regression test case prioritization (TCP) techniques for object-oriented programs. We seed bugs into a program to create large number of mutants. We execute each mutant with the originally designed test suite. From this, we record the number of mutants for which a test case fails. Based on this, we prioritize the test cases using four base fault-based prioritization techniques that we have proposed. Finally, we combine the results of our four base prioritizers using three ensemble methods. We have conducted experimental studies to determine the effectiveness of our proposed approaches. Our experimental results show that our proposed TCP techniques exhibit superior performance over related techniques.

Retracted: A Systematic Analysis of Regression Test Case Selection: A Multi-Criteria-Based Approach

Retracted: A Systematic Analysis of Regression Test Case Selection: A Multi-Criteria-Based Approach

A review paper: optimal test cases for regression testing using artificial intelligent techniques

<span lang="EN-US">The goal of the testing process is to find errors and defects in the software being developed so that they can be fixed and corrected before they are delivered to the customer. Regression testing is an essential quality testing technique during the maintenance phase of the program as it is performed to ensure the integrity of the program after modifications have been made. With the development of the software, the test suite becomes too large to be fully implemented within the given test cost in terms of budget and time. Therefore, the cost of regression testing using different techniques should be reduced, here we dealt many methods such as retest all technique, regression test selection technique (RTS) and test case prioritization technique (TCP). The efficiency of these techniques is evaluated through the use of many metrics such as average percentage of fault detected (APFD), average percentage block coverage (APBC) and average percentage decision coverage (APDC). In this paper we dealt with these different techniques used in test case selection and test case prioritization and the metrics used to evaluate their efficiency by using different techniques of artificial intelligent and describe the best of all.</span>

T-Evos: A Large-Scale Longitudinal Study on CI Test Execution and Failure

Continuous integration is widely adopted in software projects to reduce the time it takes to deliver the changes to the market. To ensure software quality, developers also run regression test cases in a continuous fashion. The CI practice generates commit-by-commit software evolution data that provides great opportunities for future testing research. However, such data is often unavailable due to space limitation (e.g., developers only keep the data for a certain period) and the significant effort involved in re-running the test cases on a per-commit basis. In this paper, we present T-Evos, a dataset on test result and coverage evolution, covering 8,093 commits across 12 open-source Java projects. Our dataset includes the evolution of statement-level code coverage for every test case (either passed and failed), test result, all the builds information, code changes, and the corresponding bug reports. We conduct an initial analysis to demonstrate the overall dataset. In addition, we conduct an empirical study using T-Evos to study the characteristics of test failures in CI settings. We find that test failures are frequent, and while most failures are resolved within a day, some failures require several weeks to resolve. We highlight the relationship between code changes and test failure, and provide insights for future automated testing research. Our dataset may be used for future testing research and benchmarking in CI. Our findings provide an important first step in understanding code coverage evolution and test failures in a continuous environment.

Demystifying Performance Regressions in String Solvers

Over the past few years, SMT string solvers have found their applications in an increasing number of domains, such as program analyses in mobile and Web applications, which require the ability to reason about string values. A series of research has been carried out to find quality issues of string solvers in terms of its correctness and performance. Yet, none of them has considered the performance regressions happening across multiple versions of a string solver. To fill this gap, in this paper, we focus on solver performance regressions (SPRs), i.e., unintended slowdowns introduced during the evolution of string solvers. To this end, we develop <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">SPRFinder</i> to not only generate test cases demonstrating SPRs, but also localize the probable causes of them, in terms of commits. We evaluated the effectiveness of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">SPRFinder</i> on three state-of-the-art string solvers, i.e., Z3Seq, Z3Str3, and CVC4. The results demonstrate that <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">SPRFinder</i> is effective in generating SPR-inducing test cases and also able to accurately locate the responsible commits. Specifically, the average running time on the target versions is 13.2× slower than that of the reference versions. Besides, we also conducted the first empirical study to peek into the characteristics of SPRs, including the impact of random seed configuration for SPR detection, understanding the root causes of SPRs, and characterizing the regression test cases through case studies. Finally, we highlight that 149 unique SPR-inducing commits were discovered in total by <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">SPRFinder</i> , and 27of them have been confirmed by the corresponding developers.

Application of the Law of Minimum and Dissimilarity Analysis to Regression Test Case Prioritization

Regression testing is one of the most expensive processes in testing. Prioritizing test cases in regression testing is critical for the goal of detecting the faults sooner within a large set of test cases. We propose a test case prioritization (TCP) technique for regression testing called <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">LoM-Score</i> inspired by the Law of Minimum (LoM) from biology. This technique calculates the impact probabilities of methods calculated by change impact analysis with forward slicing and orders test cases according to LoM. However, this ordering doesn’t consider the possibility that consecutive test cases may be covering the same methods repeatedly. Thereby, such ordering can delay the time of revealing faults that exist in other methods. To solve this problem, we enhance the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">LoM-Score</i> TCP technique with an adaptive approach, namely with a dissimilarity-based coordinate analysis approach. The dissimilarity-based coordinate analysis uses Jaccard Similarity for calculating the similarity coefficients between test cases in terms of covered methods and the enhanced technique called Dissimilarity-LoM-Score ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Dis-LoM-Score</i> ) applies a penalty with respective on the ordered test cases. We performed our case study on 10 open-source Java projects from Defects4J, which is a dataset of real bugs and an infrastructure for controlled experiments provided for software engineering researchers. Then, we hand-seeded multiple mutants generated by Major, which is a mutation testing tool. Then we compared our TCP techniques <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">LoM-Score</i> and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Dis-LoM-Score</i> with the four traditional TCP techniques based on their Average Percentage of Faults Detected (APFD) results.

An Optimized Test Case Minimization Technique Using Genetic Algorithm for Regression Testing

Regression testing is a widely used approach to confirm the correct functionality of the software in incremental development. The use of test cases makes it easier to test the ripple effect of changed requirements. Rigorous testing may help in meeting the quality criteria that is based on the conformance to the requirements as given by the intended stakeholders. However, a minimized and prioritized set of test cases may reduce the efforts and time required for testing while focusing on the timely delivery of the software application. In this research, a technique named TestReduce has been presented to get a minimal set of test cases based on high priority to ensure that the web application meets the required quality criteria. A new technique TestReduce is proposed with a blend of genetic algorithm to find an optimized and minimal set of test cases. The ultimate objective associated with this study is to provide a technique that may solve the minimization problem of regression test cases in the case of linked requirements. In this research, the 100-Dollar prioritization approach is used to define the priority of the new requirements.

Metaheuristics for Multi Criteria Test Case Prioritization for Regression Testing

Under the constraints of project deliveries, it is too costly to frequency run a large number of test cases. Test case prioritization is needed to rank the test cases. The prioritization must be done in such way that it is able to detect maximum number of faults in available time. Though many test case prioritization techniques have been proposed they have not considered the risk is skipping the test cases. This work solves the test case prioritization or test case subset selection as a multi criteria optimization problem. A meta heuristics algorithm in proposed in this work combining Particle swarm optimization (PSO) with Bat algorithm is proposed to solve the problem of finding best subset of regression test cases with multi objectives of reducing risk in skipping test cases, maximizing the number of faults within the constraints of time

Comparative Analysis of Efficacious Metaheuristic Technique with Genetically Modified- Flower Pollination Algorithm (GM-FPA) for Test Case Prioritization in Regression Testing

Abstract: Regression Testing is most imperative activity of software development life cycle. Test case prioritization being one of the most adopted branch for regression testing and with the invent of nature inspired metaheuristic techniques in optimization, this study makes an attempt to augments the features of test case prioritization with nature inspired metaheuristic techniques to determine the most efficacious metaheuristic techniques from Cuckoo Searh (CS) algorithm, Genetic Algorithm (GA) and Flower Pollination Algorithm (FPA) for three different case studies. APFD metrics is used to compare the algorithms. Further the study compares the most efficacious technique with Genetically Modified- Flower Pollination Algorithm (GM-FPA) to identify the most efficient technique for regression test case prioritization.

A novel chaotic archimedes optimization algorithm and its application for efficient selection of regression test cases

A novel chaotic archimedes optimization algorithm and its application for efficient selection of regression test cases

Analytic hierarchy process-based regression test case prioritization technique enhancing the fault detection rate

Analytic hierarchy process-based regression test case prioritization technique enhancing the fault detection rate

Developer-centric test amplification

Automatically generating test cases for software has been an active research topic for many years. While current tools can generate powerful regression or crash-reproducing test cases, these are often kept separately from the maintained test suite. In this paper, we leverage the developer’s familiarity with test cases amplified from existing, manually written developer tests. Starting from issues reported by developers in previous studies, we investigate what aspects are important to design a developer-centric test amplification approach, that provides test cases that are taken over by developers into their test suite. We conduct 16 semi-structured interviews with software developers supported by our prototypical designs of a developer-centric test amplification approach and a corresponding test exploration tool. We extend the test amplification tool DSpot, generating test cases that are easier to understand. Our IntelliJ plugin TestCube empowers developers to explore amplified test cases from their familiar environment. From our interviews, we gather 52 observations that we summarize into 23 result categories and give two key recommendations on how future tool designers can make their tools better suited for developer-centric test amplification.

ExVivoMicroTest: ExVivo Testing of Microservices

AbstractMicroservice‐based applications consist of multiple services that can evolve independently. When a service must be updated, it is first tested with in‐house regression test suites. However, the test suites that are executed are usually designed without the exact knowledge about how the services will be accessed and used in the field; therefore, they may easily miss relevant test scenarios, failing to prevent the deployment of faulty services. To address this problem, we introduce ExVivoMicroTest, an approach that analyzes the execution of deployed services at run‐time in the field, in order to generate test cases for future versions of the same services. ExVivoMicroTest implements lightweight monitoring and tracing capabilities, to inexpensively record executions that can be later turned into regression test cases that capture how services are used in the field. To prevent accumulating an excessive number of test cases, ExVivoMicroTest uses a test coverage model that can discriminate the recorded executions between the ones that are worth to be turned into test cases and the ones that should be discarded. The resulting test cases use a mocked environment that fully isolates the service under test from the rest of the system to faithfully reply interactions. We assessed ExVivoMicroTest with the PiggyMetrics and Train Ticket open source microservice applications and studied how different configurations of the monitoring and tracing logic impact on the capability to generate test cases.

Validating Switch Variable of Flower Pollination Algorithm in Regression Test Case Prioritization

This study aims at analysing and validating the impact of switch variable of Flower Pollination Algorithm (FPA) in regression test case prioritization. The value of switch variable decides whether local or global optimization will take place. The present study makes an attempt to analyse the impact of switch variable with values ranging from 0.8 to 0.1 on test case prioritization in regression testing. The present study is an extension of the prior work done to implement FPA for test case prioritization in regression testing. To validate the study two case studies written in Java programming were used. The results were validated using the APFD metrics. From the empirical results it was deduced that values ranging between 0.8 and 0.5 of switch variable gave promising results for test case prioritization.

A Framework for Optimizing Software Regression Test Case based on Modified-Ant Colony Optimization (M-ACO)

Software Regression Testing (SRT) helps with the modified code to ensure that no other new defects have been trained and significantly impact the existing code. SRT is a time and cost taking process. Test Case Prioritization (TCP) helps to reschedule test cases (TC’S) and picks according to the prioritization, which allows the software testers to maximize the number of faults detected within a given time frame. Ant Colony Optimization (ACO) is an optimization algorithm motivated by the searching behavior of insects. This paper proposes a simple framework for the novel Modified-ACO (m-ACO) to prioritize the TC’s. m-ACO is planned as the adjusted type of the ACO method, which reschedules the execution sequence of TC’S by changing the wonder of characteristic insects for choosing their food source. The proposed Framework will help the software testers save their time to execute SRT TC’s. It has been tested for Zasta Billing Web Application and compared the time taken to complete the SRT manually and using the proposed Framework.The proposed Framework helps to reduce the time taken from 90 days to 45 days to achieve the SRT.

Systematic Literature Review on Test Case Selection and Prioritization: A Tertiary Study

Software testing is undertaken to ensure that the software meets the expected requirements. The intention is to find bugs, errors, or defects in the developed software so that they can be fixed before deployment. Testing of the software is needed even after it is deployed. Regression testing is an inevitable part of software development, and must be accomplished in the maintenance phase of software development to ensure software reliability. The existing literature presents a large amount of relevant knowledge about the types of techniques and approaches used in regression test case selection and prioritization (TCS&amp;P), comparisons of techniques used in TCS&amp;P, and the data used. Numerous secondary studies (surveys or reviews) have been conducted in the area of TCS&amp;P. This study aimed to provide a comprehensive examination of the analysis of the enhancements in TCS&amp;P using a thorough systematic literature review (SLR) of the existing secondary studies. This SLR provides: (1) a collection of all the valuable secondary studies (and their qualitative analysis); (2) a thorough analysis of the publications and the trends of the secondary studies; (3) a classification of the various approaches used in the secondary studies; (4) insight into the specializations and range of years covered in the secondary texts; (5) a comprehensive list of statistical tests and tools used in the area; (6) insight into the quality of the secondary studies based on the seven selected Research Paper Quality parameters; (7) the common problems and challenges encountered by researchers; (8) common gaps and limitations of the studies; and (9) the probable prospects for research in the field of TCS&amp;P.

Comparison of Multi-Objective Evolutionary Algorithms to Prioritize Regression Test Cases

Regression testing is one of the most critical testing activities among software product verification activities. Nevertheless, resources and time constraints could inhibit the execution of a full regression test suite, hence leaving us in confusion on what test cases to run to preserve the high quality of software products. Different techniques can be applied to prioritize test cases in resource-constrained environments, such as manual selection, automated selection, or hybrid approaches. Different Multi-Objective Evolutionary Algorithms (MOEAs) have been used in this domain to find an optimal solution to minimize the cost of executing a regression test suite while obtaining maximum fault detection coverage as if the entire test suite was executed. MOEAs achieve this by selecting set of test cases and determining the order of their execution. In this paper, three Multi Objective Evolutionary Algorithms, namely, NSGA-II, IBEA and MoCell are used to solve test case prioritization problems using the fault detection rate and branch coverage of each test case. The paper intends to find out what’s the most effective algorithm to be used in test cases prioritization problems, and which algorithm is the most efficient one, and finally we examined if changing the fitness function would impose a change in results. Our experiment revealed that NSGA-II is the most effective and efficient MOEA; moreover, we found that changing the fitness function caused a significant reduction in evolution time, although it did not affect the coverage metric.

A Prioritization Approach for Regression Test Cases Based on a Revised Genetic Algorithm

The regression testing is a software-based testing approach executed to verify that changes made to the softwaredo not affect the existing functionality of the product. On account of the constraints of time and cost, it isimpractical to re-execute all the test cases for software whenever a change occurs. In order to overcome sucha problem in the selection of regression test cases, a prioritization technique should be employed. On the basisof some predefined criterion, the prioritization techniques create an execution schedule for the test cases, sothe higher priority test cases can be performed earlier than the lower priority test cases in order to improvethe efficiency of the software testing. Many prioritization criteria for regression test cases have been proposedin software testing literature; however, most of such techniques are code-based. Keeping in view this fact, thisresearch work has proposed a prioritization approach for regression test cases generated from software specificationswhich are based on the criterion of the Average Percentage Transition Coverage (APTC) by using arevised genetic algorithm. This criterion evaluates the rate of transitions coverage by incorporating knowledgeabout the significance of transitions between activates in the form of weights. APTC has been used as a fitnessevaluation function in a genetic algorithm to measure the effectiveness of a test cases sequence. Moreover, inorder to improve the coverage percentage, the proposed approach has revised the genetic algorithm by solvingthe problem of the optimal local solution. The experimental results show that the proposed approach demonstratesa good coverage performance with less execution time as compared to the standard genetic algorithmand some other prioritization techniques.