Software Regression Testing Research Articles

Optimizing regression testing with AHP-TOPSIS metric system for effective technical debt evaluation

Regression testing is essential to ensure that the actual software product confirms the expected requirements following modification. However, it can be costly and time-consuming. To address this issue, various approaches have been proposed for selecting test cases that provide adequate coverage of the modified software. Nonetheless, problems related to omitting and/or rerunning unnecessary test cases continue to pose challenges, particularly with regard to technical debt (TD) resulting from code coverage shortcomings and/or overtesting. In the case of testing-related shortcomings, incurring TD may result in cost and time savings in the short run, but it can lead to future maintenance and testing expenses. Most prior studies have treated test case selection as a single-objective or two-objective optimization problem. This study introduces a multi-objective decision-making approach to quantify and evaluate TD in regression testing. The proposed approach combines the analytic-hierarchy-process (AHP) method and the technique of order preference by similarity to an ideal solution (TOPSIS) to select the most ideal test cases in terms of objective values defined by the test cost, code coverage, and test risk. This approach effectively manages the software regression testing problems. The AHP method was used to eliminate subjective bias when optimizing objective weights, while the TOPSIS method was employed to evaluate and select test-case alternatives based on TD. The effectiveness of this approach was compared to that of a specific multi-objective optimization method and a standard coverage methodology. Unlike other approaches, our proposed approach always accepts solutions based on balanced decisions by considering modifications and using risk analysis and testing costs against potential technical debt. The results demonstrate that our proposed approach reduces both TD and regression testing efforts.

Method level static source code analysis on behavioral change impact analysis in software regression testing

Though a myriad of changes take place in a software system during maintenance, behavioral changes carry the bulk of the reasons of software modifications. In assessing the impact of the changes made in software, static source code analysis plays a key role. However, static source code analysis can be a little complex depending on the reason for the expedition. Despite the work done so far, little focus has been made on the potential of changed methods analysis during static source code analysis in assessing the impact of the changes made in a software system. We propose and investigate a static source code analysis technique that would generate information on the modified methods in the source code. This study analyzes four aThough a myriad of changes take place in a software system during maintenance, behavioral changes carry the bulk of the reasons for software modifications. In assessing the impact of the changes made in the software, static source code analysis can be a little complex depending on the reason for the expedition. Despite the works done so far, little focus has been directed on the potential of changed methods during static source code analysis, in assessing the impact of the changes made in software. This study investigates a method-level static source code analysis technique that would generate information on the methods affected by changes made in the software. The work analyzed three Java projects. The results indicate an improvement in leveraging on the knowledge of edited methods in change impact assessment during regression testing. The approach enhances code review efforts in light of assessing operational behavior impacted by the changes made.Java projects and shows that an analysis of the changed methods reveals the level of regression testing that ought to be conducted for the changes made.

Prioritizing software regression testing using reinforcement learning and hidden Markov model

Software regression testing is an essential testing practice that ensures that changes made to the source code of an application do not affect its functionality and quality. Within this research, we introduce a novel method for prioritizing software test cases using a fusion of reinforcement learning and hidden Markov model to enhance the efficiency of the testing process. The primary objective of this research paper is to maximize the likelihood of selecting test cases that have the highest priority of uncovering defects in new code changes introduced into the codebase. To assess the efficacy of our suggested methodology, we experimented on the test cases of five web applications. Our results demonstrate that our proposed approach can accurately identify critical test cases while minimizing false positives, as evidenced by an F1 score of 0.849. This outcome can help prioritize testing efforts, saving time, and resources while improving the overall efficiency of the testing process.

State of Practical Applicability of Regression Testing Research: A Live Systematic Literature Review

Context: Software regression testing refers to rerunning test cases after the system under test is modified, ascertaining that the changes have not (re-)introduced failures. Not all researchers’ approaches consider applicability and scalability concerns, and not many have produced an impact in practice. Objective: One goal is to investigate industrial relevance and applicability of proposed approaches. Another is providing a live review, open to continuous updates by the community. Method: A systematic review of regression testing studies that are clearly motivated by or validated against industrial relevance and applicability is conducted. It is complemented by follow-up surveys with authors of the selected papers and 23 practitioners. Results: A set of 79 primary studies published between 2016–2022 is collected and classified according to approaches and metrics. Aspects relative to their relevance and impact are discussed, also based on their authors’ feedback. All the data are made available from the live repository that accompanies the study. Conclusions: While widely motivated by industrial relevance and applicability, not many approaches are evaluated in industrial or large-scale open-source systems, and even fewer approaches have been adopted in practice. Some challenges hindering the implementation of relevant approaches are synthesized, also based on the practitioners’ feedback.

A Framework for Optimizing Software Regression Test Case based on Modified-Ant Colony Optimization (M-ACO)

Software Regression Testing (SRT) helps with the modified code to ensure that no other new defects have been trained and significantly impact the existing code. SRT is a time and cost taking process. Test Case Prioritization (TCP) helps to reschedule test cases (TC’S) and picks according to the prioritization, which allows the software testers to maximize the number of faults detected within a given time frame. Ant Colony Optimization (ACO) is an optimization algorithm motivated by the searching behavior of insects. This paper proposes a simple framework for the novel Modified-ACO (m-ACO) to prioritize the TC’s. m-ACO is planned as the adjusted type of the ACO method, which reschedules the execution sequence of TC’S by changing the wonder of characteristic insects for choosing their food source. The proposed Framework will help the software testers save their time to execute SRT TC’s. It has been tested for Zasta Billing Web Application and compared the time taken to complete the SRT manually and using the proposed Framework.The proposed Framework helps to reduce the time taken from 90 days to 45 days to achieve the SRT.

Test case prioritization techniques in software regression testing: An overview

The importance of Software Testing (ST) in the Software Development Life Cycle (SDLC) can never be ignored. Smarter ST can give us more relievable and defect-free products which are as per our stakeholder demand. That is the reason it takes more time and resources. But unfortunately, the limited time has left when the product reached the testing stage, especially in Regression Testing (RT). That is the reason proper planning is required in each SDLC phase, especially in the testing phase. Regression testing is an essential part of the testing phase. It guarantees defect-free software after any changes made to the requirement or software product. Because of the limited time available, it is impossible to execute all the test cases every time any changes made in the code there comes the role of the test case prioritization. TCP chooses only test cases that are most important to execute. The priority of test cases could be based on code, requirement, defects, execution time, cost, etc. After TCP the developer and tester have a minimum test case suite with better coverage in all respects. TCP definitely improves the quality of software and brings the best product within a limited time and cost. In this paper, we have provided a detailed survey of the TCP methodologies. This survey includes 2010 to the most recent studies. In this study, we have select carefully the most recent and relevant study of our topic.

Performance comparison of Bat search and Cuckoo search using software artefact infrastructure repository and regression testing

Performance comparison of Bat search and Cuckoo search using software artefact infrastructure repository and regression testing

Performance comparison of Bat search and Cuckoo search using software artefact infrastructure repository and regression testing

Regression testing is conducted to ensure that no new errors have been introduced into the software as a result of the maintenance activity performed. Re-executing all the existing test cases is however a highly expensive and time consuming approach to gain this confidence. Previous research revealed that nature inspired algorithms have vast application in this area. In this paper, Bat search algorithm is tested against Cuckoo search algorithm to solve regression test case selection problem. Two factors: number of faults covered and computational time are considered for the comparison. Extensive experiments have been conducted over the objects adopted from benchmarked software artifact infrastructure repository. Rigorous statistical tests are conducted to draw a conclusion that Cuckoo search is marginally advantageous over Bat search algorithm with respect to performance parameters. We believe that the results reported in this paper will enable researchers to develop more powerful algorithm for testing in near future.

I-Rex

We demonstrate I-Rex 1 , a system designed to help users understand SQL query evaluation and debug SQL queries. I-Rex lets users interactively "trace" the evaluation of complex SQL queries, including those with correlated subqueries. I-Rex also explains why a query returns an incorrect answer with respect to a reference query over a test database instance---a common use case in education and software regression testing. To avoid the cognitive overload caused by debugging over a large database instance, I-Rex lets users focus on smaller instances contained in the large one (which we call "counterexamples") that still distinguish the two queries. Supporting these features for SQL queries poses two key challenges. First, unlike debugging for procedural languages, it is not clear how to trace a declarative SQL query, because its execution plan often differs from how it was originally written. I-Rex offers a novel interface for tracing SQL query evaluation in a way faithful to how queries are written syntactically, even for complex queries involving multiple levels of nesting and correlation. Second, we need a method for finding small counterexamples that handles the complexity of practical SQL. I-Rex extends provenance support for SQL in non-trivial ways to work with various query constructs. This demonstration walks through use cases in which I-Rex helps users understand and debug SQL queries.

Nowatorska metoda priorytetyzacji przypadków testowych oparta na prognozowaniu błędów instrukcji kodu oprogramowania numerycznego

Test case prioritization (TCP) has been considerably utilized to arrange the implementation order of test cases, which contributes to improve the efficiency and resource allocation of software regression testing. Traditional coverage-based TCP techniques, such as statement-level, method/function-level and class-level, only leverages program code coverage to prioritize test cases without considering the probable distribution of defects. However, software defect data tends to be imbalanced following Pareto principle. Instinctively, the more vulnerable the code covered by the test case is, the higher the priority it is. Besides, statement-level coverage is a more fine-grained method than function-level coverage or class-level coverage, which can more accurately formulate test strategies. Therefore, we present a test case prioritization approach based on statement software defect prediction to tame the limitations of current coverage-based techniques in this paper. Statement metrics in the source code are extracted and data pre-processing is implemented to train the defect predictor. And then the defect detection rate of test cases is calculated by combining the prioritization strategy and prediction results. Finally, the prioritization performance is evaluated in terms of average percentage faults detected in four open source datasets. We comprehensively compare the performance of the proposed method under different prioritization strategies and predictors. The experimental results show it is a promising technique to improve the prevailing coverage-based TCP methods by incorporating statement-level defect-proneness. Moreover, it is also concluded that the performance of the additional strategy is better than that of max and total, and the choice of the defect predictor affects the efficiency of the strategy.

A Review of Automated Testing Approach for Software Regression Testing

One of the important parts of software development life-cycle is software testing and one of which is regression testing. Regression testing is done to verify that previously detected errors have been corrected correctly and no new errors arise as a result of incorrect corrections so that the quality of the software being tested is maintained. It is important and must be done. However, the conventional method is not efficient as it is time-consuming, not reusable, and prone to error. In this paper, a comprehensive review of the automated testing approach is presented to be used by other researchers in this field of study. The review result shows that the automated testing approach is suitable to enhance the regression testing with some plausible options of tools e.g. Selenium, SAHI, and Robot-framework. Furthermore, some plausible options of execution methods are also discussed in this paper. This review further concludes that the parallel execution method is considered as a promising choice to conduct the most efficient regression testing process.

Regression Testing of Service-Oriented Software

Regression testing is considered as a separate forms of testing attached with performance testing where tester runs old test suits after each change made to system. It will be a big problem in testing the service-oriented software (SOS), where each system component is inherently agile and changes its behavior dynamically. These agile component gives a big rise to big problem in the regression testing process with respect to complexity, time complexity and cost complexity. A service-oriented software may change in case of bug fixing, adaptation of new environment, upgrading or updating functionality in order to improve performance or it is demanded by customer. After the software is delivered to customer the service oriented software must be regressed to validate that there is no defects. We present a hierarchical regression test selection algorithm for service-oriented software, and evaluate it in service-oriented environment along with results.

Mahtab: Phase-wise acceleration of regression testing for C

Software regression testing consists of offline, online, and execution phases which are executed sequentially. The offline phase involves code instrumentation and test-coverage collection. Subsequently, the online phase performs program differencing, test-suite selection and prioritization. Finally, the selected test-cases are executed against the new version of software for its re-validation. Regression testing is a time-consuming process and is often on the critical path of the project. To improve the turn-around time of software development cycle, our goal is to reduce regression testing time across all phases using multi-core parallelization. This poses several challenges that stem from I/O, dependence on third-party libraries, and inherently sequential components in the overall testing process. We propose parallelization test-windows to effectively partition test-cases across threads. To measure the benefit of prioritization coupled with multi-threaded execution, we propose a new metric, EPSilon, for rewarding failure observation frequency in the timeline of test-execution. To measure the rate of code-change coverage due to regression test prioritization, we introduce ECC, a variant of the widely used APFD metric. We illustrate the effectiveness of our approach using the popular Software-artifact Infrastructure Repository (SIR) and five real-world projects from GitHub.

Framework for Optimizing Test Cases in Regression Testing

Software once developed is subject to continuous changes. Software Regression Testing thus can be used to reduce the efforts of testing the software by selecting only the required number of test cases and ordering them to test the software after changes have been made to it. In order to improve the fault detection rate, the selection of efficient test cases and order of execution of these tests is important. Here is when the test case selection comes into picture where in, the fault detection rate during the working of any software has to be improved. The test case selection process will find the most efficient test cases which can fully functionally test the software that has been modified. This indeed will contribute to an improved fault detection rate which can provide faster feedback on the system under test and let software engineers begin correcting faults as early as possible. In this paper, an approach for test case selection is proposed which takes into consideration the effect of three parameters History, Coverage and Requirement all together in order to improve the selection process. This will also ensure that the rejection of some efficient test cases is reduced when compared to the selection process in conventional methods, most of them making use of a single parameter for test case selection. These Test cases are further optimized using Genetic Algorithm. Results indicate that the proposed technique is much more efficient in terms of selecting the test cases when compared to conventional techniques, thereby improving fault detection rate.

Regression Testing of Database Applications Under an Incremental Software Development Setting

Software regression testing verifies previous features on a software product when it is modified or new features are added to it. Because of the nature of regression testing it is a costly process. Different approaches have been proposed to reduce the costs of this activity, among which are: minimization, prioritization, and selection of test cases. Recently, soft computing techniques, such as data mining, machine learning, and others have been used to make regression testing more efficient and effective. Currently, in different contexts, to a greater or lesser extent, software products have access to databases (DBs). Given this situation, it is necessary to consider regression testing also for software products such as information systems that are usually integrated with or connected to DBs. In this paper, we present a selection regression testing approach that utilizes a combination of unsupervised clustering with random values, unit tests, and the DB schema to determine the test cases related to modifications or new features added to software products connected to DBs. Our proposed approach is empirically evaluated with two database software applications in a production context. Effectiveness metrics, such as test suite reduction, fault detection capability, recall, precision, and the F-measure are examined. Our results suggest that the proposed approach is enough effective with the resulting clusters of test cases.

An Efficient Regression Testing Test Suite Optimization System with Quality Metrics

To make certain the software Quality, Software testing is one of the most significant processes in Software Development Life Cycle (SDLC). Software systems develop regularly to offer the necessary functionalities and to acclimatize to ever-changing customer necessitates. Regression testing points out to the portion of the test cycle in which a syllabus is experimented to make certain that alters do not influence characteristics such as adding novel characteristics or adapting presented characteristics that are not supposed to be influenced. The Regression testing is used to modified versions of the software to make certain that the modified characteristics perform equally and that the changes did not generate unexpected faults, moreover known as deterioration errors. As a result, in order to spot such errors, Software regression testing is necessary. The most important plan of the suggested system is to execute the software regression testing with a reduced amount of time and cost devoid of changing the quality. For this, we will promise the quality of the software tool in terms of quality metrics such as dependability and maintainability, etc after optimizing the test suite. By using Improved Particle Swarm Optimization Algorithm (IPSO), the test suite optimization will be prepared. With different devices such as JDepend and Rayleigh’s model, the quality metrics will be computed.

15 Years of Software Regression Testing Techniques — A Survey

Software regression testing techniques verify previous functionalities each time software modifications occur or new characteristics are added. With the aim of gaining a better understanding of this subject, in this work we present a survey of software regression testing techniques applied in the last 15 years; taking into account its application domain, kind of metrics they use, its application strategies and the phase of the software development process where they are applied. From an outcome of 460 papers, a set of 25 papers describing the use of 31 software testing regression techniques were identified. Results of this survey suggest that at the time of applying a regression testing techniques, metrics like cost and fault detection efficiency are the most relevant. Most of the techniques were assessed with instrumented programs (experimental cases) under academic settings. Conversely, we observe a minimum set of software regression techniques applied in industrial settings, mainly, under corrective and maintenance approaches. Finally, we observe a trend using some regression techniques under agile approaches.

Software Regression Test Case Prioritization for Object-Oriented Programs using Genetic Algorithm with Reduced-Fitness Severity

One of the most important activities in software maintenance is Regression Testing. The re-execution of all test cases during the regression testing is costly. And even though several of the code based proposed techniques by researchers address procedural programs. Some researchers addressed the issue of test case prioritization using Genetic Algorithm, but most of the authors do not select modification-revealing before prioritization, and used the same fault severity even when a fault was executed by the preceding test case. This paper propose a regression test case selection for object-oriented software based on dependence graph model analysis of the source code and optimizes the selected test case using Genetic Algorithm. The goal is to select modification-revealing test cases and order them based on their fitness value computed using previous history of the fault severity. The effectiveness of the approach was evaluated using Average Percentage of Rate of Fault Detection (APFD) metric with vending machine program. Our Proposed Approach (PA) provides considerably better results in term of average percentage of rate of fault detection. Based on the measured performance obtained from the results, GA with reduced severity of fault prioritize selected test cases more effectively compared to using GA with the same severity of fault and non-prioritize, which may result in reducing the cost of Regression Testing.

A Systematic State-Based Approach to Regression Testing of Component Software

Today, component-based software engineering has been widely used in software construction to reduce project cost and speed up software development cycle. Due to software changes in new release or update of components, regression testing is needed to assure system quality. When changes made to a component, the component could be affected, moreover, the changes could bring impacts on the entire system. We firstly identify diverse changes made to components and system based on models, then perform change impact analysis, and finally refresh regression test suite using a state-based testing practice. Related existing research did not address the issue of systematic regression testing of component-based software, especially at system level. The paper also reports a case study based on a realistic component-based software system using a, which shows that the approach is feasible and effective.

Study on Optimization of Software Regressive Testing Based on RBF Neural Networks

In order to reduce times of software regression testing, a new research idea and method is proposed based on RBFN (Radial Basis Function Network). Using the adaptive ability of network study, regression testing is optimized by its learning strategy. The simulation results demonstrate the new method can forecast regressive testing effectively, and implement very little error. It means an important meaning for developing new effective method of soft testing in the future.